FixedSizeMemory — High-Performance Thread-Local Memory Pool

FixedSizeMemory is a ultra-fast, highly scalable thread-local fixed-size memory pool engineered for modern high-concurrency systems. By combining TLS (Thread-Local Storage) fast paths, segmented page locking, and Cache Line alignment, it eliminates global lock contention and false sharing, delivering billions of allocations per second.

🌟 Key Features

• ⚡ Thread-Local Fast Path (TLS): Zero-lock allocation and deallocation paths via local thread caches (LOCAL_CACHE_SIZE = 256), minimizing global synchronization.
• 🔒 Fine-Grained Page Locks (alignas(64) Page): Avoids global lock contention during cache refills/drains by using per-page mutexes.
• 🛡️ False-Sharing Prevention: Global thread slots (ThreadOpSlot) and internal page structures are aligned to 64-byte Cache Lines (alignas(64)).
• ♻️ Lazy Page Decommissioning: Automatically reclaims and manages completely free memory pages (is_empty_candidate) to maintain low memory footprints.
• 🧹 Explicit TLS Flushing: Built-in support for flushing thread caches upon thread termination (flush_thread_cache()) to ensure zero memory leaks.
• 📊 Deep Diagnostics: Thread-safe runtime tracking for total active operations, allocated block counts, free blocks, and total system pages.

🏗️ Architectural Overview

 +-----------------------------+
| FixedSizeMemory Pool |
+-----------------------------+
|
+---------------------+---------------------+
| |
[ Thread-Local Cache ] [ Thread-Local Cache ]
+--------------------+ +--------------------+
| Thread 1 Free List | | Thread N Free List |
+--------------------+ +--------------------+
| (Refill / Drain) | (Refill / Drain)
v v
+---------------------------------------------------------------+
| Segmented Page Array |
| +--------------------+ +--------------------+ |
| | Page 1 (Mutex 1) | . . . . | Page M (Mutex M) | |
| +--------------------+ +--------------------+ |
+---------------------------------------------------------------+

1. Allocations/Deallocations: First attempt to hit the thread-local free list (Node*).
2. Refill/Drain: When the local cache is exhausted or overfilled, memory blocks are transferred in batches (REFILL_COUNT = 128) between the thread cache and a page using fine-grained locks.
3. Expansion: When all pages are full, a new Page is lazily allocated.

🚀 Benchmark Performance

Benchmarking results measured on an 8-core/16-thread Cloud Server (4 Physical Cores @ 2.45 GHz with SMT):

1. Micro-Benchmark (Google Benchmark)

• Standard Metric: Total system throughput across all threads (1 Operation = 1 allocate() + 1 deallocate() pair).

Concurrent Threads	System Throughput (QPS)	Single Allocation Latency
1 Thread	195 Million ops/s	2.56 ns
4 Threads	3.11 Billion ops/s	0.64 ns
8 Threads (SMT)	6.32 Billion ops/s	0.63 ns
16 Threads	13.29 Billion ops/s	0.60 ns

⚡ 3.2x faster than system std::malloc under multi-threaded concurrency.

2. Soak / Stability Benchmark (10-Minute Stress Test)

• Operations Executed: 234+ Billion allocations & deallocations.
• Interval QPS: Maintained a rock-solid 390 Million ops/s under real-world workloads with payload construction, verification, and random pauses.
• Stability: 0% Performance Degradation, 0 Memory Leaks, and 0 Corruption.

📦 Usage Guide

Prerequisites

• C++17 or later compatible compiler (GCC 8+, Clang 7+, MSVC 2019+)
• Standard C++ Threads and Atomics support

Quick Example

C++

#include "memory.hpp"
#include <iostream>
#include <thread>
#include <vector>
struct Packet {
uint32_t id;
char payload[28];
};
int main() {
// Instantiate memory pool: 64 bytes per block, 1024 blocks per page
FixedSizeMemory pool(sizeof(Packet), 1024);
std::vector<std::thread> workers;
for (int t = 0; t < 4; ++t) {
workers.emplace_back([&pool]() {
// 1. Allocate block
void* raw_mem = pool.allocate();
// 2. Placement new
Packet* pkt = new (raw_mem) Packet{1001, "Hello Pool"};
// 3. Do work...
// 4. Destruct and deallocate
pkt->~Packet();
pool.deallocate(pkt);
// 5. Clean up thread cache upon thread exit
pool.flush_thread_cache();
});
}
for (auto& w : workers) {
w.join();
}
std::cout << "Active allocations remaining: " << pool.allocated_count() << std::endl; // Output: 0
return 0;
}

🛠️ API Reference

Core Allocation API

Signature	Description
explicit FixedSizeMemory(size_t block_size, size_t blocks_per_page = 1024)	Construct memory pool with block size and page capacity.
void* allocate()	Allocate a memory block from TLS or Page.
void deallocate(void* p) noexcept	Return a memory block to TLS or Page.
void flush_thread_cache()	Flush current thread's TLS cache back to global pages.

Diagnostics & Inspection

Signature	Description
size_t allocated_count() const	Returns total currently unreturned allocated blocks.
size_t page_count() const noexcept	Returns total allocated system pages.
size_t free_count() const noexcept	Returns total currently available free blocks.

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.