Inspired by https://x.com/BenjDicken note about date page buffering
https://x.com/BenjDicken/status/1960737322636599695
I decided to dive into PostgreSQL’s buffer management internals. Most of this comes from the excellent page in Postgres repository
https://github.com/postgres/postgres/blob/master/src/backend/storage/buffer/README
but I’ll try to explain it in a more approachable way. If you want the full technical deep dive, definitely check out the original file.
TL;DR 📝
- Shared Buffers: PostgreSQL keeps a chunk of RAM as a cache of 8KB pages from disk. All backends share this pool.
- Pins (reference counts): Protect a buffer from being evicted while it’s in use.
- Buffer content locks: Protect the contents of a page from corruption during concurrent writes.
- Free list: Linked list of completely unused buffers.
- Clock-sweep algorithm: When no free buffers exist, Postgres cycles through the buffer array with a “clock hand” (nextVictimBuffer), using a usage counter to approximate LRU:
- Pinning a buffer increments its usage counter.
- The clock hand decrements usage counters until it finds a buffer with usage = 0 and no pins → that buffer can be recycled.
- Background writer: Proactively writes dirty pages to disk so backends don’t stall when evicting them.
👉 Together, pins + usage counters + free list + clock-sweep + background writer = a simple, efficient, and concurrency-friendly buffer manager.
For starters, let’s look at the basics:
- Shared Buffers is a chunk of memory (RAM) allocated from the OS when PostgreSQL starts.
- It’s used as a cache for database pages (8 KB blocks by default) that are read from disk.
- When a query needs data:
- PostgreSQL first checks shared_buffers.
- If the page is there → it’s a cache hit (fast).
- If not → it’s read from disk into shared_buffers, and then served.
- Modified pages (called dirty pages) are also kept in shared_buffers until they’re eventually written back to disk (via checkpoints or background writer).
- It’s called shared because all backend processes (connections) use the same buffer pool.