The Quiet Configuration That Keeps Databases Alive

Picture this: your database is humming along at 2 AM, handling overnight batch jobs. Then one process asks for a chunk of memory the system cannot deliver. Linux panics, picks a victim, and kills it. Your entire PostgreSQL cluster goes down. Every connection drops. Crash recovery starts. And nobody finds out until the morning.

This is not a hypothetical. It is a well-documented failure mode that database teams have wrestled with for years. The culprit is Linux memory overcommit, a kernel-level behavior that can silently destroy your database's availability. Understanding how to control it is one of the most valuable skills a DBA or infrastructure engineer can develop.

The PostgreSQL documentation touches on memory configuration, but the kernel-level side of things often gets skipped in tutorials. This article fills that gap. We will look at what overcommit actually does, how to tune it, and what complementary strategies protect your data in production.

What Memory Overcommit Actually Means

When a program calls malloc(), Linux does not hand over real physical memory right away. It reserves virtual address space and promises to back it with real pages later, when the program actually touches that memory. This is called lazy allocation, and it works well most of the time.

The kernel bets that not every process will use all the memory it requested at once. Usually, that bet pays off. But when too many processes fault in memory at the same time, the system runs out of physical RAM and swap space. At that point, the kernel invokes the OOM killer (Out Of Memory killer), a process that picks a running program and terminates it to free up pages.

For most services, an OOM kill is inconvenient but survivable. The process restarts, reconnects, and carries on. PostgreSQL is a different story entirely. Its architecture relies on shared memory segments that multiple backend processes read and write simultaneously. If the OOM killer terminates a backend mid-write, that shared segment can be left in a corrupt state. PostgreSQL's postmaster detects this and responds by killing every other backend process to prevent data corruption from spreading.

The result is a total cluster outage from a single memory event. If write volume was high before the kill, crash recovery can take minutes or longer. That is a painful lesson to learn in production.

The Three Overcommit Modes in Linux

Linux exposes overcommit behavior through the vm.overcommit_memory kernel parameter. There are three settings:

Mode 0 (Heuristic): The default. The kernel uses guesswork to decide whether to allow an allocation. It blocks obviously absurd requests but otherwise permits overcommitting freely.
Mode 1 (Always): Every allocation request succeeds, no matter how much memory has already been committed. The OOM killer handles the fallout later.
Mode 2 (Strict): The kernel calculates a commit limit and refuses allocations that would exceed it. Processes fail early with a clean error instead of dying unexpectedly later.

Mode 2 is what most serious PostgreSQL operators reach for. It makes memory allocation predictable. Processes that ask for too much memory get a refusal right away, not a death notice at 2 AM.

How to Configure Strict Overcommit for PostgreSQL

Switching to strict overcommit is a one-line kernel change, but you need to pick the right commit limit to go with it. Set the mode too tight and legitimate allocations start failing. Set it too loose and you lose the protection you wanted.

Diagram showing Linux kernel memory overcommit modes and their effect on PostgreSQL process management

The commit limit is controlled by vm.overcommit_ratio (a percentage of RAM) or vm.overcommit_kbytes (a fixed size). The kernel calculates total allowed commits as:

Key Memory Overcommit Facts

Lazy Allocation

Memory Allocation Type

Shared Memory Segments

PostgreSQL Architecture Dependency

Process Termination

Common Result of OOM

Kernel-Level Tuning

Crucial DBA Skill

# View current overcommit settings
sysctl vm.overcommit_memory
sysctl vm.overcommit_ratio

# Check how much memory is currently committed
grep -E 'CommitLimit|Committed_AS' /proc/meminfo

To apply strict mode and set a ratio, you can run:

# Apply immediately (does not survive reboot)
sysctl -w vm.overcommit_memory=2
sysctl -w vm.overcommit_ratio=80

# Persist across reboots
echo 'vm.overcommit_memory=2' >> /etc/sysctl.conf
echo 'vm.overcommit_ratio=80' >> /etc/sysctl.conf
sysctl -p

The ratio of 80 is a common starting point. It leaves headroom for the operating system, kernel threads, and monitoring agents. Adjust it based on your actual workload. Check /proc/meminfo after a full load test to see how close Committed_AS gets to CommitLimit.

Tuning PostgreSQL Memory Settings Alongside the Kernel

Kernel tuning alone is not enough. You also need PostgreSQL itself to be a good citizen with memory. The two most important settings are shared_buffers and work_mem.

shared_buffers controls how much RAM PostgreSQL reserves for its buffer cache. A common rule of thumb is 25 percent of total RAM, but this varies a lot by workload. The buffer is allocated at startup using shared memory, so it is committed immediately and counts against your overcommit limit.

work_mem is trickier. It is the memory available per sort or hash operation, and one query can use multiple operations in parallel. With 100 connections each running a complex query, you could see 100 times your work_mem value consumed simultaneously. Keep this number conservative unless you have very few connections and a read-heavy workload.

-- Check current PostgreSQL memory settings
SHOW shared_buffers;
SHOW work_mem;
SHOW maintenance_work_mem;

-- See total memory committed by PostgreSQL
SELECT pg_size_pretty(sum(size)) AS total
FROM pg_shmem_allocations;

For deeper query-level tuning, the resource Use The Index, Luke covers how poor index choices force PostgreSQL to do expensive in-memory sorts that blow up work_mem usage.

Swap Space: The Other Half of the Equation

Many cloud servers ship with no swap configured at all. This is a mistake when running PostgreSQL. Without swap, the system has nowhere to offload cold memory pages, and the OOM killer gets triggered at a much lower memory pressure threshold.

Adding swap does not mean your system will use it constantly. Modern kernels with vm.swappiness set low will prefer to keep hot data in RAM. But swap gives the kernel breathing room during memory spikes, which buys time for graceful handling instead of a kill.

A practical setup for a database server with 16 GB of RAM might look like this:

Add 4 to 8 GB of swap space (a swapfile works fine on modern kernels)
Set vm.swappiness=10 to discourage aggressive swapping
Set vm.overcommit_memory=2 with a ratio that accounts for swap in the commit limit

The kernel includes swap in its commit limit calculation under Mode 2. So adding swap directly increases how much memory your processes can allocate before hitting a refusal. This is one reason strict overcommit and a small swap partition work well together.

Cgroups: A More Surgical Memory Control Tool

Kernel-wide overcommit settings protect the whole system, but they are blunt instruments. Control groups (cgroups) let you set hard memory limits on individual processes or process groups. This is particularly powerful in containerized environments or when you run multiple services on the same host.

With cgroups v2 (the current default on most Linux distributions), you can cap PostgreSQL's total memory usage and define what happens when it hits the limit. Instead of the OOM killer picking a random victim, you control which process group gets constrained.

# Create a cgroup for PostgreSQL (cgroups v2)
mkdir /sys/fs/cgroup/postgres

# Set a 12 GB memory limit
echo $((12 * 1024 * 1024 * 1024)) > /sys/fs/cgroup/postgres/memory.max

# Move the PostgreSQL postmaster into the cgroup
echo $(pgrep -x postgres | head -1) > /sys/fs/cgroup/postgres/cgroup.procs

This approach is especially useful in Kubernetes environments where PostgreSQL runs inside a pod. Setting resource limits at the pod level translates directly to cgroup limits. The key is to set the limit high enough that normal workloads never hit it, but low enough that a runaway query cannot consume all available system memory.

Many teams combine cgroup limits with strict overcommit at the kernel level. The cgroup acts as a per-service guardrail. The kernel setting acts as a system-wide safety net. Together, they provide layered protection without relying on any single mechanism.

Monitoring Memory Pressure Before It Becomes a Crisis

The best memory protection is catching pressure early, before the OOM killer wakes up. Several tools give you visibility into how close the system is to the edge.

The /proc/meminfo file is your first stop. Watch MemAvailable rather than MemFree. Available memory accounts for reclaimable caches, which the kernel can drop under pressure. If MemAvailable drops below 10 to 15 percent of total RAM, it is time to investigate.

Steps to Prevent OOM Killer

1. Understand Overcommit

Learn how Linux handles memory requests.

2. Configure vm.overcommit_memory

Set it to avoid overcommitting memory.

3. Monitor Memory Usage

Use tools to track memory pressure.

4. Implement cgroups

Limit memory for PostgreSQL processes.

5. Tune PostgreSQL Parameters

Adjust work_mem and shared_buffers.

For PostgreSQL-specific memory monitoring, Plotly is a solid choice for building custom dashboards that visualize memory trends over time. Pair it with data from pg_stat_activity and system-level metrics to correlate query patterns with memory spikes.

Here is a quick SQL query to spot sessions consuming the most memory:

-- Find top memory-consuming queries
SELECT
  pid,
  usename,
  application_name,
  state,
  query_start,
  left(query, 80) AS query_preview
FROM pg_stat_activity
WHERE state != 'idle'
ORDER BY query_start ASC
LIMIT 20;

Set alerts when Committed_AS in /proc/meminfo crosses 85 percent of CommitLimit. That threshold gives you time to react before processes start getting allocation failures or the OOM killer activates. If you work with data distributions and want to understand where your memory usage falls relative to historical baselines, a percentile calculator can help you make sense of metric distributions from your monitoring data.

Using PSI (Pressure Stall Information) for Early Warnings

Linux kernels from version 4.20 onward support Pressure Stall Information (PSI). This feature tracks how much time processes spend stalled waiting for memory. It is a much more sensitive signal than simple memory usage percentages.

Read PSI memory pressure from /proc/pressure/memory. The some line shows the percentage of time at least one process was stalled. The full line shows when all processes were stalled. Any non-zero value on the full line is a serious warning sign worth investigating immediately.

Alternative Approaches Teams Use in 2026

Strict overcommit is not the only path. Different teams take different approaches based on their infrastructure and risk tolerance. Here are the strategies that come up most often in production environments:

PostgreSQL shared memory architecture showing postmaster and backend process relationships

Managed database services: Offloading memory management entirely to a cloud provider. Services like Amazon RDS, Google Cloud SQL, and managed PostgreSQL platforms handle kernel tuning internally. You trade control for convenience.
Dedicated database hosts: Running PostgreSQL on a server with no other significant workloads. This eliminates memory contention between services and makes overcommit settings less critical.
Connection pooling: Tools like PgBouncer reduce the number of backend processes PostgreSQL needs to maintain. Fewer backends means less shared memory pressure and lower peak memory usage.
Huge pages: Enabling Linux huge pages for PostgreSQL's shared memory allocation. This reduces TLB pressure and can improve memory efficiency on large-RAM systems, though it requires careful configuration.
Vertical scaling with headroom: Simply buying more RAM than you expect to need. It is not elegant, but a server with 128 GB of RAM running a 40 GB working set rarely needs kernel-level memory tricks.

None of these approaches are mutually exclusive. The most resilient production setups combine several of them. Connection pooling plus strict overcommit plus cgroup limits is a common combination that handles most failure scenarios gracefully.

If you are evaluating managed PostgreSQL options, check out resources on TechCrunch for coverage of how cloud database providers are evolving their offerings in 2026. The gap between self-managed and managed PostgreSQL in terms of operational complexity has narrowed considerably.

For teams building infrastructure tooling or dashboards to visualize these memory metrics, good design matters too. Tools like Figma's design learning resources can help your team build clearer monitoring interfaces. When sharing documentation or reports about your memory configuration strategy, a reverse image search tool can verify that any diagrams or screenshots you use are properly sourced. And if you are building color-coded memory dashboards, a hex color finder tool helps you keep your visual design consistent across alert thresholds and status indicators.

Practical Checklist for PostgreSQL Memory Safety

Before you call your PostgreSQL memory configuration production-ready, run through this checklist. Each item addresses a specific failure mode that teams have hit in real deployments.

Set vm.overcommit_memory=2 and tune vm.overcommit_ratio based on actual workload measurements
Configure shared_buffers to a fixed, appropriate value and restart to commit it at startup
Keep work_mem conservative, especially if connection counts are high
Add swap space even on cloud instances, and set vm.swappiness=10
Use cgroup memory limits to contain PostgreSQL within a defined ceiling
Enable PSI monitoring and alert on any full stall time greater than zero
Deploy a connection pooler like PgBouncer to reduce the number of active backends
Run load tests and check Committed_AS vs. CommitLimit under peak conditions

If you are managing multiple PostgreSQL instances and tracking operational metrics across them, a percentile calculator can help you benchmark memory usage across instances and spot outliers quickly. Want to share your configuration findings with your team in a visually engaging format? A free video to boomerang converter can help you create looping visual demos of your monitoring dashboards for team presentations or documentation.

Start With the Kernel, Not the Application

Most database performance guides start with query optimization and index tuning. Those matter enormously. But if your kernel is silently killing processes in the background, no amount of query tuning will save you from a 2 AM outage.

The good news is that fixing this is not complicated. A few sysctl settings, a swap partition, and some cgroup limits get you most of the way there. The PostgreSQL-specific tuning on top of that is well-documented in the official PostgreSQL docs, and the community around the project is active and helpful.

Start by checking your current overcommit settings right now. Run sysctl vm.overcommit_memory on any PostgreSQL host you manage. If it returns 0 or 1, your database is vulnerable to the exact failure mode described in this article. That is worth fixing before anything else on your to-do list today.

Have questions about your specific setup? The PostgreSQL tag on Stack Overflow has a large community of engineers who have seen most of these failure modes firsthand. Get started with the checklist above, test your changes under realistic load, and verify your commit limits before declaring victory. Your on-call rotation will thank you.