About PocketXRP

A node shouldn't need the ledger's whole state tree in RAM to process transactions that touch a small part of it, and it shouldn't need it to sync the whole thing either.

Why

The ledger keeps growing. Stock rippled builds the state tree up in memory as it works, and that in-memory tree is gone when the process stops (the ledger on disk isn't): rebuilding it means finding nodes one by one through a hash table on disk, where a miss can take a 4 KB index read before the node itself. In our test on a Mac, stock rippled 3.4.1's footprint was 15 GB when it finished its first sync.

So keep the tree on disk, read only what a ledger touches, and leave the cold rest alone: a state tree ten times larger, mostly dormant accounts, shouldn't need ten times the memory. PocketXRP is where we're working that out. The node itself uses 75–500 MB; about 2 GB including macOS's file cache is enough to run comfortably, though macOS will cache more when RAM is free and take it back when needed.

Chasing memory limits that look absurd is the point: they expose inefficiencies that plenty of RAM would hide.

How

• KeyDom, its storage engine. Parent nodes record where their children live on disk, so walking down the tree follows direct file offsets instead of looking up every step in the hash table. New ledger data is appended to the end of the store and trimmed in the background, and the node reads only the branches a ledger touches.
• Depth-first first sync. A new node works on a few adjacent branches of the state tree at a time rather than the whole tree at once. What it's reading and writing stays close together in memory and on disk, making good use of a small file cache. (It was forced on us by getting a cold sync to full inside a 1 GB Linux cgroup.)
• Streaming catch-up. After your Mac sleeps, the node streams the newest ledger straight to disk instead of acquiring every one it missed.
• mimalloc for memory. The node allocates through mimalloc instead of fixed slab pools that dirtied about 370 MB on first use even on a tiny node. Finding the right balance between CPU work and a low memory footprint is ongoing.

How it's tested

Besides solo cold syncs on a quiet M3 Mac, we test on Linux inside 1 and 2 GB memory-capped cgroups that count both the node's memory and the file cache. The store runs on latfs, a small filesystem we wrote that delays the reads that reach the disk, so fast storage flatters the numbers less.

Where it's going

• Index memory. Keep every node findable by hash as the ledger grows, within the same RAM budget.
• Paged sync. Today a new node fetches the ledger's state node by node, from the top down. We're exploring pages instead: a peer serves the ledger's entries in key order, each page with the hashes that prove it against the validated ledger, so a node could fetch pages from several peers at once, in any order.