RUM—Storing More in the Index

This series of posts traces the evolution from GIN to RUM to Extended RUM, showing how a single architectural idea—store more in the index to do less work at query time—unlocks major performance improvements at each step.

RUM was created by Alexander Korotkov, Oleg Bartunov, and Teodor Sigaev at Postgres Professional. It started as a direct response to the GIN limitations that users kept hitting in production — particularly in full-text search scenarios where ranking and ordering dominate query time.

The name is a nod: GIN (the drink) → RUM (a stronger drink). The index is stronger, too.

RUM's innovation is exactly one architectural change: each entry in a posting list can carry an additional datum alongside the TID:

GIN posting list entry:  [TID]
RUM posting list entry:  [TID, addInfo]

A 2016 thread on pgsql-general captures the exact pain point. Andreas Krogh was building a web-based email system — millions of messages, users expecting millisecond response — and wanted a single index that could:

1. Match full-text search terms (fts_all @@ to_tsquery(...))
2. Filter by folder (folder_id = ANY(ARRAY[2,3]))
3. Sort by timestamp (ORDER BY sent DESC)
4. Stop early (LIMIT 101)

With GIN, steps 1 and 2 worked (via btree_gin), but step 3 always required a separate Sort node — the index couldn't deliver results in timestamp order. The planner had to scan all matching TIDs, fetch every heap tuple, and sort them. For a mailbox with 100K matches, that's unacceptable.

Oleg Bartunov's response was direct: "We are [working] hard on our internal version of rum." RUM was designed to solve exactly this class of problem.

RUM extends the GIN framework with an order_by_attach option:

CREATE INDEX idx ON documents USING rum (tsv rum_tsvector_addon_ops, created_at)
  WITH (attach = 'created_at', to = 'tsv', order_by_attach = true);

This tells RUM: "for every TID in the tsv posting list, also store the corresponding created_at value." The posting list entries become:

"postgresql" → [(0,1, 2024-01-15), (3,7, 2024-02-20), (5,2, 2024-03-01)]

Now the posting list is sorted by addInfo (the timestamp), not by the physical order of TIDs.

What RUM Enables Over GIN:

1. Ordered results without sorting. The index walks the posting list in addInfo order and stops after N entries. No Sort node, no full scan.
2. Distance-based ordering with <=>. RUM introduces distance operators (<=>, <=|, |=>). The <=> computes ABS(a - b) — nearest-first retrieval. The <=| and |=> variants restrict to one direction. For full-text ranking, RUM's <=> operator has a built-in ranking function that combines ts_rank and ts_rank_cd semantics and handles OR queries better than either function alone.
3. Depth-first traversal: first results immediately. Unlike GIN's bitmap approach (collect all TIDs, then access the heap), RUM performs a depth-first traversal. It can return first results immediately — critical for LIMIT queries.
4. Phrase search without recheck. RUM's rum_tsvector_ops stores word positions as addInfo. The index verifies phrase adjacency during the scan — no heap rechecks needed.

Here are the operator classes:

• rum_tsvector_ops stores lexemes with positional information, supports ordering (<=>) by relevance
• rum_tsvector_hash_ops stores hashed lexemes with positions, supports ordering (<=>) with no prefix search
• rum_tsvector_addon_ops stores lexemes + any attached column, supports ordering (<=>) on attached column
• rum_anyarray_ops stores array elements + array length, supports ordering (<=>) by similarity
• rum_anyarray_addon_ops stores array elements + attached column, supports ordering (<=>) on attached column
• rum_timestamp_ops stores scalar values, supports ordering (<=>)

Using the same articles table from the previous post, I add a RUM index that adds the published timestamp to tsv:

postgres=# CREATE EXTENSION IF NOT EXISTS rum;

postgres=# CREATE INDEX idx_rum_addon ON articles
    USING rum (tsv rum_tsvector_addon_ops, published)
    WITH (attach = 'published', to = 'tsv');

I searched for the five most recent articles by publication date using the same text query as in the previous post.

postgres=# EXPLAIN (COSTS OFF, ANALYZE, BUFFERS, VERBOSE)
SELECT id, title, published,
       published <=> '2020-06-01'::timestamp AS distance
FROM articles
WHERE tsv @@ to_tsquery('simple', 'postgresql & article')
ORDER BY published <=> '2020-06-01'::timestamp
LIMIT 5;

QUERY PLAN                                                  
--------------------------------------------------------------------------------------------------------------
 Limit (actual time=89.101..89.552 rows=5 loops=1)
   Output: id, title, published, ((published <=> '2020-06-01 00:00:00'::timestamp without time zone))
   Buffers: shared hit=1233 read=1514, temp read=1303 written=1303
   ->  Index Scan using idx_rum_addon on documentdb_core.articles (actual time=89.097..89.542 rows=5 loops=1)
         Output: id, title, published, (published <=> '2020-06-01 00:00:00'::timestamp without time zone)
         Index Cond: (articles.tsv @@ '''postgresql'' & ''article'''::tsquery)
         Order By: (articles.published <=> '2020-06-01 00:00:00'::timestamp without time zone)
         Buffers: shared hit=1233 read=1514, temp read=1303 written=1303
 Planning:
   Buffers: shared hit=3
 Planning Time: 0.401 ms
 Execution Time: 89.747 ms
(12 rows)

No Sort node. No Bitmap. A true Index Scan with Order By pushed into the index. Stops after 5 results.

I add a RUM index on tsv that adds the position, for phrase search without recheck:

postgres=# CREATE INDEX idx_rum_pos ON articles USING rum (tsv rum_tsvector_ops);

postgres=# EXPLAIN (COSTS OFF, ANALYZE, BUFFERS, VERBOSE)
SELECT id, title
FROM articles
WHERE tsv @@ to_tsquery('simple', 'postgresql <-> article')
LIMIT 5;

QUERY PLAN                                                  
--------------------------------------------------------------------------------------------------------------
 Limit (actual time=116.196..116.198 rows=0 loops=1)
   Output: id, title
   Buffers: shared hit=661
   ->  Index Scan using idx_rum_pos on documentdb_core.articles (actual time=116.192..116.193 rows=0 loops=1)
         Output: id, title
         Index Cond: (articles.tsv @@ '''postgresql'' <-> ''article'''::tsquery)
         Buffers: shared hit=661
 Planning:
   Buffers: shared hit=3
 Planning Time: 0.223 ms
 Execution Time: 117.838 ms
(11 rows)

No "Rows Removed by Index Recheck". Positions stored in the index enable direct verification of adjacency. The same idea applies beyond full‑text search, for example, with JSONB, where RUM can store element positions to reduce rechecks and improve performance.

The same index is used for relevance ranking:

postgres=# EXPLAIN (COSTS OFF, ANALYZE, BUFFERS, VERBOSE)
SELECT id, title,
       tsv <=> to_tsquery('english', 'postgresql | optimization') AS rank
FROM articles
WHERE tsv @@ to_tsquery('simple', 'postgresql | article')
ORDER BY tsv <=> to_tsquery('english', 'postgresql | optimization')
LIMIT 10;

QUERY PLAN                                                   
---------------------------------------------------------------------------------------------------------------
 Limit (actual time=543.958..544.006 rows=10 loops=1)
   Output: id, title, ((tsv <=> '''postgresql'' | ''optim'''::tsquery))
   Buffers: shared hit=783, temp read=13289 written=15244
   ->  Index Scan using idx_rum_pos on documentdb_core.articles (actual time=543.954..543.977 rows=10 loops=1)
         Output: id, title, (tsv <=> '''postgresql'' | ''optim'''::tsquery)
         Index Cond: (articles.tsv @@ '''postgresql'' | ''article'''::tsquery)
         Order By: (articles.tsv <=> '''postgresql'' | ''optim'''::tsquery)
         Buffers: shared hit=783, temp read=13289 written=15244
 Planning:
   Buffers: shared hit=3
 Planning Time: 0.198 ms
 Execution Time: 548.884 ms
(12 rows)

Ranking is computed directly during the index scan. No Sort node, no heap access for computing the score, enabling early termination with LIMIT. This contrasts with GIN, where ranking is computed after fetching all matching rows from the heap and sorting them.

Early versions of RUM allowed multi-column indexes, but only the main inverted column participated in the ordered scan (2017 thread). Additional columns were applied as post-filters, which limited the benefit of LIMIT queries. Currently filter columns can participate in the index scan itself, allowing pruning during traversal of the ordered posting list — but ordering is still driven by a single attached column. Here is an example:

postgres=# CREATE INDEX idx_rum_multi ON articles
    USING rum (tsv rum_tsvector_addon_ops, category, published)
    WITH (attach = 'published', to = 'tsv');

postgres=# EXPLAIN (COSTS OFF, ANALYZE, BUFFERS, VERBOSE)
SELECT id, title, published
FROM articles
WHERE tsv @@ to_tsquery('english', 'postgresql')
  AND category = 'tech'
ORDER BY published <=> '2020-06-01'::timestamp
LIMIT 5;

QUERY PLAN                                                  
--------------------------------------------------------------------------------------------------------------
 Limit (actual time=50.212..51.269 rows=5 loops=1)
   Output: id, title, published, ((published <=> '2020-06-01 00:00:00'::timestamp without time zone))
   Buffers: shared hit=117 read=112, temp read=550 written=550
   ->  Index Scan using idx_rum_multi on documentdb_core.articles (actual time=50.209..51.258 rows=5 loops=1)
         Output: id, title, published, (published <=> '2020-06-01 00:00:00'::timestamp without time zone)
         Index Cond: ((articles.tsv @@ '''postgresql'''::tsquery) AND (articles.category = 'tech'::text))
         Order By: (articles.published <=> '2020-06-01 00:00:00'::timestamp without time zone)
         Buffers: shared hit=117 read=112, temp read=550 written=550
 Planning:
   Buffers: shared hit=37 read=9 dirtied=3
 Planning Time: 7.805 ms
 Execution Time: 51.667 ms
(12 rows)

Here, category = 'tech' is in Index Cond, not a Filter, so pre-filtering on multiple columns is possible. RUM’s <=> operator uses a ranking model (ts_score) that combines ts_rank and ts_rank_cd, addressing their respective limitations (logical operators and OR queries).

The RUM Data Structure includes addInfo and may be sorted on it instead of TID:

RUM Posting List (leaf page):
┌─────────────────────────────────────────────┐
│ [TID₁, addInfo₁] [TID₂, addInfo₂] ...       │
│ sorted by addInfo (when order_by_attach=on) │
│ or by TID (when order_by_attach=off)        │
└─────────────────────────────────────────────┘

One drawback of the RUM index is that it has slower build and insert times than the GIN index. This is because RUM stores additional information alongside keys and utilizes generic Write-Ahead Logging (WAL) records, which can significantly increase WAL volume compared to GIN. It fits workloads where data is written once, such as append-only or event-driven use cases, and is queried many times later. Unlike GIN, RUM does not implement a pending list for fast updates, which contributes to its higher write and build costs. RUM performs best when dealing with highly repetitive keys, such as those found in natural language text or denormalized documents. However, for high-cardinality unique keys, like UUIDs, the inverted index structure offers little advantage over a B-tree.

The next post will cover Extended RUM, which enables additional indexing and ordering capabilities in Microsoft's contribution to DocumentDB.