[paimon] Add bitmap global index

docs/docs/multimodal-table/global-index.mdx

@@ -30,13 +30,15 @@ Global Index is a powerful indexing mechanism for Data Evolution (append) tables
 without full-table scans. Paimon supports multiple global index types:
 
 - **[BTree Index](./global-index/btree)**: A B-tree based index for scalar column lookups. Supports equality, IN, range predicates, and can be combined across multiple columns with AND/OR logic.
+- **[Bitmap Index](./global-index/bitmap)**: A bitmap based index for enum-like scalar dimensions and tag columns. Supports equality, IN, prefix match on string columns, complement predicates, and null checks with compressed row-id bitmaps.
 - **[Vector Index](./global-index/vector)**: An approximate nearest neighbor (ANN) index powered by Paimon's vector index library for vector similarity search.
 - **[Full-Text Index](./global-index/full-text)**: A full-text search index powered by Tantivy for text retrieval. Supports term matching and relevance scoring.
 - **[Hybrid Search](./global-index/hybrid-search)**: A multi-route search API that combines results from multiple vector routes, multiple full-text routes, or both before reading table rows.
 
 | Index Type | Best For | Notes |
 |---|---|---|
 | BTree | Scalar filters on numeric, string, date, and timestamp columns | Best when predicates are selective, such as equality, IN, range, and null checks. |
+| Bitmap | Enum-like dimensions and tag columns | Best for equality, IN, string prefix match, complement predicates, and null checks over compressed row-id bitmaps. |
 | Vector | Top-K similarity search on embeddings | Uses ANN algorithms. Tune build-time and search-time options to balance recall, latency, and index size. |
 | Full-Text | Keyword search over text columns | Uses Tantivy scoring and tokenizer configuration stored with each index file. |
 | Hybrid Search | Combining multiple vector routes, multiple full-text routes, or vector and full-text retrieval together | Runs multiple scored routes and merges them with a ranker before reading rows. |
@@ -158,6 +160,12 @@ These table options affect global index build and read behavior:
 Use BTree indexes for scalar column lookups and range predicates. See [BTree Index](./global-index/btree)
 for build and query examples.
 
+## Bitmap Index
+
+Use Bitmap indexes for enum-like dimensions and tag columns queried by equality, IN, string prefix
+match, complement, or null predicates. See [Bitmap Index](./global-index/bitmap) for build examples,
+options, and file format details.
+
 ## Vector Index
 
 Use Vector indexes for approximate nearest neighbor (ANN) search. See [Vector Index](./global-index/vector)

docs/docs/multimodal-table/global-index/bitmap.mdx

@@ -0,0 +1,215 @@
+---
+title: "Bitmap Index"
+sidebar_position: 2
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# Bitmap Index
+
+A bitmap index maps each distinct scalar value to a compressed 64-bit row-id bitmap.
+Use it for enum-like dimensions and tag columns where queries often use equality,
+`IN`, complement, or null predicates, for example `status`, `country`, `tenant_type`,
+or business tags.
+
+Compared with [BTree Index](./btree), bitmap index is optimized for set operations over
+exact row-id bitmaps. It is usually a better fit for low-cardinality or medium-cardinality
+dimensions, especially when `IN`, `NOT IN`, or `!=` predicates are common. BTree index is
+still better for range predicates and high-cardinality columns where each value has only a
+few rows and range pruning is important.
+
+Supported predicate shapes include:
+
+| Predicate | Example |
+|---|---|
+| Equality | `tag = 'vip'` |
+| IN | `tag IN ('vip', 'trial')` |
+| Not equal | `tag != 'blocked'` |
+| NOT IN | `tag NOT IN ('blocked', 'test')` |
+| Null checks | `tag IS NULL`, `tag IS NOT NULL` |
+| String predicates | `tag LIKE 'vip%'`, `tag LIKE '%vip%'`, `tag startsWith 'vip'`, `tag contains 'vip'` |
+| Range predicates | `tag >= 'a'`, `tag BETWEEN 'a' AND 'm'` |
+| AND / OR combinations | `tag = 'vip' OR tag = 'trial'` |
+
+Equality, `IN`, null checks, and string prefix predicates use direct dictionary lookup.
+Other predicates such as `endsWith`, `contains`, general `LIKE`, and range predicates
+fall back to scanning bitmap dictionary entries only when the total size of candidate
+bitmap index files is within the configured fallback scan budget. If the budget is
+exceeded, Paimon falls back to other matching indexes or regular table scans.
+
+## Build Bitmap Index
+
+```sql
+-- Create bitmap index on 'tag' column
+CALL sys.create_global_index(
+ table => 'db.my_table',
+ index_column => 'tag',
+ index_type => 'bitmap'
+);
+```
+
+You can build only selected partitions:
+
+```sql
+CALL sys.create_global_index(
+ table => 'db.my_table',
+ index_column => 'tag',
+ index_type => 'bitmap',
+ partitions => 'dt=2026-06-18;dt=2026-06-19'
+);
+```
+
+## Bitmap Options
+
+| Option | Default | Description |
+|---|---|---|
+| `bitmap-index.dictionary-block-size` | `16 kb` | Target size of dictionary blocks in bitmap global index files. Smaller blocks reduce dictionary read amplification for high-cardinality columns; larger blocks reduce dictionary block index size. |
+| `bitmap-index.compression` | `none` | Compression algorithm for bitmap dictionary blocks and the dictionary block index. Supported values are the same block codecs as BTree index, such as `none`, `lz4`, `lzo`, and `zstd`. |
+| `bitmap-index.compression-level` | `1` | Compression level used by codecs that support levels, such as `zstd`. |
+| `bitmap-index.fallback-scan-max-size` | `256 mb` | Maximum total size of bitmap global index files in one reader to allow fallback dictionary scans for predicates that cannot use direct bitmap lookup. Set to `0 b` to disable fallback scans. |
+
+## Query with Bitmap Index
+
+Once a bitmap index is built, it is automatically used during scan when a filter
+predicate matches the indexed column.
+
+```sql
+SELECT * FROM my_table WHERE tag IN ('vip', 'trial');
+```
+
+For complement predicates such as `tag != 'blocked'` or
+`tag NOT IN ('blocked', 'test')`, bitmap index evaluates the complement against each
+index file's own non-null row-id bitmap. This keeps results correct when one logical
+query unions multiple bitmap index files.
+
+## File Format
+
+A bitmap global index file stores exact row-id bitmaps and a block-indexed dictionary.
+The reader opens a file by reading only its fixed-length footer. Null row sets,
+non-null row sets, and the dictionary block index are loaded lazily when a matching
+predicate needs them. Point lookups then read the dictionary block containing the
+target value and the corresponding bitmap block. The format is footer-driven: the
+footer stores the version, magic number, and offsets to all metadata blocks.
+
+```text
++----------------------------------------------+
+| null rows bitmap block |
++----------------------------------------------+
+| non-null rows bitmap block |
++----------------------------------------------+
+| value bitmap and dictionary payload blocks |
++----------------------------------------------+
+| ... |
++----------------------------------------------+
+| dictionary block index |
++----------------------------------------------+
+| footer |
++----------------------------------------------+
+```
+
+Each bitmap block is a serialized `RoaringNavigableMap64`. Bitmap blocks are not wrapped
+with an additional compression layer because Roaring already stores row ids compactly.
+Value bitmap blocks are written for non-null values in serialized-key order. Dictionary
+blocks are emitted as groups reach the configured target size, so the physical payload
+area can contain both value bitmap blocks and dictionary blocks. Readers use the stored
+`offset` and `length` fields and do not require value bitmap blocks or dictionary blocks
+to be physically contiguous.
+
+Each dictionary block stores sorted value entries. Keys are serialized with the same
+key serializer as BTree global index keys, and are ordered by serialized bytes:
+
+```text
++----------------------------------------------+
+| entry count (var-length int) |
++----------------------------------------------+
+| key length (var-length int), key bytes |
+| bitmap block offset (var-length long) |
+| bitmap block length (var-length int) |
++----------------------------------------------+
+| ... |
++----------------------------------------------+
+```
+
+The dictionary block body above is stored as a BTree-style compressed block. The
+configured dictionary compression is attempted when writing, and the uncompressed body is
+kept if compression does not save enough space. A 5-byte block trailer follows each
+dictionary block body and records the actual compression type and CRC. The stored
+dictionary block `length` is the block body length and does not include the trailer.
+
+The dictionary block index stores one entry per dictionary block and is small enough to
+load when the index file is opened:
+
+```text
++----------------------------------------------+
+| block count (var-length int) |
++----------------------------------------------+
+| first key length (var-length int), key bytes |
+| dictionary block offset (var-length long) |
+| dictionary block length (var-length int) |
++----------------------------------------------+
+| ... |
++----------------------------------------------+
+```
+
+The dictionary block index uses the same compressed-block encoding and 5-byte trailer as
+dictionary blocks. The footer's dictionary block index `length` also excludes this
+trailer.
+
+The footer has fixed length and points to the main metadata blocks:
+
+```text
++----------------------------------------------+
+| null rows block offset (long) |
+| null rows block length (int) |
+| non-null rows block offset (long) |
+| non-null rows block length (int) |
+| dictionary block index offset (long) |
+| dictionary block index length (int) |
+| value count (int) |
+| version (int) = 1 |
+| magic (int) |
++----------------------------------------------+
+```
+
+The sorted dictionary block index lets equality and `IN` predicates locate candidate
+values by binary search without deserializing every dictionary block or value bitmap in
+the file. Null checks can read only the needed null/non-null bitmap.
+
+Each bitmap index file also stores manifest-level metadata with the logical minimum
+non-null key, maximum non-null key, and whether the file contains null values. The scanner
+uses this metadata to skip impossible files before opening bitmap index files, similar to
+BTree global index file pruning.
+
+```text
++----------------------------------------------+
+| first key length (int), first key bytes |
+| last key length (int), last key bytes |
+| has nulls (byte) |
+| metadata version (byte) = 1 |
+| null key flags (byte) |
++----------------------------------------------+
+```
+
+For predicates that cannot be resolved by point or prefix lookup, the reader may scan all
+dictionary blocks and read matching bitmap blocks. This fallback is guarded by
+`bitmap-index.fallback-scan-max-size`, which compares against the total size of candidate
+bitmap index files handled by the reader. This keeps point lookup read amplification
+bounded for high-cardinality tag or dimension columns while allowing small bitmap indexes
+to answer broader predicates directly.

docs/docs/multimodal-table/global-index/btree.mdx

@@ -42,6 +42,8 @@ Supported predicate shapes include:
 | AND / OR combinations | `name = 'a200' OR name = 'a300'` |
 
 `LIKE`, `startsWith`, `contains`, and `NOT IN` predicates may still need broader index file reads.
+Use `btree-index.fallback-scan-max-size` to cap fallback range and string scans by the total
+candidate index file size.
 For keyword-style text retrieval, use [Full-Text Index](./full-text) instead.
 
 ## Build BTree Index
@@ -74,6 +76,7 @@ CALL sys.create_global_index(
 | `btree-index.build.max-parallelism` | `4096` | Maximum Flink or Spark parallelism for building BTree indexes. |
 | `btree-index.block-size` | `64 kb` | Block size used by BTree index files. |
 | `btree-index.cache-size` | `128 mb` | Cache size used by BTree index readers. |
+| `btree-index.fallback-scan-max-size` | `256 mb` | Maximum total size of candidate BTree global index files to allow fallback index scans. Set to `0 b` to disable fallback scans. |
 | `btree-index.compression` | `none` | Compression algorithm used by BTree index blocks. |
 
 ## Query with BTree Index

docs/docs/multimodal-table/global-index/full-text.mdx

@@ -1,6 +1,6 @@
 ---
 title: "Full-Text Index"
-sidebar_position: 3
+sidebar_position: 4
 ---
 
 import Tabs from '@theme/Tabs';

docs/docs/multimodal-table/global-index/hybrid-search.mdx

@@ -1,6 +1,6 @@
 ---
 title: "Hybrid Search"
-sidebar_position: 4
+sidebar_position: 5
 ---
 
 import Tabs from '@theme/Tabs';

docs/docs/multimodal-table/global-index/vector.mdx

@@ -1,6 +1,6 @@
 ---
 title: "Vector Index"
-sidebar_position: 2
+sidebar_position: 3
 ---
 
 import Tabs from '@theme/Tabs';

docs/sidebars.js

@@ -124,6 +124,7 @@ const sidebars = {
  },
  "items": [
  "multimodal-table/global-index/btree",
+ "multimodal-table/global-index/bitmap",
  "multimodal-table/global-index/vector",
  "multimodal-table/global-index/full-text",
  "multimodal-table/global-index/hybrid-search"

paimon-common/src/main/java/org/apache/paimon/globalindex/btree/KeySerializer.java → paimon-common/src/main/java/org/apache/paimon/globalindex/KeySerializer.java

@@ -16,7 +16,7 @@
  * limitations under the License.
  */
 
-package org.apache.paimon.globalindex.btree;
+package org.apache.paimon.globalindex;
 
 import org.apache.paimon.data.BinaryString;
 import org.apache.paimon.data.Decimal;
@@ -45,7 +45,7 @@
 
 import java.util.Comparator;
 
-/** This interface provides core methods to ser/de and compare btree index keys. */
+/** This interface provides core methods to ser/de and compare global index keys. */
 @ThreadSafe
 public interface KeySerializer {
 
@@ -61,7 +61,7 @@ static KeySerializer create(DataType type) {
  @Override
  public KeySerializer defaultMethod(DataType dataType) {
  throw new UnsupportedOperationException(
- "DataType: " + dataType + " is not supported by btree index now.");
+ "DataType: " + dataType + " is not supported by global index now.");
  }
 
  @Override