LightRAG Java Implementation

LightRAG is fabulus which is implemented in Python. This is a Java implementation of LightRAG.

Requirement

• Java 21+. (You may download from https://www.azul.com/downloads/?package=jdk#zulu)

Build

Take postgres implementation as an example.

git clone https://github.com/ShanGor/light-rag-java.git
cd light-rag-java
mvn clean install -DskipTests=true
cd light-rag-core
mvn clean install -DskipTests=true
cd ../feature/light-rag-postgres
mvn clean install -DskipTests=true
cd ../../example/light-rag-example-postgres
mvn clean package -DskipTests=true

For Postgres

PGVector and Apache AGE should be used.
Windows Release as it is easy to install for Linux/Mac.
If you prefer docker, please start with this image if you are a beginner to avoid hiccups (DO read the overview): https://hub.docker.com/r/shangor/postgres-for-rag

Create index for AGE:

load 'age';
SET search_path = ag_catalog, "$user", public;
CREATEINDEXCONCURRENTLY entity_p_idx ON dickens."Entity" (id);
CREATEINDEXCONCURRENTLY vertex_p_idx ON dickens."_ag_label_vertex" (id);
CREATEINDEXCONCURRENTLY directed_p_idx ON dickens."DIRECTED" (id);
CREATEINDEXCONCURRENTLY directed_eid_idx ON dickens."DIRECTED" (end_id);
CREATEINDEXCONCURRENTLY directed_sid_idx ON dickens."DIRECTED" (start_id);
CREATEINDEXCONCURRENTLY directed_seid_idx ON dickens."DIRECTED" (start_id,end_id);
CREATEINDEXCONCURRENTLY edge_p_idx ON dickens."_ag_label_edge" (id);
CREATEINDEXCONCURRENTLY edge_sid_idx ON dickens."_ag_label_edge" (start_id);
CREATEINDEXCONCURRENTLY edge_eid_idx ON dickens."_ag_label_edge" (end_id);
CREATEINDEXCONCURRENTLY edge_seid_idx ON dickens."_ag_label_edge" (start_id,end_id);
createINDEXCONCURRENTLY vertex_idx_node_id ON dickens."_ag_label_vertex" (ag_catalog.agtype_access_operator(properties, '"node_id"'::agtype));
createINDEXCONCURRENTLY entity_idx_node_id ON dickens."Entity" (ag_catalog.agtype_access_operator(properties, '"node_id"'::agtype));
CREATEINDEXCONCURRENTLY entity_node_id_gin_idx ON dickens."Entity" using gin(properties);
ALTERTABLE dickens."DIRECTED" CLUSTER ON directed_sid_idx;
-- drop if necessarydropINDEX entity_p_idx;
dropINDEX vertex_p_idx;
dropINDEX directed_p_idx;
dropINDEX directed_eid_idx;
dropINDEX directed_sid_idx;
dropINDEX directed_seid_idx;
dropINDEX edge_p_idx;
dropINDEX edge_sid_idx;
dropINDEX edge_eid_idx;
dropINDEX edge_seid_idx;
dropINDEX vertex_idx_node_id;
dropINDEX entity_idx_node_id;
dropINDEX entity_node_id_gin_idx;

Create index for pgvector:

You need to know your vector dimension for your embedding model. For example, the dimension of nomic-embed-text is 768, bge-m3 is 1024.

-- vector indexes, you have to define dimension before building indexesCREATEINDEXONpublic.lightrag_vdb_entity USING hnsw (content_vector vector_cosine_ops);
CREATEINDEXONpublic.lightrag_vdb_relation USING hnsw (content_vector vector_cosine_ops);
CREATEINDEXONpublic.lightrag_doc_chunks USING hnsw (content_vector vector_cosine_ops);
-- drop them if necessary (for example, change dimension)dropindexpublic.lightrag_vdb_entity_content_vector_idx;
dropindexpublic.lightrag_vdb_relation_content_vector_idx;
dropindexpublic.lightrag_doc_chunks_content_vector_idx;

How to define / change the dimension?

By default, there is no dimension. and you cannot create index. When you have data, and you want to define dimension and build index, you can simply run below SQLs;

-- Change the 1024 to fit your dimension.ALTERTABLE lightrag_vdb_entity ALTER COLUMN content_vector TYPE vector(1024);
ALTERTABLE lightrag_vdb_relation ALTER COLUMN content_vector TYPE vector(1024);
ALTERTABLE lightrag_doc_chunks ALTER COLUMN content_vector TYPE vector(1024);

If you want to change the dimension, and you've got data, you need to remove all the existing vector data, and then you can change the dimension.

• Do the following SQLs

  -- If your data is already has the same dimension, you can just change the dimension without data cleanup.update lightrag_vdb_entity set content_vector=null; commit;
  update lightrag_vdb_relation set content_vector=null; commit;
  update lightrag_doc_chunks set content_vector=null; commit;
  ALTERTABLE lightrag_vdb_entity ALTER COLUMN content_vector TYPE vector(1024);
  ALTERTABLE lightrag_vdb_relation ALTER COLUMN content_vector TYPE vector(1024);
  ALTERTABLE lightrag_doc_chunks ALTER COLUMN content_vector TYPE vector(1024);

• Call the API to re-embed all the data.

curl -i http[s]://{server}:{port}/vector/update