Use Chroma DB vector database in RAG application using llama index & Deepseek R1 local model

 In previous article I have explained how to install Deepseek R1 model locally and create the RAG application using the locally downloaded models step by step.

In this article I will explain Step-by-step to use the Chroma DB vector database in RAG application using llama index & Deepseek R1 local model with Ollama. Chroma DB is open source vector db which can be used to store the vector embeddings of documents and query the embeddings based on user query. I have used the same example used in previous article and added only Chroma DB integration code.

 

Prerequisites for this example is as follows:

1. Visual studio code
2. Python
3. Ollama 
   

Open visual studio code and create the file with name "sample.py". Now in visual studio code and go to terminal menu and click on New terminal link it will open new terminal. In terminal enter below command to install the LlamaIndex library, Chroma DB library, LlamaIndex Ollama and LlamaIndex embedding Ollama library in your machine.

 pip install llama-index llama-index-llms-ollama llama-index-embeddings-ollama llama-index-vector-stores-chroma chromadb  

Create the folder named "doc" in root directory of the application as shown in below image and store the documents you want to query.

Now in sample.py enter the code below.

 from llama_index.embeddings.ollama import OllamaEmbedding  
 from llama_index.llms.ollama import Ollama  
 from llama_index.core import Settings,SimpleDirectoryReader,VectorStoreIndex,StorageContext  
 from llama_index.core.node_parser import SentenceSplitter  
 import chromadb   
 from llama_index.vector_stores.chroma import ChromaVectorStore  
    
   
 Embeddingmodel="nomic-embed-text:latest"  
 llmModel="deepseek-r1:1.5b"  
   
 embedObj=OllamaEmbedding(Embeddingmodel)  
 llmnObj=Ollama(llmModel,request_timeout=360.0)  
   
 Settings.llm=llmnObj  
 Settings.embed_model=embedObj  
 #Settings.node_parser=SentenceSplitter(chunk_size=1024,chunk_overlap=20)  
   
   
 #document injection  
   
 documents=SimpleDirectoryReader("doc").load_data()  
   
 db=chromadb.PersistentClient(path="./chroma_db/WC_db")  
   
 chroma_collection=db.get_or_create_collection(name="WC_db")  
   
 vector_store=ChromaVectorStore(chroma_collection=chroma_collection)  
   
 storageContext=StorageContext.from_defaults(vector_store=vector_store)  
   
 index=VectorStoreIndex.from_documents(documents,storage_context=storageContext,  
                    transformations=[SentenceSplitter(chunk_size=1024,chunk_overlap=20)]  
                    )  
   
 #Query document  
 queryengineObj=index.as_query_engine()  
   
 inputString=input("Enter the query: ")  
   
 results=queryengineObj.query(inputString)    
   
 print(results.response)  
   
   

for above code there are three sections which is key for the RAG application

1. The yellow colored code is for the LLM and embedding models initialization with llamaindex
2.  The orange colored code is for document ingestion using embedding model and storing embedded chunks in Chroma DB vector DB.
3. The green colored code is for document querying part which includes the getting relevant chunk of data based on user query using embedding model for answer generation   
   

Now enter below command to run the above application

  python sample.py  

it will run the application and now you can enter your query as follows in image with output of your query.

 

Happy coding with Generative AI applications.😀