GraphRAG结合普通RAG，打造Hybrid RAG

HybridRAG：RAG与GraphRAG的强强联合。

RAG在生成式AI领域取得了重大进展，用户可以将自己的个人文档，比如文本文件、PDF、视频等，与大型语言模型（LLMs）连接起来进行互动。最近，RAG的进阶版GraphRAG也亮相了，它通过知识图谱和LLMs来执行RAG的检索任务。

RAG和GraphRAG各有所长，也各有局限。RAG擅长利用向量相似性技术，而GraphRAG则依赖图分析和知识图谱来提供更精确的答案。那么，如果将两者结合起来进行检索，会擦出怎样的火花呢？

1 HybridRAG

HybridRAG是一个高级框架，它合并了RAG和GraphRAG。这种集成旨在提高信息检索的准确性和上下文相关性。简单来说，HybridRAG使用来自两个检索系统（RAG和GraphRAG）的上下文，最终输出是两个系统的混合。

2 HybridRAG的优势

• 提高准确性： 通过利用结构化推理和灵活检索，HybridRAG提供的答案比单独使用VectorRAG或GraphRAG更精确。

• 增强上下文理解： 通过整合不同系统，HybridRAG能更深入地理解实体间的关系及其出现的上下文。

• 动态推理能力： 知识图谱可以动态更新，使系统能够适应新信息的可用性。

3 使用LangChain来构建HybridRAG系统

这里使用一个名为“Moon.txt”的文件进行这个演示，这是一个超级英雄故事。请查看以下内容。

In the bustling city of Lunaris, where the streets sparkled with neon lights and the moon hung low in the sky, lived an unassuming young man named Max. By day, he was a mild-mannered astronomer, spending his hours studying the stars and dreaming of adventures beyond Earth. But as the sun dipped below the horizon, Max transformed into something extraordinary—Moon Man, the guardian of the night sky.
Max’s transformation began with a mysterious encounter. One fateful evening, while gazing through his telescope, a brilliant flash of light erupted from the moon. A celestial being, shimmering with silver light, descended and bestowed upon him a magical amulet. “With this, you shall harness the power of the moon,” the being declared. “Use it wisely, for the night sky needs a hero.”
With the amulet around his neck, Max felt energy coursing through him. He could leap great distances, manipulate moonlight, and even communicate with nocturnal creatures. He vowed to protect his city from the shadows that lurked in the night.
As Moon Man, Max donned a sleek, silver suit adorned with celestial patterns that glimmered like the stars. With his newfound abilities, he patrolled the city, rescuing lost pets, helping stranded motorists, and even thwarting petty criminals. The citizens of Lunaris began to whisper tales of their mysterious hero, who appeared under the glow of the moon.
One night, as he soared through the sky, he encountered a gang of thieves attempting to steal a priceless artifact from the Lunaris Museum. With a flick of his wrist, he summoned a beam of moonlight that blinded the thieves, allowing him to swoop in and apprehend them. The city erupted in cheers, and Moon Man became a beloved figure.
However, peace in Lunaris was short-lived. A dark force emerged from the depths of the cosmos—an evil sorceress named Umbra, who sought to extinguish the moon’s light and plunge the world into eternal darkness. With her army of shadow creatures, she began to wreak havoc, stealing the moon’s energy and spreading fear among the citizens.
Moon Man knew he had to confront this new threat. He gathered his courage and sought the wisdom of the celestial being who had granted him his powers. “To defeat Umbra, you must harness the full power of the moon,” the being advised. “Only then can you restore balance to the night sky.”
With determination in his heart, Moon Man prepared for the ultimate battle. He climbed to the highest peak in Lunaris, where the moon shone brightest, and focused on channeling its energy. As Umbra and her shadow creatures descended upon the city, Moon Man unleashed a magnificent wave of moonlight, illuminating the darkness.
The battle raged on, with Umbra conjuring storms of shadows and Moon Man countering with beams of silver light. The clash of powers lit up the night sky, creating a dazzling display that captivated the citizens below. In a final, desperate move, Moon Man summoned all his strength and unleashed a powerful blast of moonlight that enveloped Umbra, banishing her to the farthest reaches of the cosmos.
With Umbra defeated, the moon’s light returned to its full glory, and the city of Lunaris rejoiced. Max, still in his Moon Man guise, stood atop the highest building, watching as the citizens celebrated their hero. They had learned the importance of hope and courage, even in the darkest of times.
From that day forward, Moon Man became a symbol of resilience and bravery. Max continued to protect Lunaris, knowing that as long as the moon shone brightly, he would always be there to guard the night sky. And so, the legend of Moon Man lived on, inspiring generations to look up at the stars and believe in the extraordinary.
As the years passed, stories of Moon Man spread beyond Lunaris, becoming a beacon of hope for those who felt lost in the darkness. Children would gaze at the moon, dreaming of adventures, and Max would smile, knowing that he had made a difference. For in the heart of every dreamer, the spirit of Moon Man lived on, reminding them that even the smallest light can shine brightly against the shadows.

导入包并设置LLM端嵌入模型（用于标准RAG）

import os
from langchain_experimental.graph_transformers import LLMGraphTransformer
from langchain_core.documents import Document
from langchain_community.graphs.networkx_graph import NetworkxEntityGraph
from langchain.chains import GraphQAChain
from langchain.text_splitter import CharacterTextSplitter
from langchain.document_loaders import TextLoader
from langchain.chains import RetrievalQA
from langchain.vectorstores import Chroma
from langchain_google_genai import GoogleGenerativeAI,GoogleGenerativeAIEmbeddings

GOOGLE_API_KEY=''

embeddings = GoogleGenerativeAIEmbeddings(model="models/embedding-001",google_api_key=GOOGLE_API_KEY)
llm = GoogleGenerativeAI(model="gemini-pro",google_api_key=GOOGLE_API_KEY)

接者，为GraphRAG实现（链对象）创建函数，覆盖文件“Moon.txt”

def graphrag():
    with open('Moon.txt', 'r') as file:
        content = file.read()

    documents = [Document(page_content=content)]
    llm_transformer = LLMGraphTransformer(llm=llm)
    graph_documents = llm_transformer.convert_to_graph_documents(documents)

    graph = NetworkxEntityGraph()

    # 添加节点到图
    for node in graph_documents[0].nodes:
        graph.add_node(node.id)

    # 添加边到图
    for edge in graph_documents[0].relationships:
        graph._graph.add_edge(
                edge.source.id,
                edge.target.id,
                relation=edge.type,
            )

        graph._graph.add_edge(
                edge.target.id,
                edge.source.id,
                relation=edge.type+" by",
            )

    chain = GraphQAChain.from_llm(
        llm=llm, 
        graph=graph, 
        verbose=True
    )
        
    return chain

同样，为同一文件实现标准RAG创建函数

def rag():
    # 文档加载器
    loader = TextLoader('Moon.txt')
    data = loader.load()

    # 文档转换器
    text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
    texts = text_splitter.split_documents(data)

    # 向量数据库
    docsearch = Chroma.from_documents(texts, embeddings)

    # 需要知道的超参数
    retriever = docsearch.as_retriever(search_type='similarity_score_threshold',search_kwargs={"k": 7,"score_threshold":0.3})
    qa = RetrievalQA.from_chain_type(llm=llm, chain_type="stuff", retriever=retriever)
        
    return qa

为两种类型的RAG创建对象

standard_rag = rag()
graph_rag = graphrag()

现在是时候实现HybridRAG了

def hybrid_rag(query,standard_rag,graph_rag):
    result1 = standard_rag.run(query)
    
    print("Standard RAG:",result1)
    result2 = graph_rag.run(query)
    
    
    print("Graph RAG:",result2)
    prompt = "Generate a final answer using the two context given : Context 1: {} \n Context 2: {} \n Question: {}".format(result1,result2,query)
    return llm(prompt)

query = "Some characteristics of Moon Man"
hybrid = hybrid_rag(query,standard_rag,graph_rag)
print("Hybrid:",hybrid)

如你所见，我们对给定的提示分别独立执行了标准RAG和GraphRAG。一旦找到答案，我们就会利用这两个响应作为上下文，来生成最终的答案。

谈到输出，最终的HybridRAG确实从两次检索中获取了上下文，并产生了更好的结果。有些点被两个RAG系统 遗漏了，但最终HybridRAG结合并给出了完美的答案。

STANDARD RAG: 
 Here are some characteristics of Moon Man, based on the story:

* **Brave:** He confronts danger and fights villains like Umbra.
* **Powerful:** He has superhuman abilities granted by the amulet.
* **Protective:** He safeguards Lunaris and its citizens.
* **Determined:** He doesn't give up, even when facing powerful enemies.
* **Compassionate:** He helps those in need, like rescuing lost pets.
* **Humble:** Despite his powers, he remains grounded and dedicated to his city. 



> Entering new GraphQAChain chain...
Entities Extracted:
Moon Man

Full Context:
Moon Man PROTECTS night sky
Moon Man WEARS silver suit
Moon Man PROTECTED Lunaris
Moon Man CAPTURED thieves
Moon Man DEFEATED Umbra
Moon Man INSPIRES hope
Moon Man INSPIRES courage

> Finished chain.

 @@ 
 GRAPH RAG: 
 Helpful Answer: 
* Protective (protects night sky, protected Lunaris)
* Courageous and Inspiring (inspires hope, inspires courage)
* Strong (captured thieves, defeated Umbra) 


 @@ 
 HYBRID RAG: 
 Moon Man is the **protective** champion of Lunaris, using his **strength** and **courage** to defend its citizens and the night sky. He is **powerful** and **determined**, facing down villains like Umbra without giving up. Yet, despite his abilities, he remains **humble** and **compassionate**, always willing to help those in need. Moon Man is a true inspiration, reminding everyone that even in darkness, hope and heroism can shine through. 

推荐书单

《生成式AI实战基于Transformer、Stable Diffusion、LangChain和AI Agent》

本书由浅入深地介绍了生成式AI的理论与实践，内容涉及从基础原理到前沿应用，为读者提供了一个系统的认知框架。本书从生成式AI技术的基础工具入手，逐步深入到Transformer模型与GPT的原理和应用，详细介绍了图像生成模型Stable Diffusion，以及LangChain与AI Agent的相关知识。书中结合开源代码分析，展示了生成式AI在各行各业的实际应用，并探讨了其在高速发展过程中所面临的伦理和隐私风险。 本书适合对生成式AI感兴趣的读者阅读，无论你是初学者还是有一定编程基础的人士都能从中获得宝贵的知识和经验。对于编程基础的读者，本书提供了跳过代码实现的理论学习路径。