Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add function get_similar_content_by_query_with_vectorstore. #961

Merged
merged 1 commit into from
Oct 29, 2024

Conversation

dzerkes
Copy link

@dzerkes dzerkes commented Oct 28, 2024

Fixes a bug when calling GPTResearcher with report_source='langchain_vectorstore'.

code to test:

import asyncio 
from gpt_researcher import GPTResearcher
import os 
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_openai import OpenAIEmbeddings 
from langchain_community.vectorstores import FAISS 
from langchain_core.documents import Document 
from gpt_researcher.document import DocumentLoader, LangChainDocumentLoader 

os.environ['OPENAI_API_KEY']="add your key"
os.environ['TAVILY_API_KEY']="add your key"

async def main():
    document_data = await DocumentLoader("./my-docs").load()
    langchain_docs = [Document(page_content=item["raw_content"], metadata={"source":item["url"]}) 
                      for item in document_data]
    text_splitter = RecursiveCharacterTextSplitter(chunk_size=1400, chunk_overlap=200)
    splitted_docs = text_splitter.split_documents(langchain_docs)
    vector_store = FAISS.from_documents(splitted_docs, OpenAIEmbeddings())
    
    researcher = GPTResearcher(
        query="Write me a summary about formalin or formaldehyde", 
        report_source="langchain_vectorstore",
        vector_store=vector_store
    )
    
    context = await researcher.conduct_research()

if __name__ == "__main__":
    asyncio.run(main())

Without the addition of the function get_similar_content_by_query_with_vectorstore, it would throw an error.

P.S.: Before some weeks, the problem was that while this function existed it used double underscore, hence when it was called, it was not called correctly. But now, I see that this function disappeared completely.

…ug when calling GPTResearcher with report_source='langchain_vectorstore'
Copy link
Owner

@assafelovic assafelovic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks wonder how this was missing. Appreciate it!

@assafelovic assafelovic merged commit 248e364 into assafelovic:master Oct 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants