Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
liukidar authored Dec 3, 2024
1 parent 0fcb4cf commit 62e2d11
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion benchmarks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ We also briefly report the insertion times for the 2wikimultihopqa benchmark (~8
|**Circlemind**| ~1.5|

### Run it yourself
The scripts in this directory will generate and evaluate the 2wikimultihopqa datasets on a subsets of 51 and 101 queries with the same methodology as in the HippoRAG paper. In particular, we evaluate the retrieval capabilities of each method, mesauring the percentage of queries for which all the required evidence was retrieved. We preloaded the results so it is enough to run `evaluate.xx` to get the numbers. You can also run `create_dbs.xx` to regenerate the databases for the different methods.
The scripts in this directory will generate and evaluate the 2wikimultihopqa datasets on a subsets of 51 and 101 queries with the same methodology as in the HippoRAG paper. In particular, we evaluate the retrieval capabilities of each method, mesauring the percentage of queries for which all the required evidence was retrieved. We preloaded the results so it is enough to run `evaluate_dbs.xx` to get the numbers. You can also run `create_dbs.xx` to regenerate the databases for the different methods.
A couple of NOTES:
- you will need to set an OPENAI_API_KEY;
- LightRAG and GraphRAG could take a while (~1 hour) to process;
Expand Down

0 comments on commit 62e2d11

Please sign in to comment.