Skip to content

Commit

Permalink
Fix formatting
Browse files Browse the repository at this point in the history
  • Loading branch information
liukidar committed Dec 3, 2024
1 parent 4915e13 commit 7e65ad1
Showing 1 changed file with 8 additions and 10 deletions.
18 changes: 8 additions & 10 deletions benchmarks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,12 @@ Loading dataset...
LightRAG [local mode]
Loading dataset...
Percentage of queries with perfect retrieval: 0.47058823529411764
[multihop] Percentage of queries with perfect retrieval: 0.32432432432432434
[multihop only] Percentage of queries with perfect retrieval: 0.32432432432432434
GraphRAG [local mode]
Loading dataset...
Percentage of queries with perfect retrieval: 0.7450980392156863
[multihop] Percentage of queries with perfect retrieval: 0.6756756756756757
[all questions] Percentage of queries with perfect retrieval: 0.7450980392156863
[multihop only] Percentage of queries with perfect retrieval: 0.6756756756756757
Circlemind
Loading dataset...
Expand All @@ -40,28 +40,26 @@ Loading dataset...
LightRAG [local mode]
Loading dataset...
Percentage of queries with perfect retrieval: 0.44554455445544555
[multihop] Percentage of queries with perfect retrieval: 0.2753623188405797
[all questions] Percentage of queries with perfect retrieval: 0.44554455445544555
[multihop only] Percentage of queries with perfect retrieval: 0.2753623188405797
GraphRAG [local mode]
Loading dataset...
Percentage of queries with perfect retrieval: 0.7326732673267327
[multihop] Percentage of queries with perfect retrieval: 0.6376811594202898
[all questions] Percentage of queries with perfect retrieval: 0.7326732673267327
[multihop only] Percentage of queries with perfect retrieval: 0.6376811594202898
Circlemind
Loading dataset...
[all questions] Percentage of queries with perfect retrieval: 0.9306930693069307
[multihop only] Percentage of queries with perfect retrieval: 0.8985507246376812
```

We also quickly benchmarked on the HotpotQA dataset (we will soon release the code for that as well). Here's a preview of the results (101 queries):
We also benchmarked on the HotpotQA dataset (we will soon release the code for that as well). Here's a preview of the results (101 queries):

```
VectorDB: 0.78
LightRAG [local mode]: 0.55
GraphRAG [local mode]: - (crashed after half an hour of processing)
Circlemind: 0.84
```

Expand Down

0 comments on commit 7e65ad1

Please sign in to comment.