Add evaluate RAG with LlamaIndex #253
Conversation
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
github-actions
bot
added
status:awaiting review
| @@ -0,0 +1,550 @@ | |||
| { | |||
There was a problem hiding this comment.
High level comment: Cool notebook! I like the eval part especially. It'd be awesome to deep dive on that (as another task, another day)
Reply via ReviewNB
| @@ -0,0 +1,550 @@ | |||
| { | |||
There was a problem hiding this comment.
Is it worth adding a link or two here that can point users off to relevant pre-reading? e.g. make LlamaIndex link to their site, and make RAG link to something with background info on RAG?
Reply via ReviewNB
| @@ -0,0 +1,550 @@ | |||
| { | |||
There was a problem hiding this comment.
Clear outputs for pip install and for llamaindex specifically can you pin the version? It's <1.0 so they can and have made breaking API changes, so it would help keep the guide working.
Reply via ReviewNB
| @@ -0,0 +1,550 @@ | |||
| { | |||
There was a problem hiding this comment.
Line #6. "Response": [response.response],
According to the type annotations response is a string, so has no .response property. Ditto for response.source_nodes
Reply via ReviewNB
| @@ -0,0 +1,550 @@ | |||
| { | |||
There was a problem hiding this comment.
Line #8. "Evaluation Result": [eval_result.feedback],
eval_result is a string too, so has no .feedback property.
Reply via ReviewNB
| @@ -0,0 +1,550 @@ | |||
| { | |||
There was a problem hiding this comment.
Line #12. eval_df = eval_df.style.set_properties(
Could you use the colab formatter instead?
from google import colab colab.data_table.enable_dataframe_formatter()
Reply via ReviewNB
| @@ -0,0 +1,550 @@ | |||
| { | |||
There was a problem hiding this comment.
Line #9. display_eval_df(question, llm_response, eval_result)
I don't think we want to create 1-row dataframes in a loop. This would look increasingly strange as N gets bigger. Can you assemble a single dataframe instead?
Reply via ReviewNB
markmcd
added
status:awaiting response
|
Marking this pull request as stale since it has been open for 14 days with no activity. This PR will be closed if no further activity occurs. |
github-actions
bot
added
the
status:stale
No description provided.