[DRAFT] Add notebook for snowpark example #1188

sfc-gh-pdharmana · 2024-06-07T16:35:39Z

Items to add to release announcement:

Heading: delete this list if this PR does not introduce any changes that need announcing.

Other details that are good to know but need not be announced:

There should be something here at least.

* first * typos * typehint

* import llama only if needed * use optional imports instead --------- Co-authored-by: Piotr Mardziel <[email protected]>

* fix * typo * don't print external if internal is available

Co-authored-by: Piotr Mardziel <[email protected]>

* adjust docstring for select_context * langchain select_context, update quickstarts * undo app name change * remove dev cell * generalized langchain select_context (#711) * generalized langchain select_context * typo * typo in string * update langchain example to pass app in select_context --------- Co-authored-by: Josh Reini <[email protected]> * comments, clarity updates to quickstarts * add lib-independent select_context * update lc li quickstarts --------- Co-authored-by: Piotr Mardziel <[email protected]>

* add optional * bug class_info fix

* update configs * bugfix * dont add class info to dicts

* Fix correctness prompt Fixes #718 * Update base.py

* Bump suggested notebook versions * Combine notebooks and py files --------- Co-authored-by: Shayak Sen <[email protected]>

* Bump suggested notebook versions * Combine notebooks and py files * Update __init__.py --------- Co-authored-by: Shayak Sen <[email protected]>

* debug * display python version * python version * PromptTemplate update import * bad escape fix * add msg to exception * pass kwargs in Groundedness * pass kwargs with GroundTruthAgreement * give default value to ground_truth_imp * migrate db on reset

* fix example notebook * fixes * remove commented out

* always use prompt instead of messages * use messages in base * use prompt in bedrock * move score to top of cot template, request entire template be used * remove dev * add TODO

* update langchain instrumentation page * include instrumented methods * llama-index instrumentation updates * update the overview * change path to instrumentation overview * add some more info in appendices and line space --------- Co-authored-by: Piotr Mardziel <[email protected]>

* add instructions and text wrapping * format * clean up github scripts and update README sources * typo --------- Co-authored-by: Josh Reini <[email protected]>

Co-authored-by: joshreini1 <[email protected]>

* fix * remove redundant --------- Co-authored-by: Josh Reini <[email protected]>

* adjusted * fix typo

Co-authored-by: joshreini1 <[email protected]>

* add instructions and text wrapping * format * debugging * making obj arg no longer required * remove obj and add documentation for WithClassInfo * remove IPython from most notebooks and organize imports * fix test errors * forgot warning --------- Co-authored-by: Josh Reini <[email protected]>

* update notebooks to test * rehack * update langchain requirement * add core lowerbound

* fix instance check on generic types * remove original check --------- Co-authored-by: Piotr Mardziel <[email protected]>

* check for langchain legacy * few more * typo * rephrase * rephrase

* pass bedrock provider to ground truth eval * Update bedrock_finetuning_experiments.ipynb --------- Co-authored-by: Piotr Mardziel <[email protected]>

* first * temp format * minor * more notes * minor fixes

* first * temp format * minor * more notes * minor fixes * more fixes

* bump version 0.20.1 * Update langchain_quickstart.ipynb * Update llama_index_quickstart.ipynb * bump py scripts

retreivers -> retrievers

* unify groundedness start * remove groundedness.py * groundedness nli moves * remove custom aggregator * groundedness aggregator to user code * move agg to trulens side by default (groundedness) * remove extra code * remove hf key setting * remove hf import * add comment about aggregation for context relevance * update init * remove unneeded import * use generate_score_and_reasons for groundedness internally * f-strings for groundedness prompts * docstring * docstrings formatting * groundedness reasons template * remove redundant prompt * update quickstarts * llama-index notebooks * rag triad helper update * oai assistant nb * update readme * models notebooks updates * iterate nbs * mongo, pinecone nbs * update huggingface docstring * remove outdated docstring selector notes * more docstring cleaning

* open ai streaming adjustments in cost tracking * notes * delete outputs

Co-authored-by: joshreini1 <[email protected]> Co-authored-by: Josh Reini <[email protected]>

* Update selecting_components.md * Update MultiQueryRetrievalLangchain.ipynb * Update random_evaluation.ipynb * Update canopy_quickstart.ipynb

Co-authored-by: joshreini1 <[email protected]>

* update comprehensiveness + nb * nb expansion * fix typo * meetingbank transcript data * oss models in app * test * benchmarking gpt-3.5-turbo, gpt-4-turbo, and gpt-4o * update path * comprehensiveness benchmark * updated summarization_eval nb * fix normalization * show improvement in comprehensiveness feedback functions --------- Co-authored-by: Daniel <[email protected]>

* version bump * simpler lc quickstart * update installs and imports * update langchain instrumentation docs * remove groundedness ref from providers.md * build docs fixes * remove key cell * fix docs build * firx formatting for stock.md * remove extra spaces * undo format change * update docstrings for hugs and base provider * openai docstring updates * hugs docstring update * update context relevance hugs docstring * more docstring updates * remove can be changed messages from openai provider docstrings

Co-authored-by: joshreini1 <[email protected]>

* add to glossary * finish some terms

* add httpx to requirements * undo httpx add * test add langchain-community * change langchain-community required version

…nai endpoint (#1154) * first * adjust docs * logic * add langhain_community to pinecone reqs too * adjust trubot example with new reqs * more reqs * remove extra text * add langchain_community to a few imports in notebooks --------- Co-authored-by: Piotr Mardziel <[email protected]> Co-authored-by: Josh Reini <[email protected]>

Co-authored-by: Josh Reini <[email protected]>

* fix logo formatting * fix url --------- Co-authored-by: Josh Reini <[email protected]>

review-notebook-app · 2024-06-07T16:35:44Z

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

joshreini1 · 2024-06-07T16:39:34Z

Can you move this to /expositional instead of /quickstart?

Also would be great if this used the standard TruLens QuickStart (with TruCustom) rather than TruBasic

joshreini1 · 2024-06-19T15:19:50Z

trulens_eval/examples/quickstart/snowpark_text2text_quickstart.ipynb

@@ -0,0 +1,280 @@
+{


Should include some info about snowpark here and why people care.

Reply via ReviewNB

I'm generally unclear on what the purpose of this notebook is. To show cortex LLM? Logging to snowflake table? Why not cortex feedback provider, cortex search ,etc?

joshreini1 · 2024-06-19T15:19:50Z

trulens_eval/examples/quickstart/snowpark_text2text_quickstart.ipynb

@@ -0,0 +1,280 @@
+{


Is 'It should look like this in' truncated? What does this mean?

Reply via ReviewNB

joshreini1 · 2024-06-19T15:19:50Z

trulens_eval/examples/quickstart/snowpark_text2text_quickstart.ipynb

@@ -0,0 +1,280 @@
+{


nit: change to = "sk-..."

Reply via ReviewNB

sfc-gh-jreini · 2024-06-19T15:20:32Z

This should not be a quickstart. Move to /expositional

sfc-gh-jreini · 2024-07-17T20:48:37Z

@sfc-gh-pdharmana can we close this? Seems stale

piotrm0 and others added 30 commits December 21, 2023 10:40

allow debug timeout to be adjusted (#713)

5fa4b82

* first * typos * typehint

import llama only if needed (#714)

332858b

* import llama only if needed * use optional imports instead --------- Co-authored-by: Piotr Mardziel <[email protected]>

fix dashboard starts for colab (#721)

426fa70

* fix * typo * don't print external if internal is available

savE (#719)

37772e2

Co-authored-by: Piotr Mardziel <[email protected]>

first (#720)

c26988c

add optional (#723)

a791a96

* add optional * bug class_info fix

pydantic2 deprecation fix to model config (#724)

7e024c3

* update configs * bugfix * dont add class info to dicts

Fix correctness prompt (#725)

f2eefbd

* Fix correctness prompt Fixes #718 * Update base.py

Releases/rc trulens eval 0.20.0 (#727)

0c0b484

* Bump suggested notebook versions * Combine notebooks and py files --------- Co-authored-by: Shayak Sen <[email protected]>

Releases/rc trulens eval 0.20.0 (#729)

27b664c

* Bump suggested notebook versions * Combine notebooks and py files * Update __init__.py --------- Co-authored-by: Shayak Sen <[email protected]>

azureopenai fixes (#735)

d7290be

* fix example notebook * fixes * remove commented out

fix typo (#739)

5c84ecc

Update extract_score_and_reasons to work across providers (#732)

b8d9303

* always use prompt instead of messages * use messages in base * use prompt in bedrock * move score to top of cot template, request entire template be used * remove dev * add TODO

add instructions for installing from github (#740)

e2b5ad1

* add instructions and text wrapping * format * clean up github scripts and update README sources * typo --------- Co-authored-by: Josh Reini <[email protected]>

Automated File Generation from Docs Notebook Changes (#744)

7e2d753

Co-authored-by: joshreini1 <[email protected]>

adjust optional llama (#745)

1b7c516

* fix * remove redundant --------- Co-authored-by: Josh Reini <[email protected]>

adjust human feedback notebook (#746)

c28c0af

* adjusted * fix typo

Automated File Generation from Docs Notebook Changes (#749)

caa9205

Co-authored-by: joshreini1 <[email protected]>

update notebooks to test (#753)

b9a2fc9

langchain thread executor rehack (#755)

98d9623

* update notebooks to test * rehack * update langchain requirement * add core lowerbound

Fix subscripted generics typechecking for Python<3.10 (#754)

7bdf01f

* fix instance check on generic types * remove original check --------- Co-authored-by: Piotr Mardziel <[email protected]>

check for langchain legacy (#757)

feabdba

* check for langchain legacy * few more * typo * rephrase * rephrase

pass bedrock provider to ground truth eval (#743)

02b11f0

* pass bedrock provider to ground truth eval * Update bedrock_finetuning_experiments.ipynb --------- Co-authored-by: Piotr Mardziel <[email protected]>

convert structures to str in feedback result tables (#758)

3d58eee

* first * temp format * minor * more notes * minor fixes

langchain provider fix (#759)

4989a14

* first * temp format * minor * more notes * minor fixes * more fixes

Releases/rc trulens eval 0.20.1 (#761)

fb2f424

* bump version 0.20.1 * Update langchain_quickstart.ipynb * Update llama_index_quickstart.ipynb * bump py scripts

joshreini1 and others added 22 commits May 14, 2024 11:34

Update custom_feedback_functions.ipynb

e365a6e

Update README.md (#1136)

744ea34

retreivers -> retrievers

dont iterate streams in openai cost tracking (#1138)

78dbb12

* open ai streaming adjustments in cost tracking * notes * delete outputs

Automated File Generation from Docs Notebook Changes (#1137)

95d8d0b

Co-authored-by: joshreini1 <[email protected]> Co-authored-by: Josh Reini <[email protected]>

Fix a few old groundedness references (#1139)

c424cd4

* Update selecting_components.md * Update MultiQueryRetrievalLangchain.ipynb * Update random_evaluation.ipynb * Update canopy_quickstart.ipynb

Update all_tools.py

e557792

Update llama_index_quickstart.ipynb

9a6edec

Automated File Generation from Docs Notebook Changes (#1141)

e41e513

Co-authored-by: joshreini1 <[email protected]>

Automated File Generation from Docs Notebook Changes (#1143)

0c1d745

Co-authored-by: joshreini1 <[email protected]>

glossary additions (#1144)

43bfb74

* add to glossary * finish some terms

Update ollama_quickstart.ipynb

e06afaf

Patch imports error (#1146)

cf8cd67

* add httpx to requirements * undo httpx add * test add langchain-community * change langchain-community required version

0.30.0 bump (#1158)

e92b4a2

Co-authored-by: Josh Reini <[email protected]>

0.30.1 bump (#1159)

b53c597

Co-authored-by: Josh Reini <[email protected]>

update ref to groundedness fn (#1161)

94fb515

very minor fix of print statement (#1173)

cc00ec6

Fix sidebar logo formatting (#1169)

9f2eea2

* fix logo formatting * fix url --------- Co-authored-by: Josh Reini <[email protected]>

Add notebook for snowpark example

755b182

dosubot bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Jun 7, 2024

dosubot bot added the documentation Improvements or additions to documentation label Jun 7, 2024

joshreini1 reviewed Jun 19, 2024

View reviewed changes

sfc-gh-dhuang force-pushed the main branch from 7e0fbf3 to 26848c6 Compare June 29, 2024 04:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[DRAFT] Add notebook for snowpark example #1188

[DRAFT] Add notebook for snowpark example #1188

sfc-gh-pdharmana commented Jun 7, 2024

review-notebook-app bot commented Jun 7, 2024

joshreini1 commented Jun 7, 2024 •

edited

Loading

joshreini1 Jun 19, 2024 •

edited

Loading

sfc-gh-jreini Jun 19, 2024

joshreini1 Jun 19, 2024 •

edited

Loading

joshreini1 Jun 19, 2024 •

edited

Loading

sfc-gh-jreini commented Jun 19, 2024

sfc-gh-jreini commented Jul 17, 2024

[DRAFT] Add notebook for snowpark example #1188

Are you sure you want to change the base?

[DRAFT] Add notebook for snowpark example #1188

Conversation

sfc-gh-pdharmana commented Jun 7, 2024

review-notebook-app bot commented Jun 7, 2024

joshreini1 commented Jun 7, 2024 • edited Loading

joshreini1 Jun 19, 2024 • edited Loading

Choose a reason for hiding this comment

sfc-gh-jreini Jun 19, 2024

Choose a reason for hiding this comment

joshreini1 Jun 19, 2024 • edited Loading

Choose a reason for hiding this comment

joshreini1 Jun 19, 2024 • edited Loading

Choose a reason for hiding this comment

sfc-gh-jreini commented Jun 19, 2024

sfc-gh-jreini commented Jul 17, 2024

joshreini1 commented Jun 7, 2024 •

edited

Loading

joshreini1 Jun 19, 2024 •

edited

Loading

joshreini1 Jun 19, 2024 •

edited

Loading

joshreini1 Jun 19, 2024 •

edited

Loading