v0.8.0: Introduction of Data Interpreter, Integration with RAG, and Expanded Support for LLMs

Release Highlights

Data Interpreter: Open Source and Better "Devin"

Data Interpreter is an autonomous agent that uses notebook, browser, shell, stable diffusion, and any custom tool to complete tasks.

Data Interpreter has achieved state-of-the-art scores in machine learning, mathematical reasoning, and open-ended tasks, and can analyze stocks, imitate websites, and train models.

It can debug code by itself, fix failures by itself, and solve a large number of real-life problems by itself.

We open-source our code and provide a wealth of working examples to give everyone access to state-of-the-art AI capabilities.

📝 Paper
🔗 Examples
📖 How to use

RAG

RAG module integrated! Common features such as indexing, retrieval, and rank are supported. Explore this example for a quick start.

New LLMs

New members for the supported LLM family: This time, Claude, QianFan, DashScope, and Yi. Check out here for configuration.

What's Changed

• Add ut mocker for mermaid ink by @shenchucheng in #884
• Unify Interpreter naming by @garylin2099 in #901
• Bump ipykernel to 6.27.1 by @revupst in #938
• Update role.py : Role.set_actions() by @lixiangtiandashen in #922
• Fix action.llm.model in examples/debate_simple by @BaoLiqi in #930
• Feat mock openai embed for document_store and memory UTs by @better629 in #939
• Feat add qianfan and dashscope by @better629 in #940
• Changed requirements.txt by @Abhishek0075 in #933
• Feat simplify provider ut code by @better629 in #942
• Update prompt of ocr example by @mannaandpoem in #944
• Feat: Wrap Openllm, Fireworks and other services into the OpenAILLM class. by @YangQianli92 in #946
• Doc: add docker installation section and fold it by @voidking in #950
• Feat fix missing mistral by @better629 in #953
• Feat add anthropic_api by @better629 in #961
• Update code plan and change and Improve test coverage by @mannaandpoem in #948
• Rename interpreter by @garylin2099 in #952
• Feat: RFC197 by @iorisa in #960
• Feat: +SECURITY.md by @iorisa in #970
• Feat: fix JSONDecodeError error caused by incorrect indentation used … by @moyitech in #963
• Feat: +rfc197 example by @iorisa in #968
• Improvement made : by automating search engine configuration in example search_in_search_engine.py by @Abhishek0075 in #965
• Restructure DataInterpreter by @garylin2099 in #995
• Restore WalmartSalesForecast example. by @orange-crow in #1007
• Feat add offcial yi-34b by @better629 in #1010
• Fix bug#979 : RuntimeError: fail to reduce message length by @azurewtl in #986
• Feat:add user llm config by @liujun3660105 in #997
• Feat add rag by @better629 in #974
• Fixbug: #1031 by @iorisa in #1035
• Update gpt-3.5-turbo-0125 in count_message_tokens by @RuifengFu in #1020
• Fix: typos by @testwill in #1027
• Fix bug: #1018 by @iorisa in #1037
• Fix critical bug: human prior not injected by @garylin2099 in #1043
• Correct bad URL in token_counter by @jinchihe in #1042
• Update readme with data interpreter usage by @garylin2099 in #1047
• Fix text ut error by @shenchucheng in #1049
• Update process for "Fix bug" by @mannaandpoem in #1054
• Fixbug: #1016 by @iorisa in #1058
• Add params to search engine by @geekan in #1065
• Feat: + tree command by @iorisa in #985
• Fix state value extract for issues/1067 by @better629 in #1069
• Fix bug: llm.timeout not working by @iorisa in #1060
• Feat: repo to markdown by @iorisa in #1061
• Fdd custom tool example by @geekan in #1071
• Make the rag module optional by @seehi in #1070
• Fix minecraft typo by @geekan in #1072
• Fixbug: #1076 by @iorisa in #1080
• Replace *.deepwisdom.ai with a local server for scraping by @shenchucheng in #1081
• Fixbug: #958 by @iorisa in #1082
• Updata di example and change the location of the .ipynb locations by @luxiangtaoya in #1090
• Fix issue 868 by @orange-crow in #1098
• Fix provider failed ut by @better629 in #1107
• Skip rebuild seq view by @geekan in #1108
• Fixbug: #1100 by @iorisa in #1105
• Changed concatenation of strings to f-strings to improve readability by @eukub in #648
• Feat rag add more features by @better629 in #1114
• Update paddleocr version to fix its bug by @garylin2099 in #1133

New Contributors

• @revupst made their first contribution in #938
• @lixiangtiandashen made their first contribution in #922
• @BaoLiqi made their first contribution in #930
• @Abhishek0075 made their first contribution in #933
• @YangQianli92 made their first contribution in #946
• @moyitech made their first contribution in #963
• @azurewtl made their first contribution in #986
• @liujun3660105 made their first contribution in #997
• @RuifengFu made their first contribution in #1020
• @testwill made their first contribution in #1027
• @jinchihe made their first contribution in #1042
• @luxiangtaoya made their first contribution in #1090
• @eukub made their first contribution in #648

Kudos to all the contributors for making their first mark!