[Frontend] improve hermes_tool_parser.py #11444
Conversation
Using json_repair can improve the robustness of parsing the JSON output
|
👋 Hi! Thank you for contributing to the vLLM project. Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can do one of these:
🚀 |
| try: | ||
| from json_repair import json_repair | ||
| except ImportError: | ||
| # might remove codes below by adding json_repair in requirement.txt | ||
| import subprocess | ||
| subprocess.check_call(["pip", "install", "json_repair"]) | ||
| from json_repair import json_repair |
There was a problem hiding this comment.
Let's not install things automatically during runtime. Can you add this to the requirements file instead?
There was a problem hiding this comment.
Let's not install things automatically during runtime. Can you add this to the requirements file instead?
I noticed there are a lot of requirements files, and I'm not familiar with their specific uses. It might need some assistance with it.
There was a problem hiding this comment.
The base dependencies of vLLM are contained in requirements-common.txt. Since tool parsing is applicable to all backends, you can add the dependency there.
Using json_repair can improve the robustness of parsing the JSON output