Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Retry when structured output fails #86

Open
RyanMarten opened this issue Nov 13, 2024 · 1 comment
Open

Retry when structured output fails #86

RyanMarten opened this issue Nov 13, 2024 · 1 comment
Assignees

Comments

@RyanMarten
Copy link
Contributor

Thing things we need to check and then retry if they fail
(1) the response is valid JSON (try-catch current in parse_response_message)
(2) the response JSON matches the Pydantic model schema (base_request_processor in create_dataset_files)

Right now these checks happen after the LLM responds (successfully sends back a string) so we don't have it within the retry logic. For batch, we would need to accumulate all the failed responses and send a new batch.
For Online, we can add these checks before writing to responses.jsonl file. However the logic might be weird since everything about this parsing was abstracted away from the requests --> responses logic in online. Might need to rethink the pathways a little bit.

Lessons learned from #85

@RyanMarten
Copy link
Contributor Author

Discussion on strict: True in this comment: #85 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants