feat: Ported back new grammar changes from C++ to Python implementati…

…on (#1637) * Backported . (any chat) from llama.cpp * unfinished {count,optionalmax) * implemented slice function in std:vector * fixed mistake done while reading * ported ggerganov/llama.cpp#7194 * multiple fixes, var copy * Rewrite LlamaGrammar internals in python style * bugfix --------- Co-authored-by: Andrei <[email protected]>
abetlen · Aug 7, 2024 · dff186c · dff186c
1 parent 5575fed
commit dff186c
Show file tree

Hide file tree

Showing 2 changed files with 682 additions and 972 deletions.
diff --git a/llama_cpp/llama_cpp.py b/llama_cpp/llama_cpp.py
@@ -3002,7 +3002,7 @@ def llama_grammar_init(
     n_rules: Union[ctypes.c_size_t, int],
     start_rule_index: Union[ctypes.c_size_t, int],
     /,
-) -> llama_grammar_p:
+) -> Optional[llama_grammar_p]:
     """Initialize a grammar from a set of rules."""
     ...