Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix gguf-py: Conversion error when multiple licenses are configured #9807

Merged
merged 8 commits into from
Nov 24, 2024

Conversation

mmngays
Copy link
Contributor

@mmngays mmngays commented Oct 9, 2024

In the write_kv_data_to_file function of GGUFWriter, when there are multiple licenses, they were being passed as a list, which was causing an error. Therefore, I have joined them into a string.

general.license ['llama3.1', 'gemma']

Traceback (most recent call last):
  File "/home/mmnga/llm-data/./llama.cpp/convert_hf_to_gguf.py", line 4433, in <module>
    main()
  File "/home/mmnga/llm-data/./llama.cpp/convert_hf_to_gguf.py", line 4427, in main
    model_instance.write()
  File "/home/mmnga/llm-data/./llama.cpp/convert_hf_to_gguf.py", line 436, in write
    self.gguf_writer.write_kv_data_to_file()
  File "/home/mmnga/llm-data/llama.cpp/gguf-py/gguf/gguf_writer.py", line 243, in write_kv_data_to_file
    kv_bytes += self._pack_val(val.value, val.type, add_vtype=True)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mmnga/llm-data/llama.cpp/gguf-py/gguf/gguf_writer.py", line 882, in _pack_val
    kv_data += encoded_val
TypeError: can't concat list to bytearray

I have confirmed this error in this model.
license is llama3.1 and gemma.
https://huggingface.co/tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.1

@github-actions github-actions bot added the python python script changes label Oct 9, 2024
@mmngays
Copy link
Contributor Author

mmngays commented Oct 23, 2024

Sorry for the delay in making the corrections. As you pointed out, I moved it to the location where add_license in metadata.py is being called.

@compilade compilade added the merge ready indicates that this may be ready to merge soon and is just holding out in case of objections label Nov 22, 2024
@slaren slaren merged commit 96fa2c5 into ggerganov:master Nov 24, 2024
8 checks passed
arthw pushed a commit to arthw/llama.cpp that referenced this pull request Dec 20, 2024
…gerganov#9807)

* fix general.license list to str

* fix join license list

---------

Co-authored-by: momonga <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
merge ready indicates that this may be ready to merge soon and is just holding out in case of objections python python script changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants