Vicuna issue

[tuandattt]

HI, I am trying to train this model but I have some issues when the model run Vicuna model:
RuntimeError: Internal: src/sentencepiece_processor.cc(1101) [model_proto->ParseFromArray(serialized.data(), serialized.size())]

Do you know how to fix this?

[SaraAlthubaiti]

HI, I am trying to train this model but I have some issues when the model run Vicuna model: RuntimeError: Internal: src/sentencepiece_processor.cc(1101) [model_proto->ParseFromArray(serialized.data(), serialized.size())]

Do you know how to fix this?

I had the chance to train the model, so I can help; could you please elaborate more on how and where exactly this error started to show? and which dataset/s did you use?

[hbellafkir]

HI, I am trying to train this model but I have some issues when the model run Vicuna model: RuntimeError: Internal: src/sentencepiece_processor.cc(1101) [model_proto->ParseFromArray(serialized.data(), serialized.size())]
Do you know how to fix this?

I had the chance to train the model, so I can help; could you please elaborate more on how and where exactly this error started to show? and which dataset/s did you use?

models/salmonn.py line 110:
self.llama_tokenizer = LlamaTokenizer.from_pretrained(llama_path, use_fast=False)

llama_path is the path to vicuna-13b-v1.1 where the tokenizer.model is located

[hbellafkir]

Ok, I think the error is due to large files. I used Git to clone Whisper and Vicuna from Hugging Face, which results in incomplete files if git-lfs is missing.