Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

how to use GGML-Tips-&-Tricks in new code? #4819

Closed
hariji814 opened this issue Jan 8, 2024 · 6 comments
Closed

how to use GGML-Tips-&-Tricks in new code? #4819

hariji814 opened this issue Jan 8, 2024 · 6 comments

Comments

@hariji814
Copy link

https://github.com/ggerganov/llama.cpp/wiki/GGML-Tips-&-Tricks
If you follow this link, an error will appear after you modify it.

/media/harijizhang/793D8F946E3470B9/llama.cpp/llama.cpp: In function ‘int llama_decode_internal(llama_context&, llama_batch)’:
/media/harijizhang/793D8F946E3470B9/llama.cpp/llama.cpp:7229:9: error: ‘n_past’ was not declared in this scope
 7229 |     if (n_past % 45 == 0) {
      |         ^~~~~~
gmake[2]: *** [CMakeFiles/llama.dir/build.make:76:CMakeFiles/llama.dir/llama.cpp.o] 错误 1
gmake[1]: *** [CMakeFiles/Makefile2:732:CMakeFiles/llama.dir/all] 错误 2
gmake: *** [Makefile:146:all] 错误 2

The compilation failed because it is incompatible with the current llama.cpp

@bullno1
Copy link
Contributor

bullno1 commented Jan 8, 2024

After the move to batch, n_past doesn't exactly exist anymore.

I guess batch.pos[batch.n_tokens - 1] is an approximation. It should work for a single sequence batch.

I don't know why it's % N though, that would output the graph every few tokens?

Copy link
Contributor

This issue is stale because it has been open for 30 days with no activity.

@github-actions github-actions bot added the stale label Mar 18, 2024
Copy link
Contributor

github-actions bot commented Apr 4, 2024

This issue was closed because it has been inactive for 14 days since being marked as stale.

@github-actions github-actions bot closed this as completed Apr 4, 2024
@Javipove
Copy link

Can any one explain how to solve this problem?

@hariji814
Copy link
Author

Can any one explain how to solve this problem?

hey , bro, just n_past is now replaced with batch.pos[] ,it will be okay

https://github.com/ggerganov/llama.cpp/wiki/GGML-Tips-&-Tricks

@Javipove
Copy link

But then how it should look the resulting modification to print the computational graph? Because performing that change it doesn't compile and the other part of the code to modify do only save one layer doesn't exist. Sorry I am a bit new in this but the guide for this "simple" modification is not correct

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants