How can I achieve GPU acceleration by Offloading few layers to GPU ?

[nikhildr22]

I'm trying to use Local LLaMa models. But Im not able to find any info about GPU Offloading capability using Langflow.
Is it possible to achieve this if I've installed Langflow using the pip?

[rodgermoore]

I have exactly the same question. What are you trying to use? The CTransformers wrapper?

[ogabrielluiz]

I'm trying to implement this in this PR #519 but I can't test it. Could you try it out?

[Cristhianzl]

Hi! 👋

We are using other channels as our official means of communication with users. We apologize for the delayed response.
For faster assistance, please direct your questions to our Discord channel or open a GitHub issue, as we are much more active on these platforms.

Langflow Discord Server

Thank you for your understanding.

Best regards,
The Langflow Team