Skip to content

Issues: vllm-project/vllm

[Roadmap] vLLM Roadmap Q4 2024
#9006 opened Oct 1, 2024 by simon-mo
Open 26
vLLM's V1 Engine Architecture
#8779 opened Sep 24, 2024 by simon-mo
Open 9
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1,253 Open 4,379 Closed
1,253 Open 4,379 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

[Performance]: vllm0.6.5加载GLM4-9B-Chat，动态加载lora，输入长文本时推理性能下降较多 performance Performance-related issues
#11317 opened Dec 19, 2024 by zh19980310
1 task done
@jeejeelee
[Bug]: gpt2 输出只有！ bug Something isn't working
#11316 opened Dec 19, 2024 by G1017
1 task done
[Bug]: Chat with n>1 breaks xgrammar bug Something isn't working
#11312 opened Dec 18, 2024 by joerunde
1 task done
1
[New Model]: RLHFlow/ArmoRM-Llama3-8B-v0.1 new model Requests to new models
#11306 opened Dec 18, 2024 by gx-ai-architect
1 task done
[Feature]: Publish arm64 image to dockerhub feature request
#11296 opened Dec 18, 2024 by ciarancourtney
1 task done
[Performance]: decoding speed on long context performance Performance-related issues
#11286 opened Dec 18, 2024 by 155394551lzk
1 task done
2
[Usage]: Is pipeline parallelism supported on machines that are not in the same local network? usage How to use vllm
#11285 opened Dec 18, 2024 by oldcpple
1 task done
1
[Bug]: Issues with vLLM tool call functionality leading to abnormal requests bug Something isn't working
#11284 opened Dec 18, 2024 by yumc2573
1 task done
[Bug]: [RuntimeError: CUDA error: unspecified launch failure ]int8 w8a8 quantization data set to generate model data, an error occurred when changing the specified data set bug Something isn't working
#11281 opened Dec 18, 2024 by bunengshuo1
1 task done
1
[Bug]: Invalid tool arguments generated in v0.6.5 bug Something isn't working
#11279 opened Dec 18, 2024 by marcelodiaz558
1 task done
[Usage]: How to set a dynamic temperature when sampling? (decrease gradually as more tokens generated) usage How to use vllm
#11276 opened Dec 18, 2024 by StarDewXXX
1 task done
1
[Bug]: AttributeError: 'CachedPreTrainedTokenizerFast' object has no attribute 'default_chat_template'. Did you mean: 'get_chat_template'? bug Something isn't working
#11271 opened Dec 17, 2024 by ziyang-arch
1 task done
4
[Doc]: update deploying with docker to include vllm TPU images documentation Improvements or additions to documentation
#11270 opened Dec 17, 2024 by annapendleton
1 task done
1
[Bug]: sentence_bert_config.json 404 Client Error bug Something isn't working
#11268 opened Dec 17, 2024 by shaowei-su
1 task done
[Bug]: vLLM on TPU does not support --pipeline-parallel-size with Ray bug Something isn't working
#11260 opened Dec 17, 2024 by totorochina
1 task done
[Feature]: LoRA support for qwen2-vl Models feature request
#11255 opened Dec 17, 2024 by xlg-go
1 task done
12
[Bug]: Qwen2vl vllm grounding任务效果不如transformers推理 bug Something isn't working
#11254 opened Dec 17, 2024 by nansanhao
1 task done
4
[Bug]: torch.OutOfMemoryError for 0.6.4.post1 but 0.6.3.post1 is working bug Something isn't working
#11251 opened Dec 17, 2024 by mces89
1 task done
4
[Bug]: Hermes tool choice can not supprot format 'string' bug Something isn't working
#11250 opened Dec 17, 2024 by warlockedward
1 task done
7
[Bug]: Using lm_format_enforcer, or using certain schemas, with Llama-3.2-90B-Vision-Instruct causes a crash bug Something isn't working
#11248 opened Dec 17, 2024 by npt
1 task done
3
[Bug]: disaggregated prefilling hangs when TP=2 bug Something isn't working
#11247 opened Dec 17, 2024 by Louis-99
1 task done
[New Model]: Support Phi-4 new model Requests to new models
#11238 opened Dec 16, 2024 by maziyarpanahi
1 task done
1
[New Model]: DeepSeek-VL2 new model Requests to new models
#11236 opened Dec 16, 2024 by mgoin
1 task done
[Bug]: With ROCm and certain HF models that require 'trust-remote-code', you get VLLM_RPC_TIMEOUT and failure to finish loading. bug Something isn't working rocm
#11232 opened Dec 16, 2024 by philtimmes
1 task done
@hongxiayang
2
[Bug]: ValueError: This model does not support the 'reward' task. Supported tasks: {'embedding'} bug Something isn't working
#11231 opened Dec 16, 2024 by wccccp
1 task done
1
ProTip! Adding no:label will show everything without a label.