Skip to content

[Feature Request] support vLLM AsyncLLM for rollout #488

@ZongqiangZhang

Description

@ZongqiangZhang

vLLM AsyncLLM mode is already available in verl, whose performance is better than vLLM sync mode.
Any plan to support AsyncLLM ?

related code:
https://github.com/volcengine/verl/blob/main/verl/workers/rollout/vllm_rollout/vllm_async_server.py#L181
verl-project/verl@aacd366

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions