Skip to content

vllm.kernels.vllm_c ¶

CUDA_ALIKE `module-attribute` ¶

CUDA_ALIKE = is_cuda_alike()

Most kernels in this file are supported on all CUDA-alike platforms.

rms_no_var_size `module-attribute` ¶

rms_no_var_size = (
    lambda x, weight, epsilon, variance_size=None: (
        variance_size is None
        and (weight is None or dtype == dtype)
    )
)

vLLM kernel requires no variance_size override and matching input/weight dtype.