Skip to content

vllm.v1.executor.utils

get_and_update_mm_cache 

get_and_update_mm_cache(
    receiver_cache: ShmObjectStoreReceiverCache,
    args: tuple[SchedulerOutput],
) -> None

For each MultiModalKwargsItem in SchedulerOutput, fetch from shared memory cache as needed.

Parameters:

Name Type Description Default
receiver_cache ShmObjectStoreReceiverCache

The receiver cache to update.

 required
args tuple[SchedulerOutput]

According to the collective_rpc call of execute_model method in executor, args is a tuple of only one SchedulerOutput element.

 required
Source code in vllm/v1/executor/utils.py 
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
def get_and_update_mm_cache(
    receiver_cache: ShmObjectStoreReceiverCache,
    args: tuple[SchedulerOutput],
) -> None:
    """
    For each MultiModalKwargsItem in SchedulerOutput, fetch from shared memory
    cache as needed.

    Args:
        receiver_cache: The receiver cache to update.
        args: According to the collective_rpc call of execute_model method in
            executor, args is a tuple of only one SchedulerOutput element.
    """
    scheduler_output = args[0]
    for request_data in scheduler_output.scheduled_new_reqs:
        request_data.mm_features = receiver_cache.get_and_update_features(
            request_data.mm_features)