vllm.transformers_utils.configs.speculators.base ¶
SpeculatorsConfig ¶
Bases: PretrainedConfig
Source code in vllm/transformers_utils/configs/speculators/base.py
14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 |
|
build_vllm_speculative_config classmethod
¶
Build vLLM-compatible speculative configuration from speculators format.
This method extracts and transforms speculative configuration from the speculators format into the structure expected by vLLM.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
config_dict | dict[str, Any] | Configuration dictionary in speculators format | required |
Returns:
Type | Description |
---|---|
dict[str, Any] | Dictionary with vLLM-compatible speculative configuration |
Source code in vllm/transformers_utils/configs/speculators/base.py
extract_vllm_speculative_config classmethod
¶
Source code in vllm/transformers_utils/configs/speculators/base.py
from_pretrained classmethod
¶
from_pretrained(
pretrained_model_name_or_path: Union[str, PathLike],
**kwargs,
) -> SpeculatorsConfig
Load speculators Eagle config and convert to vLLM format.