easydel.layers.quantization.quantizers

easydel.layers.quantization.quantizers#

class easydel.layers.quantization.quantizers.EasyQuantizer(quantization_method: EasyDeLQuantizationMethods = EasyDeLQuantizationMethods.NF4, quantization_platform: Optional[EasyDeLPlatforms] = EasyDeLPlatforms.JAX, quantization_pattern: Optional[str] = None, block_size: int = 256, **kwargs)[source]#

Bases: object

quantize_linears(model: Module, /, *, quantization_pattern: Optional[str] = None, verbose: bool = True) → Module[source]#

Quantize parameters to requested precision, excluding specified layers.

Parameters

  • model – The model to quantize.

  • quantization_pattern (str) – re pattern for layers to be quantized.

  • verbose (bool) – whenever to use tqdm for logging stuff.

Returns

Quantized parameters in the same structure as the input.

Contents