easydel.modules.glm4.modeling_glm4

class easydel.modules.glm4.modeling_glm4.Glm4Attention(*args: Any, **kwargs: Any)

Bases: UnifiedAttention

Multi-head attention block configured for GLM-4.

class easydel.modules.glm4.modeling_glm4.Glm4DecoderLayer(*args: Any, **kwargs: Any)

Bases: Module

Single GLM-4 decoder block combining attention and MLP.

class easydel.modules.glm4.modeling_glm4.Glm4ForCausalLM(*args: Any, **kwargs: Any)

Bases: BaseCausalLMModule[Glm4Model, Glm4Config]

GLM4 model with a language modeling head for causal language modeling tasks.

class easydel.modules.glm4.modeling_glm4.Glm4ForSequenceClassification(*args: Any, **kwargs: Any)

Bases: BaseSequenceClassificationModule[Glm4Model, Glm4Config]

GLM4 model for sequence classification tasks.

class easydel.modules.glm4.modeling_glm4.Glm4MLP(*args: Any, **kwargs: Any)

Bases: Module

Feed-forward network used inside GLM-4 decoder layers.

class easydel.modules.glm4.modeling_glm4.Glm4Model(*args: Any, **kwargs: Any)

Bases: EasyDeLBaseModule

GLM4 model implementation.

get_decoder()

Return the decoder component of the model.

This method should be overridden by encoder-decoder models to return their decoder component. Useful for tasks that need access to the decoder separately from the encoder.

Returns

The decoder module.

Return type

nn.Module | EasyDeLBaseModule

Raises

NotImplementedError – If the model does not implement a decoder.

get_embedding()

Return the input embedding layer of the model.

This method should be overridden by models to return their token embedding layer. Useful for weight tying or accessing embeddings directly.

Returns

The embedding layer.

Return type

nn.Module | nn.Embed

Raises

NotImplementedError – If the model does not have an embedding layer.

get_encoder()

Return the encoder component of the model.

This method should be overridden by encoder-decoder models to return their encoder component. Useful for tasks that only need the encoder, such as feature extraction or embedding generation.

Returns

The encoder module.

Return type

nn.Module | EasyDeLBaseModule

Raises

NotImplementedError – If the model does not implement an encoder.

get_lm_head()

Return the language model head of the model.

This method should be overridden by language models to return their output projection layer that maps hidden states to vocabulary logits.

Returns

The language model head layer.

Return type

ParallelLinear

Raises

NotImplementedError – If the model does not have a language model head.