easydel.modules.roberta.modeling_roberta

class easydel.modules.roberta.modeling_roberta.RobertaAttention(*args: Any, **kwargs: Any)

Bases: Module

Full attention module combining self-attention and its output projection.

class easydel.modules.roberta.modeling_roberta.RobertaClassificationHead(*args: Any, **kwargs: Any)

Bases: Module

Classifier head used for sequence-level classification tasks.

class easydel.modules.roberta.modeling_roberta.RobertaEmbeddings(*args: Any, **kwargs: Any)

Bases: Module

Construct the embeddings from word, position, and token_type embeddings for RoBERTa.

class easydel.modules.roberta.modeling_roberta.RobertaEncoder(*args: Any, **kwargs: Any)

Bases: Module

Stack of RoBERTa encoder layers with optional gradient checkpointing.

class easydel.modules.roberta.modeling_roberta.RobertaForCausalLM(*args: Any, **kwargs: Any)

Bases: EasyDeLBaseModule

RoBERTa repurposed for causal language modeling with an LM head.

get_decoder()

Returns the decoder part of the model’s graph definition.

get_embedding()

Returns the embedding layer of the module.

get_encoder()

Returns the encoder part of the model’s graph definition. This model is adapted as a decoder, so it has no separate encoder.

get_lm_head()

Returns the language model head of the module.

class easydel.modules.roberta.modeling_roberta.RobertaForMultipleChoice(*args: Any, **kwargs: Any)

Bases: EasyDeLBaseModule

RoBERTa encoder adapted for multiple-choice tasks with per-option scoring.

get_decoder()

Returns the decoder part of the model’s graph definition. RoBERTa is an encoder-only model.

get_embedding()

Returns the embedding layer of the module.

get_encoder()

Returns the encoder part of the model’s graph definition.

get_lm_head()

Returns the language model head of the module. This model has a multiple choice classification head, not an LM Head.

class easydel.modules.roberta.modeling_roberta.RobertaForQuestionAnswering(*args: Any, **kwargs: Any)

Bases: EasyDeLBaseModule

RoBERTa encoder with start/end span heads for extractive QA.

get_decoder()

Returns the decoder part of the model’s graph definition. RoBERTa is an encoder-only model.

get_embedding()

Returns the embedding layer of the module.

get_encoder()

Returns the encoder part of the model’s graph definition.

get_lm_head()

Returns the language model head of the module. This model has a question answering head, not an LM Head.

class easydel.modules.roberta.modeling_roberta.RobertaForSequenceClassification(*args: Any, **kwargs: Any)

Bases: EasyDeLBaseModule

RoBERTa backbone with a classification head for sequence-level labels.

get_decoder()

Returns the decoder part of the model’s graph definition. RoBERTa is an encoder-only model.

get_embedding()

Returns the embedding layer of the module.

get_encoder()

Returns the encoder part of the model’s graph definition.

get_lm_head()

Returns the language model head of the module. This model has a sequence classification head, not an LM Head.

class easydel.modules.roberta.modeling_roberta.RobertaForTokenClassification(*args: Any, **kwargs: Any)

Bases: EasyDeLBaseModule

RoBERTa encoder with token classification head for per-token labels.

get_decoder()

Returns the decoder part of the model’s graph definition. RoBERTa is an encoder-only model.

get_embedding()

Returns the embedding layer of the module.

get_encoder()

Returns the encoder part of the model’s graph definition.

get_lm_head()

Returns the language model head of the module. This model has a token classification head, not an LM Head.

class easydel.modules.roberta.modeling_roberta.RobertaIntermediate(*args: Any, **kwargs: Any)

Bases: Module

First feed-forward layer of the RoBERTa transformer MLP.

class easydel.modules.roberta.modeling_roberta.RobertaLMHead(*args: Any, **kwargs: Any)

Bases: Module

Language modeling head for masked language modeling on top of RoBERTa.

class easydel.modules.roberta.modeling_roberta.RobertaLayer(*args: Any, **kwargs: Any)

Bases: Module

Single RoBERTa transformer encoder layer.

class easydel.modules.roberta.modeling_roberta.RobertaModel(*args: Any, **kwargs: Any)

Bases: EasyDeLBaseModule

RoBERTa encoder composed of embeddings, stacked layers, and pooling.

get_decoder()

Returns the decoder part of the model’s graph definition. RoBERTa is an encoder-only model.

get_embedding()

Returns the embedding layer of the module.

get_encoder()

Returns the encoder part of the model’s graph definition.

get_lm_head()

Returns the language model head of the module. Base Models don’t have a Language Model Head.

class easydel.modules.roberta.modeling_roberta.RobertaOutput(*args: Any, **kwargs: Any)

Bases: Module

Output feed-forward layer with dropout and residual connection.

class easydel.modules.roberta.modeling_roberta.RobertaPooler(*args: Any, **kwargs: Any)

Bases: Module

Pooling layer that projects the first token representation.

class easydel.modules.roberta.modeling_roberta.RobertaSelfAttention(*args: Any, **kwargs: Any)

Bases: AttentionModule

Multi-head self-attention used throughout RoBERTa layers.

class easydel.modules.roberta.modeling_roberta.RobertaSelfOutput(*args: Any, **kwargs: Any)

Bases: Module

Dense projection and dropout following RoBERTa self-attention.