easydel.trainers.prompt_utils

easydel.trainers.prompt_utils.apply_chat_template(example: dict[str, list[dict[str, str]]], tokenizer: Any, tools: Optional[list[Union[dict, Callable]]] = None) → dict[str, str]

Apply a chat template to a conversational example along with the schema for a list of functions in tools.

For more details, see [maybe_apply_chat_template].

easydel.trainers.prompt_utils.extract_prompt(example: dict[str, Sequence]) → dict[str, Sequence]

Extracts the shared prompt from a preference data example, where the prompt is implicit within both the chosen and rejected completions.

easydel.trainers.prompt_utils.is_conversational(example: dict[str, Any]) → bool

Check if the example is in a conversational format.

easydel.trainers.prompt_utils.maybe_apply_chat_template(example: dict[str, list[dict[str, str]]], tokenizer: Any, tools: Optional[list[Union[dict, Callable]]] = None) → dict[str, str]

If the example is in a conversational format, apply a chat template to it.

easydel.trainers.prompt_utils.maybe_extract_prompt(example: dict[str, list]) → dict[str, list]

Extracts the shared prompt from a preference data example, where the prompt is implicit within both the chosen and rejected completions.

easydel.trainers.prompt_utils.maybe_unpair_preference_dataset(dataset: DatasetType, num_proc: Optional[int] = None, desc: Optional[str] = None) → DatasetType

Unpair a preference dataset if it is paired.

easydel.trainers.prompt_utils.unpair_preference_dataset(dataset: DatasetType, num_proc: Optional[int] = None, desc: Optional[str] = None) → DatasetType

Unpair a preference dataset.