easydel.inference.tools.abstract_tool

easydel.inference.tools.abstract_tool#

class easydel.inference.tools.abstract_tool.ToolParser(tokenizer: AutoTokenizer)[source]#

Bases: object

Abstract base class for tool call parsing from LLM outputs.

This class provides the foundation for extracting function/tool calls from language model outputs. Subclasses implement model-specific parsing logic for different tool call formats (JSON, XML, pythonic, etc.).

The parser maintains state for streaming responses and provides methods for both batch and streaming extraction of tool calls.

prev_tool_call_arr#

History of previously parsed tool calls

Type

list[dict]

current_tool_id#

ID counter for tool calls in current session

Type

int

current_tool_name_sent#

Flag indicating if current tool name was sent

Type

bool

streamed_args_for_tool#

Buffer for streaming tool arguments

Type

list[str]

model_tokenizer#

Tokenizer instance for the model

Type

AnyTokenizer

Note

This is an abstract class - use model-specific subclasses like HermesToolParser, MistralToolParser, etc.

adjust_request(request: ChatCompletionRequest) → ChatCompletionRequest[source]#

Adjust request parameters for model-specific requirements.

Override this method to modify request parameters like system prompts, tool definitions, or formatting before processing. Default implementation returns the request unchanged.

Parameters

request – Original chat completion request

Returns

Potentially modified request

Return type

ChatCompletionRequest

Example

Some models may need to reformat tool definitions or add specific system instructions for proper tool calling.

extract_tool_calls(model_output: str, request: ChatCompletionRequest) → ExtractedToolCallInformation[source]#

Extract tool calls from complete model output (batch mode).

Parses the entire model response to identify and extract tool/function calls. This method is used for non-streaming responses where the complete output is available.

Parameters

  • model_output – Complete text generated by the model

  • request – Original request containing tool definitions

Returns

Parsed tool calls and remaining content

Return type

ExtractedToolCallInformation

Raises

NotImplementedError – Must be implemented by subclasses

Note

This method is stateless - it doesn’t use instance state. Each parser implements model-specific extraction logic.

extract_tool_calls_streaming(previous_text: str, current_text: str, delta_text: str, previous_token_ids: Sequence[int], current_token_ids: Sequence[int], delta_token_ids: Sequence[int], request: ChatCompletionRequest) → easydel.inference.openai_api_modules.DeltaMessage | None[source]#

Extract tool calls from streaming model output.

Processes incremental model output to identify partial tool calls and emit appropriate streaming updates. Maintains state across calls to handle incomplete JSON/XML structures.

Parameters

  • previous_text – Text accumulated up to previous call

  • current_text – Text accumulated including current chunk

  • delta_text – New text in current chunk

  • previous_token_ids – Token IDs up to previous call

  • current_token_ids – Token IDs including current chunk

  • delta_token_ids – New token IDs in current chunk

  • request – Original request with tool definitions

Returns

Incremental tool call update, or None if no update

Return type

DeltaMessage

Raises

NotImplementedError – Must be implemented by subclasses

Note

This method is stateful - it uses instance variables to track parsing progress across streaming chunks.

property vocab: dict[str, int]#

Get the tokenizer vocabulary.

Returns

Mapping of tokens to their IDs

Return type

dict[str, int]

Note

Only PreTrainedTokenizerFast is guaranteed to have .vocab

class easydel.inference.tools.abstract_tool.ToolParserManager[source]#

Bases: object

Registry and manager for tool parser implementations.

This class provides a centralized registry for tool parsers, allowing dynamic registration and retrieval of parser implementations. It supports both decorator-based and direct registration patterns.

The manager enables: - Registration of custom parser implementations - Retrieval of parsers by name - Dynamic loading of parser plugins from external files - Validation that parsers inherit from ToolParser base class

tool_parsers#

Registry mapping parser names to classes

Type

dict[str, type]

Example

```python # Register using decorator @ToolParserManager.register_module(“my_parser”) class MyCustomParser(ToolParser):

…

# Retrieve parser parser_class = ToolParserManager.get_tool_parser(“my_parser”) parser = parser_class(tokenizer) ```

classmethod get_tool_parser(name: str) → type[source]#

Retrieve a registered tool parser by name.

Parameters

name – Name of the parser to retrieve

Returns

The parser class registered with the given name

Return type

type

Raises

KeyError – If no parser is registered with the given name

Example

`python HermesParser = ToolParserManager.get_tool_parser("hermes") parser_instance = HermesParser(tokenizer) `

classmethod import_tool_parser(plugin_path: str) → None[source]#

Import and register a tool parser from an external file.

Dynamically loads a Python module containing tool parser definitions. The module should contain parser classes decorated with @register_module or manually register them upon import.

Parameters

plugin_path – File path to the Python module containing parser(s)

Note

The parser class in the file should use the @register_module decorator or call register_module during module initialization.

Example

```python # In external_parser.py: @ToolParserManager.register_module(“external”) class ExternalParser(ToolParser):

…

# Load it: ToolParserManager.import_tool_parser(“/path/to/external_parser.py”) ```

classmethod register_module(name: str | list[str] | None = None, force: bool = True, module: type | None = None) → type | collections.abc.Callable[source]#

Register a tool parser module.

Can be used as a decorator or called directly. Supports registering a parser under multiple names.

Parameters

  • name – Name(s) to register the parser under (defaults to class name)

  • force – If True, overwrites existing; if False, raises error on conflict

  • module – Parser class to register (if None, returns decorator)

Returns

The registered module or a decorator function

Return type

type | Callable

Raises

  • TypeError – If arguments have incorrect types

  • KeyError – If name conflict and force=False

Examples

```python # As decorator @ToolParserManager.register_module(“custom”) class CustomParser(ToolParser):

…

# Direct registration ToolParserManager.register_module(

name=”alternate”, module=CustomParser

)

# Multiple names @ToolParserManager.register_module([“v1”, “version1”]) class V1Parser(ToolParser):

…

```

tool_parsers: dict[str, type] = {'deepseek_v3': <class 'easydel.inference.tools.parsers.deepseekv3_tool_parser.DeepSeekV3ToolParser'>, 'glm-4.5': <class 'easydel.inference.tools.parsers.glm4_moe_tool_parser.Glm4MoeModelToolParser'>, 'granite': <class 'easydel.inference.tools.parsers.granite_tool_parser.GraniteToolParser'>, 'granite-20b-fc': <class 'easydel.inference.tools.parsers.granite_20b_fc_tool_parser.Granite20bFCToolParser'>, 'hermes': <class 'easydel.inference.tools.parsers.hermes_tool_parser.HermesToolParser'>, 'hunyuan_a13b': <class 'easydel.inference.tools.parsers.hunyuan_a13b_tool_parser.HunyuanA13BToolParser'>, 'internlm': <class 'easydel.inference.tools.parsers.internlm2_tool_parser.Internlm2ToolParser'>, 'jamba': <class 'easydel.inference.tools.parsers.jamba_tool_parser.JambaToolParser'>, 'kimi_k2': <class 'easydel.inference.tools.parsers.kimi_k2_tool_parser.KimiK2ToolParser'>, 'llama3_json': <class 'easydel.inference.tools.parsers.llama_tool_parser.Llama3JsonToolParser'>, 'llama4_json': <class 'easydel.inference.tools.parsers.llama_tool_parser.Llama3JsonToolParser'>, 'llama4_pythonic': <class 'easydel.inference.tools.parsers.llama4_pythonic_tool_parser.Llama4PythonicToolParser'>, 'minimax': <class 'easydel.inference.tools.parsers.minimax_tool_parser.MinimaxToolParser'>, 'mistral': <class 'easydel.inference.tools.parsers.mistral_tool_parser.MistralToolParser'>, 'phi4_mini_json': <class 'easydel.inference.tools.parsers.phi4mini_tool_parser.Phi4MiniJsonToolParser'>, 'pythonic': <class 'easydel.inference.tools.parsers.pythonic_tool_parser.PythonicToolParser'>, 'qwen3_coder': <class 'easydel.inference.tools.parsers.qwen3coder_tool_parser.Qwen3CoderToolParser'>, 'seed_oss': <class 'easydel.inference.tools.parsers.seed_oss_tool_parser.SeedOssToolParser'>, 'step3': <class 'easydel.inference.tools.parsers.step3_tool_parser.Step3ToolParser'>}#
easydel.inference.tools.abstract_tool.import_from_path(module_name: str, file_path: str | os.PathLike)[source]#

Import a Python module from a file path.

Dynamically imports a Python file and registers it in sys.modules. Used for loading external tool parser plugins.

Parameters

  • module_name – Name to register the module under in sys.modules

  • file_path – Path to the Python file to import

Returns

The imported module object

Return type

module

Raises

ModuleNotFoundError – If the file cannot be loaded as a module

Note

Based on the official Python importlib recipe: https://docs.python.org/3/library/importlib.html

easydel.inference.tools.abstract_tool.is_list_of(value: object, typ, *, check: Literal['first', 'all'] = 'first') → bool[source]#

Check if a value is a list of specific type.

Parameters

  • value – Value to check

  • typ – Expected type of list elements

  • check – “first” to check only first element, “all” to check all elements

Returns

True if value is a list of the specified type

Return type

bool

Examples

`python is_list_of(["a", "b"], str)  # True is_list_of(["a", 1], str, check="all")  # False is_list_of([], str)  # True (empty list) is_list_of("not a list", str)  # False `

Contents