easydel.inference.tools.utils

easydel.inference.tools.utils#

Utility functions for tool call parsing in EasyDeL inference.

This module provides essential helper functions for parsing and processing tool calls from Large Language Model outputs. It includes utilities for:

  • Partial JSON parsing and validation

  • String manipulation for streaming token differences

  • Common prefix/suffix extraction for incremental parsing

  • Whitespace handling in structured outputs

These utilities are fundamental to the streaming tool call extraction process, enabling parsers to handle incomplete JSON/XML structures as they are being generated token-by-token.

Key Functions:

find_common_prefix: Find shared prefix between strings find_common_suffix: Find shared suffix between strings extract_intermediate_diff: Extract new content for streaming partial_json_loads: Parse potentially incomplete JSON is_complete_json: Validate JSON completeness

easydel.inference.tools.utils.consume_space(i: int, s: str) → int[source]#

Skip whitespace characters starting from index.

Parameters

  • i – Starting index

  • s – String to process

Returns

Index of first non-whitespace character, or len(s) if all whitespace

easydel.inference.tools.utils.extract_intermediate_diff(curr: str, old: str) → str[source]#

Extract the difference between two strings with common prefix/suffix.

Used for streaming partial JSON to extract only new tokens that should be sent to the client. Critical for proper streaming of tool arguments without duplicating or missing content.

Parameters

  • curr – New/current version of partially-parsed JSON (must be first)

  • old – Previous version from earlier generation (must be second)

Returns

The intermediate difference that should be streamed

Example

>>> extract_intermediate_diff('{"fruit": "apple"}', '{"fruit": "ap"}')
'ple'

Note

Argument order is important - new version must be first.

easydel.inference.tools.utils.find_all_indices(string: str, substring: str) → list[int][source]#

Find all starting indices of a substring in a string.

Useful for locating multiple tool call boundaries or markers in model output.

Parameters

  • string – String to search in

  • substring – Substring to find

Returns

List of starting indices where substring appears

easydel.inference.tools.utils.find_common_prefix(s1: str, s2: str) → str[source]#

Find common prefix shared between two strings.

Used for extracting information from JSON generated by partial_json_parser to ensure proper token streaming without premature closure of quotes, brackets, or braces. Order of arguments does not matter.

Parameters

  • s1 – First string to compare

  • s2 – Second string to compare

Returns

The common prefix substring, empty string if none

Example

>>> find_common_prefix('{"fruit": "ap"}', '{"fruit": "apple"}')
'{"fruit": "ap'
easydel.inference.tools.utils.find_common_suffix(s1: str, s2: str) → str[source]#

Find common suffix shared between two strings.

Stops when suffix ends or hits an alphanumeric character. Order of arguments does not matter.

Parameters

  • s1 – First string to compare

  • s2 – Second string to compare

Returns

The common suffix substring, empty string if none

Example

>>> find_common_suffix('{"fruit": "ap"}', '{"fruit": "apple"}')
'"}'
easydel.inference.tools.utils.is_complete_json(input_str: str) → bool[source]#

Check if a string is complete, valid JSON.

Parameters

input_str – String to validate

Returns

True if string is complete valid JSON, False otherwise

easydel.inference.tools.utils.partial_json_loads(input_str: str, flags: Allow) → tuple[Any, int][source]#

Load partial JSON with fallback to standard decoder.

Attempts to parse potentially incomplete JSON using partial_json_parser, falling back to standard JSON decoder for “Extra data” errors.

Parameters

  • input_str – JSON string to parse (may be incomplete)

  • flags – Parsing flags from partial_json_parser.Allow

Returns

Tuple of (parsed object, characters consumed)

Raises

JSONDecodeError – If JSON is malformed (not just incomplete)

Contents