easydel.inference.esurge.scheduler.request_queue

easydel.inference.esurge.scheduler.request_queue#

Request queue implementations for scheduling.

Provides different queueing strategies for managing inference requests, including FCFS (first-come-first-served) and priority-based scheduling.

Classes:: RequestQueue: Abstract base class for request queues FCFSRequestQueue: First-come-first-served queue PriorityRequestQueue: Priority-based request queue SchedulingPolicy: Enum of scheduling policies

Example

>>> queue = FCFSRequestQueue()
>>> queue.add_request(request1)
>>> queue.add_request(request2)
>>> next_request = queue.pop_request()

>>> priority_queue = PriorityRequestQueue()
>>> priority_queue.add_request(high_priority_request)
>>> priority_queue.add_request(low_priority_request)
>>> next_request = priority_queue.pop_request()  # Gets high priority first

class easydel.inference.esurge.scheduler.request_queue.FCFSRequestQueue[source]#

Bases: deque[EngineRequest], RequestQueue

A first-come-first-served queue that supports deque operations.

add_request(request: EngineRequest) → None[source]#: Add a request to the queue according to FCFS policy.

peek_request() → EngineRequest[source]#: Peek at the next request in the queue without removing it.

pop_request() → EngineRequest[source]#: Pop a request from the queue according to FCFS policy.

prepend_request(request: EngineRequest) → None[source]#: Prepend a request to the front of the queue.

prepend_requests(requests: RequestQueue) → None[source]#: Prepend all requests from another queue to the front of this queue.

remove_request(request: EngineRequest) → None[source]#: Remove a specific request from the queue.

remove_requests(requests: Iterable[EngineRequest]) → None[source]#: Remove multiple specific requests from the queue.

class easydel.inference.esurge.scheduler.request_queue.PriorityRequestQueue[source]#

Bases: RequestQueue

A priority queue that supports heap operations.

Requests with a smaller value of priority are processed first. If multiple requests have the same priority, the one with the earlier arrival_time is processed first.

add_request(request: EngineRequest) → None[source]#: Add a request to the queue according to priority policy.

peek_request() → EngineRequest[source]#: Peek at the next request in the queue without removing it.

pop_request() → EngineRequest[source]#: Pop a request from the queue according to priority policy.

prepend_request(request: EngineRequest) → None[source]#

Add a request to the queue according to priority policy.

Note: In a priority queue, there is no concept of prepending to the front. Requests are ordered by (priority, arrival_time).

prepend_requests(requests: RequestQueue) → None[source]#

Add all requests from another queue according to priority policy.

Note: In a priority queue, there is no concept of prepending to the front. Requests are ordered by (priority, arrival_time).

remove_request(request: EngineRequest) → None[source]#: Remove a specific request from the queue.

remove_requests(requests: Iterable[EngineRequest]) → None[source]#: Remove multiple specific requests from the queue.

class easydel.inference.esurge.scheduler.request_queue.RequestQueue[source]#

Bases: ABC

Abstract base class for request queues.

Defines the interface for different request queueing strategies. Implementations must provide methods for adding, removing, and inspecting requests in the queue.

abstract add_request(request: EngineRequest) → None[source]#

Add a request to the queue according to the policy.

Parameters: request – The engine request to add.

abstract peek_request() → EngineRequest[source]#: Peek at the request at the front of the queue without removing it.

abstract pop_request() → EngineRequest[source]#: Pop a request from the queue according to the policy.

abstract prepend_request(request: EngineRequest) → None[source]#: Prepend a request to the front of the queue.

abstract prepend_requests(requests: RequestQueue) → None[source]#: Prepend all requests from another queue to the front of this queue.

abstract remove_request(request: EngineRequest) → None[source]#: Remove a specific request from the queue.

abstract remove_requests(requests: Iterable[EngineRequest]) → None[source]#: Remove multiple specific requests from the queue.

class easydel.inference.esurge.scheduler.request_queue.SchedulingPolicy(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]#

Bases: Enum

Enum for scheduling policies.

FCFS#: First-come-first-served scheduling.

PRIORITY#: Priority-based scheduling.

FCFS = 'fcfs'#

PRIORITY = 'priority'#

easydel.inference.esurge.scheduler.request_queue.create_request_queue(policy: SchedulingPolicy) → RequestQueue[source]#: Create request queue based on scheduling policy.

easydel.inference.esurge.scheduler.request_queue

Contents

easydel.inference.esurge.scheduler.request_queue#