jobshoplab.state_machine.middleware package¶
Submodules¶
jobshoplab.state_machine.middleware.middleware module¶
Middleware components for connecting the state machine to reinforcement learning environments.
This module provides middleware classes that serve as an interface between the JobShopLab state machine and reinforcement learning environments, particularly those compatible with StableBaselines3. It translates RL actions to state machine transitions and converts state machine results into observations for the learning algorithms.
- class StableBaselines3ActionProtocol[source]¶
Bases:
ProtocolProtocol defining the expected interface for StableBaselines3 actions.
This protocol ensures that actions can be accessed via indexing, which is required for handling various action formats from StableBaselines3.
- class SubTimeStepper[source]¶
Bases:
objectTracks and manages operation counts during simulation time steps.
This class monitors the number of no-operations and actual actions performed during simulation time steps, and determines if the current step should be truncated based on predefined rules.
-
__init__(truncation_active=
True)[source]¶ Initialize a new SubTimeStepper instance.
- add_operation(action)[source]¶
Increment the appropriate counter based on action type.
This method categorizes actions as either no-operations or actual actions and increments the corresponding counter.
-
__init__(truncation_active=
- class Middleware[source]¶
Bases:
ABCAbstract base class for state machine middleware components.
Middleware components provide the interface between the state machine and reinforcement learning environments. They translate RL actions into state machine transitions and convert state machine results into observations suitable for RL algorithms.
This abstract class defines the required interface for all middleware implementations in the system.
- __init__(loglevel, config, instance, action_factory, observation_factory, state_machine_step=<function step>, *args, **kwargs)[source]¶
Initialize the middleware component.
- Parameters:¶
- loglevel : int | str¶
Logging level for middleware operations
- config : __SPHINX_IMMATERIAL_TYPE_VAR__V_Config¶
Global configuration settings
- instance : InstanceConfig¶
Problem instance configuration
- action_factory : ActionFactory¶
Factory for creating/interpreting actions
- observation_factory : ObservationFactory¶
Factory for creating observations from states
- state_machine_step : Callable
Function to execute state machine steps (defaults to step)
- *args
Additional positional arguments for extensions
- **kwargs
Additional keyword arguments for extensions
- class EventBasedBinaryActionMiddleware[source]¶
Bases:
MiddlewareEvent-based middleware for handling binary actions in a state machine environment.
This middleware processes binary actions and manages state transitions based on events. It handles both regular operations and no-operation (NO-OP) cases, with support for truncation through a joker system that limits consecutive no-op actions.
The middleware provides an interface between RL algorithms (particularly StableBaselines3) and the JobShopLab state machine, translating actions and observations between them.
- __init__(loglevel, config, instance, truncation_joker, truncation_active, action_factory, observation_factory, state_machine_step=<function step>, *args, **kwargs)[source]¶
Initialize the event-based binary action middleware.
- Parameters:¶
- loglevel : int | str¶
Logging level for middleware operations
- config : __SPHINX_IMMATERIAL_TYPE_VAR__V_Config¶
Global configuration settings
- instance : InstanceConfig¶
Problem instance configuration
- truncation_joker : int¶
Number of allowed truncations before episode termination
- truncation_active : bool¶
Whether to enable truncation
- action_factory : BinaryJobActionFactory¶
Factory for creating/interpreting binary actions
- observation_factory : ObservationFactory¶
Factory for creating observations from states
- state_machine_step : Callable
Function to execute state machine steps
- *args
Additional positional arguments for extensions
- **kwargs
Additional keyword arguments for extensions
- reset(init_state)[source]¶
Reset the middleware to an initial state.
This method initializes the state machine with the given initial state, performs a dummy action to set up the initial state machine result, and generates an initial observation. It also resets the truncation joker to its initial value.
- step(state, action)[source]¶
Execute one step in the state machine based on the given action.
This method processes an action through the state machine, updates the state accordingly, and generates a corresponding observation for the RL algorithm.
The method handles regular actions and no-operations differently: - Regular actions: Applied directly to the state machine - No-operations: Processed through special handling based on context
- Parameters:¶
- state : StateMachineResult¶
Current state of the state machine
- action : StableBaselines3ActionProtocol | int | float | ndarray¶
Action from the RL agent to execute
- Returns:¶
New state and corresponding observation
- Return type:¶
- Raises:¶
InvalidValue – If the provided state is not a valid StateMachineResult