Generic Multi-Agent Reinforcement Learning Approach for Flexible Job-Shop Scheduling by Schirin Bär

Author:Schirin Bär
Language: eng
Format: epub
ISBN: 9783658391799
Publisher: Springer Fachmedien Wiesbaden

Applied MARL Concept

We apply the presented concept to the introduced example FMS to conduct the evaluation in the following chapters based on this. Therefore, the state vector and the action space is specified, whereas the reward design is presented in the parameter study in section 5.​2. One component of the state vector is the locations of all agents in the FMS. We use a 12 x 3 x 3 matrix to replicate the locations of the agents, and to provide information regarding their queue position and their ID in the first, second, and third axis respectively. The size is dependent on the queue length of each place, and on the maximum number of agents in the FMS, which we both limited to three to start with a complexity that is challenging enough to test the concept. Lastly, we include the machine topology, which defines the order of all available and unavailable machines around the manufacturing plant as a 7 x 6 matrix with six columns representing the machine interface places where a machine can be connected to the FMS. We concatenate the one-hot encoded components and flatten the resulting matrix to shape the naive state input vector that we can feed to the agent at each decision-making point. Our final vector has a length of 166 bits with the determined number of three agents and unused agents padded by zeros. For the advanced state design, we add the encoded job specifications. We therefore defined the range of 1 to 9, as the processing times of the machines in our FMS vary from 10 seconds to 90 seconds. We assume that a maximum deviation of 5 seconds is still accurate enough when we round the processing time to the nearest tens and divide it by 10 to transfer it into an integer value in the defined range from 1 to 9. The value zero is used for a machine that cannot process the operation. The processing time is also one-hot encoded, as it is added to the other one-hot encoded components, requiring 10 input nodes for each value of . Although the advanced job specification encoding is designed to keep the agent’s state as small as possible, thereby avoiding redundant information, it still requires substantial space. Considering three agents in the advanced job specification encoding with a lookahead of two, six machines, and ten time units for each operation requires 360 additional input nodes for the neural network. A lookahead of three would require an additional 180 input nodes. Adding the 360 nodes to the already existing 166 input nodes, the state input results in a size of 526 bits. Considering the state input to select an action from the defined action set, the agent must be able to evaluate their selected action that is done using the reward design of the following section.

For the action space variant of actions, the agent has to select one out of 24 actions because this is the total number of transitions in the Petri net of figure 4.

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.