Overview of The Process
Understanding where you are in this process
Last updated
Understanding where you are in this process
Last updated
Training a DQN model for process control begins with the historical data. In practice, industrial facilities constantly monitor and store real-time process data in time series databases. These databases are the genesis for building the digital environments required to safely and consistently train neural networks and reinforced learning (RL) models. In this lesson, it is the goal to go through as generically, but as thoroughly as possible the steps required to build a DQN model for control using real-time historical time series data.
In a standard industrial environment there are several key personnel and processes that must be understood. The main reason for this section is that, during this lesson, we are going to bypass these essential processes and personnel engagements for the purpose of illustration.
Figure 1 above shows a typical process for how Ai-Ops is solving process control issues utilizing existing industrial real-time, time series data as the backbone of the Machine Learning and RL training process. This process is not a process of solitude but requires real engagement between the subject matter experts (SME(s)), and the Machine Learning Application engineer. These SMEs are usually, but not limited too the following:
Business Site Leadership and Economists
Process Engineers
Operations Managers
Front-Line and/or Board Operators
Controls Systems Engineers
Instrumentation and Electrical
Lab Technicians
Understanding the key players, their roles, the data they collect, their desired control, and monitoring outcomes is absolutely necessary in how Machine Learning can help or even if it can be applied to help. It is imperative to understand that not all the desires of an industrial client can be met with a Machine Learning solution.
In this lesson, all the data manufactured is considered to be useful and of good quality. In practice, and as shown in Figure 1, Step 1, there is much to consider when deciding which data should be included when developing digital environments. This is a key step in the process of becoming a Machine Learning Application engineer for industrial clients, and will be discussed in other lessons. However, for the purposes of this lesson, all data manufactured in our coded lesson is of good quality and is useful to the training scenarios.
A controls narrative is the documentation that expresses the objective or philosophy used in order to establish a desired outcome in a control program. It also provides the context, naming conventions, limits, and all the other parts and pieces used to achieve the controls objective.
Figure 2 shows a diagram of a standard controls scenario associated with a water storage tank. In the above figure it can be shown that there are three (3) flow controllers, FIC-1, 2, and 3, feeding into water storage tank T-1. FIC-1, 2, and 3 have the control function of supplying a set amount of flow to the water storage tank, T-1. Each controller controls its flow by modulating its respective control valve. The control valve will modulate based on how far above or below the actual flow reading is versus that of the flow set point desired by its associated flow controller.
Separately, the level controller, LIC-101 has the control function of controlling a set amount of water in tank T-1, by controlling how much water is drained from the tank. LIC-101 will control the level of T-1 by modulating its control valve by a set amount depending on how far below or above the actual level reading is above or below the desired set point (lets say its desired set point is 50%).
All four (4) controllers shown in the diagram are standard PID controllers with optimized tuning parameters. A PID Controller is a Single-In, Single-Out (SISO) control algorithm that is widely used by industrial control systems. Some key terms to understand:
PID : Proportional, Integral Derivative Controller, is a stand-alone control function that uses a mathematical algorithm to regulate a process variable at a desired set point by means of a modulated variable.
PV : Process Variable, or actual reading from instrumentation (i.e. flow and level meters)
MV : Modulated Variable, or the PID control signal wired to a field device (i.e. control valve)
As is normal with PID tuning parameters, the tuning can work well for one given set of process conditions, but perform very poorly when process conditions change caused overshoot and oscillations in the process. (i.e. more or less flow from one or more of the flow controllers is introduced causing the drain valve to get overwhelmed).
It is desired to build a more robust controls solution that will keep T-1's level at 50% regardless of how much or how fast the flows from FIC-1, 2, or 3 change. LIC-101 should anticipate any changes such that the tank level does not oscillate due to changes in incoming flow.
In this lesson, it has been proposed that by using a combination of neural networks and reinforcement learning, we can achieve an optimization that will meet the desired goal listed above. In this lesson, the models that will be built during the steps, will ultimately allow the flow controllers FIC-1, 2, and 3 as well as the level controller LIC-101 to function as independent variables that through a reinforced learning model will operate the drain valve associated with LIC-101. The end result will produce an Ai controller denoted as AI-101 in the figure below that can be used to provide the control signal and modulate the drain valve.
In four (4) steps, you will:
Generate a make-shift time series historian with real-time data in for the form of .csv files. This step combines Steps 1 and 2 from Figure 1 in the above section.
Build and validate a replicated 'digital twin' of the data captured, this is Step 3 from Figure 1.
Train a DQN model to control a certain aspect of this digital environment created, this is Step 4 from Figure 1.
Validate the Trained Agent for control quality and capability
