Stability Prediction and Control of Anaerobic Digestion Process Based on Artificial Intelligence = 인공지능 기반 혐기성소화 공정의 안정성 예측 및 제어

다국어 초록 (Multilingual Abstract)

This study investigated the process stability of anaerobic digestion and enhanced intelligent control through applications of machine learning. One stability indicator characterized the dynamic balance of anaerobic biochemical reactions by introducing Recovery Potential (RP) and Deterioration Potential (DP). Another stability indicator diagnosed the state of anaerobic digestion through a comprehensive indicator derived using Principal Component Analysis (PCA). A deep learning model combining a Convolutional Neural Network (CNN) and Bidirectional Long Short-Term Memory (BdLSTM) utilized real-time sensor data to provide insights into system state and performance, ensuring economical and stable digester operation. Finally, intelligent control of anaerobic digestion was implemented using a Deep Q-Network (DQN) reinforcement learning model, integrating stability indicators with the CNN-BdLSTM network.
RP and DP were formulated to shed light on the kinetic balance between anaerobic biochemical reactions. RP is gauged by the ratio of the methanogenesis rate (MR) to the acidogenesis rate (AR), while the DP is the sum of the accumulation rate (AcR) and dilution rate (DR) of total VFAs, normalized using the AR. In an anaerobic digester for a mixture of pulverized food waste and liquified sewage sludge, an RP above 1.0 signifies a restorative state in the kinetic balance of anaerobic biochemical reactions across various operational phases, including startup and steady state, and shifts in organic loading rate. Conversely, a DP value of 0.0 or higher denotes a deterioration in the kinetic balance. The instability index (ISI), calculated as the DP to RP ratio, serves as an indicator of an anaerobic digestion state. When the standard deviation of ISI surpasses 0.2, it signifies instability in biochemical reactions; however, an average ISI below 0.05 indicates a stable digestion process. The study underscores the efficacy of RP, DP, and ISI as robust indicators for assessing the stability of anaerobic digestion based on the kinetics of biochemical reactions.
A comprehensive indicator based on PCA has been proposed for diagnosing the state of anaerobic digestion. Various state and performance variables were monitored under different operational modes, including start-up, interruption and resumption of substrate supply, and impulse organic loading rates. While these individual variables are useful for estimating the state of anaerobic digestion, they must be interpreted by experts. Coupled indicators combine these variables with the effect of offering more detailed insights, but they are limited in their universal applicability. Time-series eigenvalues reflected the anaerobic digestion process occurring in response to operational changes: Stable states were identified by eigenvalue peaks below 1.0, and they had an average below 0.2. Slightly perturbed states were identified by a consistent decrease in eigenvalue peaks from a value of below 4.0 or by observing isolated peaks below 3.0. Disturbed states were identified by repeated eigenvalue peaks over 3.0, and they had an average above 0.6. The long-term persistence of these peaks signals an increasing kinetic imbalance, which could lead to process failure. Ultimately, this study demonstrates that time-series eigenvalue analysis is an effective comprehensive indicator for identifying kinetic imbalances in anaerobic digestion.
The immediate response to the state disturbances of anaerobic digestion is essential to prevent anaerobic digestion failure. However, frequent monitoring of the state and performance of anaerobic digestion is challenging. Thus, deep learning models were investigated to predict the state and performance variables from online sensor data. The online sensor data, including pH, electric conductivity, and oxidation-reduction potential, were used as the input features to build deep learning models. The state and performance data measured offline were used as the labels. The model performance was compared for several deep learning models of CNN, LSTM, dense layer, and their combinations. The combined model of CNN and BdLSTM was robust and well-generalized in predicting the state and performance variables (R2=0.978, root mean square error=0.031). The combined model is an excellent soft sensor for monitoring the state and performance of anaerobic digestion from electrochemical sensors.
Reinforcement learning (RL) based on a deep Q-network (DQN) was studied to enable intelligent control of anaerobic digestion processes. Anaerobic digesters operated under statistically designed organic loading rate (OLR) conditions provided sensor data on process states and performance. Variable importance analysis identified key RL components—pH, EC, and ORP as states; OLR (flow rate and COD) as actions; and total reward combining stability and methane production. A deep learning-based environment model was trained to simulate process dynamics, predicting the next states and total reward based on the current states and actions. The architecture of the DQN with ε-greedy and prioritized experience replay was optimized by interacting with the environment model. Offline training effectively pre-trained model parameters, enhancing initial learning performance. The pre-trained DDQN was activated above a total reward threshold, stabilizing process instability and improving methane production under variable OLR conditions. The dueling DDQN (TDQN) showed slower pre-training but rapidly adapted to variability, stabilizing the process and significantly improving methane production. Both pre-trained DDQN and TDQN provide intelligent control frameworks for optimizing anaerobic digestion performance under variable OLR conditions.

목차 (Table of Contents)

• Chapter 1: Introduction 1
• 1.1 Background 1
• 1.2 Objective 3
• 1.3 Content 4
• Chapter 2: Literature Review 7
• 2.1 Anaerobic digestion 7
• 2.1.1 Fundamentals of anaerobic digestion 7
• 2.1.2 Stability of anaerobic digestion 10
• 2.2 Artificial intelligence 13
• 2.2.1 Machine learning basics 13
• 2.2.2 Deep learning basics 16
• 2.2.3 Reinforcement learning basics 17
• 2.3 Application of artificial intelligence in anaerobic digestion 20
• 2.3.1 Application of machine learning in anaerobic digestion 20
• 2.3.2 Application of deep learning in anaerobic digestion 21
• 2.3.3 Application of reinforcement learning in anaerobic digestion 22
• Chapter 3: Unraveling Anaerobic Digestion Instability: A Simple Index Based on the Kinetic Balance of Biochemical Reactions 24
• 3.1. Introduction 24
• 3.2. Materials and Methods 26
• 3.2.1. Substrate, anaerobic digester, and operation 26
• 3.2.2. Analysis 27
• 3.2.3. VFA balance in anaerobic digestion 28
• 3.3. Results 29
• 3.3.1. Kinetic response of anaerobic digestion to the fluctuations in OLR 29
• 3.3.2. Restoration and deterioration of the kinetic balance in the biochemical reactions 33
• 3.3.3. Instability index of anaerobic digestion 37
• 3.4. Discussion 40
• 3.5. Conclusions 44
• Chapter 4: A New Comprehensive Indicator for Monitoring Anaerobic Digestion: A Principal Component Analysis Approach 45
• 4.1. Introduction 45
• 4.2. Materials and methods 48
• 4.2.1 Setup and operation of an anaerobic digester 48
• 4.2.2 Single and coupled indicators for the AD state 49
• 4.2.3 Principal component analysis for comprehensive indicators 50
• 4.3. Results and discussion 50
• 4.3.1 Single indicators for the state of anaerobic digestion 50
• 4.3.2 Coupled indicators for the state of anaerobic digestion 54
• 4.3.3 Comprehensive indicators based on PCA 58
• 4.3.4 Implications 63
• 4.4. Conclusions 66
• Chapter 5: Exploration of Deep Learning Models for real-time monitoring of state and performance of anaerobic digestion with online sensors 68
• 5.1. Introduction 69
• 5.2. Materials and methods 72
• 5.2.1 Features and labels for deep learning 72
• 5.2.2 Relevance analysis of feature and label data 73
• 5.2.3 Deep learning models for prediction 73
• 5.3. Results and discussion 78
• 5.3.1 State and performance of the anaerobic digestion 78
• 5.3.2 Relevance of sensor data as the features 81
• 5.3.3 Performance of deep learning models 84
• 5.3.4 Generalizability of predictive deep learning models 87
• 5.3.5 Issues related to malfunction of sensors 89
• 5.3.6 Implications 91
• 5.4. Conclusions 91
• Chapter 6: A Reinforcement Learning Model Based on Deep Q-Learning Network for the Control of Anaerobic Digester 93
• 6.1. Introduction 93
• 6.2. Q-Learning-based algorithms: overview and limitations 95
• 6.2.1 Q-learning 95
• 6.2.2 DQN 96
• 6.2.3 Double DQN (DDQN) 96
• 6.2.4 Dueling network 97
• 6.3. Materials and methods 97
• 6.3.1. Experimental setup, operation, and data collection 97
• 6.3.2 Data preprocessing and model parameter selection 99
• 6.3.3 Environment model 99
• 6.3.4 Architectures and training of DQN-based RL models 100
• 6.3.5 Validation of DQN-based RL models for applicability 102
• 6.4. Results and discussion 103
• 6.4.1. Behavior of anaerobic digestion process under statistical conditions 103
• 6.4.2. States, actions, and rewards 105
• 6.4.3. Environment model 107
• 6.4.4. DQN-based RL model 108
• 6.4.5. Application of optimized DQN based models 112
• 6.4.6. Implication 115
• 6.5. Conclusions 118
• Chapter 7: Conclusion and Future Study 119
• 7.1. Conclusion 119
• 7.2. Future study and prospects 121
• References 123