Keras2c: A library for converting Keras neural networks to real-time compatible C

doi:10.1016/j.engappai.2021.104182

Engineering Applications of Artificial Intelligence

Volume 100, April 2021, 104182

https://doi.org/10.1016/j.engappai.2021.104182 Get rights and content

Abstract

With the growth of machine learning models and neural networks in measurement and control systems comes the need to deploy these models in a way that is compatible with existing systems. Existing options for deploying neural networks either introduce very high latency, require expensive and time consuming work to integrate into existing code bases, or only support a very limited subset of model types. We have therefore developed a new method called Keras2c, which is a simple library for converting Keras/TensorFlow neural network models into real-time compatible C code. It supports a wide range of Keras layers and model types including multidimensional convolutions, recurrent layers, multi-input/output models, and shared layers. Keras2c re-implements the core components of Keras/TensorFlow required for predictive forward passes through neural networks in pure C, relying only on standard library functions considered safe for real-time use. The core functionality consists of $\sim 1500$ lines of code, making it lightweight and easy to integrate into existing codebases. Keras2c has been successfully tested in experiments and is currently in use on the plasma control system at the DIII-D National Fusion Facility at General Atomics in San Diego.

Section snippets

Motivation

TensorFlow (Abadi et al., 2015) is one of the most popular libraries for developing and training neural networks. It contains a high level Python API called Keras (Chollet et al., 2015) that has gained popularity due to its ease of use and rich feature set. An example of using Keras to make a simple neural net is shown in Listing 1. As the use of machine learning and neural networks grows in the field of diagnostic and control systems (Hunt et al., 1992, Jin et al., 2018, Liu et al., 2017,

Method

Keras2c is based around the “layer” API of Keras, which treats each layer of a neural network as a function. This makes calculating the forward pass through the network a simple matter of calling the functions in the correct order with the correct inputs. The process of converting a model using Keras2c is shown in Fig. 1. The primary functionality can be broken into four primary components: weight and parameter extraction, graph parsing, a small C backend, and automatic testing.

Usage

An example of using Keras2c from within Python to convert a trained model is shown below in Listing 4. Here my_model is the Keras model to be converted (or a path to a saved model on disk in HDF5 format) and ‘‘my_converted_model’’ is the name that will be used for the generated C function and source files.

The command shown will generate three files:my_converted_model.c containing the main neural net function, my_converted_model.h containing the necessary declarations for including the neural

Benchmarks

Though the current C backend is not designed explicitly for speed, Keras2c has been benchmarked against Python Keras/TensorFlow for single CPU performance, and the generated code has been shown to be significantly faster for small to medium sized models while being competitive against other methods of implementing neural networks in C such as FANN and TensorFlow Lite. Results for several generic network types are shown in Fig. 2. They show that for fully connected, 1 dimensional convolutions,

Real-time applications

While not designed explicitly for speed, Keras2c was designed with real-time applications in mind, so efforts were made to ensure the generated code is deterministic and thread-safe, segmenting all non-deterministic behavior such as system calls to dedicated initialization and cleanup routines. The core of the neural net function is fully deterministic. All functions are re-entrant, and all mutable data is explicitly passed into and out of each function, allowing multiple calls to the neural

Discussion

As discussed in Section 4, the backend code as currently implemented is not optimized for execution speed, especially for large models. Convolutions are currently implemented using a direct calculation as opposed to the “im2col $+$ GEMM” approach more commonly used in deep learning libraries (Chetlur et al., 2014). Planned future work will include implementing this and other modifications to improve calculation speed and better support the larger model sizes becoming common in state of the art

Conclusion

Keras2c allows for the straightforward and simple conversion of Keras neural networks to pure C code, in a form that can be easily deployed to real-time control systems, or anywhere that a C executable can be run. By relying only on C standard library functions, it avoids any complicated dependencies that can make deploying any program a challenge. The generated code is designed to be human readable and editable, to allow for custom modifications and extensions if necessary. Despite not making

CRediT authorship contribution statement

Rory Conlin: Conceptualization, Methodology, Software, Validation, Writing - original draft, Visualization, Investigation. Keith Erickson: Methodology, Software, Validation, Writing - review & editing. Joseph Abbate: Conceptualization, Methodology, Validation, Investigation. Egemen Kolemen: Conceptualization, Supervision, Project administration.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

The authors thank Mitchell Clement, Ge Dong, and Mark D. Boyer for their help in beta testing and bug fixing. This material is based upon work supported by the U.S. Department of Energy, Office of Science, Office of Fusion Energy Sciences , using the DIII-D National Fusion Facility, a DOE Office of Science user facility, under Awards DE-FC02-04ER54698, DE-SC0015878, DE-AR0001166, and Field Work Proposal No. 1903

References (30)

AbiodunO.I. et al.
State-of-the-art in artificial neural network applications: A survey
Heliyon
(2018)
HuntK.J. et al.
Neural networks for control systems—a survey
Automatica
(1992)
JinL. et al.
Robot manipulator control using neural networks: A survey
Neurocomputing
(2018)
LiuW. et al.
A survey of deep neural network architectures and their applications
Neurocomputing
(2017)
PenaflorB. et al.
Extending the capabilities of the DIII-D plasma control system for worldwide fusion research collaborations
Fusion Eng. Des.
(2009)
AbadiM.
Tensorflow: Large-scale machine learning on heterogeneous systems
(2015)
AbbateJ. et al.
Fully data-driven profile prediction for DIII-D
Nucl. Fusion
(2021)
Amazon sagemaker
(2020)
BaiJ. et al.
ONNX: Open neural network exchange
(2019)
BehnelS. et al.
Cython: The best of both worlds
Comput. Sci. Eng.
(2011)

ChetlurS. et al.

CuDNN: Efficient primitives for deep learning

(2014)

CholletF.

Keras

(2015)

CurtinR.R. et al.

Mlpack 3: a fast, flexible machine learning library

J. Open Source Softw.

(2018)

FerronJ.R. et al.

Flexible software architecture for tokamak discharge control systems

FuY. et al.

Machine learning control for disruption and tearing mode avoidance

Phys. Plasmas

(2020)

Cited by (30)

Combining knowledge extension with convolution neural network for diabetes prediction
2023, Engineering Applications of Artificial Intelligence
Prediction and diagnosis of diabetes are critical issues in the field of smart healthcare. However, the dependence of large-scale annotated diabetes data and the lack of diabetes knowledge represent significant challenges for diabetes prediction. To address these challenges, we propose a new diabetes prediction model named KE-CNN, which combines knowledge extension and convolution neural network. The KE-CNN model first extracts abnormal indicator features from physical examination index data of diabetic patients and uses Word2vec to embed the feature words. We then employ entity recognition technique named BERT-BiLSTM-CRF to identify medical entities in the condition description text and utilize a knowledge graph to extend the knowledge of each medical entity, followed by using pre-trained Chinese word vectors to embed the extended description text. Finally, we construct a semantic enhanced convolutional neural network model with word embedding vectors and text embedding vectors as dual-channel input, aiming to enhance the feature expression of the KE-CNN model. Our model not only learns and captures more fine-grained features of diabetes information, but also significantly reduces the amount of data required for model training and improves the prediction performance of convolutional neural network models. Our experiments show that the KE-CNN model effectively improves the accuracy of diabetes prediction compared with the benchmark model.
Forecasting long-term world annual natural gas production by machine learning
2023, Resources Policy
Citation Excerpt :
Among the hyperparameters of the neural network model, the mean square error was chosen as the loss function. In addition to this, the number of epochs that controls the complete passes of the training set was chosen as 11 with 30 layers (Conlin et al., 2021). First, the prediction errors of each machine learning model were compared with one another using the Diebold-Mariano statistic (Table 1), which tests whether or not the forecasts differ by a statistically significant amount (Gao et al., 2019).
The goal of this study is to model the global annual natural gas production using a variety of machine learning models in order to predict future production and determine a peak production date. World gross domestic product (GDP) based on purchasing power parities (at PPPs), inflation percentage, Henry Hub Price, Euro Price, cumulative natural gas resources, and annually discovered new resources were taken as descriptor variables, and Shapley analysis was conducted to observe the importance of features on the dataset. It was revealed according to this analysis that, Henry Hub price, inflation percentage, and newly discovered resources had minor effects on natural gas production, so they were left out. Then, a variety of machine learning algorithms were employed and the one with the highest prediction ability was found to be the stochastic gradient descent (SGD) algorithm. Next, this model was tested under four different scenarios, each with different GDP and natural gas price projections. Finally, natural gas production was found to reach its peak sometime between 2034 and 2046. It was then concluded that rather than relying on a traditional approach based on the Hubbert Curve, a machine learning model that takes into account all relevant factors can be used to accurately forecast natural gas production and its peak time, allowing governments and policymakers to make the necessary preparations.
LOWESS smoothing and Random Forest based GRU model: A short-term photovoltaic power generation forecasting method
2022, Energy
Citation Excerpt :
We add RepeatVector layer and TimeDistributed layer to make the GRU layer more diversified. First, we add the RepeatVector layer to the GRU hierarchy to ensure the same vector in each time step, which specifically refers to increase the dimension of input data and add attribute dimension, then the model can be analyzed in all aspects from various dimensions [34]. The parameter is represented by n. For example, when n = 3, it means that the dimension of input data increases to 2 dimensions.
Accurate prediction of photovoltaic power generation is vital to guarantee smooth operation of power stations and ensure users’ electricity consumption. As a good forecasting tool, Gated Recurrent Unit method has been widely used in different forecasting areas. However, the existing studies ignore the impact of data fluctuations on prediction accuracy, to fill the gaps and enhance prediction accuracy, several different data smoothing techniques are introduced and compared to reduce fluctuations, Random Forest method is used for feature selection, and RepeatVector layer extended by attribute dimensions and TimeDistributed layer with full connectivity are utilized to optimize the Gated Recurrent Unit model. A real-world case from the photovoltaic power plant in Xuhui District, Shanghai, China, is adopted to evaluate the performance of proposed method. The comparing results with Recurrent Neural Networks and Long Short-Term Memory, and the actual data as well, show that the proposed prediction method can effectively improve the prediction accuracy of photovoltaic power generation. We also use the daily and monthly data of The Desert Knowledge Australia Solar Centre in Australia to investigate whether the proposed method is suitable for short-term or medium and long-term prediction. The results indicate that our method is more appropriate for short-term prediction.
A general infrastructure for data-driven control design and implementation in tokamaks
2023, Journal of Plasma Physics
Avoiding fusion plasma tearing instability with deep reinforcement learning
2024, Nature
Machine learning-based real-time kinetic profile reconstruction in DIII-D
2024, Nuclear Fusion

View all citing articles on Scopus

View full text

Keras2c: A library for converting Keras neural networks to real-time compatible C

Abstract

Section snippets

Motivation

Method

Usage

Benchmarks

Real-time applications

Discussion

Conclusion

CRediT authorship contribution statement

Declaration of Competing Interest

Acknowledgments

Heliyon

Automatica

Neurocomputing

Neurocomputing

Fusion Eng. Des.

Tensorflow: Large-scale machine learning on heterogeneous systems

Fully data-driven profile prediction for DIII-D

Nucl. Fusion

Amazon sagemaker

ONNX: Open neural network exchange

Cython: The best of both worlds

Comput. Sci. Eng.