Graph convolutional networks for enhanced resolution 3D Electrical Capacitance Tomography image reconstruction
Graphical abstract
Introduction
Electrical Capacitance Tomography (ECT) is a non-intrusive low-cost cross-sectional imaging technique dedicated to the monitoring of industrial processes in pipelines, containers, and reactors, wherever nonconducting dielectric nature components are used. Typical examples of ECT usage may be shown as gas–fluid flows [1], solid particles flows [2] or various reservoirs [3], [4]. A typical ECT system measures mutual capacitance changes between pairs of electrodes distributed around the circumference of an industrial process container or pipe [5]. The amount of independent measurement data depends on used hardware (typically 28–496) [6] and its acquisition rate may vary from a few to thousands of images per second [7]. Thereafter, collected data can be processed by a high-performance PC system(s) using mathematical modeling [8] and specialized image reconstruction algorithms [9], [10], analyzing raw data [11] and finally making a proper diagnostic decision for efficient process control and automation [12], [13]. ECT tomography can be used in both: real-time [14] and offline mode [15]. In the past few years, there has been a great deal of interest in developing some extensions to classic cross-sectional ECT tomography. The most important one seems to be a 3D capacitance measurement concept [16], [17], [18], [19], [20] which has also evolved to 4D dynamic mode [21]. A new adaptive concept has also been introduced [22]. Spatial (volumetric) ECT aims to image dielectric industrial process components that differ in dielectric permittivity by measuring capacitances spatially from a set of copper electrodes arranged as a multi-plane sensor [16]. A big challenge for implementing 3D capacitance tomography in a real industrial environment is the volumetric image reconstruction aspect, including the forward modeling and the inverse modeling working with complex unstructured grids with hundreds of thousands of elements [23]. That is what makes the difference between 3D capacitance tomography and other 3D computed tomography methods where volumetric images are grid-free and built using tomographic scans. However, forward and inverse modeling is a significant computational hurdle to the existing ECT image reconstruction approaches in volumetric grids, making them very cost full in terms of time and memory resources. As a result, the best-known 3D ECT image reconstruction techniques have already reached their limits.
This paper proposes a new postprocessing technique for 3D ECT reconstructed image quality improvement. The method is loosely inspired by works on image superresolution deep neural networks [24], [25], [26]. Particularly, we hypothesize that it is possible to train a graph neural network to predict 3D ECT high-resolution images from the incomplete or limited number of capacitance measurements. We approach spatial improvement of ECT image reconstruction quality as membership prediction of consecutive nodes in the volumetric finite element grid concerning their electric permittivity distribution. Two memberships are considered, namely, object and non-object membership. First, we use a weighted and undirected graph to encode the ECT 3D finite element grid’s geometrical relationships. Graph nodes represent grid vertices and store the corresponding electric permittivity values. A graph convolutional network (GCN) is then trained in a graph-to-graph framework to predict high-resolution images from lower resolution reconstructions. Particularly, the 3D image reconstruction quality improvement is achieved by classifying each node in the input graph, either as belonging or not belonging to an object. By aggregating permittivities from neighboring nodes and exploiting its local correlation, the GCN model captures structural patterns of electric permittivity spatial distribution along the ECT grid. As a result, when trained with pairs of graphs representing volumetric ECT image reconstructions of a simulated phantom and a phantom itself, the GCN model learns how to classify input graph nodes as object or non-object, and thus finetune the resolution of inputs. Consequently, the proposed approach aims to go a step beyond the best, recently achievable, 3D ECT image reconstruction results of complex, time-consuming non-linear reconstruction methods, moving the reconstructed image closer towards the reference quality of the phantom. This work demonstrates the performance of the graph convolutional network when applied to finetune the results of the non-linear reconstruction algorithm [23], which is one of the leading state-of-the-art methods for 3D ECT image reconstruction. However, the proposed technique is universal and can be used to finetune images reconstructed by any finite element modeling-based relevant algorithm.
The contributions of this paper can thus be summarized as follows:
- •
graph convolutional networks are applied in the problem of ECT 3D image reconstruction quality improvement for the first time;
- •
a new approach to the considered problem is proposed aimed at predicting enhanced ECT 3D image reconstructions from the lower quality ones; this is a new concept that has not been considered so far;
- •
our GCN-based approach offers qualitative and quantitative enhancement of ECT 3D image resolution beyond level available only for time-consuming, complicated non-linear reconstruction algorithms.
The later part of this paper is organized as follows. First, in Section 2, we briefly review the related works on ECT image reconstruction and graph convolutional networks. Then follow the details of the proposed GCN-based approach to ECT 3D image reconstruction quality improvement given in Section 3. Visual results of the proposed method, together with the corresponding numerical assessment, are presented in Section 4 and discussed in Section 5. Finally, Section 6 concludes the paper.
Section snippets
ECT image reconstruction
3D ECT tomography reconstruction model generally uses dependencies between an electric field’s spatial distribution inside a capacitance sensor and the spatial electric permittivity distribution that exits in a sensor volume to visualize a content of a pipe. This idea is sketched in Fig. 1.
In the assumption of lack of spatial electric charge, the electric field behavior in a 3D ECT sensor can be described by Laplace equation (see Eq. (1)) [17], [18]. where is an
General idea
The main idea behind the introduced approach is to improve the 3D ECT image reconstruction quality by classifying nodes in a grid representing the domain of the reconstructed image and generated by the finite element method (FEM) either as an object or as air. This idea is schematized in Fig. 3.
The classification of nodes is performed concerning their electric permittivities, as well as spatial and geometrical relationships encoded in the 3D ECT FEM grid. Thus we train a graph convolutional
Input data
For this study, 1012 electric permittivity phantoms were generated, with 1000 phantoms used for training, and 12 phantoms used for testing. In the phantoms, object nodes were assigned the relative electric permittivity of 3, whereas the air nodes’ electric permittivity was set to 1. For each phantom the corresponding NLECTCM reconstruction (considering 496 measurements simulated using 32 electrodes) was also provided. The resulting FEM-based graphs representing the corresponding 3D ECT grids
Discussion
The results of the proposed GCN-based approach to enhance 3D ECT image reconstruction are promising. Visual assessment (see Fig. 7) clearly shows that regardless of the variant used (i.e., Hard or Soft), for the proposed approach, the reconstructed ECT images are of better quality than for the NLECTCM method. The advantage of the GCN model over the SVM-based approach is also apparent. The improved quality of ECT images finetuned by the GCN mainly manifests itself by a sharper transition between
Conclusions
This pioneering study on using graph convolutional networks in the 3D Electrical Capacitance Tomography (ECT) image reconstruction problem shows promising results. Notably, the proposed GCN-based framework proved successful in improving the nonlinear NLECTCM method’s reconstruction results; currently, one of the best and most accurate capacitance tomography image reconstruction approaches. The proposed approach used as a postprocessing step moved the NLECTCM’s reconstructed images closer
CRediT authorship contribution statement
Anna Fabijańska: Methodology, Software, Validation, Formal analysis, Investigation, Writing. Robert Banasiak: Conceptualization, Validation, Formal analysis, Investigation, Resources, Data curation, Writing, Visualization.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgment
This research was supported by the Smart Growth Operational Programme 2014–2020 project no POIR.04.01.02-00-0089/17-00, Poland. The project was conducted in the Institute of Applied Computer Science at the Lodz University of Technology.
References (57)
- et al.
Analysis of the bulk solid flow during gravitational silo emptying using x-ray and ect tomography
Powder Technol.
(2012) - et al.
A high-speed digital electrical capacitance tomography system combining digital recursive demodulation and parallel capacitance measurement
IEEE Sens. J.
(2017) - et al.
Study on two-phase flow regime visualization and identification using 3d electrical capacitance tomography and fuzzy-logic classification
Int. J. Multiph. Flow.
(2014) - et al.
Measurement of real-time flow structures in gas-liquid and gas-liquid-solid flow systems using electrical capacitance tomography (ect)
Chem. Eng. Sci.
(2001) - et al.
A deep learning method for image super-resolution based on geometric similarity
Signal Process., Image Commun.
(2019) Neural networks and stochastic search methods applied to industrial capacitive tomography
Control Eng. Pract. - Control Eng. Pract.
(1999)- et al.
An autoencoder based image reconstruction for electrical capacitance tomography
IEEE Sens. J.
(2018) - et al.
Deep learning-based inversion method for imaging problems in electrical capacitance tomography
IEEE Trans. Instrum. Meas.
(2018) - et al.
Ect image reconstruction algorithm based on multiscale dual-channel convolutional neural network
Complexity
(2020) - et al.
Graph kernel based measure for evaluating the influence of patents in a patent citation network
Expert Syst. Appl.
(2015)
Extended co-citation search: Graph-based document retrieval on a co-citation network containing citation context information
Inf. Process. Manage.
Predicting station-level hourly demand in a large-scale bike-sharing network: A graph convolutional neural network approach
Transp. Res. C
3d shape segmentation via shape fully convolutional networks
Comput. Graph.
Imaging Industrial Flows: Applications of Electrical Process Tomography
Application of electrical capacitance tomography for measurement of gas-solids flow characteristics in a pneumatic conveying system
Meas. Sci. Technol.
Analysis and interpretation of hopper flow behaviour using electrical capacitance tomography
Part. Part. Syst. Charact.
Non-invasive process imaging – principles and applications of industrial process tomography
Chem. Eng. Technol.
Architecture of evt4 data acquisition system for electrical capacitance tomography
Measurement
Nonlinear image reconstruction for electrical capacitance tomography using experimental data
Meas. Sci. Technol.
Image reconstruction algorithms for electrical capacitance tomography: State of the art
Qinghua Daxue Xuebao/J. Tsinghua Univ.
A review on image reconstruction algorithms for electrical capacitance/resistance tomography
Sens. Rev.
Use of raw capacitance tomography data for flow pattern control
Optimizing of baking processes by means of electrical capacitance tomography
Modelling, measurement and analysis of fluidised bed dryer using an ect sensor
Environ. Sci.
Spatial imaging with 3d capacitance measurements
Meas. Sci. Technol.
Three-dimensional electrical capacitance tomography imaging
Insight
Towards 3d-electrical capacitance tomography for interface detection
Johnson Matthey Technol. Rev.
Cited by (20)
Learnable bilevel optimization method for electrical capacitance tomography
2024, Signal ProcessingData and measurement mechanism integrated imaging method for electrical capacitance tomography
2024, Applied Soft ComputingTransfer learning-driven inversion method for the imaging problem in electrical capacitance tomography
2023, Expert Systems with ApplicationsPhysics-informed multi-fidelity learning-driven imaging method for electrical capacitance tomography
2022, Engineering Applications of Artificial IntelligenceCitation Excerpt :Machine learning has emerged as popular candidates for solving tomography problems or inverse problems. Lei et al. (2018) developed the deep extreme learning machine method, Zhu et al. (2020) proposed the deep neural network for reconstruction, Zheng and Peng (2020) combined the linear back projection method and a deep fully-connected neural network, Anna and Banasiak (2021) proposed the graph convolutional network, and Zhu et al. (2022) reported an imaging algorithm for complex-valued multi-frequency electrical capacitance tomography using deep learning technique. There are many variants of this type of approach, and many interesting algorithms are being proposed.
A hybrid deep learning model for ECT image reconstruction of cryogenic fluids
2022, Flow Measurement and InstrumentationCitation Excerpt :Zhu et al. [23] proposed a deep learning net and used a hybrid training strategy to optimize the image reconstructed by the traditional method. Fabijańska and Banasiak [24] used the results of the nonlinear reconstruction method of the 3D ECT sensor model (NLECTCM) as the initial value, and successfully improved the image quality and resolution based on the graph convolutional neural network (GCN). The expressive ability of the neural network is dramatically improved with the deepening of the network.
- 1
Both authors contributed equally to this work.