Next Article in Journal
How to Construct Polar Codes for Ring-LWE-Based Public Key Encryption
Previous Article in Journal
Long-Term Impact of Wind Erosion on the Particle Size Distribution of Soils in the Eastern Part of the European Union
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Application of Structural Entropy and Spatial Filling Factor in Colonoscopy Image Classification

1
Department of Computer Science, Széchenyi István University, Egyetem tér 1, H-9026 Gyor, Hungary
2
Department of Telecommunications, Széchenyi István University, Egyetem tér 1, H-9026 Gyor, Hungary
3
Institute for Computer Science and Control (SZTAKI), Eötvös Loránd Research Network (ELKH), 13-17 Kende utca, H-1111 Budapest, Hungary
*
Author to whom correspondence should be addressed.
Entropy 2021, 23(8), 936; https://doi.org/10.3390/e23080936
Submission received: 26 April 2021 / Revised: 15 July 2021 / Accepted: 18 July 2021 / Published: 22 July 2021

Abstract

:
For finding colorectal polyps the standard method relies on the techniques and devices of colonoscopy and the medical expertise of the gastroenterologist. In case of images acquired through colonoscopes the automatic segmentation of the polyps from their environment (i.e., from the bowel wall) is an essential task within computer aided diagnosis system development. As the number of the publicly available polyp images in various databases is still rather limited, it is important to develop metaheuristic methods, such as fuzzy inference methods, along with the deep learning algorithms to improve and validate detection and classification techniques. In the present manuscript firstly a fuzzy rule set is generated and validated. The former process is based on a statistical approach and makes use of histograms of the antecedents. Secondly, a method for selecting relevant antecedent variables is presented. The selection is based on the comparision of the histograms computed from the measured values for the training set. Then the inclusion of the Rényi-entropy-based structural entropy and the spatial filling factor into the set of input variables is proposed and assessed. The beneficial effect of including the mentioned structural entropy of the entropies from the hue and saturation (H and S) colour channels resulted in 65% true positive and 60% true negative rate of the classification for an advantageously selected set of antecedents when working with HSV images.

1. Introduction

Colorectal polyps and cancer are the third most common disease in the world nowadays [1]. The colorectal polyps–and thus the cancer developing from them–grow on the surface of the colon/rectum. The literature distinguish several types of polyps. The five most common polyps types are the inflammatory, the hyperplastic, the adenomatous, the villous or tubulovillous adenoma, and the serrated adenoma. Structurally and shape-wise two different fundamental forms can be distinguished within these types, these forms are the pedunculated and sessile forms. The pedunculated polyps hang from a bowel wall into the internal space of the bowel. Sessile polyps grow directly on the bowel wall and are often quite flat.
A portion of these colorectal polyps progress into colorectal cancer over time: approximately 10% of all these lesions [1]. This indicates that the colorectal polyp screening is extremely important as colorectal cancer affects a large population, especially the population over age 50.
Medical image processing systems help doctors in preparing their image-based diagnoses. The medical image processing procedures and methods do not make a diagnosis themselves and it is clear that they will not do so in the near future. Their present goal is to assist the doctors by drawing their attention to image–and thus body–areas which are possibly problematic. These methods are referred to as computer-aided diagnosis (CAD) methods. As the colon cancer and the colorectal cancer are among the leading causes of death worldwide, many countries carry out screening for as wide a segment of the population aged 50 and over years as possible. Nowadays, however, in most countries, early-stage screening is not extensive enough, so it does not have a satisfactory impact in the prevention of the disease. The colorectal polyps can be detected with a special endoscope called colonoscope and also with a computer tomography device. CT scans do not always provide adequate results, futhermore the doctors cannot take samples or remove polyps during the examination procedure [2,3,4]. Both the CT and the colonoscopic procedures are inconvenient for the patients and require thorough preparations from the patients. In the course of colonoscopy examination the doctor inserts and leads a movable, flexible endoscope into the bowel through the rectum. This device not only has a lighting and recording device but also has small tweezers and loops with which samples may be taken or even lesions can be removed.
Many of the colonoscopes comprise additional light sources for narrow-band imaging (NBI), and a small container with which they can squish some bluish-coloured liquid usually (indigo carmine food colour) to a designated area: this bluish-coloured liquid concentrates in the groves, pits and valleys of the uneven bowel wall, and makes the surface structure more perceptible. This branch of the endoscopy is called chromoendoscopy [5].
Capsule endoscopy is another relatively new alternative to classical colonoscopy, its main drawbacks are that it cannot intervene, and it does not facilitate distance and size estimations [6,7].
In the polyp classification the first step is to determine the shape of the polyp [8,9]. As it has been mentioned above, polyps can be depressed into the bowel wall, i.e., flat, slightly protruding sessile polyps or pedunculated polyps with expressed stalk.
As a second step the investigation of the polyp’s surface is carried out. Based on its texture, the types that are likely to be of cancerous nature can usually be distinguished from the non-cancerous polyp surfaces. The pit pattern analysis was developed by Kudo and his coworkers for this purpose. They proposed a five-grade system based on the polyps surface texture on magnified colonoscopy images [10,11,12]. Those polyps surface textures that have almost regular, perfect circular pits of neither very dense and not too sparse spatial density and have regularly distributed pattern, they are the benign polyps. These polyps constitute the class A1. If the surface of the polyp has a pattern like a star, it is light pink, and has regular arrangement, then these polyps–similarly to the A1 polyps–do not tend to develop into malignancy. These polyps belong to the class A2. However, if the surface of the polyp has tubular shapes (class B3L) or extremely small round pits with high spatial density (class B3S), then those polyps may potentially develop into carcinoma. These usually show some characteristic discoloration, at especially in NBI [5]. The class B4 consist of polyps that are dark and have elongated pits, or have meander-like pits-trenches. These have already become cancerous. Those polyps that have irregularly textured surface of black or whitish colour, are in the advanced stadium of cancer, these constitute class B5.
We note here that the question, whether a polyp is malignant or it can turn into a malignant tumor, can only be answered unambiguously by biopsy, followed by a histological examination.
Bernal and his coworkers developed a polyp detection method based on the general shapes and colour of the polyps [13]. They have set up and organized an international scientific challenge, i.e., a scientific competition, Medical Image Computing and Computer-Assisted Intervention (MICCAI) for image segmentation and polyp detection [14]. They prepared three still image databases with different emphasis for this scholarly competition. These databases are called the CVC-Colon, Etis-Larib, and CVC-Clinic databases. In these a specific binary mask is associated with each image. The mask specifies the shape(s) and location(s) of the polyp(s) in the image. The mentioned masks are drawn by human medical experts, thus–due to the limited annotation time–they might be inaccurate. Further causes of the inaccuracy are the manual processing, and the possible vagueness of the polyp perimeter (i.e., due to occlusion or suboptimal lighting conditions). In the aforementioned challenge a wide range of algorithmic methods had been applied by the competing teams, from convolutional neural network coupled with fully automated learning to various hybrid polyp detection methods. There was also absolutely handcrafted method that was based on the idealized appearances of the polyps. As in medical image processing the main issue is the low availability and high cost associated, therefore we decided to use a dedicated method that relies on fuzzy inference.
Fuzzy sets are a conceptual extension of the Boolean sets (where any object can have either 0 or 1 membership values in the sets): In 1965 Zadeh introduced memberships that have real values between 0 and 1 [15,16]. Using fuzzy memberships instead of the classical “yes” and “no” values, the “if A 1 and A 2 and …, then B i ”-type rules can be turned into more flexible and more applicable rules and associated algorithms, e.g., for decision and control. The fuzzy rules, however have to be determined, as also the membership functions of each antecedent A j for each of the consequents B i , thus the use of a training set is also necessary for fuzzy decision. If the number of training pictures is not very large, then the rules can be adjusted, made more efficient by incorporating expert knowledge. These two aspects justify the application of fuzzy inference for the purpose.
Also a technique called fuzzy rule interpolation [17] can be applied to images, for those cases when some of the antecedent parameters have no membership at any of the rules, and therefore a classical Mamdani-Assilian [18] or other commonly used inference method will not give valid output.
Based on our research interest and experience in fuzzy classification, we took an fuzzy inference based approach for detecting and classifying colonorectal polyps and developed a method that classifies image segments according to the inferred presence of one or more polyp being shown in them [19]. The aim is not a perfect segmentation of a polyp within a colonoscopic image, but to draw attention of the medical personnel to a given image segment or to several image segments, where polyps might be visible. Firstly, the images are cut into smaller segments, and ech segments is labeled with a fuzzy decision value. Initially, we computed and used certain statistical parameters of the image segments and fitted polynomials to these [20]. However, polynomials introduced rather large sensitivity–in respect of the tile-size–into the method, so we dismissed this approach, but instead applied an edge-sensitive approach [21]. Later different preprocessing methods were incorporated into the approach and their influence were also studied [22].
After some experimentation it has become evident, that a general rulebase that is derived from the whole image-base may not be effective, as there are many different types of polyps furthermore, these polyps had been photographed from fairly different angles and viewpoints, not to mention; the problem associated with the different state of cleansing, in which the bowels appear in the pictures. For these reasons grouping process we made sure to include within each group only images ta, we grouped the images within the databases into image groups. During this grouping we made sure to include within each group only images taken from fairly similar angles of the polyps. These different image groups resulted in different classification rulebases, however the performance of other rulebases were also tested on each image group. Nevertheless, many of the heterologous rulebases showed similar efficiency as the homologous ones, therefore, these could be joined or utilized in the construction of a joint rulebase, or of a multilevel fuzzy classification scheme.
Analysing the histograms of the factual (measured and computed) data for each of the antecedents, it was realized that the fuzzy membership functions corresponding to “polyp” and “no polyp” outputs were very similar for certain antecedents, suggesting that by omiting them all, or at least some of these antecedents would be a viable option without much contraproductive effect on the result.
Bernal and his coworkers [23,24,25] carried out a studies, in which they tracked, where the human experts’ eyes were looking in the images during polyp searches, they found that the circular or similarly shaped shadows, i.e., intensity valleys and intensity gradients, that are characteristic to hemispheres, had been the foci that drew the experts’ attention, and therefore, such shadows are considered the key factors in the localization, identification, recognition, characterization and classification of polyps. Guided by this study we selected such statistical features of images that are closely linked to 3D shapes like hemispheres and spherical calottes [26]. In conjuction with the image quality perceived by the experts, it turned out to be beneficial to compensate for the poorly illuminated environment by transformation from RGB to HSV colour space and increasing the V values [27]. In the work presented herein, we also rely on the HSV colour channels furthermore, this colour space have become an aspect of the investigation.
In the following conciderations, the effects of the above factors are analyzed, namely, the advantages and disadvantages of the mentioned colour spaces, the selection of the input parameters, and reduction of their number. In Section 2, the antecedents are listed and motivated, with special attention to the Rényi entropy antecedents. Next, in Section 3 the mathematical background of the fuzzy inference and fuzzy rule interpolation is summarized for the Readers’ convenience. In Section 4 the results are presented; and finally, in Section 5, the conclusions are drawn and our plans for further research in the area are outlined.

2. Antecedents

Colonoscopy images used in the present research are from the three databases published during the MICCAI endoscopic vision challenge. This international scientific challenge was dicsussed above in the Introduction. These databases are the CVC-Colon, Etis-Larib, and CVC-Clinic databases. These consist of 380, 195, and 612 colorectal endoscopic images, respectively. The images are of different size and resolution, the image sizes are 574 × 500, 1225 × 966, and 384 × 288 pixels, respectively, while the resolutions are 72, 72, and 96 dpi, respectively. All three databases have a resolution of 24-bit in the RGB colour space with the primary colours sharing equally the bits. Only the first database uses some low-level compression, the other databases store and manage uncompressed images. For each image in the databases, a binary mask had been manually created to specify the position of polyps in the image. This makes the databases suitable for training learning algorithms, as well as other intelligent methods; in the particular case a fuzzy classification scheme.
Analysing characteristics of the areas where polyps were present, in the first step, we partitioned the images into square tiles. Our primary purpose was to develop a classification algorithm that can establish if the image segment contains polyp or not. This decision is reached by means of a fuzzy classification system. The antecedents were selected as described below, in Section 2.1.

2.1. Average, Standard Deviation, Gradients

In colonoscopy, the endoscopic camera takes snapshots in an originally dark, wet and mucous environment lit by a number of point-like light sources. For this reason, the pictures usually contain lots of reflections. Moreover, as the bowel wall is originally pink, the colour range of these pictures is rather narrow (except for the cases, when either the bowel is not cleansed sufficiently, or the lesion present in the image has turned malignant). The input parameters for the fuzzy classification system need to be selected for such an unfriendly spatial, biological, optical environment considering the mentioned colour composition.
In [23] the images were classified into three classes based on the angle of the view of the polyp: zenithal, lateral, and semilateral, see Figure 1. In the zenital case, the polyps can be seen in a top view, from this view they are not significantly lighter than the environment, but their circumference and the surface texture are usually well distinguishable. A polyp when viewed from its side (i.e., from a lateral view) with a much darker–distant and therefore, worse lit–bowel wall background. Then there is also an intermediate semilateral view. In this case, the background is somewhat darker than the polyp but often the contour, especially at the polyp’s base’ is hardly visible.
The polyps are usually protruding from the bowel wall, thus they are lighter in the images than their surroundings. Based on the above viewing-direction-dependant visual attributes of the polyps, the average pixel intensity, the standard deviation, the mean of the gradients’ absolute values, the standard deviation of gradients magnitudes were chosen as antecedents, as the gradients around the polyp-blobs can be quite large.

2.2. Edge Density

If the edges are strong and distinct (i.e., in most of the zenithal and lateral pictures and in some of the semilateral ones) edge images provide much information on the location of the polyp. When computing the edge density within a picture segment, firstly an edge detection–using an edge detection filter–is carried out.
For instance, a Canny filter making use of the Sobel matrices S x and S y can be emploiyed to select pixels where a rapid change in intensity occurs [28].
S x = 1 0 1 2 0 2 1 0 1   and   S y = 1 2 1 0 0 0 1 2 1
As a simple measurable quantity, the ratio of the edges within the image segment was calculated. If there are many edges in the picture segment (e.g., due to veins, polyp texture, bowel wall folding, polyp contours), this value will be high, for smoother image segments this value tends to be low.

2.3. The Rényi Entropy Based Structural Entropy and the Spatial Filling Factor

In image processing and computer vision, entropies are proven to be useful features [29]. Beside the usual von Neumann [30] and Shannon [31] entropies, there are many other generalizations and these are also in use in the mentioned disciplines [32,33]. There are a number of algorithmic approaches that apply some sort of enropies to images, but they all need a conversion from the pixel intensities to probability distributions, i.e., firstly, intensity histograms need to be computed for the images and then these histograms need to be normalized. The entropy is defined by the probabilities of the elements of a set,
H ( S ) = i = 1 N p i log p i ,
with H being the entropy of set S where the probability of the ith element’s occurance is p i and the total number of these elements is N, e.g., in the above example N = 256. In other approaches, certain neighbourhoods are looked at and their probability distributions are derived, or even differences between corresponding elements of the neighbourhoods are analysed in this manner. As pixel intensities relate to each other within neighbourhood, and the differences of the neighbouring pixels are rather easy to calculate, a possible approach to the “image-to-probability” task is to calculate the probability distribution of these differences [34]. This results in a one-dimensional (i.e., neighbour difference) distribution. It can be extended to 2D, if x and y directions are considered separately. Moreover, many other dimensions could be introduced if greater (i.e., not just direct) neighborhoods are to be considered.
Another, straightforward “image-to-probability distribution” conversion possibility is to normalize the complete image (or a part of it) so that the sum of the pixel intensities would be 1, thus they could be viewed as a 2D probability distribution [35]. Pipek and Varga [36] introduced two entropy related quantities, that can give a quantitative measure of the overall shape. One of the quantities, the logarithm of the spatial filling factor q,
log q = S 2 S 0
was already used in quantum physics. Here S 2 and S 0 are the 2nd and 0th generalized Rényi entropies, that can be calculated according to the following formula,
S n = 1 1 n log i = 1 N p i n ,
where N is the number of pixels. The probability p i is the normalized pixel intensity,
p i = I i / j = 1 N I j ,
with I j being jth pixel’s intensity value. For the limit n 1 the Formula (4) results in the classical Shannon entropy.
The other quantity is the difference of the total entropy S 1 and the entropy that is associated with the ratio of the higher probability points, which is the 2nd Rényi entropy. This quantity was named structural entropy, and can be calculated as
S s t r = S 1 S 2 .
The position of the probability distribution on the S s t r ( log q ) map gives information about the localization, shape of the distribution. Various distribution shapes, like Gaussian, spherical, sinusoidal or power law distributions have different trend lines on the S s t r ( log q ) map, see Figure 2, for instance, if a probability distribution is an exponential function, it will have an exponential trend line. For further details and the connection between the physical quantities and the entropies, and the trend lines associated with various distribution shapes, see [37]. In image processing, based on the shape of the trend lines associated with the
p ( x i , y j ) = A · e | B · x i + C · y j | α
Bonyár introduced the notion of the localisation factor α [38], which he used for characterising roughness of micro and nano surfaces [39]. The above Rényi entropy based quantities are used in detecting the shape and the pattern on the surface of the polyp [29]. We found significant difference between the responses of corrugated bowel wall and the semisphere-like polyps with or without shadow. For this reason, we introduced the S s t r and log q quantities as antecedents.

2.4. Wavelets

Wavelet transformation is effective tool widely used in signal and image processing, if different scale behaviour need to be identified or studied. A wavelet transform generates the smoothed versions of the studied signal as well as the fine details that are necessary to move from the smoothed signal to the original signal. The fine details are usually large at the positions where rapid changes are present in the image. If the signal is 2D then the wavelet transform should be performed in both dimensions. Our colonoscopy image segments are also 2D discrete signals within each of the three colour channels. As around the polyps there are rather rapid variations, we expect the fine scale components to be relatively large near and around the polyps. The above properties served as motivation for us to include wavelet transforms of the image segments in our study. The same parameters were calculated for the wavelet transforms as for the original images themselves. See antecedent indices 16–75 and 82–99 in Table 1.
A 1D discrete wavelet analysis is a set of transformations with the following properties. The original series x n , x = 0.1 , are expanded into series c n and d n with approximately half of the original length c n and d n . The former is computed by expanding the series according to a scaling function ϕ i , that is
c n / 2 = i = 1 N s 1 ϕ i x n i if   n = 2 k ,
where N s is the size of the sliding window and · denotes floor operator, i.e., the integer part of the value. Here it is visible, that the length of the series is (approximately) halved. Also, the odd indices n are omitted for simplicity from the present paper. This transformation gives sliding-windowed weighted average over a sliding window of lenth N s . It characterizes the larger scale behaviour of the discrete signal. Practically, this operation is implemented as a low-pass convolutional filter and then downsampling. The latter series, i.e., d n is the output of a high-pass convolutional filter, that is represents the rapidly varying components of the signal. Similarly to (8), it can be represented as expansion of series according to a wavelet ψ as
d n / 2 = i = 1 N s 1 ψ i x n i if   n = 2 k .
In the case of 2D signals, such as images, the wavelet transform is usually carried out separately in both dimensions, thus producing four output pictures, see Figure 3. The low-pass–low-pass output (LL) gives the average behaviour of the image. The second and third, correspond to the mixed filters, that is low-pass–high-pass (LH) and high-pass–low-pass (HL). The respective output images have near-zero pixel intensity in the slowly varying domains, and high intensity, where vertical, or horizontal changes occur in the original image parts are present. The fourth image is the output after the two high-pass filters (HH), it has non-zero pixels where the variations are large between the neighbouring pixels in both directions. These transformed images were analyzed similarly to the original ones.

2.5. Initial Set of Antecedents

We generated 99 input parameters including also the above listed measurable quantities. The antecedents (i.e., these 99 input parameters) of the fuzzy rule base are listed in Table 1.

2.6. Colour Spaces HSV and RGB

First we carried out research on RGB (red, green, blue) images from the mentioned image-bases, then we tested our approach on images converted to HSV colour space. The RGB colour space is used primarily in computer graphics, and seemed a natural choice. However, the results derived from RGB images were not encouraging. As according to the findings in [27] the H and the S channels of the HSV colour space are suitable for both suppressing and highlighting shadows and this advantageous feature is relied on by many image processing methods in various application areas, we decided to continue our experiments in this colour space.

3. Fuzzy Inference and Interpolation

3.1. Fuzzy Sets and Inference

As mentioned in the Introduction, the fuzzy sets which had been introduced in the seminal work by Zadeh [15], are set-like constructs, whose members (in case of ordinary sets: elements) can have not only full, membership represented by (binary) 1, but also partial membership values represented by real values between 0 and 1. Such a “fuzzy” approach is used by people in conjunction with everyday notions such as ‘tall’, ‘lean’, and ‘fat’. This is expressed with words like ‘very’, ‘somewhat’ and so on. For example a human can be considered tall with 1 membership value if they are taller than 2 m, but in the case of 1.7 m, they can be considered only with, say, 0.1 or 0.3 membership values. The membership function μ ( x ) associates membership values with all the possible values x of the set X being studied, like in our above example the possible human heights. The interval I α of the base set x X that arises if the membership function is thresholded at a given membership value α , i.e., the { x : μ ( x ) α } is called the α -cut.
Using fuzzy membership functions as rules for decision makes it possible to classify some measured data with respect to overlapping intervals and regions. Furthermore it makes the decision process more flexible. A statement like “if a man is tall and has slow reaction time then he can not be a successful Formula 1 pilot” could be a rule-of-thumb for the selection of the Formula 1 pilots. However, the separation between being tall and not being tall is far from being distinct. Similar sentences that are understandable for humans and common are not interpretable by computers, using only Boolean logic, such sentences obtain meaning vialy used in everyday talk fuzzy logic. Nonetheless, in order to construe a meaning to the above example, also the logical operators “and” and “or” need to be adapted to the fuzzy logic, i.e., need to be fuzzified. In the fuzzy logic literature, there are a number of different adaptations, definitions of these operators/operations. One of the most widespread is the Zadeh t-norm and s-norms, where the “and” operation between two membership values is their minimum, and the “or” operation is their maximum. See the relevant definitions in [15] and some applications of these in [40,41].
Using these operations complex decisions can be carried out in a straightforward manner. For the purpose of fuzzy inference, for example, Mamdani and Assilian suggested in [18] a fairly simple scheme: the antecedents are fuzzy-logically connected with an “and” while the consequents are connected in the mentioned sense by an “or”; see Figure 4.
In the computer-aided diagnosis application considered herein, the rules that are used in decision are represented by membership functions similar to the ones shown in Figure 4. In this work, a method relying on triangular fuzzy sets and their characteristic points (infima and suprema of the supports and the highest value points) was used. These values were computed from the minimum, maximum and mean of the empirical measured data from the training set [20], later also these triangles were fitted properly to the shape of the training set’s measured data’s histograms [42].
Herein we also apply such fitted membership functions. However, the histograms in some cases have very low numbers outside of a narrow interval, furthermore, in some cases very sparse fuzzy rules arise, with small or no overlap at all between the supports. Such an example is given in the 2nd column of Figure 4. In such cases, i.e., if the measured data falls into a domain where both of the rules have zero membership. Hence, the fuzzy inference can not be carried out. For such cases the fuzzy rule interpolation can help to come up with some meaningful membership values for the purpose of inference.

3.2. Fuzzy Rule Interpolation

As it was discussed above there are cases when conventional inference methods do not lead to valid results in the inference, thus fuzzy rule interpolation is necessary. One of the possible interpolation methods was developed by Kóczy and Hirota [17,43]. This method uses the natural concept, that if an observed value (even fuzzy value) is more similar to one of the antecedent rules, then its output will also be more similar the corresponding consequent. To implement this approach, it is necessary to measure the distance of the observed value from the fuzzy sets. In case of a simple triangular membership functions, such a distance can be calculated simply from the characteristic points of the membership functions, i.e., from the vertices of the corresponding triangle. In order to turn this vague idea into a viable practically usable (e.g., stabil) method ita modification was necessary [44]. According to this modified method the characteristic α -cuts ( α = 0 and α = 1 ) of the conclusion B * based on the antecedent sets A i , and consequent sets B j can be calculated as
inf { B α * } = i = 1 2 n 1 d α L ( A * , A i ) k inf { B i α * } i = 1 2 n 1 d α L ( A * , A i ) k ,
and
sup { B α * } = i = 1 2 n 1 d α U ( A * , A i ) k sup { B i α * } i = 1 2 n 1 d α U ( A * , A i ) k ,
if the measured data is A * , the number of the antecedents is k, and the Euclidean distance operator is denoted by d ( · , · ) . This type of rule interpolation was applied in the present study. Furthermore, also the original, triangular rules were used in a Mamdani-like inference system, as well as three types of membership functions with extended supports. This approach was necessary as the mentioned Kóczy-Hirota interpolation only fills the gaps around the narrow rules that belong to different consequents in a given antecedent dimension.

4. Experimental Setup

It has become an accepted development practice usual to test the classification methods to be used on colonoscopy images on public annoted databases and develop and invest in the method further only after these tests are succesful. In [14] many methods and their respective results are listed therefore it is convinient to compare the results arrived to via the methods proposed herein and use the databases that had been provided for the MICCAI colonoscopy challenge mentioned in the Introduction [13,14,24]. Some information has been already given in Section 2 on the image sizes used by the mentioned databases. In line with this image-base oriented approach all the three MICCAI image-bases have been used.
As a first step, each colonoscopy image was cut into roughly the same number of tiles. The tile sizes were chosen in the following manner for ETIS-Larib [24] it was 200 by 200 pixel, for the CVC Colon [23] 100 by 100, while for the CVC Clinic [13,14] 50 by 50. From every second tile a training set was built, and the other half of the tiles was used for testing the derived rulebases. The number of the non-polyp tiles vastly outnumbered those with one or more polyp. There were 13733 from the former, while only 4066 from the latter in the test set respectively. In the two reference cases, we used all the 99 antecedents from Table 1, i.e., for RGB and HSV colour spaces. Later, however, during the experimentation with various antecedent sets some of the antecedents were omitted.
As a second step, the fuzzy rulebases were generated from the data gained from the training set according to the method outlined at the end of Section 3.1. Thus three sets of rules were created, the first corresponds to the triangle’s core at the mean (Figure 5), the second at the median of the measured data, and the third rulebase was fitted to the maximum of the histogram. The suprema and infima of the triangles in this case were set to the first points where the historgram (that is normalized to 1 as its maximal column value) surpasses the 0.01 limit starting form both sides.
As a last step we evaluated the tiles in the test set, and compared their inferred polyp-content to that of the associated image masks which had been drawn by experts. Figure 6 shows the plan of the method that can help in finding polyps and our present work is still a part of the preparatory phase: the selection of the antecedents and building the fuzzy rules.

4.1. Using All the Initial Antecedents

If all the initially selected input variables were used in the inference, both for the mean centered triangles and for the median centered ones, the results turned out to be not very encouraging. In the case of mean and median centered types of rulebases the results were practically always “no polyp”. In case of the histogram fitted rulebase, both the true positive and the true negative rates (TPR and TNR) were just marginally better than the the corresponding rates associated with a random classification ( T P R = 0.9567 , T N R = 0.1987 , M C C = 0.18 , P P V = 0.26 , N P V = 0.94 , κ = 0.08 ). The results in case of HSV were either almost all positive, almost all negative. Furthermore, the results depend on whether interpolation or classical inference was used with extended supports.

4.2. Using Only the Rényi Entropy-Based Antecedents in the Rulebase

As the structural entropy and the spatial filling factor seemed to be promising in this type of classification [29], therefore we decided to check, whether using only these variables from the image tiles and their four wavelet transformed versions can lead to more probable decisions. When this approach was followed then the histogram fitted results improved significantly. For the HSV images, the TP and TN rates achieved 55.80% (95% CI [0.5572, 0.5587]) and 65.79% (95% CI [0.6575, 0.6583]), respectively, which means, that these entropies enable a better classification than all the previously selected variables together, even though they are still low, the improvement is visible. The other performance metrics for this case were M C C = 0.1857 (95% CI [0.1850, 0.1864]), P P V = 0.3257 (95% CI [0.3253, 0.3261]), N P V = 0.8341 (95% CI [0.8338, 0.8343]), κ = 0.1726 (95% CI [0.1719, 0.1732]). The 95% confidence intervals (95% CI) were calculated according to formula
M z * · σ n , M + z * · σ n
where M is the mean, σ the standard deviation of the measured values, n is the number of samples and z * is 1.96 for the confidence level of 95%. The generation of the population was carried out by randomly selecting 80% of the possible test image segments, and the population element number was n = 100 .
As it can be seen from Figure A1, Figure A2, Figure A3, Figure A4, Figure A5, Figure A6, Figure A7 and Figure A8 in Appendix A, the histograms of structural entropies and the spatial filling factors calculated from the HSV channels i.e., the 1st and 3rd columns of antecedent figures corresponding to the mentioned quantities. According to Table 1, the antecedents concerned were those marked with 10 to 15, 25 to 30, 40 to 45, 55 to 60, and 70 to 75). For the tiles–both–with and without polyps are centered at one side of the value set very near to each other, but the width of the supports for the two membership functions that belong to the two consequents differ significantly.
It seemed to be a reasonable approach to select those antecedents, where the histograms differ as much as possible, and the above finding about the structural entropy and spatial filling factor corroborates this approach. After some consideration we extended this set of antecedents somewhat. This extended set is detailed in the next subsection.

4.3. Antecedents Resulting in Considerably Differing Histograms for Images with and without Polyps

In order to select those antecedents where the histograms, and thus the resulting rules differ the most, the total absolute distance between the three characteristic points of the respective approximated histograms (i.e., the distances between the vertices of the purple and the cyan triangles in Figure A1, Figure A2, Figure A3, Figure A4, Figure A5, Figure A6, Figure A7, Figure A8, Figure A9 and Figure A10) were calculated. The input variables where this total distance was larger than 0.2 are as follows (in decreasing total distance order), see Table 2.
For these antecedents, the best result for the RGB colour space was achieved by interpolation, but it still was very poor, namely only 35% TPR and 75% TNR. The other performance metrics were M C C = 0.1 , P P V = 0.29 , N P V = 0.795 , κ = 0.35 . For the HSV colours the results were even worse, almost all tiles in the test set were listed as negative. Therefore, we investigated the reasons for this situation and found that this set contains all the very narrow triangle rules (like antecedents 31, 48, 61, 63, 84, 94), and after some closer investigation, we found that excluding these antecedents considerably improves the results (40% TPR and 75% TNR), not only for the classical triangular Mamdani-type classification, but even for interpolation and rules with extended support. We tested this rather counter intuitive result for many original sets, and excluding these narrow, and further placed triangle rules always increased the results.
The antecedents with total distance larger than 0.35 are presented in Table 3. This set includes only 13 antecedents from the 99, however, in this case almost all the classification results become negative (in medical sense), omitting the mentioned narrow triangle rules it is still 67.16% TPR (95% CI [0.6708, 0.6725]) and 48.52% TNR (95% CI [0.4848, 0.4857]). The other performance metrics were M C C = 0.1324 (95% CI [0.1316, 0.1332]), P P V = 0.2787 (95% CI [0.2784, 0.2790]), N P V = 0.8327 (95% CI [0.8327, 0.8335]), κ = 0.1049 (95% CI [0.1042, 0.1055]). As this approach did not seem very successful, we decided to use the difference of the centers of the histograms as selection rule.

4.4. Antecedents with Histograms Resulting in Considerable Distance between the Histogram Centres for Images with and without Polyps

The antecedents, where the centers of the histograms for the two consequents’ training set were fairly distant from each other, were those identified by the following numbers, see Table 4. According to Table 1 these numbers correspond to the input variables listed in Table 5.
For all the input parameters listed in Table 5 the resulting TPR was 64%, and the TNR 60%. Without the gradients (i.e. excluding last row denoted by * in Table 5) 65% and 60% was the TPR and TNR, respectively. The Matthews correlation coefficient was with gradient 0.1999 (95% CI [0.1991, 0.2007]), without gradient it improved slightly to M C C = 0.2119 (95% CI [0.2112, 0.2125]). The predictive rates behaved very similarly; for the case with the gradients P P V = 0.3196 (95% CI [0.3192, 0.3200]), N P V = 0.8488 (95% CI [0.8485, 0.8492]), while for the case without the gradients they are only slightly better, P P V = 0.3247 (95% CI [0.3244, 0.3250]), N P V = 0.8538 (95% CI [0.8535, 0.8540]). The Cohen- κ increase also a little, from κ = 0.1751 (95% CI [0.1744, 0.1758]), to κ = 0.1852 (95% CI [0.1847, 0.1858]).
From this point the results did not improve any further, including all the previously omitted entropies affect the results by less than a percent. The resulting indices are not too high, so our research results are not applicable for real diagnosis. It is clear, that a further search for efficient antecedents is needed, as well as a systematic optimization of the antecedents should be included [45], or perhaps some bacterial or other nature inspired algorithms, it is worth to start in this direction, and whether the Rényi entropy based quantities are applicable for this purpose.
In [13] recently finished colonoscopy challenges the performance of the methods were quite varied, the TPR was between 71.4% and 16.7%, the TNR was between 98.6% and 26.6% and the PPV was between 93.5% and 13.6%. In the more up-to-date review [46] the performances improved to (ETIS-Larib dataset) the TPR was between 54.2% and 23.43%, the PPV was between 70.23% and 25.83%, and (CVC-ColonDB dataset) the TPR was between 60.46% and 12.93%, the PPV was between 76.06% and 37.46%.

5. Conclusions

A fuzzy inference method–making use of fuzzy interpolation and histogram estimation–for colorectal polyp detection was proposed in this paper. Among the inputs were the Rényi entropy based structural entropies along with other statistical parameters characterizing the image and its wavelet transforms. The method partitions each image into a number of tiles and suggests whether a given tile contains polyp, or not. In earlier works [19,22] a simple triangular rulebase using the minimum, maximum, and mean or median of the training sets’s empirical data was created, which turned out to be not very effective. Later a histogram fitted triangular rulebase provided much better results, so these types of antecedents were employed herein. Our study focused on the selection of the relevant antecedents.
Intuitively, those antecedents seem to be essential for an effective inference system, where the rules derived from the images with and without a polyp respectively, differ as much as possible. However, we found, that even with rule interpolation, the too narrow supported antecedents have disadvantageous influence on the classification results. The antecedents with larger distance between the α cuts in the membership function corresponding to the positive and negative output rules, but still overlapping supports are the most effective.
Using the mean, standard deviation, edge density and Rényi entropy based structural entropy and spatial filling factor, and the gradients were selected as possible antecedents, however, most of these antecedents, like the means and the gradients had no or had negative influence on the classification results. Edge densities were important only in the case of the mixed wavelet transform outputs.
The Rényi entropy based structural entropy from the H and S colour channels of the original and low-pass wavelet transformed image segment, as well as of the (S) of the high-pass wavelet transform was proven beneficial for the classification, but the others could not improve the 65% TPR and 60% TNR of the classification.The influence of changing the colour channel from RGB to HSV was proven to be positive.
In the future we plan to change the method of prepairing tiles, we would like to introduce overlaps between the tiles as too small polyp percentage is not beneficial for the detection efficiency. Also, The results presented in [29] showed that the sliding windows can uncover such properties that can be used for distinguishing between bowel wall and polyp-like structures.

Author Contributions

Conceptualization, B.S., S.N. and Z.F.; methodology, B.S.; software, B.S.; writing—original draft preparation, B.S. and S.N.; writing—review and editing, Z.F.; visualization, B.S., S.N. and Z.F.; supervision, Z.F.; project administration, B.S.; funding acquisition, B.S. All authors have read and agreed to the published version of the manuscript.

Funding

Supported by the ÚNKP-20-4-II-SZE-69 New National Excellence Program of the Ministry for Innovation and Technology from the source of the National Research, Development and Innovation Fund.

Data Availability Statement

We used anonymized images from databases that are public and openly available at http://www.cvc.uab.es/CVC-Colon/index.php/databases/ (accessed on 26 April 2021), and https://polyp.grand-challenge.org/EtisLarib/ (accessed on 26 April 2021).

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A. Figures of Histograms and Rules

Table A1. The rule membership function parameters for the first 33 HSV antecedents. The following notation was used. The first column, i means the antecedent number, x i 0 m i n and x i 0 m a x are the infimum and supremum of the support of the rule corresponding to the non-polyp case, similaly, x i 1 m i n and x i 1 m a x are the infimum and supremum of the support of the triangular membership function of the polyp containing tiles. The columns x i 0 c o r e and x i 1 c o r e give the center points, i.e., the α = 1 α -cuts.
Table A1. The rule membership function parameters for the first 33 HSV antecedents. The following notation was used. The first column, i means the antecedent number, x i 0 m i n and x i 0 m a x are the infimum and supremum of the support of the rule corresponding to the non-polyp case, similaly, x i 1 m i n and x i 1 m a x are the infimum and supremum of the support of the triangular membership function of the polyp containing tiles. The columns x i 0 c o r e and x i 1 c o r e give the center points, i.e., the α = 1 α -cuts.
i x i 0 min x i 0 core x i 0 max x i 1 min x i 1 core x i 1 max
100.07070.14140.03030.06060.1212
2000.2323000.0404
300.55560.96970.09090.55560.8485
400.03030.343400.04040.3939
50.01010.07070.92930.02020.62631.0000
600.02020.242400.05050.3030
700.19190.757600.57580.8283
800.58590.82830.15150.64650.8889
900.06060.76770.03030.36360.7273
10000.0101000.0101
110.98991.00001.00000.98991.00001.0000
12000.070700.01010.0909
130.92931.00001.00000.90910.98991.0000
14000.090900.01010.1111
150.90911.00001.00000.89900.98991.0000
1600.07070.14140.03030.06060.1212
17000.2424000.0808
1800.55560.96970.09090.55560.8485
1900.03030.292900.04040.3939
200.01010.07070.92930.02020.62631.0000
2100.02020.222200.03030.2929
2200.27270.767700.28280.8485
2300.13130.85860.13130.49490.8182
2400.13130.59600.12120.28280.7374
25000.0101000
260.98991.00001.00000.98991.00001.0000
2700.01010.090900.01010.1010
280.88890.98991.00000.89900.98991.0000
29000.141400.01010.2020
300.85860.98991.00000.84850.98991.0000
310.26260.27270.28280.39390.40400.4242
32000.0202000.0101
330.43430.48480.56570.36360.42420.5051
Table A2. The rule membership function parameters for the second 33 HSV antecedents. The following notation was used. The first column, i means the antecedent number, x i 0 m i n and x i 0 m a x are the infimum and supremum of the support of the rule corresponding to the non-polyp case, similarly, x i 1 m i n and x i 1 m a x are the infimum and supremum of the support of the triangular membership function of the polyp containing tiles. The columns x i 0 c o r e and x i 1 c o r e give the center points, i.e., the α = 1 α -cuts.
Table A2. The rule membership function parameters for the second 33 HSV antecedents. The following notation was used. The first column, i means the antecedent number, x i 0 m i n and x i 0 m a x are the infimum and supremum of the support of the rule corresponding to the non-polyp case, similarly, x i 1 m i n and x i 1 m a x are the infimum and supremum of the support of the triangular membership function of the polyp containing tiles. The columns x i 0 c o r e and x i 1 c o r e give the center points, i.e., the α = 1 α -cuts.
i x i 0 min x i 0 core x i 0 max x i 1 min x i 1 core x i 1 max
3400.01010.383800.04040.3333
350.60610.70710.74750.47470.62630.7374
3600.02020.131300.01010.2626
3700.24240.767700.25250.8687
38000.94950.07070.56570.9596
39000.91920.12120.62630.9293
40000.0202000.0101
410.97981.00001.00000.98991.00001.0000
42000.0606000.0606
430.93941.00001.00000.92931.00001.0000
44000.030300.01010.1414
450.96971.00001.00000.88891.00001.0000
460.51520.53540.54550.42420.43430.4545
47000.0101000.0101
480.22220.26260.30300.41410.47470.5354
4900.01010.242400.02020.2626
500.41410.53540.59600.37370.50510.6061
5100.01010.121200.01010.2323
5200.24240.808100.38380.9091
53000.94950.17170.64650.9495
5400.23230.92930.22220.56570.9293
55000.0202000.0101
560.97981.00001.00000.98991.00001.0000
57000.0707000.0505
580.91921.00001.00000.93941.00001.0000
59000.0202000.0404
600.97981.00001.00000.95961.00001.0000
610.48480.49490.49490.34340.34340.3535
62000.0101000.0101
630.44440.45450.47470.43430.45450.4848
6400.01010.141400.01010.1313
650.60610.62630.64650.66670.69700.7273
6600.01010.101000.01010.1515
Table A3. The rule membership function parameters for the thirs 33 HSV antecedents. The following notation was used. The first column, i means the antecedent number, x i 0 m i n and x i 0 m a x are the infimum and supremum of the support of the rule corresponding to the non-polyp case, similarly, x i 1 m i n and x i 1 m a x are the infimum and supremum of the support of the triangular membership function of the polyp containing tiles. The columns x i 0 c o r e and x i 1 c o r e give the center points, i.e., the α = 1 α -cuts.
Table A3. The rule membership function parameters for the thirs 33 HSV antecedents. The following notation was used. The first column, i means the antecedent number, x i 0 m i n and x i 0 m a x are the infimum and supremum of the support of the rule corresponding to the non-polyp case, similarly, x i 1 m i n and x i 1 m a x are the infimum and supremum of the support of the triangular membership function of the polyp containing tiles. The columns x i 0 c o r e and x i 1 c o r e give the center points, i.e., the α = 1 α -cuts.
i x i 0 min x i 0 core x i 0 max x i 1 min x i 1 core x i 1 max
67000.7475000.8283
68000.94950.10100.51520.9596
69000.95960.11110.62630.9596
70000.0202000.0202
710.97981.00001.00000.97981.00001.0000
72000.0404000.0303
730.95961.00001.00000.96971.00001.0000
74000.0101000.0404
750.98991.00001.00000.95961.00001.0000
76000.1313000.0707
77000.0202000.0101
7800.17170.42420.05050.20200.5051
7900.01010.353500.04040.5556
8000.06060.23230.05050.12120.5051
8100.01010.151500.01010.3636
820.61620.79800.81820.46460.69700.7980
8300.17170.55560.06060.11110.4545
840.56570.64650.69700.35350.50510.6667
8500.16160.58590.05050.11110.4141
860.13130.70710.83840.13130.72730.8788
8700.17170.59600.05050.14140.5152
880.47470.49490.52530.53540.55560.5758
89000.1919000.0101
900.41410.46460.51520.35350.43430.5051
9100.02020.353500.03030.3333
920.42420.49490.61620.41410.52530.6667
9300.02020.181800.01010.3030
940.52530.54550.60610.71720.74750.7778
95000.0202000.0202
960.34340.40400.47470.35350.42420.5152
9700.02020.434300.05050.4545
980.38380.44440.56570.36360.44440.6566
9900.01010.14140.01010.02020.3333
Figure A1. The measured histograms of the HSV antecedent parameters. The histograms are normalized in both directions. The measured values are represented by black circles in the case of the tiles with polyp (positive case), while blue asterisk means the number of tiles without polyp (negative case). The rules are also plotted for the histogram based triangular membership functions with the same colour (black for positive and blue for negative), just dashed lines were used, while the skirted triangle and half Gaussian histogram fitted rules were shown in magenta colour in the case of the tiles with polyp, and cyan for the polypless tiles.
Figure A1. The measured histograms of the HSV antecedent parameters. The histograms are normalized in both directions. The measured values are represented by black circles in the case of the tiles with polyp (positive case), while blue asterisk means the number of tiles without polyp (negative case). The rules are also plotted for the histogram based triangular membership functions with the same colour (black for positive and blue for negative), just dashed lines were used, while the skirted triangle and half Gaussian histogram fitted rules were shown in magenta colour in the case of the tiles with polyp, and cyan for the polypless tiles.
Entropy 23 00936 g0a1
Figure A2. The measured histograms of the HSV antecedent parameters. The histograms are normalized in both directions. The measured values are represented by black circles in the case of the tiles with polyp (positive case), while blue asterisk means the number of tiles without polyp (negative case). The rules are also plotted for the histogram based triangular membership functions with the same colour (black for positive and blue for negative), just dashed lines were used, while the skirted triangle and half Gaussian histogram fitted rules were shown in magenta colour in the case of the tiles with polyp, and cyan for the polypless tiles.
Figure A2. The measured histograms of the HSV antecedent parameters. The histograms are normalized in both directions. The measured values are represented by black circles in the case of the tiles with polyp (positive case), while blue asterisk means the number of tiles without polyp (negative case). The rules are also plotted for the histogram based triangular membership functions with the same colour (black for positive and blue for negative), just dashed lines were used, while the skirted triangle and half Gaussian histogram fitted rules were shown in magenta colour in the case of the tiles with polyp, and cyan for the polypless tiles.
Entropy 23 00936 g0a2
Figure A3. The measured histograms of the HSV antecedent parameters. The histograms are normalized in both directions. The measured values are represented by black circles in the case of the tiles with polyp (positive case), while blue asterisk means the number of tiles without polyp (negative case). The rules are also plotted for the histogram based triangular membership functions with the same colour (black for positive and blue for negative), just dashed lines were used, while the skirted triangle and half Gaussian histogram fitted rules were shown in magenta colour in the case of the tiles with polyp, and cyan for the polypless tiles.
Figure A3. The measured histograms of the HSV antecedent parameters. The histograms are normalized in both directions. The measured values are represented by black circles in the case of the tiles with polyp (positive case), while blue asterisk means the number of tiles without polyp (negative case). The rules are also plotted for the histogram based triangular membership functions with the same colour (black for positive and blue for negative), just dashed lines were used, while the skirted triangle and half Gaussian histogram fitted rules were shown in magenta colour in the case of the tiles with polyp, and cyan for the polypless tiles.
Entropy 23 00936 g0a3
Figure A4. The measured histograms of the HSV antecedent parameters. The histograms are normalized in both directions. The measured values are represented by black circles in the case of the tiles with polyp (positive case), while blue asterisk means the number of tiles without polyp (negative case). The rules are also plotted for the histogram based triangular membership functions with the same colour (black for positive and blue for negative), just dashed lines were used, while the skirted triangle and half Gaussian histogram fitted rules were shown in magenta colour in the case of the tiles with polyp, and cyan for the polypless tiles.
Figure A4. The measured histograms of the HSV antecedent parameters. The histograms are normalized in both directions. The measured values are represented by black circles in the case of the tiles with polyp (positive case), while blue asterisk means the number of tiles without polyp (negative case). The rules are also plotted for the histogram based triangular membership functions with the same colour (black for positive and blue for negative), just dashed lines were used, while the skirted triangle and half Gaussian histogram fitted rules were shown in magenta colour in the case of the tiles with polyp, and cyan for the polypless tiles.
Entropy 23 00936 g0a4
Figure A5. The measured histograms of the HSV antecedent parameters. The histograms are normalized in both directions. The measured values are represented by black circles in the case of the tiles with polyp (positive case), while blue asterisk means the number of tiles without polyp (negative case). The rules are also plotted for the histogram based triangular membership functions with the same colour (black for positive and blue for negative), just dashed lines were used, while the skirted triangle and half Gaussian histogram fitted rules were shown in magenta colour in the case of the tiles with polyp, and cyan for the polypless tiles.
Figure A5. The measured histograms of the HSV antecedent parameters. The histograms are normalized in both directions. The measured values are represented by black circles in the case of the tiles with polyp (positive case), while blue asterisk means the number of tiles without polyp (negative case). The rules are also plotted for the histogram based triangular membership functions with the same colour (black for positive and blue for negative), just dashed lines were used, while the skirted triangle and half Gaussian histogram fitted rules were shown in magenta colour in the case of the tiles with polyp, and cyan for the polypless tiles.
Entropy 23 00936 g0a5
Figure A6. The measured histograms of the RGB antecedent parameters. The histograms are normalized in both directions. The measured values are represented by black circles in the case of the tiles with polyp (positive case), while blue asterisk means the number of tiles without polyp (negative case). The rules are also plotted for the histogram based triangular membership functions with the same colour (black for positive and blue for negative), just dashed lines were used, while the skirted triangle and half Gaussian histogram fitted rules were shown in magenta colour in the case of the tiles with polyp, and cyan for the polypless tiles.
Figure A6. The measured histograms of the RGB antecedent parameters. The histograms are normalized in both directions. The measured values are represented by black circles in the case of the tiles with polyp (positive case), while blue asterisk means the number of tiles without polyp (negative case). The rules are also plotted for the histogram based triangular membership functions with the same colour (black for positive and blue for negative), just dashed lines were used, while the skirted triangle and half Gaussian histogram fitted rules were shown in magenta colour in the case of the tiles with polyp, and cyan for the polypless tiles.
Entropy 23 00936 g0a6
Figure A7. The measured histograms of the RGB antecedent parameters. The histograms are normalized in both directions. The measured values are represented by black circles in the case of the tiles with polyp (positive case), while blue asterisk means the number of tiles without polyp (negative case). The rules are also plotted for the histogram based triangular membership functions with the same colour (black for positive and blue for negative), just dashed lines were used, while the skirted triangle and half Gaussian histogram fitted rules were shown in magenta colour in the case of the tiles with polyp, and cyan for the polypless tiles.
Figure A7. The measured histograms of the RGB antecedent parameters. The histograms are normalized in both directions. The measured values are represented by black circles in the case of the tiles with polyp (positive case), while blue asterisk means the number of tiles without polyp (negative case). The rules are also plotted for the histogram based triangular membership functions with the same colour (black for positive and blue for negative), just dashed lines were used, while the skirted triangle and half Gaussian histogram fitted rules were shown in magenta colour in the case of the tiles with polyp, and cyan for the polypless tiles.
Entropy 23 00936 g0a7
Figure A8. The measured histograms of the RGB antecedent parameters. The histograms are normalized in both directions. The measured values are represented by black circles in the case of the tiles with polyp (positive case), while blue asterisk means the number of tiles without polyp (negative case). The rules are also plotted for the histogram based triangular membership functions with the same colour (black for positive and blue for negative), just dashed lines were used, while the skirted triangle and half Gaussian histogram fitted rules were shown in magenta colour in the case of the tiles with polyp, and cyan for the polypless tiles.
Figure A8. The measured histograms of the RGB antecedent parameters. The histograms are normalized in both directions. The measured values are represented by black circles in the case of the tiles with polyp (positive case), while blue asterisk means the number of tiles without polyp (negative case). The rules are also plotted for the histogram based triangular membership functions with the same colour (black for positive and blue for negative), just dashed lines were used, while the skirted triangle and half Gaussian histogram fitted rules were shown in magenta colour in the case of the tiles with polyp, and cyan for the polypless tiles.
Entropy 23 00936 g0a8
Figure A9. The measured histograms of the RGB antecedent parameters. The histograms are normalized in both directions. The measured values are represented by black circles in the case of the tiles with polyp (positive case), while blue asterisk means the number of tiles without polyp (negative case). The rules are also plotted for the histogram based triangular membership functions with the same colour (black for positive and blue for negative), just dashed lines were used, while the skirted triangle and half Gaussian histogram fitted rules were shown in magenta colour in the case of the tiles with polyp, and cyan for the polypless tiles.
Figure A9. The measured histograms of the RGB antecedent parameters. The histograms are normalized in both directions. The measured values are represented by black circles in the case of the tiles with polyp (positive case), while blue asterisk means the number of tiles without polyp (negative case). The rules are also plotted for the histogram based triangular membership functions with the same colour (black for positive and blue for negative), just dashed lines were used, while the skirted triangle and half Gaussian histogram fitted rules were shown in magenta colour in the case of the tiles with polyp, and cyan for the polypless tiles.
Entropy 23 00936 g0a9
Figure A10. The measured histograms of the RGB antecedent parameters. The histograms are normalized in both directions. The measured values are represented by black circles in the case of the tiles with polyp (positive case), while blue asterisk means the number of tiles without polyp (negative case). The rules are also plotted for the histogram based triangular membership functions with the same colour (black for positive and blue for negative), just dashed lines were used, while the skirted triangle and half Gaussian histogram fitted rules were shown in magenta colour in the case of the tiles with polyp, and cyan for the polypless tiles.
Figure A10. The measured histograms of the RGB antecedent parameters. The histograms are normalized in both directions. The measured values are represented by black circles in the case of the tiles with polyp (positive case), while blue asterisk means the number of tiles without polyp (negative case). The rules are also plotted for the histogram based triangular membership functions with the same colour (black for positive and blue for negative), just dashed lines were used, while the skirted triangle and half Gaussian histogram fitted rules were shown in magenta colour in the case of the tiles with polyp, and cyan for the polypless tiles.
Entropy 23 00936 g0a10

References

  1. Bosman, F.T. Chapter 5.5: Colorectal Cancer. In World Cancer Report the International Agency for Research on Cancer; Stewart, B.W., Wild, C.P., Eds.; World Health Organization: Geneva, Switzerland, 2014; p. 392402. ISBN 978-92-832-0443-5. [Google Scholar]
  2. Fenlon, H.M.; Nunes, D.P.; Schroy, P.C.; Barish, M.A.; Clarke, P.D.; Ferrucci, J.T. A Comparison of Virtual and Conventional Colonoscopy for the Detection of Colorectal Polyps. N. Engl. J. Med. 1999, 341, 1496–1503. [Google Scholar] [CrossRef]
  3. Pickhardt, P.J.; Choi, J.R.; Hwang, I.; Butler, J.A.; Puckett, M.L.; Hildebrandt, H.A.; Wong, R.K.; Nugent, P.A.; Mysliwiec, P.A.; Schindler, W.R. Computed Tomographic Virtual Colonoscopy to Screen for Colorectal Neoplasia in Asymptomatic Adults. N. Engl. J. Med. 2003, 349, 2191–2200. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  4. Menardo, G. Sensitivity of diagnostic examinations for colorectal polyps. Tech. Coloproctol. 2004, 8 (Suppl. S2), S273–S275. [Google Scholar] [CrossRef]
  5. Tischendorf, J.J.W.; Wasmuth, H.E.; Koch, A.; Hecker, H.; Trautwein, C.; Winograd, R. Value of magnifying chromoendoscopy and narrow band imaging (NBI) in classifying colorectal polyps: A prospective controlled study. Endoscopy 2007, 39, 1092–1096. [Google Scholar] [CrossRef] [PubMed]
  6. Song, H.J.; Shim, K.N. Current status and future perspectives of capsule endoskopy. Intest. Res. 2016, 14, 21–29. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  7. Enns, R.A.; Hookey, L.; Armstrong, D.; Bernstein, C.N.; Heitman, S.J.; Teshima, C.; Leontiadis, G.I.; Tse, F.; Sadowski, D. Clinical practice guidelines for the use of video capsule endoscopy. Gastroenterology 2017, 152, 497–514. [Google Scholar] [CrossRef] [Green Version]
  8. Søreide, K.; Nedrebø, B.S.; Reite, A.; Thorsen, K.; Kørner, H. Endoscopy Morphology, Morphometry and Molecular Markers: Predicting Cancer Risk in Colorectal Adenoma. Expert Rev. Mol. Diagn. 2009, 9, 125137. [Google Scholar] [CrossRef]
  9. Jass, J.R. Classication of colorectal cancer based on correlation of clinical, morphological and molecular features. Histopathology 2006, 50, 113130. [Google Scholar]
  10. Kudo, S.; Hirota, S.; Nakajima, T.; Hosobe, S.; Kusaka, H.; Kobayashi, T.; Himori, M.; Yagyuu, A. Colorectal tumours and pit pattern. J. Clin. Pathol. 1994, 47, 880–885. [Google Scholar] [CrossRef] [Green Version]
  11. Kudo, S.; Tamura, S.; Nakajima, T.; Yamano, H.O.; Kusaka, H.; Watanabe, H. Diagnosis of colorectal tumorous lesions by magnifying endoscopy. Gastrointest. Endosc. 1996, 44, 8–14. [Google Scholar] [CrossRef]
  12. Kudo, S.; Rubio, C.A.; Teixeira, C.R.; Kashida, H.; Kogure, E. Pit pattern in colorectal neoplasia: Endoscopic magnifying view. Endoscopy 2001, 33, 367–373. [Google Scholar] [CrossRef] [PubMed]
  13. Bernal, J.; Sánchez, F.J.; Vilariño, F. Towards Automatic Polyp Detection with a Polyp Appearance Model. Pattern Recognit 2012, 45, 3166–3182. [Google Scholar] [CrossRef]
  14. Bernal, J.; Tajkbaksh, N.; Sánchez, F.J.; Matuszewski, B.; Chen, H.; Yu, L.; Angermann, Q.; Romain, O.; Rustad, B. Comparative Validation of Polyp Detection Methods in Video Colonoscopy: Results from the MICCAI 2015 Endoscopic Vision Challenge. IEEE Trans. Med. Imaging 2017, 36, 1231–1249. [Google Scholar] [CrossRef] [PubMed]
  15. Zadeh, L.A. Fuzzy sets. Inf. Control 1965, 8, 338–353. [Google Scholar] [CrossRef] [Green Version]
  16. Zadeh, L.A. Fuzzy algorithms. Inf. Control 1968, 12, 94–102. [Google Scholar] [CrossRef] [Green Version]
  17. Kóczy, L.T.; Hirota, K. Approximate reasoning by linear rule interpolation and general approximation. Int. J. Approx. Reason. 1993, 9, 197–225. [Google Scholar] [CrossRef] [Green Version]
  18. Mamdani, E.H.; Assilian, S. An experiment in linguistic synthesis with a fuzzy logic controller. Int. J. Man-Mach. Stud. 1975, 7, 1–13. [Google Scholar] [CrossRef]
  19. Nagy, S.Z.; Sziová, B.; Kóczy, L.T. The effect of image feature qualifiers on fuzzy colorectal polyp detection schemes using KH interpolation—Towards hierarchical fuzzy classification of coloscopic still images. In Proceedings of the FuzzIEEE 2018, Rio de Janeiro, Brazil, 8–13 July 2018; pp. 1–7. [Google Scholar]
  20. Nagy, S.Z.; Lilik, F.; Kóczy, L.T. Applicability of various wavelet families in fuzzy classification of access networks’ telecommunication lines. In Proceedings of the 2017 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), Naples, Italy, 9–12 July 2017. [Google Scholar]
  21. Nagy, S.Z.; Sziová, B.; Kóczy, L.T. The effect of wavelet analyis on entropy based fuzzy classification of colonoscopy images. In Proceedings of the 5th International Workshop on Advanced Computational Intelligence and Intelligent Informatics (IWACIII 2017), Beijing, China, 2–5 November 2017. [Google Scholar]
  22. Sziová, B.; Nagy, S.Z.; Kóczy, L.T. The Effects of Preprocessing on Colorectal Polyp Detecting by Fuzzy Algorithm. In Recent Developments and the New Direction in Soft-Computing Foundations and Applications; Springer: Cham, Switzerland, 2018; pp. 347–357. [Google Scholar]
  23. Bernal, J.; Sánchez, F.; Esparrach, G.; Gil, D.; Rodríguez, C.; Vilariño, F. WM-DOVA maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physicians. Comput. Med. Imaging Graph. 2015, 43, 99–111. [Google Scholar] [CrossRef]
  24. Silva, J.S.; Histace, A.; Romain, O.; Dray, X.; Grando, B. Towards embedded detection of polyps in WCE images for early diagnosis of colorectal cancer. Int. J. Comput. Assist. Radiol. Surg. 2014, 9, 283–293. [Google Scholar] [CrossRef] [PubMed]
  25. Yuji, I.; Akira, H.; Yoshinori, A.; Bhuyan, M.; Robert, J.; Kunio, K. Automatic Detection of Polyp Using Hessian Filter and HOG Features. Procedia Comput. Sci. 2015, 60, 730–739. [Google Scholar]
  26. Soumelidis, A.; Fazekas, Z.; Schipp, F. Geometrical description of quasi-hemispherical and calotte-like surfaces using discretised argument-transformed Chebyshev-polynomials. In Proceedings of the 44th IEEE Conference on Decision and Control, Seville, Spain, 15 December 2005. [Google Scholar]
  27. Wu, F.; KinTak, U. Low-Light image enhancement algorithm based on HSI color space. In Proceedings of the 10th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), Shanghai, China, 14–16 October 2017. [Google Scholar] [CrossRef]
  28. Canny, J. A Computational Approach to Edge Detection. IEEE Trans. Pattern Anal. Mach. Intell. 1986, PAMI-8, 679–698. [Google Scholar] [CrossRef]
  29. Nagy, S.Z.; Sziová, B.; Pipek, J. On Structural Entropy and Spatial Filling Factor Analysis of Colonoscopy Pictures. Entropy 2019, 21, 256. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  30. Von Neumann, J. Thermodynamik quantenmechanischer Gesamtheiten. Nachr. Ges. Wiss. Gött. Math.-Phys. Kl. 1927, 102, 273–291. [Google Scholar]
  31. Shannon, C.E. A mathematic theory of communication. Bell Syst. Tech. J. 1948, 27, 379–423. [Google Scholar] [CrossRef] [Green Version]
  32. Rényi, A. On measures of information and entropy. In Proceedings of the fourth Berkeley Symposium on Mathematics, Statistics and Probability, Berkeley, CA, USA, 20 June–30 July 1960; pp. 547–561. [Google Scholar]
  33. Amigó, J.M.; Balogh, S.G.; Hernández, S. A Brief Review of Generalized Entropies. Entropy 2018, 20, 813. [Google Scholar] [CrossRef] [Green Version]
  34. Stantchev, I. Structural Entropy: A New Approach for Systems Structure’s Analysis. In Cybernetics and Systems ’86; Trappl, R., Ed.; Springer: Dordrecht, The Netherlands, 1986; pp. 139–186. [Google Scholar] [CrossRef]
  35. Mojzes, I.; Dominkovics, C.S.; Harsányi, G.; Nagy, S.Z.; Pipek, J.; Dobos, L. Heat treatment parameters effecting the fractal dimensions of AuGe metallization on GaAs. Appl. Phys. Lett. 2007, 91, 073107. [Google Scholar] [CrossRef] [Green Version]
  36. Pipek, J.; Varga, I. Universal classification scheme for the spatial-localization properties of one-particle states in finite, d-dimensional systems. Phys. Rev. A 1992, 46, 3148–3163. [Google Scholar] [CrossRef] [PubMed]
  37. Varga, I.; Pipek, J. Rényi entropies characterizing the shape and the extension of the phase space representation of quantum wave functions in disordered systems. Phys. Rev. E 2003, 68, 026202. [Google Scholar] [CrossRef] [Green Version]
  38. Bonyár, A.; Molnár, L.M.; Harsányi, G. Localization factor: A new parameter for the quantitative characterization of surface structure with atomic force microscopy (AFM). Micron 2012, 43, 305–310. [Google Scholar] [CrossRef]
  39. Bonyár, A. AFM characterization of the shape of surface structures with localization factor. Micron 2016, 87, 1–9. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  40. Fodor, J.C. A remark on constructing t-norms. Fuzzy Sets Syst. 1991, 41, 195–199. [Google Scholar] [CrossRef]
  41. Weber, S. A general concept of fuzzy connectives, negations and implications based on t-norms and t-conorms. Fuzzy Sets Syst. 1983, 11, 115–134. [Google Scholar] [CrossRef]
  42. Sziová, B.; Ismail, R.; Lilik, F.; Kóczy, L.T.; Nagy, S.Z. Fuzzy rulebase parameter determination for stabilized KH interpolation based detection of colorectal polyps on colonoscopy images. In Proceedings of the 2020 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), Glasgow, UK, 19–24 July 2020; pp. 1–6. [Google Scholar] [CrossRef]
  43. Kóczy, L.T.; Hirota, K. Interpolative reasoning with insufficient evidence in sparse fuzzy rule bases. Inf. Sci. 1993, 71, 169–201. [Google Scholar] [CrossRef]
  44. Tikk, D.; Joó, I.; Kóczy, L.T.; Várlaki, P.; Moser, B.; Gedeon, T.D. Stability of interpolative fuzzy KH-controllers. Fuzzy Sets Syst. 2002, 125, 105–119. [Google Scholar] [CrossRef]
  45. Takacs, O.; Varkonyi-Koczy, A.R. SVD-based complexity reduction of rule-bases with nonlinear antecedent fuzzy sets. IEEE Trans. Instrum. Meas. 2002, 51, 217–221. [Google Scholar] [CrossRef]
  46. Brandao, P.; Zisimopoulos, O.; Mazomenos, E.; Ciuti, G.; Bernal, J.; Visentini-Scarzanella, M.; Menciassi, A.; Dario, P.; Koulaouzidis, A.; Arezzo, A.; et al. Towards a computed-aided diagnosis system in colonoscopy: Automatic polyp segmentation using convolution neural networks. J. Med. Robot. Res. 2021, 3, 1–10. [Google Scholar] [CrossRef] [Green Version]
Figure 1. Sample pictures from widely used colorectal image-bases [13,23,24]. The left image shows the zenithal view of a colorectal polyp, the second one is a lateral view, while the third is a semilateral view. The resolution differences of the three databases are also visible on the images.
Figure 1. Sample pictures from widely used colorectal image-bases [13,23,24]. The left image shows the zenithal view of a colorectal polyp, the second one is a lateral view, while the third is a semilateral view. The resolution differences of the three databases are also visible on the images.
Entropy 23 00936 g001
Figure 2. Structural entropy ( S S t r = S 1 S 2 ) vs. spatial filling factor (− log q = S 0 S 2 ) for various types of probability distributions. The thick solid line shows the theoretical limit: all the distributions have to be below this line, i.e., S S t r log q . The other curves show some examples of trend lines.
Figure 2. Structural entropy ( S S t r = S 1 S 2 ) vs. spatial filling factor (− log q = S 0 S 2 ) for various types of probability distributions. The thick solid line shows the theoretical limit: all the distributions have to be below this line, i.e., S S t r log q . The other curves show some examples of trend lines.
Entropy 23 00936 g002
Figure 3. A picture presented in greyscale from database [24], and its respective wavelet transforms. In this case only the LH wavelet transformed version carries perceptable information.
Figure 3. A picture presented in greyscale from database [24], and its respective wavelet transforms. In this case only the LH wavelet transformed version carries perceptable information.
Entropy 23 00936 g003
Figure 4. Graphical representation of the Mamadani-Assilian type inference. The antecedents are denoted by A i j , their actual measured value by x i , their membership value in the rule corresponding to the consequent B j with w i j . The defuzzified decision is shown as B * .
Figure 4. Graphical representation of the Mamadani-Assilian type inference. The antecedents are denoted by A i j , their actual measured value by x i , their membership value in the rule corresponding to the consequent B j with w i j . The defuzzified decision is shown as B * .
Entropy 23 00936 g004
Figure 5. The histograms of a training set of measured data. Black circles mark the no polyp results, while the blue circles the tiles with polyp. The rulebases resulting from all three of the methods are plotted. The black and blue continious poly-lines–used as above–denote the mean centered rules, and the dashed lines the histogram based rules. Also, for the histogram based rules the Gaussian and half Gaussian membership functions with extended support are plotted with cyan and magenta colours. The half Gaussian extended rule membership function consists of two half Gaussians fitted to the asymmetrical triangle membership functions.
Figure 5. The histograms of a training set of measured data. Black circles mark the no polyp results, while the blue circles the tiles with polyp. The rulebases resulting from all three of the methods are plotted. The black and blue continious poly-lines–used as above–denote the mean centered rules, and the dashed lines the histogram based rules. Also, for the histogram based rules the Gaussian and half Gaussian membership functions with extended support are plotted with cyan and magenta colours. The half Gaussian extended rule membership function consists of two half Gaussians fitted to the asymmetrical triangle membership functions.
Entropy 23 00936 g005
Figure 6. Workflow chart. The first two boxes represent the development of the method, the boxes with italic type letters represent those parts of method which can be automatically performed without human interaction, the others need human control or in the case of the detail medical analysis it should be performed and overseen by human medical experts.
Figure 6. Workflow chart. The first two boxes represent the development of the method, the boxes with italic type letters represent those parts of method which can be automatically performed without human interaction, the others need human control or in the case of the detail medical analysis it should be performed and overseen by human medical experts.
Entropy 23 00936 g006
Table 1. The indices for the initial antecedent parameters. The indices are given for both the RGB and the HSV colour representations, e.g., antecedents 1 and 2 are the mean and standard deviation of the first colour channel, i.e., of the red (R) and of the hue (H) channel, respectively. The wavelet transformed images are denoted by their respective low-pass and high-pass filters, i.e., LL, LH, HL, and HH.
Table 1. The indices for the initial antecedent parameters. The indices are given for both the RGB and the HSV colour representations, e.g., antecedents 1 and 2 are the mean and standard deviation of the first colour channel, i.e., of the red (R) and of the hue (H) channel, respectively. The wavelet transformed images are denoted by their respective low-pass and high-pass filters, i.e., LL, LH, HL, and HH.
IndexAntecedent Name
1–2mean and standard deviation, R/H
3–4mean and standard deviation, G/S
5–6mean and standard deviation, B/V
7edge density, R/H
8edge density, G/S
9edge density, B/V
10–11 S s t r , ln q , R/H
12–13 S s t r , ln q , G/S
14–15 S s t r , ln q , B/V
16–30similar to 1–15, wavelet transform LL
31–45similar to 1–15, wavelet transform LH
46–60similar to 1–15, wavelet transform HL
61–75similar to 1–15, wavelet transform HH
76–77gradient magnitude’s mean and standard deviation, R/H
78–79gradient magnitude’s mean and standard deviation, G/S
80–81gradient magnitude’s mean and standard deviation, B/V
82–87similar to 76–81, gradient direction
88–93similar to 76–81, gradient x component
94–99similar to 76–81, gradient y component
Table 2. Antecedents where this total distance was larger than 0.2 in decreasing order of the total distance.
Table 2. Antecedents where this total distance was larger than 0.2 in decreasing order of the total distance.
51, 4, 61, 19, 85, 29, 73, 27, 35, 34, 32, 82, 97, 43, 72, 64, 13, 12, 94, 28, 36, 99, 78, 49, 84, 91,
31, 48, 80, 63, 79, 45, 74, 75, 44
Table 3. The antecedents with total distance larger than 0.35 .
Table 3. The antecedents with total distance larger than 0.35 .
12, 28, 36, 44, 45, 49, 74, 75, 78, 79, 80, 91, 99
Table 4. Antecedents, where the centers of the histograms for the two consequents’ training set were fairly distant from each other.
Table 4. Antecedents, where the centers of the histograms for the two consequents’ training set were fairly distant from each other.
2, 3, 5, 10, 11, 12, 13, 17, 18, 20, 25, 26, 27, 28, 37, 38, 39, 52,
53, 54, 62, 63, 64, 68, 69, 72, 73, 77, 79, 83, 85, 86, 89, 91, 95, 97
Table 5. The antecedents of the reduced set. The colour channels were denoted the following way: H: hue, S: saturation, V: value or intensity. The last group containts the gradients. As to different cases were studied: one, which included these gradients, the other which did not, the last row was denoted by *.
Table 5. The antecedents of the reduced set. The colour channels were denoted the following way: H: hue, S: saturation, V: value or intensity. The last group containts the gradients. As to different cases were studied: one, which included these gradients, the other which did not, the last row was denoted by *.
Antecedent GroupAntecedent
Original tilestandard deviation H
mean of S and V
structural entropy of H and S
ln q of H and S
Low-pass–low-pass wavelet transformedge density of H, S and V
Low-pass–high-pass wavelet transformedge density of H, S and V
High-pass–low-pass wavelet transformedge density of H, S and V
High-pass–high-pass wavelet transformstandard deviation of H and S
edge densities of S and V
structural entropies of S
(Gradients) *(standard deviation of H and S)
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Sziová, B.; Nagy, S.; Fazekas, Z. Application of Structural Entropy and Spatial Filling Factor in Colonoscopy Image Classification. Entropy 2021, 23, 936. https://doi.org/10.3390/e23080936

AMA Style

Sziová B, Nagy S, Fazekas Z. Application of Structural Entropy and Spatial Filling Factor in Colonoscopy Image Classification. Entropy. 2021; 23(8):936. https://doi.org/10.3390/e23080936

Chicago/Turabian Style

Sziová, Brigita, Szilvia Nagy, and Zoltán Fazekas. 2021. "Application of Structural Entropy and Spatial Filling Factor in Colonoscopy Image Classification" Entropy 23, no. 8: 936. https://doi.org/10.3390/e23080936

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop