Fish freshness categorization from eyes and gills color features using multi-class artificial neural network and support vector machines
Introduction
Aquatic products are popular among consumers and their visual quality used to be detected manually for sorting, grading, species classification, and freshness assessment. Machine vision, as a non-destructive method, has been used in quality detection of aquatic products due to its efficiency, objectiveness, consistency, and reliability (Hong et al., 2014).
Fish is a valuable seafood in terms of nutritional value. The fish protein, containing essential amino acids and growth promoters, has a positive effect on human health. The deficiency of this protein could cause nervous system problems, mental retardation, and physical weakness in human as well as reduction in body resistance to infectious diseases (Skuland, 2015).
Taste, health, quality, and freshness of fish are among the factors motivating its consumption (Tomić et al., 2016). Recent changes in consumers’ lifestyle demand high-quality and immune fish products. Fish quality parameters such as safety, nutritional value, availability, and freshness are mainly influenced by storing and processing methods from harvesting until consumption. Moreover, production, transportation, retail, home-keeping, and final food provision could have a negative impact on the fish quality and freshness (Dutta et al., 2016; Shi et al., 2018).
The changes during the storage of fish can be categorized based on apparent features including color, stiffness, odor, secretions, scales, skin, flesh, abdomen, eyes, and gills. These features can be used individually to assess the fish freshness (Issac et al., 2017). There exist various sensory (quality index method and torry scheme), physical (tissue analysis, torrymeter, electronic nose, and near-infrared reflection spectroscopy), and chemical and biochemical (measurement of total volatile nitrogen, trimethylamine, pH, and adenosine triphosphate) methods for this purpose (Nollet and Toldra, 2009). However, most of these methods are either destructive or expensive. In addition, they cannot be used to determine the fish freshness in the early stages of storage, as they are sensitive to the latter phase of deterioration (Dowlati et al., 2013). To overcome these issues, machine vision, as a non-destructive method, has been used in external quality detection of aquatic products for its efficiency, objectiveness, consistency, and reliability (Hong et al., 2014).
The colorimetric technique is known as a determinative indicator for grading food products as fresh, moderately non-fresh or completely non-fresh (McCaig, 2002). Color is one of the most important quality attributes of the fish having a direct relation with its freshness and consumer’s acceptance (Lawless and Heymann, 2010). Gill is a red respiration organ in the fish body in which blood flows directly, and hence its odor and color patterns could be used as an indicator for evaluating the fish freshness. The color of the fish eye is another determinant index for evaluating its freshness. On the other hand, changes in protein and fat, along with the changes in biogenic amines and hypoxanthine that significantly influence fish corruption, occur after the animal’s death. These in turn cause changes in the taste, quality and color during storage (Issac et al., 2017; Olafsdottir et al., 2004). Therefore, the analysis of the color and odor of the gills and eyes could be used as the main factors for identifying the freshness of the harvested fish.
Studies have been done in the field of image processing by capturing the images of whole fish (Cakli et al., 2006; Kılınc et al., 2007; Luten et al., 2003), the fillets (Balaban et al., 2005; Barat et al., 2008; Hernández et al., 2009; Kohler et al., 2002; Korel et al., 2001, 2006; Mateo et al., 2006; Misimi et al., 2007; Oliveira and Balaban, 2006; Quevedo and Aguilera, 2010; Quevedo et al., 2010; Roth et al., 2007; Stien et al., 2006; Yagiz et al., 2009), and the skin (Erikson and Misimi, 2008). However, online and automatic use of machine vision through imaging the whole fish, fillets or skin is inadequate to put a value on fish freshness due to the product degradation and inaccuracy of the method. In this regard, the fish freshness has been investigated by using artificial neural network regression based on the color indices (L*, a*, b*, c*, and total color difference (ΔE)) of the eyes and gills (Dowlati et al., 2013), fuzzy logic classification based on RGB values of the eyes and gills (Muhamad et al., 2009), wavelet-based classification based on RGB values of the gills (Dutta et al., 2016), classification based on the segmented gill in RGB, Lab, and HSV color spaces (Issac et al., 2017), and multiple regression approach based on pupils and gills color in RGB, HSI, and L*a*b* color spaces (Shi et al., 2018).
In most researches conducted on the eyes and gills color changes, simple color indices have been used to assess the fish freshness and little information is available on the machine vision technology in terms of the color changes in the eyes and gills of the stored fish. The aim of this study is to evaluate the freshness of rainbow trout as one of the most cultivated fish for feeding (Miranda and Romero, 2017; Skuland, 2015), based on its eyes and gills color changes by means of image processing method. The effectiveness and applicability of the method is examined by a thorough investigation on features extracted from the eyes and gills images in different color spaces, and the classification of the ice-storage duration using artificial neural networks (ANNs) and support vector machines (SVMs) techniques.
Section snippets
Sample preparation
A total number of 20 rainbow trout (Oncorhynchus mykiss) fish with market size of 300–400 g produced in about 5–6 months were prepared from a pool located in Cheshmeh Dimeh, Chaharmahal and Bakhtiari province, Iran. The samples were immediately transferred to the laboratory after harvest. The samples were first numbered and placed in Styrofoam to control their temperature for optimum maintenance. Ice packs were used to cool the samples. The fishes were arranged into 5 layers from the bottom of
Effect of storage duration on eyes and gills color
Based on the analysis of variance (ANOVA), ice-storage duration had a significant effect on the RGB color components of the rainbow trout eyes and gills (P < 0.01). Fig. 6a and b show the changes in the color components of the eyes and gills with respect to the storage day, respectively. As shown, the values of all components increased for both the organs with increasing the ice-storage duration. However, the rate of changes in the color components of the gills was lower and more uniform than
Conclusions
The potential of the computer vision system based on imaging from the gills and eyes was evaluated for the non-destructive detection of the rainbow trout fish freshness by using two machine learning tools of ANN and SVM. Between the two classifiers, the ANN resulted in somewhat better classification accuracy. Furthermore, the variations in the gills color with the storage day were more effective than the changes in the eyes color to discriminate the storage days. The findings of this research
CRediT authorship contribution statement
Hosna Mohammadi Lalabadi: Methodology, Software, Formal analysis, Investigation, Writing - original draft. Morteza Sadeghi: Conceptualization, Methodology, Software, Validation, Formal analysis, Resources, Writing - review & editing, Supervision, Project administration. Seyed Ahmad Mireei: Software, Validation, Formal analysis, Writing - review & editing, Supervision.
Declaration of Competing Interest
The authors have no conflict of interest related to this study to disclose, and declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgment
This research was supported by Isfahan University of Technology, which is gratefully acknowledged.
References (39)
- et al.
Freshness monitoring of sea bream (Sparus aurata) with a potentiometric sensor
Food Chem.
(2008) - et al.
Freshness assessment of gilthead sea bream (Sparus aurata) by machine vision based on gill and eye color changes
J. Food Eng.
(2013) - et al.
Image processing based method to assess fish quality and freshness
J. Food Eng.
(2016) - et al.
Is the Atlantic surface temperature a good proxy for forecasting the recruitment of European eel in the Guadalquivir estuary?
Progress Oceanogr.
(2015) - et al.
Sensory, physical, chemical and microbiological changes in aquacultured meagre (Argyrosomus regius) fillets during ice storage
Food Chem.
(2009) - et al.
Visual quality detection of aquatic products using machine vision
Aquacult. Eng.
(2014) - et al.
Computer vision based method for quality and freshness check for fish from segmented gills
Comp. Electron. Agricult.
(2017) - et al.
Comparison of effects of slurry ice and flake ice pretreatments on the quality of aquacultured sea bream (Sparus aurata) and sea bass (Dicentrarchus labrax) stored at 4°C
Food Chem.
(2007) - et al.
Sorting salted cod fillets by computer vision: a pilot study
Comp. Electron. Agricult.
(2002) - et al.
Discriminating varieties of tea plant based on Vis/NIR spectral characteristics and using artificial neural networks
Biosyst. Eng.
(2008)
Quality analysis of tuna meat using an automated color inspection system
Aquacult. Eng.
Extending the use of visible/near-infrared reflectance spectrophotometers to measure colour of food and agricultural products
Food Res. Int.
A prototype to measure rainbow trout’s length using image processing
Aquacult. Eng.
Comparing data mining classifiers for grading raisins based on visual features
Comput. Electron. Agricult.
Early detection of freezing damage in sweet lemons using Vis/SWNIR spectroscopy
Biosyst. Eng.
Multisensor for fish quality determination
Trends Food Sci. Technol.
Developing a machine vision system for simultaneous prediction of freshness indicators based on tilapia (Oreochromis niloticus) pupil and gill color during storage at 4°C
Food Chem.
Healthy eating and barriers related to social class. The case of vegetable and fish consumption in Norway
Appetite
Image analysis as a tool to quantify rigor contraction in pre-rigor-filleted fillets
Comput. Electron. Agric.
Cited by (55)
Standardizing the fish freshness class during ice storage using clustering approach
2024, Ecological InformaticsIntelligent detection and control of quality deterioration of fresh aquatic products in the supply chain: A review
2024, Computers and Electronics in AgricultureIntelligent fish feeding based on machine vision: A review
2023, Biosystems EngineeringEffects of image data quality on a convolutional neural network trained in-tank fish detection model for recirculating aquaculture systems
2023, Computers and Electronics in AgricultureDetection of small yellow croaker freshness by hyperspectral imaging
2023, Journal of Food Composition and AnalysisCitation Excerpt :Because light can only be captured by the fish once it has passed through the retina, the amount of oxygen required in the fish's eyes is significantly higher than the amount required by other tissues. However, the gills, as the respiratory organ of the fish, are able to provide the necessary oxygen for the fish's own activities in other tissues (e.g., muscle tissue) even in the absence of oxygen or in low oxygen conditions (Lalabadi et al., 2020). Therefore, the change in the eye of fish samples is more obvious than other tissues.