Vision-Based Lettuce Growth Stage Decision Support System Using Artificial Neural Networks

Machine vision approaches for lettuce growth stage prediction are continuously being developed. Previous works suggest further extensive study of computer vision features in determining plant growth. This paper presented an ANN-based decision support system of classifying lettuce growth stage by using extracted vision features that included two morphological features (area, perimeter), 12 color features (RGB, HSV, YCbCr, Lab), and five textural features (contrast, energy, correlation, entropy, and homogeneity). Image processing techniques were used to extract the required vision features, and the neural network was trained using scaled conjugate gradient back propagation. The decision support system exhibited promising results in lettuce growth stage classification.


I. INTRODUCTION
Controlled-environment agriculture (CEA) is a technology-based approach toward horticulture crop production. Supported by science and engineering, the aim of CEA is to provide and maintain best conditions for crops by using controlled environments that optimize environmental parameters, energy, space, labor, and money [1].
One of the CEA technologies is hydroponics. In this hydroponics setup, lettuce is planted hydroponically on rockwool that floats on top of holes on pipes. Nutrient solution flows through the pipes. This system improves the control of pests, conserves space, and eliminates the need for constant watering [2].
In conventional greenhouse production, environmental parameters for growing conditions are managed by farmer's experience or learned preset favorable conditions instead of using the actual growth stage of the plant at that given time. The identification of growth stage of horticultural crops, such as lettuce, is vital to the proper allocation of environmental requirements, such as temperature, humidity, radiation, water, and nutrients [3]. Different lettuce growth stages may require distinct proportions of these environmental factors to ensure optimum yield of harvestable products. Growth stage analysis of plant entails measurements and allometric functions for estimations of plant parameters to characterize Manuscript  lettuce [4].
One method of identifying plant growth characteristics is physical and manual measurement. This methodology is labor intensive, tedious, and sometimes destructive [5]. Thus, an automated machine-vision-based decision support system that can monitor plant growth should be developed. Recently, computer vision approaches for plant growth monitoring are being increasingly employed [6]- [8]. This approach makes the plant growth monitoring continuous, automated, and noninvasive. This kind of intelligent monitoring in greenhouse farming allows the precise understanding and analysis of various environmental factors attributed to growth and farm management deficiencies.
Recent developments on machine vision applications for monitoring lettuce growth have existed. For example, [9] presented a computer-vision-based method for predicting the crop coefficient (Kc) from the estimated ratio of the percentage of ground cover and height PGC/h. The PGC was computed using digital image processing techniques.
In another study, [10] developed a computer stereo-vision system using two cameras with parallel optical axes coupled with an image processing algorithm to monitor Boston lettuce growth automatically in a plant factory. Vision features included projected leaf area, plant height, diameter, and volume.
Moreover, [11] proposed a scale invariant leaf area calculation using machine vision and knowledge-based methods. The leaf area in turn can be used to determine lettuce growth stage. Li X. et al. (2016) [12] presented a system that determines lettuce plant growth based on vision features. Six parameters were considered, namely, plant height, top projected canopy area, length and width of a certain leaf, number of leaves, length of the X-axis direction of the top projected canopy, and length of the Y-axis direction of the top projected canopy. Image processing techniques and principal component analysis method were used for the algorithm. Experimental results showed significant correlation between the total lettuce growth information and image parameters for a single lettuce plant and a group of lettuce plants. These studies are some of the previous efforts concerning machine vision as applied in lettuce growth monitoring. However, they suggest further extensive study of computer vision features in determining plant growth.
In this study, the objectives are (1) to develop a decision support system for classifying lettuce growth stage by using 15 different vision features that include morphology, color information, and texture features with artificial neural networks (ANNs) and (2) to evaluate the capability of the machine vision system. This paper discusses the proposed method as follows. The subsequent chapter discusses the classification of lettuce growth, the environmental conditions on the hydroponics chamber, and the camera setup of the proposed system. Section III discusses what vision features are being considered. Section IV describes the ANN design for the decision support system. Section V presents the results and analysis. Lastly, this paper ends with the conclusion and recommendations for future work.

II. HYDROPONICS SETUP
In this study, a floating hydroponic system was utilized. The plants floated on top of holes in polyvinyl chloride (PVC) pipes filled with water mixed with a mineral nutrient solution.
The hydroponics system was constructed in a testing site located in Morong, Rizal, Philippines. The setup was composed of two chambers that can provide 324 crops in one cycle of 45 days. The PVC pipes were assembled in cabinet racks following the design shown in Fig. 1(a). hydroponics with lettuce growing plants.
A lighting system was installed above the PVC pipes such that all net pots were covered by sufficient light. The greatest light was concentrated at the middle pipe for each level of the rack with a photosynthetically active radiation (PAR) value of 29-33 µmol m −2 s −1 measured from a Vernier PAR sensor. A PAR value of 22-29 µmol m −2 s −1 was measured on the pipes located at the sides. A nutrient tank with pump is located at the bottom layer of the rack. The parameters, such as pH level, electrical conductivity (EC), and water level, were monitored to determine whether to mix another nutrient solution on the tank. EC that ranges from 0.668 ppm to 0.892 ppm and pH level that ranges from 6 to 7.84 were measured in the cycle of 45 days.
The vision system was developed to monitor the lettuce growth and to identify the lettuce growth stage. Top views of the canopy of lettuce crops were captured using a positioned Logitech c920 camera, as shown in Fig. 2. Specific camera settings were used all throughout the captures to provide a standard color magnitude. The LED lights installed were the sources of light inside the chamber, not allowing sunlight during image capture. Lettuce plants were planted and grown in the hydroponics setup, as shown in Fig. 1(b). Lettuce images were captured at specified time intervals. Sample image captures are shown in Fig. 2.
Temporal information of the plant growth was noted to determine the growth stage of the lettuce plants. Lettuce growth can be classified as harvest, vegetative, and sowing stages based on the number of days [13]. Different stages of lettuce in the hydroponics setup are shown in Fig. 2 III. PROPOSED METHODOLOGY Fig. 3 shows the overall framework of the proposed methodology. Lettuce images were captured in top view. Pre-processing enhances the captured images. Lettuce plant pixels were segmented using superpixels and thresholding method. Using the methodology by   [14], Lab color information of the image extracted from a training image dataset undergoes two-level thresholding and K-means clustering through superpixels to identify each pixel class. A sample segmented result is shown in Fig. 4 The vision features were extracted on the lettuce images by using image processing techniques. Color is one of the determining factors of lettuce growth. Color has been used in several crops in determining overall plant health [15]. The 12 color features to be extracted and to be used in the decision support system includes H, S, and V (for HSV color space); R, G, and B (for RGB color space); L, a, and b (for CIE Lab color space); and Y, Cb, and Cr (for YCbCr color space).
Lettuce texture may determine the growth stage. Texture features were extracted by using computer vision. Gray-level co-occurrence matrix (GLCM) is a statistical method for analyzing the spatial distribution of pixels in the neighborhood and computes the values of texture features, including contrast, correlation, entropy, energy, and homogeneity [16]. These five texture features were extracted and used in identifying plant growth.
Basic morphological features, such as area and perimeter, were also considered in this study. These features were extracted from the binarized images of the input lettuce images.
Four ANN systems, as shown in Fig. 3, were developed using varying color spaces (RGB, HSV, Lab, and YCbCr) while setting the remaining morphological and texture features fixed among the systems. Each ANN design was subjected to similar design parameters, including the number of hidden layers, data division, training, performance, and calculation. Performance evaluation of each ANN design was conducted using validation performance using cross entropy, gradient and validation checks, confusion matrix, and receiver operating characteristic (ROC) plot.

IV. VISION FEATURES
One of the important tasks in image processing and computer vision is feature extraction. Feature extraction here refers to obtaining the image features of lettuce crops that can be useful in classifying the plant based on its growth stage.
This study in general used morphological, color, and texture features.

A. Morphological Features
This research used global shape descriptors, such as area and perimeter. Table I describes these features and the formulas used. L is the actual lettuce area in cm 2 , M is the actual lettuce perimeter in cm, P is the actual pixel area, O is the actual pixel perimeter, C is the reference component pixel area, and Q is the reference component pixel square side measurement.
The approach in [11] used in this hydroponics farm setup to address scale invariance was to embed a template or marker of known dimensions into the image, as shown in Fig.  5. These known dimensions of the marker were used for the system to scale area measurements. The number of pixels of a reference component shown in Fig. 5 was determined and related to the known area to produce the scaling factor in the formula shown in Table I. Note that one side of the black square bounded by red outline is 1.8 cm and the area therefore of the square is equal to 3.24 cm 2 .
Using binary images of the lettuce, MATLAB estimated the area of all pixels in an image by summing the areas of each pixel in the image. The area of an individual pixel was determined by looking at its 2 × 2 neighborhood. MATLAB also estimated the perimeter pixels of objects in the binary input image. A pixel is part of the perimeter if it is nonzero and is connected to at least one zero-valued pixel. The default connectivity is 4 for two dimensions [17].
The area and perimeter were related to the actual number of pixels of the reference components to produce scaling factors given in area per pixel, and the perimeter per pixel is shown in formulas in Table I. The scaling factor for area is multiplied to the number of pixels of segmented lettuce leaf to calculate the lettuce leaf area in cm 2 . On the contrary, the scaling factor for perimeter is multiplied by the number of perimeter pixels to compute the lettuce perimeter in cm.

B. Color Features
Another feature considered in this research was color. Colors are expressed numerically in different ways called as color spaces. Some of the well-known color spaces are RGB, CIE Lab, HSV, and YCbCr [13]. The visualization of these color spaces is shown in Fig. 7.
Colors are usually represented by a combination of red, green, and blue values. CIE Lab is another color space that is based on the linear color perception of humans. This color space is defined by the International Commission on Illumination in 1976 and derived from the CIE XYZ tristimulus values. The "Lab" color space consists of three components: L for lightness, "a" for chromacity from green to red, and "b" for chromacity from blue to yellow. Conversion formulas from CIE XYZ to CIE Lab are given in the following equations [13]: The YCbCr color space is another color representation whose components are Y, Cb, and Cr. The component "Y" is the brightness of Cb (blue-yellow difference) and Cr (red-green difference). Mathematically, they are defined from a coordinate transformation form (RGB), as shown by Equation 7 [13]. The last color space considered in this research is HSV. The HSV color representation is composed of hue, saturation, and value aligned to the way human vision perceives color mixing. If the components would be defined with the way paints of different colors are mixed together, then hue is a value from 0 to 1 from the neutral colors arranged in a radial slice with white on top and black at the bottom. On the contrary, saturation is a value corresponding to various shades of brightly colored paint. Finally, value resembles the mixture of those paints with varying amounts of white and black paint. The conversion formulas are shown in Equations 8-15 [13].

C. Texture Features
Texture features are developed in this research by using GLCM, which computes the spatial dependence of gray values. GLCM calculates the frequencies of two pixels in a particular spatial relationship. The GLCM values in the equations use the term Pi,j as element in a normalized gray-tone spatial-dependence matrix and N as the number of elements. Texture feature values that are included in this research are contrast, correlation, entropy, energy, and homogeneity [20].
Energy, also known as angular second moment, computes the summation of squared elements in the GLCM. It can be calculated as Homogeneity calculates the closeness of the distribution of elements with respect to the diagonal of GLCM. The homogeneity texture can be computed using Equation 17.
Correlation computes the linear dependency of the gray levels with surrounding pixels. The correlation texture can be computed using Equation 18. i j汨͵ N P i j Ͳ log P(i j) (19) where p contains the histogram, and counts returned from the histogram.
Contrast pertains to the local variations in the GLCM. Equation 20 shows the formula for contrast.

V. ANNS
ANNs have been applied in various fields, such as classification, identification, pattern recognition, and image processing [19]. In this research, ANN was trained using vision features to classify input lettuce images based on their growth stage.
ANN for the lettuce growth decision support system. The training and testing data were collected from a smart farm hydroponics setup. Vision features were extracted using image processing techniques. Lettuce images undergone segmentation followed by RGB color information extraction of the segmented lettuce pixels. These RGB values were then converted to its equivalent color space. Average, minimum, and maximum values were calculated for the lettuce training and test images. The images were then binarized and the corresponding area and perimeter were calculated using [11].
The inputs to each ANN system were 16 in total, as presented in Table II with their respective range of values.    9 shows the ANN block diagram. The input was composed of 16 features using morphological features, area and perimeter, color information in a selected color space, and texture features. The input parameters used in this study are shown in Table II. The outputs were lettuce growth stages: sowing, vegetative, and harvest. As can be seen in the ANN block diagram, the ANN system consisted of hidden and output layers constructed to form a two-layer feedforward network. Transfer functions used were sigmoid and linear for hidden and output layers, respectively. The system consisted of a single hidden layer of 20 neurons. This value was selected for high network training performance. composed of a 3 × 90 matrix representing static data: 90 samples of three elements, as shown in Table IV.  Table V shows the general parameters of the ANN design. This study used lettuce image dataset consisting of 90 distinct images with 15 features for each color space considered and three-element target vectors. The data involved here were distributed randomly as follows: 70% or 62 samples were used for training, 15% or 14 samples were used for validating, and 15% or 14 samples were used for testing. The systems were trained using scaled conjugate gradient back propagation.

VI. RESULT AND DISCUSSION
MATLAB neural network fitting tool was used to simulate the expected output of the decision support system. Fig. 10 and Fig. 11 show how each network's performance improved during training. As shown in Fig. 10, each training of neural network stopped for each system when the validation error reached its minimum and started to increase for six iterations (validation checks), which occurred at iteration 14 for  The magnitudes of the gradient for each network were 0.0300 for network 1, 0.0883 for network 2, 0.0444 for network 3, and 0.0311 for network 3, as shown in Fig. 11. Furthermore, in general, for the four networks, except network 2, the test and validation curves were very similar, indicating that the set errors have similar characteristics. One measure of how well the neural network fits the data is the confusion matrix, which shows the percentages of correct and incorrect classifications. It also shows how the data are distributed for each class. Correct classifications are the green squares on the matrix's diagonal. Incorrect classifications form the red squares. As presented in the red squares of Fig. 12, the percentages of misclassifications for networks 2 and 3 among the four networks were relatively small. Table VI summarizes the overall correct and incorrect classifications of the test data from the confusion matrix. Based on Table VI, networks 1 and 2 outperformed the rest in terms of correct classification. These networks involved the vision features of HSV and Lab color information.
The results are consistent with the ROC obtained for the four networks shown in Fig. 13. The ROC plot shows how the false-positive and true-positive rates relate, as the thresholding of outputs is varied from 0 to 1.  Fig. 12. Test confusion matrix of (a) morphological, RGB, and texture features; (b) morphological, HSV, and texture features; (c) morphological, Lab, and texture features; and (d) morphological, YCbCr, and texture features.

VII. CONCLUSIONS
In this study, the proposed ANN techniques for identifying lettuce growth stage were simulated using MATLAB. This work was conducted using lettuce images obtained in a smart farm hydroponics setup. Based on morphological features, various color information, and texture features, the best ANN design is composed of morphological information (area, perimeter), Lab/HSV color information, and texture features (energy, entropy, homogeneity, contrast, and correlation), thereby outperforming the other ANN designs described in this study. Based on the results, ANN can be used to classify lettuce growth stage based on the input color, morphological features, and texture features. A research extension may be performed by increasing the number of hidden neurons, training vectors, and relevant information for the ANN design. Along with these increase in data vectors, feature selection is another important work.

CONFLICT OF INTEREST
The authors, Loresco and Dadios, declare no conflict of interest.

AUTHOR CONTRIBUTIONS
The authors, Loresco and Dadios, conducted the research design and analysis, wrote the paper and approved the final version of the manuscript.