Towards a generalized colour image segmentation for kiwifruit detection

Developing robust computer vision algorithms to detect fruit in trees is challenging due to less controllable conditions, including variation in illumination within an image as well as between image sets. There are two classes of techniques: local-feature-based techniques and shape-based techniques, which have been used extensively in this application domain. Out of the two classes, the local-feature-based techniques have shown higher accuracies over shape-based techniques, but are less desirable due to the requirement of repeated calibration. In this paper, we investigate the potential of developing a generalized colour pixel classifier that can be employed to detect kiwifruit on vines, under variable fruit maturity levels and imaging conditions. First, we observed the colour data patterns of fruit and nonfruit regions from different image sets. With consistant data patterns it was found that a suitable normalization could produce an invariant colour descriptor. Then, a neural network self-organizing map (SOM) model, which has a hierarchical clustering ability was used to investigate the potential of developing a generalized neural network model to classify pixels under variable conditions. Models were built for colour features extracted in CIELab space for both absolute colour values and relative colour descriptors. The paper presents the positive results of the preliminary investigations. The conditions for a successful application of the approach as well as the potential for extending it for automatic calibration will also be discussed.