D Stereo Vision Measurements for Weed-Crop Discrimination

Computer vision research is targeting toward agricultural robotics applications. Our vision system contains two web cameras forming a binocular stereovision system. The weed-crop discrimination process is based on 3D stereo vision plant height measurements. A distance threshold can filter the plant which can be analyzed with respect to their height and colour [1]. The stereo vision system allows obtaining simultaneous information about colour and distance. The stereoscopic technique has been used by several researchers in the field of automation in agriculture [2–5]. Fig. 1 shows the plant arrangement used in our experiment for 3D stereo vision measurements.


Introduction
Computer vision research is targeting toward agricultural robotics applications.Our vision system contains two web cameras forming a binocular stereovision system.The weed-crop discrimination process is based on 3D stereo vision plant height measurements.A distance threshold can filter the plant which can be analyzed with respect to their height and colour [1].The stereo vision system allows obtaining simultaneous information about colour and distance.The stereoscopic technique has been used by several researchers in the field of automation in agriculture [2][3][4][5].Fig. 1 shows the plant arrangement used in our experiment for 3D stereo vision measurements.

Vision-based object localization
For 3D object localization, distance information is needed.This chapter discusses two techniques for object localization.First technique shows how to extract depth information from a single monocular image by using a circle with known diameter.Second technique shows how to provide depth information from a binocular stereovision system.

Distance measurement using circle (first technique).
This technique estimates distance from a sin-gle monocular image.For this task, we use a circle with known diameter."Why use a circle?"Answer: The geome-tric properties of circle are useful for distance estimation.Circle projection onto picture plane is an ellipse.We started algorithms development from the observation that the major axis length value of ellipse (circle projection) remains constant for all circle positions and orientations, if circle center (P_world) is placed on the same plane, which is parallel to the image plane.All circles having circle centers on the same plane (parallel to image plane) provides identical scale factor values.Circle centers placed on different planes (parallel to image plane) provides distinct scale factor values.Scale factors are necessary to make the conversion from 2D image coordinates to 3D world coordinates (Fig. 1).
(1)-( 7) are presented in a simplified form, based on pinhole camera model.In practice, algorithms are more complex, because it contains more intrinsic camera parameters.In most cases, it exist two focal lengths; first focal length corresponds to horizontal FOV (field of view) and the second to vertical FOV.In order to remove lens distortion, image correction must be done.The values of intrinsic camera parameters results from intrinsic camera calibration process.

Distance measurement using stereo vision (second technique).
This technique shows how to provide distance information from a binocular stereo vision system (Fig. 3).First, intrinsic and extrinsic camera calibration must be done.The values of the rotation matrix from relation (8) and the values of the translation vector from relation (9) are obtained from extrinsic camera calibration: We assume that the focal length corresponding to horizontal FOV (Field of View) is equal to the focal length corresponding to vertical FOV.Scale factors s_cam1 (related to camera 1) and s_cam2 (related to camera 2), are determined from relation (10).Using relations (4), ( 5) and ( 6), the point can be localized in the 3D scene, related to camera1 coordinate system and to camera2 coordinate system.The distance from camera focal point to the world point is computed with the relations (7).This relation is used to compute the distance to P_world from focal point F1 of camera1, and the distance to P_world from focal point F2 of camera2.The parameters can be visualized in the geometrical representation from Fig. 3 .

Camera calibration
Camera calibration is the first step towards computational computer vision.The mapping between the coordinates associated with the 2D image and the corresponding coordinates in the world system is given in terms of perspective transformations that involve the intrinsic and extrinsic camera parameters.
Intrinsic camera calibration.Intrinsic parameters are associated with optical and geometrical aspects of the camera such as focal length, principal point, skew, and lens distortions.A high precision intrinsic camera calibration can be done by using "Camera Calibration Toolbox for Matlab" [6].We determined intrinsic parameters of two 2D-color web cameras (type Logitech C160).Extrinsic camera calibration.Extrinsic parameters are referred to as the position and orientation of the camera with respect to a reference frame.We developed an original extrinsic camera calibration method to estimate the extrinsic parameters by using four circles with known diameters.The four circles are arbitrary placed in the 3D scene, but the co-planarity of all four circle centers must be avoided.A circles arrangement example is presented in Fig. 4.
The goal of extrinsic camera calibration is to determine the values of the rotation matrix and translation vector.This aim can be reached by using the proposed calibration method.The intrinsic parameters of camera1 and camera2 must be introduced as inputs.The rotation matrix and the translation vector are determined by using the mathematical development presented in [7].

Ground plane localization
Plant height is considered to be the maximum distance between identified plant points and the ground plane.For this reason, we had to determine ground plane equation with respect to camera1 coordinate system.The plane can be defined through three non-collinear points; P1(x1, y1, z1), P2(x2, y2, z2), and P3(x3, y3, z3).The determinant, expressed by the relation (11), is zero.Plane equation is expressed by the relation (12): Coefficients values are given by the determinants: The problem is how to determine the 3D coordinates of three points that are located on the ground plane.For this reason, we use three circles with known diameter, which are located on the ground plane.We consider that points P1, P2 and P3 correspond to the centers of three circles.Fig. 5 presents four samples of circles arrangements.The 3D coordinates of the circles centers can be determined; if the values of all three circle diameters are known.

Weed-crop discrimination
Weed and crop are discriminated based on plant height and green colour (Fig. 6).The distance between a point on plant and ground plane is determined by using relation (17 Function harris.m[8] detects corners."To cater for image regions containing texture and isolated features, a combined corner and edge detector based on the local autocorrelation function is utilized."[9].Function matchbycorrelation.mmatch image feature points by correlation.Only points that correlate most strongly with each other in both directions are returned.[8] Function ransacfitfundmatrix.m fits fundamental matrix, using RANSAC, to a set of putatively matched image points [8].Function distance_camera2point_calculation.m computes the value of distance from camera1 to a point placed in the 3D scene.Function distance_point2ground_calculation.m computes the value of distance from a point placed in the 3D scene to the ground plane.Function plant_detection.mdetects green objects in the image by using a green colour filter.

Conclusions
The results shows that it is possible to discriminate weed from crop using a high threshold.The method can only be used when there are significant differences between crop height and weed height.This method needs further investigations to decrease processing time per image.Also, the image matching program needs to be improved.

Fig. 1 .
Fig. 1.Plant height measurements using stereo vision Fig. 2 shows a simplified geometric representation, where circle_Diameter_world denotes circle diameter length [mm]; ellipse_MajorAxisLength_image denotes major axis length [pixel] of the ellipse that corresponds to circle projection onto image plane; M_im denotes principal point; P_im denotes image point coordinates related to image coordinate system; P_im_cam denotes image point coordinates related to camera coordinate system; s denotes scale factor [mm/pixel]; P_world denotes world point related to camera coordinate system; f denotes focal length [pixel]; d_FP_world denotes distance from focal point to circle center [mm].The algorithms are developed based on proportionalities between image elements and world elements.Scale factor, denoted by s, is the ratio of circle diameter length [mm] to ellipse major axis length [pixel]:

Fig. 6 .Camera 2 Fig. 4 .Fig. 5 .
Fig. 6.Image processing for weed-crop discrimination ). P is a point on the plant which has the coordinates