Aiming at the lack of existing unmanned express robots to measure the size of express items, a machine vision measurement method based on OpenCV is proposed. The contour size of objects is measured by preprocessing the pictures and using the Canny algorithm for edge detection. The measurement system is improved by adding correction coefficients. Through the comparison of the two measurement data before and after, the accuracy of the measurement system has been significantly improved, providing a more accurate measurement method for the unmanned express robot.

Key words

unmanned express robot; OpenCV;Canny; contour detection; size measurement

In recent years, due to the rapid development of the Internet, online shopping has become more and more frequent, and online shopping has grown at an annual rate of more than 20%. From January to June 2020, the national online sales reached 5.15 trillion yuan, a year-on-year increase of 7.3%. The growth rate has improved for four consecutive months. The rapid development of e-commerce also has higher requirements for logistics and distribution. Traditional logistics and distribution have shown some of its shortcomings: the delivery time is not flexible enough, which is embodied in:

• The delivery time of the courier does not match the time of the recipient’s receipt. There are mainly two cases. When the courier delivers, the picker has no time; when the picker has time, the express delivery does not arrive.
• Holidays, especially during the Spring Festival, the shortage of delivery staff leads to the backlog of logistics during holidays;
• Traffic accidents occur frequently due to the pursuit of delivery speed by delivery personnel;
• Since the outbreak of the new crown pneumonia epidemic, “contactless” delivery services have gradually emerged, and Alibaba and JD Company have also launched unmanned express robots for delivery.

In the future “last mile” distribution, the distribution of unmanned courier robots will become more and more popular. A key technology of the unmanned express robot is to measure the size of the express item and recommend a suitable size express box according to the measurement result. There is currently little research in this area.

Liu Shiwei from Hangzhou Dianzi University (2019) proposed a visual inspection system for ceramic substrates based on OpenCV for the measurement requirements of ceramic substrates. Guo Ruifeng, Yuan Chaofeng, etc. (2017) from Xi’an University of Architecture and Technology proposed a single-camera unit size measurement of train cars based on OpenCV for the contour measurement requirements of coal mine train cars. The measurement deviation of the built measurement system is less than 95mm, which can meet the measurement requirements of the occasion.

Jiatong Han from Inner Mongolia University of Technology (2020) proposed a machine vision-based measurement method to solve the problem of height measurement of road vehicles. At present, there has been a lot of research on the use of machine vision to detect the size of the car contour and auto parts. At this stage, the application of machine vision in measurement is becoming more and more mature, but unmanned express robots rarely use machine vision technology to measure the contour of express items. Therefore, for the measurement of the outline size of express items, the paper uses machine vision to measure the outline size of the item.

The image used by the camera for measurement has a corresponding relationship with the object itself. The model is a pinhole imaging model. Image distortion will occur when the camera is imaged. Image distortion will lead to changes in the shape and size of the object, image distortion, and distortion. In order to improve the accuracy of the measurement, it is necessary to correct the lens distortion.

At present, the calibration methods of cameras mainly include linear calibration methods, nonlinear calibration methods, and Zhang Zhengyou’s camera calibration methods. The Zhang Zhengyou camera calibration method, which is between the traditional and self-calibration methods, is one of the most classic calibration methods at present. Therefore, the MATLAB-based Zhang Zhengyou calibration method is used to calibrate the camera.

The side length of the checkerboard square of the calibration board is 17mm. In the calibration process, in order to ensure the authenticity of the results, pictures of the calibration board are collected from different angles and distances.

The pictures taken by the camera are generally saved in the computer as color images in RGB format. The pictures in RGB format have three color channels: R (red), G (green), and B (blue). The channel value of each channel is in In the range of 0-255, the grayscale image has only one color channel, which only reflects the brightness of the pixel. The larger the value, the whiter the pixel, and the smaller the value, the darker it is. Graying the image can reduce the calculation of the computer. Its conversion formula is shown in formula below:

Gray=0.299R+0.578G+0.114B

Image filtering is to suppress the noise signal of the target image under the condition that the details of the target image can be preserved. The image processing result will affect the signal to be processed later. It is an important step in image processing.

The filtering operation can remove and reduce the digitally doped noise in the image, increase the image clarity, and optimize its visual effect.

As a nonlinear filter, bilateral filtering is based on Gaussian filtering, and on this basis, the numerical difference between pixels is considered, and the weighted average value of the gray value of the surrounding pixels is used to replace the gray value in the pixel. Bilateral filtering centered on q can be expressed as:

y(p)—noisy pixel; F—field of size (2r+1)*(2r+1) centered on q; w_σs—spatial kernel; w_σr—value field kernel. The spatial kernel and the range kernel are:

|p-q|²—the distance of the space field; σ_s—the weighted value of the effect strength of the space field.

In formula above: |y(p)-y(q)|²– the difference intensity of the pixel, generally the difference between the gray values; σ_s – the weighted value of the numerical difference intensity.

Edge detection technology is the basic method of processing graphic information in visual processing, which is generally suitable for feature extraction and feature detection. Edge detection can greatly reduce the workload of the image information data that needs to be processed, and the related information is also deleted and retained as the main component of the image. At present, the common first-order differential operators mainly include the roberts operator, the sobel operator, and the prewitt operator. In addition, common detection operators in the field of machine vision include Laplacian operator and Canny operator.

The Canny algorithm is used in this paper. The Canny algorithm adds Gaussian filtering, double-threshold edge point filtering and non-maximum suppression on the basis of the Sobel operator. The basic idea is to select a certain Gaussian filter to smooth the image, and then perform non-maximum suppression processing, and finally obtain an edge image.

The experimental system includes: Operating system: 64-bit Windows 10.

Camera: LeTMC-520 with a resolution of 1280720/640480 pixels.

Computer CPU: 1.6GHz quad-core,

memory: 4G,

hard disk: 500G,

graphics card: AMD Radeon R6 M340DX.

The experiment measured a small box with a size of 8.3cm, a width of 6.6cm and a height of 2.9cm, because the length and width can represent the characteristics of the small box and can detect the measurement effect of the measurement system. Therefore, the length and width of the small box were measured 6 times. Figure 1 shows the effect of the size measurement of the object.

Fig. 1 Effect drawing of object size measurement

And Table 1 shows the size measurement data of the box.

Table 1 Box size measurement data

After 6 measurements of the box, it can be seen from Table 1 that the average measurement length is 8.67cm, and the average measurement width is 7.42cm. It has a length error of 4.27% and a width error of 11%. Through the analysis of the whole experimental process, the main reasons for the measurement error of the experimental system are as follows:

(1) The error of the LeTMC-520 camera itself
In the operation of the measurement system, most of the electronic components inside the camera are resistive components, which will generate thermal electronic noise during operation. Time operation still affects measurement accuracy.

(2) System error
The size of the item is measured by vision, and the measurement process is carried out through the function correspondence in the imaging principle. Therefore, the relevant calculations in the image processing process may bring errors, which are systematic errors and cannot be eliminated by multiple measurements.

(3) Measurement error
When measuring with a camera, the distance between the camera and the object to be measured and the angle of measurement will affect the measurement result. Through experiments, it is found that the influence of the angle has a greater impact on the measurement accuracy. When the camera is facing the object, the measurement size is more accurate, and the measurement error is larger when the angle is inclined.

Through the analysis of the measurement results, it is found that the measured data using this measurement system is larger than the actual size by a certain range, the length measurement of the measuring box is 4.27% larger, and the width size is 11% larger.

Therefore, a correction coefficient is added to the measurement value of the measurement system in this paper, so that it can be corrected in a certain proportion on the basis of the measurement data, so as to reduce the measurement error and make the measurement result more accurate. After the correction, the box was measured 6 times. The measurement effect is shown in Figure 2.

Fig. 2 The effect of size measurement after correction

And the specific data of the measurement are shown in Table 2.

Table 2 Box size measurement data after correction

It can be seen from Table 2 that the average measurement length is 8.26cm, and the average measurement width is 6.72cm. After correction, the length measurement error is reduced to 0.48%, and the width measurement error is reduced to 1.77%. Adding correction factors from measurement data descriptions improves measurement accuracy.

①In this paper, a method for measuring the size of objects with a monocular camera based on machine vision is proposed, which realizes the measurement of the size of objects by a monocular camera. Its precision meets the use of unmanned express robots.

②The innovation of this paper is that the machine vision is applied to the unmanned express robot, so that it can measure the size of the items, and according to the measurement results, the appropriate express packaging box is recommended, which improves the intelligence of the unmanned express robot.

③ The experiment in this paper is ideal for the measurement in the visual direction perpendicular to the camera and the object, but it is not ideal when the camera and the object to be measured have a certain inclination angle. Affected by the experimental equipment, the measurement speed of the existing equipment is slow. In the follow-up application, the computing power of the computer needs to be improved; for the measurement of the inclined angle, the current measurement effect is not ideal, and the algorithm needs to be optimized in the follow-up application to reduce the influence of the inclined angle on the measurement.