# Laser Triangulation and Extraction This section explains the basics of laser triangulation and laser line extraction. ## Laser Triangulation In a laser-line triangulation system, a laser line is projected on the object to measure. A camera is viewing the laser line from a different angle and position. The line deformation observed by the camera contains the shape information of the measured object.  Scanning the object consists in moving it under the laser line and recording multiple images. From the scanning you can reconstruct its 3D shape.  ### Occlusions Using the laser triangulation method, the laser may be unable to reach some parts of the object or the camera may be unable to view them. This is called occlusion. * On the left illustration, the camera does not see the bottom of the hole, inducing camera occlusion. * On the right illustration, the laser does not reach the bottom of the hole, inducing laser occlusion.  #### HINT You can limit or avoid occlusions by using advanced scanning methods, for example by using two cameras or two lasers. ## Laser Line Acquisition Pipeline The 3D acquisition pipeline starts with the acquisition of a laser line profile and ends up with the point cloud, mesh or ZMap. The source material for 3D processing is the depth map, coming from a Coaxlink Quad 3D-LLE or generated from a list of images. 3 types of depth map are available, one for each different pixel coding scheme (8, 16 or 32 bits).  Some processing methods can use the depth map directly, but most measurement and matching processes need metric, distortion-free representations. Calibration of the laser triangulation setup is therefore required. Calibration is used to turn the depth map into a point cloud or mesh expressed in a metric space that we call “world space”.  A point cloud is a list of 3D points, expressed in a world space coordinate system. The point cloud can be projected on a plane, producing a ZMap, which is a convenient and effective representation for 2D processing with a metric scale.  The following sections describe the classes and methods useful for a 3D workflow. The [`Measuring a Remote Controller`](../../../resources/application_examples/measuring_a_remote_controller.html.md) application goes through this processing pipeline. ## Laser Line Extraction A Laser Line Extraction (LLE) algorithm is required to create a depth map from a sequence of profiles of the object captured by the camera sensor. The objective of an LLE algorithm is to measure the line position along a vertical profile in every column of a sensor frame, within a user-defined region of interest (ROI). For every step of the object position, the detection analyzes each column of a frame individually and produces a row of output positions, stored as gray values. The figure below illustrates this process.  The class [`ELaserLineExtractor`](../../../doxygen_cpp/classeVision_1_1Easy3D_1_1ELaserLineExtractor.html) [_.NET](../../../doxygen_cs/classeVision_1_1Easy3D_1_1ELaserLineExtractor.html) [_Python](../../../doxygen_python/classeVision_1_1Easy3D_1_1ELaserLineExtractor.html) provides the laser line extraction functionality. Use the method [`ELaserLineExtractor::SetAnalysisMode`](../../../doxygen_cpp/classeVision_1_1Easy3D_1_1ELaserLineExtractor.html#ab4eaebb9693d2226717125aaf62ab440) [_.NET](../../../doxygen_cs/classeVision_1_1Easy3D_1_1ELaserLineExtractor.html#af967aa0e6693dea516d58238da2d803e) [_Python](../../../doxygen_python/classeVision_1_1Easy3D_1_1ELaserLineExtractor.html#ab7d4e24158053342ede62056d518830f) to select one of the following algorithms to extract the laser line (see below for more details): * **Maximum detection** returns the position of the pixel of maximum intensity. It’s the fastest method but it doesn’t support sub-pixel precision. * **Peak detection** approach detects local maxima. If several maxima are detected, the one with the highest intensity is returned. The position is returned with sub-pixel precision. * **Center of gravity** algorithm is suitable when the laser line is spread over several pixels. The position is returned with sub-pixel precision. #### HINT You can also set a threshold to exclude pixels with low intensity. The line position returned by the laser line extraction algorithms is relative to the bottom of the region of interest. So, values in the depth map range from 0 (bottom of the ROI) to the height of the ROI.  ### Laser Line Extraction Methods * Maximum detection The maximum detection algorithm analyzes all the pixels in a ROI column to determine the one with the maximum intensity. The figure below shows the laser line position on a given ROI column.  We also recommend to include in the processing chain: * A low-pass filter to reduce the high frequency variations in the image. * A threshold to eliminate the background noise from the sensor. * Peak detection The peak detection algorithm relies on a discrete simplification of the first derivative function. $\frac{df}{dx}=f(x+1)-f(x-1)=f'(x)$ The $f'(x)$ outputs the slope of a given $f(x)$ along the $x$.  We compute the line position by detecting where $f’(x)$ changes its signal based on the two-point form line equation: $y-y_1=\frac{y_2-y_1}{x_2-x_1}(x-x_1)$ where $(x_1, y_1)$ and $(x_2, y_2)$ are two points on the line with $x_2\ne x_1$, we obtain the following equation for $y = 0$: $x=\frac{x_1y_2-x_2y_1}{y_2-y_1}$ * Center of gravity The center of gravity (CoG) method uses an algorithm that calculates the center of mass of an image object. Also known as “centroid of plane figures”, the CoG is obtained by the following equations: $ \left\{ \begin{aligned} \bar{X} &= \dfrac{\sum ax}{\sum a}\\ \bar{Y} &= \dfrac{\sum ay}{\sum a} \end{aligned} \right. $ where $\overset{—}{X}$ and $\overset{—}{Y}$ are the coordinates of the CoG and $a$ is the pixel intensity along the $x$ and $y$ axes.  ### Low-Pass Linear Filter Optionally, you can apply a low-pass linear filter in front of the line extraction in order to reduce noise and high frequencies in the image. The low-pass filter applies a convolution operator on a 1 $\times$ 3 sliding window. The 3 elements of the convolution kernel (A, B and C) are configurable, accepting any positive integer. The figure below illustrates the positioning of the convolution kernel elements within a given ROI.  You can activate the low-pass filter for any of the laser line extraction methods with the method [`ELaserLineExtractor::SetEnableSmoothing`](../../../doxygen_cpp/classeVision_1_1Easy3D_1_1ELaserLineExtractor.html#abfbe03294d09d09d64097f4e8daca8e8) [_.NET](../../../doxygen_cs/classeVision_1_1Easy3D_1_1ELaserLineExtractor.html#a85f03979c0b99407ed78b1ada7856ceb) [_Python](../../../doxygen_python/classeVision_1_1Easy3D_1_1ELaserLineExtractor.html#a8342961bd3d7535a224cfa568fd0be04). Parameters A, B and C are set with [`ELaserLineExtractor::SetSmoothingParameters`](../../../doxygen_cpp/classeVision_1_1Easy3D_1_1ELaserLineExtractor.html#a5b3fa50fd1666653da186cefec6e5a80) [_.NET](../../../doxygen_cs/classeVision_1_1Easy3D_1_1ELaserLineExtractor.html#ad06e73dfebb74f8cefe97ef62f64eef2) [_Python](../../../doxygen_python/classeVision_1_1Easy3D_1_1ELaserLineExtractor.html#afb1366a23a723efa5131a571d898b44f). ### Depth Map Z Resolution * As explained above, the laser line extraction computes the sub-pixel position of the laser line profile for each column of the region of interest. * This position is encoded in a 16-bit depth map ([`EDepthMap16`](../../../doxygen_cpp/classeVision_1_1Easy3D_1_1EDepthMap16.html) [_.NET](../../../doxygen_cs/classeVision_1_1Easy3D_1_1EDepthMap16.html) [_Python](../../../doxygen_python/classeVision_1_1Easy3D_1_1EDepthMap16.html)). * The resolution controls the way the sub-pixel positions are converted to 16-bit fixed point values (that is how many bits are allocated to represent the fractional part of the sub-pixel position). * On a [`EDepthMap16`](../../../doxygen_cpp/classeVision_1_1Easy3D_1_1EDepthMap16.html) [_.NET](../../../doxygen_cs/classeVision_1_1Easy3D_1_1EDepthMap16.html) [_Python](../../../doxygen_python/classeVision_1_1Easy3D_1_1EDepthMap16.html), use the method [`EDepthMap16::GetZResolution`](../../../doxygen_cpp/classeVision_1_1Easy3D_1_1EDepthMap16.html#a06c1b8151433b868290f5f920675802c) [_.NET](../../../doxygen_cs/classeVision_1_1Easy3D_1_1EDepthMap16.html#a9ef15b113ef1062cb7cbecc17024d15c) [_Python](../../../doxygen_python/classeVision_1_1Easy3D_1_1EDepthMap16.html#a74bba02c743b0173bc19c9799a0bda6f) to retrieve the resolution. It is a floating-point value used to convert the integer pixel value to a real depth value. * By default, [`ELaserLineExtractor`](../../../doxygen_cpp/classeVision_1_1Easy3D_1_1ELaserLineExtractor.html) [_.NET](../../../doxygen_cs/classeVision_1_1Easy3D_1_1ELaserLineExtractor.html) [_Python](../../../doxygen_python/classeVision_1_1Easy3D_1_1ELaserLineExtractor.html) computes the best resolution depending on the height of the region of interest. The following table lists typical resolution values depending on the height of the processed frame or region of interest. | Frame or ROI height | Bits used for the fractional part | Z resolution | |-----------------------|-------------------------------------|--------------------| | 100 | 8 | 1/256 = 0.00390625 | | 200 | 8 | 1/256 = 0.00390625 | | 400 | 7 | 1/128 = 0.0078125 | | 800 | 6 | 1/64 = 0.015625 | | 2500 | 4 | 1/16 = 0.0625 | * Use the optional parameter `zResolution` of the constructor [`ELaserLineExtractor::ELaserLineExtractor`](../../../doxygen_cpp/classeVision_1_1Easy3D_1_1ELaserLineExtractor.html#a92b89971410f80294d51ce6ab56e1546) [_.NET](../../../doxygen_cs/classeVision_1_1Easy3D_1_1ELaserLineExtractor.html#ad3682d5066d901284457265b13afcba8) [_Python](../../../doxygen_python/classeVision_1_1Easy3D_1_1ELaserLineExtractor.html#ad1591eab3765643456bb1288234bcb8b) to fix the Z resolution. * If you do not set the parameter `zResolution`, an algorithm computes the optimal value. Use [`ELaserLineExtractor::GetDepthMap`](../../../doxygen_cpp/classeVision_1_1Easy3D_1_1ELaserLineExtractor.html#ad667525d47af4a392a80edbfa3e8ae80) [_.NET](../../../doxygen_cs/classeVision_1_1Easy3D_1_1ELaserLineExtractor.html#a22bf6187c6266e4135965bafabeb2fdc) [_Python](../../../doxygen_python/classeVision_1_1Easy3D_1_1ELaserLineExtractor.html#a77a7f94d547ab80ea73ed4a603e3ee29) to query the returned depth map and get the effective resolution. * If you are using the hardware laser line extraction with the **Coaxlink Quad 3D-LLE**, the fixed point format is set by the configuration: `8_8` or `11_5` for 8- or 5-bit fractional part. #### SEE ALSO See Coaxlink’s [documentation to LLE processing](https://documentation.euresys.com/Products/Coaxlink/Coaxlink_10_1/Content/coaxlink/functional-guide/coaxlink-lle-core/LLE_Processing_Core_Characteristics.htm).