# Easy Spot Detector - Detecting Spots with Deep Learning ## Operating Principle #### WARNING Please note that **EasySpotDetector** is deprecated and will be removed in release 27.6. ### Introduction **EasySpotDetector** uses local segmentation to extract salient objects from the background, then a deep learning classifier can sort them. It is applicable to many materials like films, coated surfaces, steel, battery foils, glass… Specific parameters allow you to select the aspect and the size of the defect, as well as the sensitivity of the detection. The segmentation is robust to noise and doesn’t require context training or calibration. The main features are: * Detection of faint defects and contamination, even in noisy images * Fast processing for in-line inspection * Compatible with acquisition from line-scan and 2D cameras * Optional pre-alignment of the region of interest on the part edges * Optional deep learning classification of the defects * Simple and comprehensive C++, C# and Python API * Full featured tool, with samples ### General Principle 1. **EasySpotDetector** performs a simple alignment of the Region of Interest (ROI) from an 8-bit gray scale image. This stage is optional. 2. The spots are extracted inside the ROI. Two types of spots are supported: * “Particle” for compact objects  * “Scratch” for elongated objects  3. Optionally, you can invoke a deep learning classifier for each detected spot at production time. * This classifier produces a label and a probability for each spot. * Using **Deep Learning Studio**, you can train your own classifier, dedicated to your specific use case. 4. At development time, **EasySpotDetector** offers an API to easily populate a classification dataset with the detected spots. * You can open this dataset file in **Deep Learning Studio** for spot annotation and training. * **EasySpotDetector** uses the resulting model at production time to classify the detections. ### Workflow  ## The Alignment The alignment feature in **EasySpotDetector** is an optional step, before the detection of the spots. It aims to locate the part to process inside the image. #### HINT To restrict the processing domain in a more flexible and generic way, use the parameter `Region` of the [`ESpotDetector::Detect`](../../doxygen_cpp/classeVision_1_1ESpotDetector.html#a3416d67167a4f86aeecb64a69e321004) [_.NET](../../doxygen_cs/classeVision_1_1ESpotDetector.html#a4be1466b4dae59820add60859175c382) [_Python](../../doxygen_python/classeVision_1_1ESpotDetector.html#a14f79385bca052894516514e59bc0c1d) method. * The alignment works by fitting an axis aligned rectangle to the object edges. * The nominal rectangle is defined by a center (X, Y) and a size (width and height). * A tolerance (in pixels) sets the search area inside and outside the nominal rectangle.  * **EasySpotDetector** searches the rectangle representing the effective ROI in the area between the inner and the outer nominal rectangles. * By default, the alignment is disabled. * Enable it with [`ESpotDetector::SetEnableAlignment`](../../doxygen_cpp/classeVision_1_1ESpotDetector.html#ae7b794aade4f6a399f661ef91624fa5e) [_.NET](../../doxygen_cs/classeVision_1_1ESpotDetector.html#a20388ac682f5416cc8ae885c0519ba98) [_Python](../../doxygen_python/classeVision_1_1ESpotDetector.html#ac8a46fe19c0b48ee88f036df9bda0e72). * Configure the nominal rectangle with [`ESpotDetector::SetAlignmentArea`](../../doxygen_cpp/classeVision_1_1ESpotDetector.html#a2a8dc0df896d64395d801d1629593c80) [_.NET](../../doxygen_cs/classeVision_1_1ESpotDetector.html#a0b5129083838a1c12a042a0967b82212) [_Python](../../doxygen_python/classeVision_1_1ESpotDetector.html#a6fea09f385549511b3761f365968ae55). * Set the tolerance with [`ESpotDetector::SetAlignmentTolerance`](../../doxygen_cpp/classeVision_1_1ESpotDetector.html#a96bdc6a4c62426f525361e9bcdd9434d) [_.NET](../../doxygen_cs/classeVision_1_1ESpotDetector.html#aa2cfe07c23e7a905db53be0f339729ee) [_Python](../../doxygen_python/classeVision_1_1ESpotDetector.html#a95334dc95e4426b868d31d8f3f608399). * The alignment process is designed to search a rectangle shape, with limited variations in position, size and orientation. * Use the parameter `Guard Band` to further adjust the ROI. * The purpose of `Guard Band` is to exclude borders where detection artifacts could appear, due to non-uniformity, noise, fuzzy edges… * Use the method [`ESpotDetector::SetGuardBand`](../../doxygen_cpp/classeVision_1_1ESpotDetector.html#a512379951a438124e9a87b31522f8e0b) [_.NET](../../doxygen_cs/classeVision_1_1ESpotDetector.html#af4741a9371061338c762253cf2fe110b) [_Python](../../doxygen_python/classeVision_1_1ESpotDetector.html#af5d12aa847e3b5478e666f3c88f7cce7) to set up the guard band parameters.  ## Detecting Particles and Scratches **EasySpotDetector** detects particles and scratches by comparing the average local surroundings with the value of the center pixel. It performs this operation at various scales to segment large and small objects. #### NOTE The definition of the surroundings is different for particles detection and for scratches detection. * To operate the detection, use the method [`ESpotDetector::Detect`](../../doxygen_cpp/classeVision_1_1ESpotDetector.html#a3416d67167a4f86aeecb64a69e321004) [_.NET](../../doxygen_cs/classeVision_1_1ESpotDetector.html#a4be1466b4dae59820add60859175c382) [_Python](../../doxygen_python/classeVision_1_1ESpotDetector.html#a14f79385bca052894516514e59bc0c1d) with a gray scale image or an EROIBW8 as parameter. You can also use an optional `ERegion` to restrict the domain. * The result is returned by [`ESpotDetector::GetSpots`](../../doxygen_cpp/classeVision_1_1ESpotDetector.html#a09da2a19b74a2d9a4cd5c3ca8d74fa5f) [_.NET](../../doxygen_cs/classeVision_1_1ESpotDetector.html#a1b2f6cb968fcc5cdb2d7862561f91c96) [_Python](../../doxygen_python/classeVision_1_1ESpotDetector.html#adc961d422e6535395e72d8a76cfb4280) as a vector of [`ESpot`](../../doxygen_cpp/classeVision_1_1ESpot.html) [_.NET](../../doxygen_cs/classeVision_1_1ESpot.html) [_Python](../../doxygen_python/classeVision_1_1ESpot.html). * An [`ESpot`](../../doxygen_cpp/classeVision_1_1ESpot.html) [_.NET](../../doxygen_cs/classeVision_1_1ESpot.html) [_Python](../../doxygen_python/classeVision_1_1ESpot.html) stores all the data about a detected object: type (particle or scratch), polarity (black or white), position and size, area (in pixels), strength (representing the contrast between the object and its surroundings), the pixel region… ### Particle Detection  * To enable the detection of particles, use [`ESpotDetector::SetEnableParticleDetection`](../../doxygen_cpp/classeVision_1_1ESpotDetector.html#a5639fa4b54b2dd5eddc8b30e4f5af5f0) [_.NET](../../doxygen_cs/classeVision_1_1ESpotDetector.html#abb39c007c3de42684a56ccf1aafa06be) [_Python](../../doxygen_python/classeVision_1_1ESpotDetector.html#a2244fd2e46f9e4756644a8f18435d9cc). * To configure the parameters controlling the detection, use [`ESpotDetector::SetParticleDetectionParameters`](../../doxygen_cpp/classeVision_1_1ESpotDetector.html#ac8f8804a497fe2d8d108155b7b366f47) [_.NET](../../doxygen_cs/classeVision_1_1ESpotDetector.html#a2794b5d4a0bb8462eede522d82bc8f0e) [_Python](../../doxygen_python/classeVision_1_1ESpotDetector.html#a4545f99cf8f32c00fa669c6192e9a9d8): * **Polarity** ([`EParticleDetectionParameters::Polarity`](../../doxygen_cpp/structeVision_1_1EParticleDetectionParameters.html#ad58570994c51001447c142357a467914) [_.NET](../../doxygen_cs/structeVision_1_1EParticleDetectionParameters.html#a66ccff611c4dcf5831d3508419f987d6) [_Python](../../doxygen_python/classeVision_1_1EParticleDetectionParameters.html#aa3b9295677f5de0187cfba76787021aa)): sets to `Black`, `White` or `Any`, depending on the gray values of the object compared to its surroundings (darker, lighter or both). * **Threshold** ([`EParticleDetectionParameters::Threshold`](../../doxygen_cpp/structeVision_1_1EParticleDetectionParameters.html#a90d1b51bb050ff4922f66ebe68502cbd) [_.NET](../../doxygen_cs/structeVision_1_1EParticleDetectionParameters.html#ac07507819b8c4ec099b715be710976f2) [_Python](../../doxygen_python/classeVision_1_1EParticleDetectionParameters.html#a7d5b3f5fb8c17689627115869168a5bb)): the required gray scale difference between the object pixels and the object surroundings. * A lower value means a higher sensitivity (detection of faint objects). * A higher value means a lower sensitivity (when the contrast between the objects and the background is significant). * **Minimum and Maximum sizes** ([`EParticleDetectionParameters::MinSize`](../../doxygen_cpp/structeVision_1_1EParticleDetectionParameters.html#acaeaa30cec19963904c662ad369860af) [_.NET](../../doxygen_cs/structeVision_1_1EParticleDetectionParameters.html#a94227d6f17899a62bac66c76dc0997ca) [_Python](../../doxygen_python/classeVision_1_1EParticleDetectionParameters.html#ad092f4705f7457ee7629abc8ea40b040) and [`EParticleDetectionParameters::MaxSize`](../../doxygen_cpp/structeVision_1_1EParticleDetectionParameters.html#a59a5927595dbcf6ca453d846f073d6e2) [_.NET](../../doxygen_cs/structeVision_1_1EParticleDetectionParameters.html#a6ccf6b61b5fe589c456a7cb121b024e5) [_Python](../../doxygen_python/classeVision_1_1EParticleDetectionParameters.html#afd753cfe4100147178f092df61a0516b)): to select the detected objects based on their pixel size. * The width and height of the oriented bounding box is considered. * To match the size criteria, the width or height must be larger or equal to the given minimum size and the width and height must be smaller or equal to the given maximum size. * **Minimum area** ([`EParticleDetectionParameters::MinArea`](../../doxygen_cpp/structeVision_1_1EParticleDetectionParameters.html#ae745a9e128bf6c1f7e1419ed32aa8ddf) [_.NET](../../doxygen_cs/structeVision_1_1EParticleDetectionParameters.html#a90b0723eedd13d2c40752074e14af0d3) [_Python](../../doxygen_python/classeVision_1_1EParticleDetectionParameters.html#ad857a1605224ab3e55ee1eea0872901a)): the minimum number of pixels of the object to be detected.  ### Scratch Detection  A scratch is an elongated object, with length largely greater than the width. * To enable the detection of scratches, use [`ESpotDetector::SetEnableScratchDetection`](../../doxygen_cpp/classeVision_1_1ESpotDetector.html#ad8377a0beb13bb7bf9aa90ed859322e1) [_.NET](../../doxygen_cs/classeVision_1_1ESpotDetector.html#a5c669cd5b3bf74486b6cc32f2dfad496) [_Python](../../doxygen_python/classeVision_1_1ESpotDetector.html#acc76829bdc723ca2ec92f4ddc80f9699). * To configure the parameters controlling the detection, use [`ESpotDetector::SetScratchDetectionParameters`](../../doxygen_cpp/classeVision_1_1ESpotDetector.html#adc37ca1d007afcb0b025e160095d4bf2) [_.NET](../../doxygen_cs/classeVision_1_1ESpotDetector.html#acf392b947a7489b2c306f6fd9d91764c) [_Python](../../doxygen_python/classeVision_1_1ESpotDetector.html#a3b3e2566145947a0c2d63d64613bda88): * **Polarity** ([`EScratchDetectionParameters::Polarity`](../../doxygen_cpp/structeVision_1_1EScratchDetectionParameters.html#a6e115603c6911a88a56c9eceeb768602) [_.NET](../../doxygen_cs/structeVision_1_1EScratchDetectionParameters.html#afcc86a2154cea23bef43f1b87a2e188d) [_Python](../../doxygen_python/classeVision_1_1EScratchDetectionParameters.html#ab302a5930e2bacf18928324e1f79312a)): sets to `Black`, `White` or `Any`, depending on the gray values of the object compared to its surroundings (darker, lighter or both). * **Threshold** ([`EScratchDetectionParameters::Threshold`](../../doxygen_cpp/structeVision_1_1EScratchDetectionParameters.html#ab836a7270035f4f6b81c6e12a355b1c7) [_.NET](../../doxygen_cs/structeVision_1_1EScratchDetectionParameters.html#a464a9b2d27dd965537f6c1d846d1eb0b) [_Python](../../doxygen_python/classeVision_1_1EScratchDetectionParameters.html#a410eb37021f4e9734a7628daee717900)): the required gray scale difference between the object pixels and the object surroundings. * A lower value means a higher sensitivity (detection of faint objects). * A higher value means a lower sensitivity (when the contrast between the objects and the background is significant). * **Nominal width** ([`EScratchDetectionParameters::NominalWidth`](../../doxygen_cpp/structeVision_1_1EScratchDetectionParameters.html#aeb5f52b8c20d67f48ebb6997c997951f) [_.NET](../../doxygen_cs/structeVision_1_1EScratchDetectionParameters.html#ad9f1888b2be60b83c2173a50d3d198c5) [_Python](../../doxygen_python/classeVision_1_1EScratchDetectionParameters.html#a53f3ba0f80041f5739100bb4481bd1bd)): the typical width of the scratch (in pixels). * **Minimum length** ([`EScratchDetectionParameters::MinLength`](../../doxygen_cpp/structeVision_1_1EScratchDetectionParameters.html#aee05e0e9ef9199fad942c13c892d6a66) [_.NET](../../doxygen_cs/structeVision_1_1EScratchDetectionParameters.html#ac4f88155d5fefe55a2757f2740c5e985) [_Python](../../doxygen_python/classeVision_1_1EScratchDetectionParameters.html#ab71f0ceb5e68a5419d2f4e20e4e7e48c)): the minimum length (in pixels) to be considered as a valid scratch.  ## Using a Deep Learning Classifier After the detection of spots, you can submit them to a custom deep learning classifier. * You must train this classifier specifically for the inspection task (inspection of battery foil or fabric will have different classifiers). The purpose of this process is to get a class that represents/qualifies the object, for example: * To confirm or discard the detection. * To evaluate the severity of the defect. * To classify objects based on their aspect.  1. In the class [`ESpotDetector`](../../doxygen_cpp/classeVision_1_1ESpotDetector.html) [_.NET](../../doxygen_cs/classeVision_1_1ESpotDetector.html) [_Python](../../doxygen_python/classeVision_1_1ESpotDetector.html), use the method [`ESpotDetector::SetDLClassifierPath`](../../doxygen_cpp/classeVision_1_1ESpotDetector.html#a860e4a808523cb6cf7a54f9eb876e489) [_.NET](../../doxygen_cs/classeVision_1_1ESpotDetector.html#a51588aad4f97c7a1fae75b31b7dbf696) [_Python](../../doxygen_python/classeVision_1_1ESpotDetector.html#a100a34f68a39b3557e81cb447b5a70d1) to configure a deep learning classifier. * All further detected spots are passed to this classifier. * The resulting label and probability are stored in the class [`ESpot`](../../doxygen_cpp/classeVision_1_1ESpot.html) [_.NET](../../doxygen_cs/classeVision_1_1ESpot.html) [_Python](../../doxygen_python/classeVision_1_1ESpot.html) (as [`ESpot::GetDLLabel`](../../doxygen_cpp/classeVision_1_1ESpot.html#aacf2209b38ce1c84b1978edf82c1d967) [_.NET](../../doxygen_cs/classeVision_1_1ESpot.html#acf63d90ee89dd5d0a3b0714794ff55bb) [_Python](../../doxygen_python/classeVision_1_1ESpot.html#a49bd3673dec69bd256329f5767a4c3cd) and [`ESpot::GetDLProbability`](../../doxygen_cpp/classeVision_1_1ESpot.html#a18246b4681da2936924b72c9de140062) [_.NET](../../doxygen_cs/classeVision_1_1ESpot.html#aecd965031afec9a799c026a864928c0d) [_Python](../../doxygen_python/classeVision_1_1ESpot.html#adbe362d673c1d22fbff774dadb9f6a73)). In **EasySpotDetector**, the typical workflow with a deep learning classifier is: 1. Gather samples of detected objects in a deep learning project. 2. Open the deep learning project in **Deep Learning Studio** to annotate the samples and train a new classifier. 3. Use the trained classifier at production time in **EasySpotDetector**.  1. After calling the method [`ESpotDetector::Detect`](../../doxygen_cpp/classeVision_1_1ESpotDetector.html#a3416d67167a4f86aeecb64a69e321004) [_.NET](../../doxygen_cs/classeVision_1_1ESpotDetector.html#a4be1466b4dae59820add60859175c382) [_Python](../../doxygen_python/classeVision_1_1ESpotDetector.html#a14f79385bca052894516514e59bc0c1d), use the method [`ESpotDetector::AddDetectedSpotsToDLProject`](../../doxygen_cpp/classeVision_1_1ESpotDetector.html#a0858b6ee3f825f75b39170518f5217b2) [_.NET](../../doxygen_cs/classeVision_1_1ESpotDetector.html#a74f9d91065dd645790ada237507435f5) [_Python](../../doxygen_python/classeVision_1_1ESpotDetector.html#ac8b4647e875a594c173787ab3b3542f0) to fill a [`EDeepLearningProject`](../../doxygen_cpp/classeVision_1_1EasyDeepLearning_1_1EDeepLearningProject.html) [_.NET](../../doxygen_cs/classeVision_1_1EasyDeepLearning_1_1EDeepLearningProject.html) [_Python](../../doxygen_python/classeVision_1_1EasyDeepLearning_1_1EDeepLearningProject.html) with the newly detected spots: * The image used by [`ESpotDetector::Detect`](../../doxygen_cpp/classeVision_1_1ESpotDetector.html#a3416d67167a4f86aeecb64a69e321004) [_.NET](../../doxygen_cs/classeVision_1_1ESpotDetector.html#a4be1466b4dae59820add60859175c382) [_Python](../../doxygen_python/classeVision_1_1ESpotDetector.html#a14f79385bca052894516514e59bc0c1d) is copied to the subdirectory `Images` and linked in the project. #### NOTE * Automatic image names are generated unless you set the parameter `imageName`. * All images must have a different name. * All the detected spots are added to the project as ROIs of the image, with a label depending on the spot type (particle or scratch). * Use the optional parameter `roiExtension` to increase the ROI size by a pixel amount. It could help the deep learning inference to have a ‘context’ around the ROI. * Use the optional parameter `noDefectSamples` to set the number of extra ROIs added to the project and representing areas without defect. The label of these areas is given by the parameter `noDefectLabel`. 2. After adding enough spots to the deep learning project (depending on the context, between 20 and 200 samples per class are recommended), you can open the project with **Deep Learning Studio**. #### NOTE To annotate and train a deep learning classifier, see:
\\\\ [`Deep Learning Tools - Inspecting Images with Deep Learning`](../deep_learning/easydeeplearning/index.html.md) > [`Workflow`](../deep_learning/easydeeplearning/index.html.md). 3. At production time, set the trained classifier tool with the method [`ESpotDetector::SetDLClassifierPath`](../../doxygen_cpp/classeVision_1_1ESpotDetector.html#a860e4a808523cb6cf7a54f9eb876e489) [_.NET](../../doxygen_cs/classeVision_1_1ESpotDetector.html#a51588aad4f97c7a1fae75b31b7dbf696) [_Python](../../doxygen_python/classeVision_1_1ESpotDetector.html#a100a34f68a39b3557e81cb447b5a70d1). ## See Also Easy to understand, ready to copy code snippets.