# EasySegment Unsupervised **EasySegment Unsupervised** is the deep learning tool part of the **EasySegment** segmentation library of **eVision**. It detects and segments defects in images. This tool trains in an unsupervised way. This means that it is trained only with good images. So it does not require any ground truth segmentation of the defects. To create an **EasySegment Unsupervised** tool: * API: [`EUnsupervisedSegmenter`](../../../doxygen_cpp/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenter.html) [.NET](../../../doxygen_cs/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenter.html) [Python](../../../doxygen_python/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenter.html) * To create an **EasySegment Unsupervised** project in **Deep Learning Studio**: 1. Start **Deep Learning Studio**. 2. Create a new project and select `EasySegment Unsupervised` in the `New deep learning project` dialog. ![image](assets/images/easydeeplearning/easysegment_1.webp) ## Properties ![image](assets/images/easydeeplearning/easysegment_3.webp) The unsupervised segmenter tool has 6 parameters: 1. The `Good label` is the name of the class that contains the good images. In the API: use [`SetGoodLabel`](../../../doxygen_cpp/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenter.html#a8750a0372d81090910ae5d11083e44c7) [.NET](../../../doxygen_cs/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenter.html#a7434495494e277e7614bff82b3c1d1eb) [Python](../../../doxygen_python/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenter.html#a754481df9457f8d4ecc9a9176e81853a). In **Deep Learning Studio**, the `Good label` is a project property that you must select in the `Dataset and annotations` tab below the list of labels. ![](assets/images/easydeeplearning/easysegment_7.webp) 2. The `Capacity` of the neural network (default: `Normal`) represents the quantity of information it is capable of learning. A larger capacity makes the tool slower. In the API: * The capacity is represented by the enumerate type [`EUnsupervisedSegmenterCapacity`](../../../doxygen_cpp/namespaceeVision_1_1EasyDeepLearning.html#ac274b87de0aedc657d1a662021f03200) [.NET](../../../doxygen_cs/namespaceeVision_1_1EasyDeepLearning.html#a36dca6debf133df55e03d52e3bc7bacd) [Python](../../../doxygen_python/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenterCapacity.html). * [`SetCapacity`](../../../doxygen_cpp/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenter.html#afc0a21715850efe2f0938042ecc1c42c) [.NET](../../../doxygen_cs/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenter.html#a8e06147a7de1eed453e539e9048096b7) [Python](../../../doxygen_python/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenter.html#abc335dfadb043e0a1bd4e1f2745b025d) sets the capacity of the tool. 3. The `Image type` (default: `Monochrome (1 channel)`): In the API: * To use monochrome (grayscale, 1 channel) images, set [`SetForceGrayscale`](../../../doxygen_cpp/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenter.html#a695437c832162d1bcead1701b7b4bb20) [.NET](../../../doxygen_cs/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenter.html#a0d0734f94def8070b908916dcdf2c5d3) [Python](../../../doxygen_python/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenter.html#af0b943a353b64003627f244fbe6359ce) to `true`. * To use color (3 channels) images, set [`SetForceGrayscale`](../../../doxygen_cpp/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenter.html#a695437c832162d1bcead1701b7b4bb20) [.NET](../../../doxygen_cs/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenter.html#a0d0734f94def8070b908916dcdf2c5d3) [Python](../../../doxygen_python/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenter.html#af0b943a353b64003627f244fbe6359ce) to `false`. 4. The `Sampling density` ([`SetSamplingDensity`](../../../doxygen_cpp/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenter.html#a71f8b8b7358c1dc42f892dcaf7318f89) [.NET](../../../doxygen_cs/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenter.html#a0ea0447dafb6227975fd02702f5dee83) [Python](../../../doxygen_python/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenter.html#af735ef9a4bcc42ec4ee363409cf72470)) is the parameter of the sliding window algorithm used to process whole images using patches of size ([`EUnsupervisedSegmenter::SetPatchSize`](../../../doxygen_cpp/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenter.html#ac071eae191fc0773b234149a75db3ba0) [.NET](../../../doxygen_cs/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenter.html#aa0fc513901b051b97c9cffdbb6c65a0e) [Python](../../../doxygen_python/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenter.html#aee296a2cf43e4e85439538adace2579a)). * It indicates how much overlap there is between the image patches: 100 - (100 / `SamplingDensity`) (%) * In practice, the stride between 2 consecutive patches is: `PatchSize` / `SampleDensity` (pixels) 5. The `Patch size` ([`SetPatchSize`](../../../doxygen_cpp/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenter.html#ac071eae191fc0773b234149a75db3ba0) [.NET](../../../doxygen_cs/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenter.html#aa0fc513901b051b97c9cffdbb6c65a0e) [Python](../../../doxygen_python/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenter.html#aee296a2cf43e4e85439538adace2579a)) is the size of the patches processed by the neural network. * By default, the patch size is determined automatically from the images in the training dataset. * You can also select the resolution of the patch size from the drop down list. ![image](assets/images/easydeeplearning/easysegment_6.webp) 6. Use the `Scale` ([`SetScale`](../../../doxygen_cpp/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenter.html#ab193047bca00915303fa20f882b99a47) [.NET](../../../doxygen_cs/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenter.html#ac31a7340dd6f3416640569d4f0ee1303) [Python](../../../doxygen_python/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenter.html#a3e1389e485adeeb1ea9de12dfff78e13)) to automatically resize your images to a lower resolution and accelerate the processing. In **Deep Learning Studio**: * If the dataset contains images with different resolutions, the `Input width` and the `Input height` indicate the range of the resolutions with the given scale. * If all the images in the dataset have the same resolution, set either the `Input width` or the `Input height` to change the scale. ## Training To train your tool, see [Training a Deep Learning Tool](../easydeeplearning/training_a_deep_learning_tool.html.md). ## Validating the Results There are 2 types of metric for the unsupervised segmentation tool: * ***Unsupervised*** metric only uses the results of the tool on good images. There is only one unsupervised metric: the error. * ***Supervised*** metrics require both good and defective images. The supervised metrics are the AUC (Area Under ROC Curve), the ROC curve, the accuracy, the good detection rate (also called the true negative rate), the defect detection rate (also called the true positive rate). The unsupervised segmentation tool computes a [`classification score`](../../../doxygen_cpp/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenterResult.html#a20e805e00f2b5ce2b19d6635d8aadfea) [.NET](../../../doxygen_cs/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenterResult.html#a75e3d442869cb477cd0b780ecf1aa7c0) [Python](../../../doxygen_python/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenterResult.html#ae62b00346cbf372dc5cbb895e1223ab9) for each image. The label of a result is obtained by thresholding this score with the segmenter [`classification threshold`](../../../doxygen_cpp/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenter.html#a66998236f889481286833b3529c62345) [.NET](../../../doxygen_cs/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenter.html#a90f6523b90e3416fcbca63483417a3af) [Python](../../../doxygen_python/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenter.html#a986015980cda0c925eb52d30e9d4f1b3). So, the supervised metrics also depends on the value of this classification threshold. The ROC curve (Receiver Operating Characteristic) is the plot of the defect detection rate (the true positive rate) against the rate of good images classified as defective (also called the false positive rate). It is obtained by varying the classification threshold. The ROC curve shows the possible tradeoffs between the good detection rate and the defect detection rate. The area under the ROC curve (AUC) is independent of the chosen classification threshold and represents the overall performance of the tool. Its value is between 0 (bad performance) and 1 (perfect performance). ### Deep Learning Studio * In the `Tools` tab, the metrics `Best validation error` and `Best validation AUC` are computed during the training on the validation dataset without using data augmentation. The validation error, the training error and the validation AUC are plotted for each iteration. ![](assets/images/easydeeplearning/easyclassify_3.webp) * In the `Validation and results` tab, various metrics, the confusion matrix, a cumulative score histogram, and the ROC curve are displayed. You can also change the classification threshold directly in this tab. * The cumulative score histogram shows the cumulative proportion of good (in green) and defective (in red) images with respect to the scores of the image. * You can change the classification threshold in 3 ways : direct input, dragging the threshold line in the score histogram and selecting a point on the ROC curve. ### API * The metrics are represented by an [`EUnsupervisedSegmenterMetrics`](../../../doxygen_cpp/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenterMetrics.html) [.NET](../../../doxygen_cs/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenterMetrics.html) [Python](../../../doxygen_python/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenterMetrics.html) object that contains the following performance metrics: * The error on good image ([`GetError`](../../../doxygen_cpp/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenterMetrics.html#a57630a0600ce1464b673df90c238e0ea) [.NET](../../../doxygen_cs/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenterMetrics.html#acd28ae45e1d647004260469fc1e924c9) [Python](../../../doxygen_python/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenterMetrics.html#a5376090469996ae504112f9c30d65d61)) * The confusion matrix ([`GetConfusion`](../../../doxygen_cpp/classeVision_1_1EasyDeepLearning_1_1EDeepLearningDefectDetectionMetrics.html#ac79cba10bd81e0de14b3d80614c53247) [.NET](../../../doxygen_cs/classeVision_1_1EasyDeepLearning_1_1EDeepLearningDefectDetectionMetrics.html#a37912166fca1367c662996895cb182fc) [Python](../../../doxygen_python/classeVision_1_1EasyDeepLearning_1_1EDeepLearningDefectDetectionMetrics.html#ab6a6045c97c2db2fa0301141387cecb4)) * If the results for bad images are included in the metrics, [`IsTotallyUnsupervised`](../../../doxygen_cpp/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenterMetrics.html#a6e1126ef5521b16264cc91005522b39c) [.NET](../../../doxygen_cs/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenterMetrics.html#ae30d92c20ec1b8d62aa9099ee239e14d) [Python](../../../doxygen_python/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenterMetrics.html#ae66edbc05dd2fb45f0d5b412991fe8ee) is `false` and the following metrics are also be accessible: * The accuracy ([`GetAccuracy`](../../../doxygen_cpp/classeVision_1_1EasyDeepLearning_1_1EDeepLearningDefectDetectionMetrics.html#a95c10b0c4a0ea7296cf6996dc6551407) [.NET](../../../doxygen_cs/classeVision_1_1EasyDeepLearning_1_1EDeepLearningDefectDetectionMetrics.html#a4e730fab4b8ac957b973bd673451c899) [Python](../../../doxygen_python/classeVision_1_1EasyDeepLearning_1_1EDeepLearningDefectDetectionMetrics.html#ae3156b90c84f109ec11f6e582d93c5d8)) * The Area under ROC curve ([`GetAreaUnderROCCurve`](../../../doxygen_cpp/classeVision_1_1EasyDeepLearning_1_1EDeepLearningDefectDetectionMetrics.html#a98f50db2886bc232bfb429ab7b56b608) [.NET](../../../doxygen_cs/classeVision_1_1EasyDeepLearning_1_1EDeepLearningDefectDetectionMetrics.html#ab52b77fc699a1d6eab2f240ac4cd2c28) [Python](../../../doxygen_python/classeVision_1_1EasyDeepLearning_1_1EDeepLearningDefectDetectionMetrics.html#a68eac005e7938bdccdfe07df62143994)) * The ROC point corresponding to the classification threshold ([`GetROCPoint`](../../../doxygen_cpp/classeVision_1_1EasyDeepLearning_1_1EDeepLearningDefectDetectionMetrics.html#a91f6548696199fbaceae0db3d19413f3) [.NET](../../../doxygen_cs/classeVision_1_1EasyDeepLearning_1_1EDeepLearningDefectDetectionMetrics.html#a2b9c0d8bbb66f15a429134ee4bbafd57) [Python](../../../doxygen_python/classeVision_1_1EasyDeepLearning_1_1EDeepLearningDefectDetectionMetrics.html#a92bc7961b68cf7a36ba13774e49f343c)) ## Applying the Tool to New Images ### Deep Learning Studio * Add new images to the dataset and refresh the results. ![image](assets/images/easydeeplearning/easysegment_8.webp) * Open the `Inference tests` tab to apply the segmenter to new images and display detailed results for these images. ### API * Once the unsupervised segmenter is trained, call [`EUnsupervisedSegmenter::Apply`](../../../doxygen_cpp/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenter.html#aebeed188b80a970d4b67a748c44c0939) [.NET](../../../doxygen_cs/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenter.html#aadb51ff560c97cb70703485ed09fdaa7) [Python](../../../doxygen_python/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenter.html#a1b7a0a94f9ff3cb894791b53c64dfccc) to detect and segment defects in an **eVision** image. This method returns a [`EUnsupervisedSegmenterResult`](../../../doxygen_cpp/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenterResult.html) [.NET](../../../doxygen_cs/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenterResult.html) [Python](../../../doxygen_python/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenterResult.html) object: * [`IsGood`](../../../doxygen_cpp/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenterResult.html#ab869209d163ef8394fc8b38164a2c952) [.NET](../../../doxygen_cs/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenterResult.html#aafdec968af13da066704b71762ef4985) [Python](../../../doxygen_python/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenterResult.html#ade0a980a0fd7915238547175a29f5eea) and [`IsDefective`](../../../doxygen_cpp/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenterResult.html#ad23aa5a7932cb0356647aff73ffeda9f) [.NET](../../../doxygen_cs/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenterResult.html#a5b155ec90ebea0e51ad051fed6d127a3) [Python](../../../doxygen_python/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenterResult.html#aad3d944b05adcf4d79f7251d2a478b9c) returns whether the tool has decided that the image is good or defective according to the [`GetClassificationScore`](../../../doxygen_cpp/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenterResult.html#a20e805e00f2b5ce2b19d6635d8aadfea) [.NET](../../../doxygen_cs/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenterResult.html#a75e3d442869cb477cd0b780ecf1aa7c0) [Python](../../../doxygen_python/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenterResult.html#ae62b00346cbf372dc5cbb895e1223ab9) and the [`EUnsupervisedSegmenter::GetClassificationThreshold`](../../../doxygen_cpp/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenter.html#a66998236f889481286833b3529c62345) [.NET](../../../doxygen_cs/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenter.html#a90f6523b90e3416fcbca63483417a3af) [Python](../../../doxygen_python/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenter.html#a986015980cda0c925eb52d30e9d4f1b3). * [`GetSegmentationMap`](../../../doxygen_cpp/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenterResult.html#aeea7ac82a3bd831c131e288733b4d835) [.NET](../../../doxygen_cs/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenterResult.html#a6b2f4adac3225b200eb7748ca7dbf627) [Python](../../../doxygen_python/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenterResult.html#a381654e7499ea8ccc8d9ca290f147963) returns an [`EImageBW8`](../../../doxygen_cpp/classeVision_1_1EImageBW8.html) [.NET](../../../doxygen_cs/classeVision_1_1EImageBW8.html) [Python](../../../doxygen_python/classeVision_1_1EImageBW8.html) image where all pixels with a value different than 0 are ***defective*** pixels. The value of a defective pixel is proportional to the importance of the defect at that position. * [`GetRegion`](../../../doxygen_cpp/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenterResult.html#a16debb2c8c93fcd5a3cfde403112b280) [.NET](../../../doxygen_cs/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenterResult.html#a33958c00b4dc210bac9b4efce645a959) [Python](../../../doxygen_python/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenterResult.html#a5441248275ca28ae4dda0f6f330d2231) returns an `ERegion` object corresponding to the segmented region of the image (all the pixels of [`GetSegmentationMap`](../../../doxygen_cpp/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenterResult.html#aeea7ac82a3bd831c131e288733b4d835) [.NET](../../../doxygen_cs/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenterResult.html#a6b2f4adac3225b200eb7748ca7dbf627) [Python](../../../doxygen_python/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenterResult.html#a381654e7499ea8ccc8d9ca290f147963) that have a value strictly higher than 0). * [`Draw`](../../../doxygen_cpp/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenterResult.html#a785c2dff4749f7be4d67edc7c2a3020d) [.NET](../../../doxygen_cs/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenterResult.html#a6a62b4867ae89d2efb929dec7ab078a5) [Python](../../../doxygen_python/classeVision_1_1EasyDeepLearning_1_1EUnsupervisedSegmenterResult.html#aeeb509f6f546d2758d21811b71a81ab2) draws the segmentation mask.