Authors: Vamsi Sravanth Mahendrada, Murali Parameswaran
Certificate: View Certificate
This comprehensive exploration centers on the pivotal role of tyres in vehicular performance, particularly in the era of autonomous vehicles. Navigating the intricate dynamics of tyre-road interactions, the study delves into a spectrum of methodologies for effective tyre defect detection. Emphasizing the evolutionary landscape from intelligent tyres with sensor technology to the transformative impact of deep learning, the focus remains on addressing challenges such as imbalanced datasets and limited labeled samples through strategies like transfer learning and domain adaptation. Key studies include innovative approaches like Hybrid-DBN for crack detection, 3D laser scanning for low-visibility defects, and the application of Wasserstein GAN for handling imbalanced datasets. These advancements signal paradigm shifts in the field, integrating hybrid methodologies, inventive data augmentation, and domain adaptation. Real-world scenarios, including tyre tread wear detection and MEMS accelerometer-based monitoring, highlight the practical implications of these methodologies. The conclusion acknowledges persistent challenges and suggests future research directions, advocating for standardized evaluation frameworks and collaborative efforts. The synthesis of insights contributes to the maturation of tyre defect detection research, offering a guide for the development of safer and more reliable vehicles on the road.
Tyres, the unsung heroes of vehicular performance, bear the weight of not just vehicles but also the safety and dynamics of our journeys. In the ever-evolving realm of autonomous vehicles navigating complex traffic scenarios, the tyre-road interaction becomes a crucial orchestrator of driving safety and precise vehicle control. At the heart of this interaction lies the contact patch—an intricate dance between tyre and road that significantly influences vehicle dynamics and trajectory planning . Decoding the language of road conditions, including roughness, potholes, and cracks, becomes pivotal for enhancing the perceptual acuity of autonomous vehicles. Moreover, accurate estimations of tyre forces, pressure, and deformations become the bedrock for dynamic control modules such as the antilock brake system (ABS) & electronic stability program (ESP).
The landscape of tyre-road interaction data collection is primarily shaped by on-board vehicle sensors, laying the foundation for various recognition models that decode these interactions. A notable stride in this trajectory is the advent of intelligent tyres—born out of the synergy between sensor technology and information technology . By embedding sensors within the tyre structure, intelligent tyres offer a direct and sensitive response to varying road conditions. However, the real-world challenges of rapidly changing environments and demanding tyre working conditions pose formidable challenges to the reliable performance of intelligent tyre systems. This literature review embarks on a focused exploration of tyre defect detection, a critical facet of ensuring tyre quality in the automotive industry. Traditional methods, reliant on manual inspection of X-ray images, ultrasound, microscopy are not only labor-intensive and time-consuming but also susceptible to subjectivity . The transformative prowess of deep learning, especially through convolutional neural networks (CNNs), has ushered in a new era for tyre defect detection, showcasing substantial improvements in both accuracy and speed.
However, the road to robust tyre quality assurance is riddled with challenges. Deep-learning algorithms face major obstacles due to the lack of identified faulty examples and the high cost of data labelling . Moreover, the domain shift problem, a consequence of changing working conditions during tyre production, adds layers of complexity to the defect detection task. In response to these challenges, transfer learning emerges as a beacon of hope, with unsupervised domain adaptation (UDA) methods proving effective, especially in scenarios where the target domain lacks labeled samples. The tyre industry, marked by variations in tread depths, patterns, and X-ray image characteristics due to shifts in working conditions, demands adaptive solutions.
The literature review draws insights from a spectrum of papers, each providing a unique perspective on the nuances of deep-learning-based methods and domain adaptation strategies, particularly within the context of tyre production. In our quest to tackle the domain shift challenges prevalent in the tyre industry, we propose a nuanced dual-domain adaptation strategy. This innovative approach aims to align both marginal and conditional distributions simultaneously, bolstered by a dynamic adaptive factor, elevating feature learning and enhancing model reusability. Additionally, we introduce a tyre defect detection feature extractor, a symbiosis of CNN and multi head self-attention (MSA). This novel fusion, enriched with global context semantic information, aspires to amplify the efficacy of transfer learning.
As we traverse the diverse landscape of tyre defect detection methodologies, the synthesis of insights from various papers not only unravels the advancements in the field but also underscores the need for more robust, context-aware approaches . In contributing to the evolving tapestry of tyre quality assurance, we envision safer and more reliable vehicles navigating our roads, fueled by the knowledge distilled from the intricate dance between tyres and technology.
II. DIFFERENT TYPES OF DEFECTS OF TYRES
Tyres are susceptible to various defects, including structural weaknesses like cracks, cuts, bulges, and separation. These issues compromise the tyre's integrity, increasing the risk of failures and accidents. Irregularities in wear, stemming from factors such as under or overinflation, poor alignment, or suspension problems, further impact vehicle stability and handling, posing safety concerns. External damage from collisions, road hazards, and harsh conditions can also lead to sidewall issues, underscoring the need for vigilance and prompt attention to ensure overall road safety.
The presence of cracks on a tyre, whether on its sidewall, tread, or within tread blocks, signals the aging and deterioration of the rubber. While minor cracks can be a result of extended use, major cracks are often a consequence of prolonged exposure to UV rays and driving on poorly maintained roads. Overlooking these cracks may compromise the tyre's structural integrity, elevating the risk of potential blowouts.
Tyre cuts are more than mere punctures and arise from encounters with sharp objects on the road. These cuts have the potential to escalate into blowouts, presenting an immediate safety hazard. Furthermore, underinflation serves as a common precursor to cuts, emphasizing the significance of maintaining proper tyre pressure to prevent such defects.
Bulges on the tyre's sidewall are a result of impact damage, typically incurred from navigating potholes, speed bumps, or other road hazards at high speeds. This impact causes internal layers to collapse, leading to air leaks and the formation of a noticeable bulge or bubble on the outer layer. Swift attention is imperative to prevent further damage and uphold road safety.
D. Bead Problems
Challenges with the tyre bead, including breakage, chafing, or deformation, may arise during careless mounting onto a rim or if an unsuitable rim is used. Bead chafing can occur when a tyre is fitted onto a dirty or mismatched rim or when operated under overloaded or under-inflated conditions. Proper mounting procedures and regular maintenance are pivotal in forestalling bead-related issues.
E. Tyre Separations
Tyre separation is a grave defect, presenting as a bulge on the shoulder or tread face or as localized wear above the separated region. The primary culprit behind this issue is abnormal heat build-up within the tyre, often triggered by high-speed driving, overloading, or under-inflation. Additionally, water or foreign materials entering the tyre carcass through external cuts can contribute to separation.
F. Sidewall Damages
Instances of sidewall damage usually occur when a tyre encounters road hazards such as metal debris or curbs. Cuts and tears result from encounters with sharp objects, while bubbles and scrapes may be consequences of impact damage or prolonged abrasion. Timely addressing of sidewall issues is critical to preserving tyre integrity and preventing further harm.
G. Sidewall Dimples
Dimples on the tyre sidewall are typically innocuous and primarily an aesthetic concern. With modern radial tyres utilizing cord layers arranged perpendicularly to the direction of travel, these dimples may emerge due to overlaps in the cord layers. While not impacting performance, it is important to distinguish these dimples from more serious sidewall issues.
Among the most perilous tyre defects, blowouts entail rapid air loss. While underinflation remains the leading cause, other contributing factors encompass overloading, unattended tyre cuts and punctures, and impact damage. Regular pressure checks to address underinflation and prompt attention to visible damages are imperative preventive measures against blowouts.
III. ABNORMAL TREAD WEAR
Unusual tread wear patterns, indicative of brake, suspension, or alignment issues, can compromise a vehicle's handling and stability. Misalignment contributes to uneven tyre wear, impacting a vehicle's responsiveness. Correcting abnormal tread wear through proper maintenance and alignment procedures is essential in accident prevention.
A. Toe Wear
Toe wear becomes evident when the tread on one side of a tyre wears down more rapidly than the opposite side along the same circumference.
This type of wear is typically linked to misalignment problems, where the wheels lack parallel alignment. Inadequate toe settings can result in uneven tyre wear due to increased friction. Consistent wheel alignments are necessary to rectify toe wear and uphold consistent tread durability.
B. Camber Wear
Camber wear is identified by uneven tread wear on one side of the tyre compared to the other. This issue arises primarily from improper wheel alignment, particularly when the wheels deviate from the vertical axis. Excessive positive or negative camber angles can create uneven pressure on the tyre, leading to accelerated wear on one shoulder. Regular alignment checks are crucial to addressing camber-related wear and ensuring balanced tyre performance.
C. Center Wear
Center wear signifies a tyre wearing down more rapidly in the center of the tread compared to the edges. Overinflation is a common contributor to center wear as it diminishes the tyre's contact with the road, concentrating pressure on the central area. To prevent center wear, it is imperative to maintain proper tyre inflation levels in accordance with the manufacturer's guidelines, ensuring an even distribution of pressure across the entyre tread.
D. Edge Wear
Edge wear, or shoulder wear, becomes apparent when the outer edges of the tyre tread wear down more quickly than the center. Underinflation is a frequent cause of edge wear, increasing friction on the tyre shoulders. Aggressive driving habits or improper tyre rotation practices can also play a role in edge wear. Regularly monitoring and maintaining the correct tyre pressure and following recommended rotation patterns are crucial steps to mitigate edge wear.
E. Patch Wear
Patch wear becomes noticeable when specific sections of the tyre tread undergo accelerated wear, resulting in patches with reduced tread depth. This type of wear is commonly associated with irregular tyre rotation practices or localized factors, such as consistently carrying heavy loads. The presence of patch wear can compromise the overall performance and traction of the tyre. To prevent and alleviate patch wear, it is crucial to adhere to regular rotation routines, maintain appropriate tyre inflation levels, and follow recommended load capacity guidelines.
F. Cup Wear
Cup wear, also referred to as cupping or scalloping, manifests as cup-shaped depressions on the tyre tread. These irregular depressions typically form at regular intervals around the tyre's circumference. Cupping is frequently linked to a combination of factors, including worn-out suspension components, imbalanced tyre/wheel assemblies, or issues with shock absorbers. Effectively addressing these factors through routine suspension inspections, wheel balancing, and timely replacement of worn components is essential to prevent cup wear, ensuring a smoother and safer driving experience.
IV. DANGERS WITH DEFECTIVE AND WORN-OUT TYRES
Defective tyres pose significant dangers on the road, compromising vehicle safety and increasing the risk of accidents. Some of the potential hazards associated with defective tyres are:
A. Loss of Traction
Defective tyres, especially those with worn-out treads or inadequate tread depth, have reduced traction. This loss of traction can result in poor grip on the road, particularly in wet or slippery conditions, leading to skidding and increased chances of accidents.
B. Increased Stopping Distance
Tyres with defects may not respond effectively to braking, resulting in an extended stopping distance. This compromised braking capability can be critical in emergency situations, increasing the likelihood of collisions.
Structural defects, such as bulges, cuts, or internal damage, can lead to tyre blowouts. A sudden blowout can cause the driver to lose control of the vehicle, resulting in accidents, especially at high speeds.
D. Poor Handling and Stability
Defective tyres can negatively impact a vehicle's handling and stability. Issues like uneven wear, misalignment, or imbalances can lead to unpredictable and unsafe driving conditions, increasing the risk of accidents.
E. Increased Heat Buildup
Tyres with structural or inflation defects may generate excessive heat during normal driving. This can lead to tyre failure, including blowouts, as the increased heat weakens the tyre's structure over time.
F. Risk of Hydroplaning
Worn-out or improperly inflated tyres are more prone to hydroplaning on wet surfaces. Inadequate tread depth reduces the tyre's ability to channel water away, increasing the chances of skidding and loss of control.
G. Handling Issues in Adverse Weather
Defective tyres can exacerbate challenges in adverse weather conditions such as rain, snow, or ice. Reduced grip and compromised performance can make driving in these conditions more hazardous.
H. Increased Likelihood of Accidents
The cumulative effect of various tyre defects can significantly raise the overall risk of accidents. These accidents can result in injuries, damage to the vehicle, and potentially involve other road users.
I. Legal Consequences
Driving with defective tyres may lead to legal consequences. In many places, vehicles with unsafe tyres may be subject to fines, penalties, or even vehicle impoundment. Additionally, accidents caused by defective tyres may result in legal liabilities.
Regular inspection, maintenance, and timely replacement of tyres are essential to mitigate these dangers and ensure optimal road safety. Motorists should be vigilant about tyre condition, including tread depth, inflation levels, and overall structural integrity, to minimize the risks associated with defective tyres.
V. DEEP LEARNING FOR TYRE DEFECT DETECTION
Al-juboori et al.'s paper introduces a groundbreaking "Hybrid-DBN" system for tyre crack detection. Overcoming challenges of overlapping textures, the approach begins with HOG feature extraction, followed by Hybrid Gradient Correlation-Based Feature Selection (HCS) to optimize discriminative features. Comparison with Naive Bayes (NB), Decision Tree (DT) & Random Forest (RF) showcases H-DBN's superiority, especially in addressing overlapping features . Adaptive Correlation Features Selection (ACFS) significantly enhances accuracy, precision, and recall for Decision Tree and Random Forest, reaching 78.8%.
A standout feature is the incorporation of Deep Belief Neural Network (DBN) as a predictive model. Despite challenges with a large feature set, ACFS boosts DBN accuracy by 1%, culminating in an impressive 88% accuracy. H-DBN outperforms standard machine learning and deep learning methods, marking a notable advancement in tyre defect detection.
Dongbeom Ko et al. propose an innovative methodology for training deep learning models to detect anomalies in tyre manufacturing, with a primary emphasis on vent spew errors. The study addresses the inherent limitations of grayscale depth images, particularly their restricted information compared to RGB images . The primary goal is to generate highlight images, input images, stack images, and train images for deep learning (DL) segmentation in four steps.
The overarching goal is to augment the accuracy of vent spew error detection, positioning the proposed methodology against a generic training approach. Evaluation metrics encompass intersection of union (IoU), precision, recall, and F1-score. The crux of the proposed method lies in preprocessing tyre depth images through anomaly detection concepts before engaging in deep learning network training. Data utilization in the study involves 18 types of tyre images, with careful cropping for training and validation. To counteract the scarcity of vent spew error instances, the train data is meticulously balanced between images with and without errors. The training process spans 1000 iterations, employing specific hyperparameters and various CNNs (MobileNets, Resnet-50 & 101).
Results indicate a notable 7% improvement in mean IoU compared to the original method, showcasing the efficacy of the proposed approach. Furthermore, the proposed method demonstrates more stable convergence during training, reducing the time required for a stable mean IoU by 80%. While precision results exhibit a marginal decrease, there is an overall enhancement in F1-score.
Gao et al., proposed a new approach is presented to tackle the issue of dataset imbalance in tyre X-ray defect detection. The proposed method involves leveraging a Wasserstein generative adversarial network (WGAN) for image classification, aiming to address the imbalance by generating high-quality X-ray defect images. Notably, a redesigned deep-CNN is created for better performance, and the WGAN model is trained based on an existing model to extract deep features. By utilizing the balanced dataset with augmentation generated through WGAN assistance, the enhanced network is trained to classify imbalanced tyre X-ray defects effectively . Results from experiments demonstrate the superiority of this method in type and grade classification compared to existing models. This research offers valuable insights into the application of WGAN-assisted image classification for non-destructive tyre defect detection, particularly in industrial contexts such as tyre production lines and repair shops, showcasing advancements in the field.
Wang et al.'s pioneering work on Prototype-Based Supervised Contrastive Learning (PSCL) tackles the crucial issue of label noise in industrial datasets, particularly focusing on tyre defect detection using X-ray imaging sensors. The paper emphasizes the shift towards deep learning for automated inspection in industrial production but acknowledges the inherent challenge of noisy labels, often stemming from manual annotation processes . PSCL introduces a novel approach, utilizing supervised contrastive learning to refine class prototypes, subsequently correcting labels and contributing to the training of robust neural networks. Extensive experiments on synthetic noisy datasets and a tyre dataset from a production line showcase the method's resilience, with PSCL achieving an impressive 89.55% accuracy in real-world industrial scenarios. The authors express optimism about the broader applications of PSCL, envisioning its potential in object detection for handling both noisy labels and anchors in industrial defect detection scenes. This research marks a significant stride in addressing label noise, fostering reliability in deep learning models for industrial quality control.
Zheng et al. conducted a groundbreaking study focusing on tyre defect detection, specifically addressing challenges associated with low-quality and ambiguously shaped defects, such as bubbles, during tyre manufacturing. In response, the researchers proposed an innovative approach utilizing 3D laser scanning technology to create a unique 3D-dataset of tyre surfaces which were scanned. This dataset formed the foundation for an unconventional tyre defect detection framework, integrating the capacity for learning of 2D-detection models with a unique 3D rendering technique .
In order to obtain pixel-by-pixel information regarding local surface orientation, the process entails extracting exact depth dispersion from raw point cloud data and transforming it into a rendered 2D feature map. Rendering 2D pictures, a transformer-based identification pipeline is then applied. This departure from conventional methods represents the first instance of tyre defect detection using 3D data. To assess how successful their approach is, the authors introduced a new 3D tyre dataset, the first of its kind for low-visibility tyre defect detection. Comparative analyses were conducted against existing tyre defect detection methods, particularly those based on X-ray images. The evaluation spanned different data configurations with the proposed pipeline demonstrating superior performance in detecting challenging tyre defects.
Four different types of datasets were used in the experiments: a point cloud dataset, rendered photos, captured RGB images & depth images. Evaluation metrics followed the COCO dataset standards, and comparisons included X-ray-based methods and mainstream 2D detectors. The proposed 3D pipeline outperformed existing X-ray-based methods, particularly excelling in detecting low-visibility defects. Additionally, the paper conducted ablation studies, justifying the effectiveness of their method, including the impact of data augmentation, query generation strategies, and the hyperparameter L. The findings revealed promising results, and the study highlighted areas for improvement, such as addressing missing and wrong detections, and the need for a larger dataset to enhance model performance. The research by Ho & Huang et al. focuses on the critical issue of tyre defect detection in scenarios characterized by limited annotation, a challenge referred to as Tyre Defect Detection with Limited Annotation (TDDLA). To address this problem, the authors propose a novel framework called Self-Supervision and Synthetic data (S3), designed around the Faster-RCNN architecture. The framework comprises two key stages: a self-supervised pretraining phase specifically utilizing the SimCLR approach and a fine-tuning phase using the Faster-RCNN architecture, incorporating limited defect annotations along with synthetic data. Quantitative results demonstrate the efficacy of the proposed approach. Self-supervised pretraining enhances the average F1 score, and the introduction of synthetic data in pretraining further boosts performance. Combining synthetic data in both pretraining and fine-tuning results in a substantial improvement over the baseline . An ablation study investigates the impact of the synthetic/regular ratio, revealing that a ratio of 2 leads to the best F1 score. Synthetic data proves to be effective in addressing data scarcity, resulting in an overall improvement in detection performance. The paper by Vinod et al., addresses the crucial need for effective tyre condition monitoring in vehicles. The proposed tyre condition monitoring system (TCMS) utilizes vibration signals obtained through a low-cost MEMS accelerometer, offering a cost-effective and accurate solution. The study introduces a novel approach by visualizing vibration data through easily interpretable radar plots, enhancing user-friendliness compared to traditional methods. A notable aspect of the study is the inclusion of different inflation pressure cases, examining high, low, and puncture conditions, along with a normal reference state. This comprehensive evaluation enhances the applicability of the proposed TCMS across various real-world scenarios . The experimental methodology involves the exploration of VGG-16, GoogLeNet, AlexNet, & ResNet-50 with systematic variations in hyperparameters. Results reveal that VGG-16 exhibits the highest overall accuracy, while ResNet-50 emerges as the most optimal network, achieving a remarkable classification accuracy of 93.80%. The study recommends ResNet-50 for effective and economical tyre condition monitoring, considering its superior performance across different inflation pressure cases. Additionally, the paper emphasizes the significance of hyperparameter tuning in optimizing pretrained networks for tyre condition monitoring. By systematically varying parameters the research provides valuable insights into enhancing the classification accuracy of deep neural networks.
Zhang et al. present a groundbreaking dual-domain adaptation model for tyre defect detection using X-ray image sensors. Unlike traditional defect detection methods susceptible to performance issues under varying conditions, their model concurrently aligns global marginal and fine-grained conditional distributions. This is facilitated by a dynamic adaptive factor, quantifying the relative importance of these distributions and enhancing feature learning for both intraclass compactness and interclass discriminative capabilities.
The study focuses on tyre defect detection across five common types, utilizing X-ray images from diverse sensors, introducing a significant domain shift . Analyzing in comparison to the most recent domain adaptation algorithms reveals the model's outstanding performance, achieving a remarkable 96.39% detection accuracy. The model proves effective in addressing domain shifts caused by diverse X-ray image sensors, showcasing its potential for superior tyre defect detection. Notably, the model enhances data reusability, reduces the cost of labeled sample collection, and improves detection of challenging defects like bubbles.
Ablation experiments confirm the significance of key components, including the optimal collaborative integration of ResNet50 with Vision Transformer, emphasizing the synergy between deep CNN & multihead self-attention (MSA)-based Vision Transformer. The dual-domain adaptation strategy, considering both marginal and conditional distributions, outperforms single-domain adaptation methods. Feature visualization using t-SNE highlights the model's effectiveness in enhancing feature discriminability between domains of source & target. The proposed model consistently aligns overall data distributions and defect category distributions, effectively addressing marginal and conditional distribution misalignments.
The research conducted by Han et al. introduces an innovative method for detecting tyre tread wear utilizing ML in real-road driving scenarios. Unlike traditional approaches that rely on mathematical models and tyre characteristics, the proposed system employs an intelligent tyre, a preprocessing component, includes a detection component that precisely classifies the amount of tyre wear using a deep-NN. The intelligent tyre samples acceleration signals, transmitting them wirelessly to the tyre wear detection component. This component converts the signals from the time domain to the frequency domain, segments the bandwidth, and extracts features based on mean amplitudes within each segment. These features serve as input for the DNN model, enabling it to classify different levels of tyre wear . In the experimental setup, an acceleration-based intelligent tyre was mounted on a passenger car. Acceleration signals were collected and processed through a data acquisition module and a data processing module. In real-world driving situations, the system showed an excellent performance of 95.51% in identifying tyre tread degradation at speeds that varied between 30 – 80 km/h. Interestingly, the system did not require complicated physical models or numerical computations; instead, it only used ML techniques & preprocessed acceleration signals. The preprocessing of acceleration signals involved multiple samplings of radial acceleration to address noise during real-road driving. The signals were then converted to the frequency domain, and statistical features were extracted, forming the input feature set for the DNN model.
The DNN model, with 11 input nodes and ReLU activation in hidden layers, demonstrated superior performance compared to other machine-learning algorithms such as multiple linear regression (MLR), SVM, KNN & CNN. The DNN achieved an overall avg accuracy of 95.51%, with acceptable running and learning times given the non-real-time nature of tyre wear detection.
The tyre defect detection method outlined in the paper by Liu et al. addresses the challenge of accurately identifying defects in tyre images. Combining characteristics from LBP & HOG, the proposed algorithm leverages the strengths of both edge and texture descriptors. The experimental setup involves a tyre image dataset, with 80% allocated for train & 20% for test, with an SVM classifier. Results indicate that the HOG + LBP + SVM approach achieves a detection accuracy of approximately 84%, showcasing its effectiveness in discerning between normal and defective tyre images. Furthermore, the paper discusses the impact of individual feature-based algorithms, namely HOG + SVM and LBP + SVM, on tyre defect detection. While HOG + SVM exhibits 70% accuracy, it struggles with some defective images being misclassified as normal due to its focus on edge information. On the other hand, LBP + SVM demonstrates higher accuracy (over 12% more than HOG + SVM) by emphasizing texture features, making it effective in identifying defects, particularly small cracks .
The combination of HOG & LBP features is strategically approached, with parameter tuning for optimal performance. The authors find that a combination of 30% weight for HOG and 70% for LBP achieves the highest accuracy, highlighting the complementary nature of these features in describing tyre images. Despite the increased computational complexity associated with feature fusion, the authors argue that the heightened accuracy justifies the method's adoption, particularly as advancements in GPU computing capabilities continue. Wei et al. tackle the intricacies of automatic defect detection within the tyre industry, acknowledging challenges stemming from diverse product textures and characteristics. They propose a Multi-scale Defect Detection Network (MDDN) that integrates semantic-aware and texture-aware networks. This innovative approach aims to capture both high-level semantic features and intricate texture details for enhanced defect detection. Additionally, the authors introduce a distinctive multi-scale preprocessing strategy, involving image blocking, to refine the model's accuracy in characterizing defects .
Ablation experiments underscore the superior performance of MDDN compared to existing methods, showcasing its effectiveness in detecting defects of varying sizes. In a comparative evaluation with cutting-edge object detectors (FCN-8s, FCN-Tyre, SegNet, ResNetDUC, and DRN), MDDN stands out, demonstrating improved accuracy and lower false alarm rates.
The MDDN model, utilizing VGG as its backbone with 300K iterations, achieves remarkable results: 85.19% mPA, 79.47% mIoU, 97.78% wIoU, 0.47 FAR, and 29.15 MAR. These findings underscore the model's robustness in precisely locating defects, positioning MDDN as a promising solution for tyre defect detection in industrial inspection applications.
TyreNet, a novel end-to-end technique created for the real-world application of automated tyre fault identification using X-ray pictures, is presented by Ying Li et al. Despite the advancements in existing tyre defect detection methods, they fall short of meeting the specific demands of real-world applications . Unsupervised methods struggle with intricate tyre specifications and patterns, while supervised methods face challenges due to limited data.
TyreNet addresses these issues by introducing a novel approach that employs object detection models as baselines. Emphasizing the importance of achieving a high recall rate, considering the distinctive characteristics of tyre defects, TyreNet utilizes the Siamese network as part of the downstream classifier, leveraging the periodic features of tyre X-ray images. Experimental findings demonstrate TyreNet's superior performance over SSD, Faster R-CNN, & YOLO in labelled datasets, achieving a significantly higher recall rate. Comparative analyses with mainstream object detection methods reveal noteworthy observations, with TyreNet, particularly when coupled with ResNet-50, emerging as the top performer.
A. Hybrid Adaptive–CFS & DBN (H-DBN)
The core of the hybrid cracked tyre detection system lies in the synergy of three robust techniques: Deep Belief Networks (DBN), Correlation-Based Feature Selection (CFS), and HOG, collectively forming the suggested Hybrid Adaptive–CFS & DBN (H-DBN) architecture.
DBN, a stalwart in neural network technology, consists of cascading RBMs, forming a Deep Neural Network (DNN). With the help of labeled data, this architecture can effectively perform unsupervised pre-training on unlabeled data, enabling later fine-tuning. RBMs provide crucial functions such collaborative feature filtering & dimensionality-reduction (DR) before providing input to a Back-Propagation Neural Network (BPNN).
2. Correlation-Based Feature Selection (CFS)
CFS, a filter method in the feature selection arsenal, operates independently of the chosen classification model. The technique relies on evaluating feature subsets based on the intrinsic correlations between variables. Features exhibiting correlations surpassing a critical value at a 0.05 significance level are cherry-picked for further analysis.
3. Histogram of Oriented Gradients (HOG)
HOG steps in as an adept feature extraction method, particularly efficient for 2D image analysis. HOG is particularly good at expressing texture because it can capture the directed gradients of colors in localized image sections. Cell size, number of blocks, and orientations are some of the criteria that determine how many features are extracted.
4. Integration in H-DBN
The proposed H-DBN model by Al-juboori et al., meticulously weaves these methodologies into a cohesive framework. Preprocessing sets the stage, followed by feature extraction through HOG, which encapsulates critical texture information. The subsequent CFS step strategically selects optimal features with high correlations, enhancing the discriminative power of the model. Finally, the DBN leverages the pre-trained knowledge, amalgamating features and predicting the category of the input data. H-DBN is a blend of powerful neural network capabilities, selective feature filtration based on correlations, and efficient texture feature extraction, collectively geared towards achieving superior accuracy in detecting cracked tyres.
B. New Image Preprocessing Technique
The study employs a comprehensive methodology for efficient tyre defect detection and training, centered around the preprocessing of depth images. Utilizing a 3D camera, the acquisition of depth images, which encapsulate tyre height information, serves as the foundational element for detecting defects. Challenges arise due to height value concentration in a limited section and variations in shape for identical tyre errors.
To mitigate these challenges, the methodology introduces three key images - the original image, a heatmap image with height information, and an image showing the histogram equalization depth. These are strategically combined into a three-channel image, laying the groundwork for the training of a DeepLabV3+ model.
The system process involves a systematic four-step depth image preprocessing method, commencing with image input, progressing through highlighted image creation and stacking, and culminating with image training. The 3D camera captures depth images with 16-bit depth information, emphasizing vent spew errors protruding from the tyre surface. Preprocessing steps include the creation of highlight images through histogram equalization and heatmap generation. These enriched images, alongside the original, are stacked and normalized for subsequent model training.
The tyre fault detection system's architecture incorporates depth image scanning, labeling, and a tyre fault inspection system. The latter includes various modules such as data loader, data matching, data slice, data stacking, inspector, model training, weight applying, fault inspecting, and visualizer. This comprehensive approach collectively contributes to effective defect detection and training in the tyre manufacturing process.
C. Wasserstein Generative Adversarial Networks (WGAN)
In the realm of tyre X-ray defect detection with the help of Generative Adversarial Networks (GANs), challenges faced by the original GAN, such as the vanishing gradient of the generator and instability in the loss function, prompted the development of an improved model known as Wasserstein GAN (WGAN). This uses the measurement function of the Wasserstein distance.
To address the issue of imbalanced classification in tyre defects, a WGAN model is introduced to produce photos of X-ray defects for each minority class, effectively transforming the dataset into a balanced state. This WGAN is further enhanced through the application of image processing methods. Subsequently, a deep classification network is trained on the balanced dataset to categorize tyre defect types based on X-ray images.
The methodology involves the establishment and training of the WGAN, dataset processing to augment the minority classes, and the subsequent training of the classification network.
Moving on to grade classification of tyre defects, a WGAN-assisted grade classification method is proposed. The WGAN, previously trained for type classification, serves as a model that has already been trained and is further refined to extract deep features unique to grade categorization. Classical classification models, including standard CNN, ResNet, GoogLeNet, and MobileNet, are initially employed.
This is followed by the restructuring of an improved network denoted as ImprovedNet, which involves the creation of a novel module called ResMob. The algorithmic implementation of this methodology includes WGAN training, dataset processing to address imbalances, and training the grade classification network. The structures of the classification models are outlined, specifying components such as standard convolution, residual connection modules, max pooling, fully connected layers, convolution modules, and depth-separable convolution.
The training process occurs in three stages, each with a specified learning rate. This comprehensive methodology revolves around leveraging and enhancing WGAN to achieve effective tyre defect type and grade classification, addressing challenges related to dataset imbalances and achieving improved model performance.
D. Prototype-Based Supervised Contrastive Learning (PSCL)
The PSCL method tackles the issue of noisy labels in industrial datasets to enhance defect detection. The concept is centered on enhancing defect detection performance and rectifying noisy labels within industrial datasets. Data augmentation modules, encoder networks, a classifier, projection networks, & class prototypes are some of its components. Specifically, for query & key data augmentation, the data augmentation modules make use of SimAugment & RandAugment. In the supervised contrastive learning (SCL) phase, the goal is to group together in the embedding space clusters of points belonging to the same class. This involves using a framework with query and key encoders, contrastive loss (LSCL) that incorporates label information for sampling positives, momentum updates for key encoder parameters, and the utilization of a queue for embedding storage.
Prototype update and label correction play a crucial role in mitigating label noise. Class prototypes are maintained as moving averages of normalized embeddings, while label correction involves adjusting labels based on the similarity between query embeddings and prototypes. The label correction process uses a moving-average mechanism to gradually increase confidence in correct labels and reduce confidence in incorrect ones. The algorithm pipeline consists of two main phases: a warm-up phase and a training phase.
The warm-up phase focuses on initial network development using noisy labels, while the training phase involves prototype updates and label correction for robust feature extraction. Positive set construction for SCL includes using noisy labels in the 1st phase & predicted labels from the classifier in the 2nd phase. The collaboration between class prototypes and label correction modules ensures effective label noise mitigation.
The training objective combines cross-entropy loss (LCE) & supervised contrastive losses (LSCL), with a weighting factor (λ) adjusting the contribution of each component. The training process involves two phases, and positive sets for SCL are constructed differently in each phase.
The collaboration between class prototypes and label correction modules facilitates network convergence with robustness to label noise. In terms of performance evaluation, the PSCL method demonstrates superiority over existing methods, particularly in scenarios with high label noise. The application of PSCL to a tyre defect dataset showcases its effectiveness in real-world industrial settings, outperforming other methods and emphasizing its robustness in practical applications.
E. HOG + LBP Feature Extraction
The proposed method for tyre appearance defect detection combines HOG & LBP features, presenting a comprehensive approach to tackle the task.
The workflow encompasses collecting a tyre image dataset, preprocessing normal and defective images, extracting HOG and LBP features, training Support Vector Machine (SVM) classifiers, and ultimately classifying tyre images as normal or defective through the combination of HOG & LBP features.
HOG feature extraction begins by dividing the tyre image into blocks and cells, applying gamma transformation for illumination robustness, calculating gradients, and generating histograms for each cell. The HOG feature map is then constructed by aggregating these histograms. Conversely, LBP feature extraction utilizes a basic LBP operator to compare pixel gray values, producing binary numbers that capture local texture features. The LBP feature map is obtained by calculating histograms for blocks and cells.
Following the extraction of HOG & LBP features, SVM classifiers are individually trained for each feature. The SVM classifier adopts a linear kernel function, and the resulting prediction scores are documented. The combination of HOG & LBP features is achieved using an additive fusion method, which comprehensively considers shape and texture information.
F. Domain Adaptive (DA) - Feature Pyramid Network (FPN)
The rendering strategy involves estimating surface normals from raw 3D scans to provide accurate depth information and identify irregular bulge-like areas. The surface normal estimation relies on efficient Kd-tree querying and least-square plane fitting algorithms.
The obtained normal information is combined with depth information to create rendered images & depth images for use in defect detection.
a. Surface Normal Estimation: Tangent planes are calculated at each point in a point cloud patch to estimate normal vectors reflecting surface orientation.
b. Projecting 3D Point Clouds into 2D Features: Depth & normal information are converted into regular 2D feature maps utilizing the technique of a 3D projection. This involves mapping point-wise depth to pixel values for depth images and projecting normal vectors into pixel values for rendered images.
2. Attention-Based Detector
Drawing inspiration from transformer architectures, an attention-based defect detector is proposed. A deformable attention operation is introduced to optimize computational costs. The detector comprises a Domain Adaptive (DA) Feature Pyramid Network (FPN) & a DA-decoder.
a. Deformable Attention: With this process, each pixel can only focus on its immediate neighbors, optimizing the receptive field & efficiently capturing long-range dependencies.
b. Defect Detector: The defect detector incorporates a DA-FPN, utilizing deformable attention for capturing local properties, and a DA-decoder with modules of cross-attention & self-attention.
G. Self-Supervision and Synthetic data (S3) Framework
The Self-Supervision and Synthetic data (S3) framework, proposed by Ho & Huang et al., is a novel approach designed to tackle the challenges of tyre defect detection in scenarios characterized by limited annotation. This framework is specifically developed to address real-world manufacturing circumstances where acquiring sufficient labeled data for defect detection proves to be a significant obstacle.
The S3 framework operates in a two-stage process, employing a combination of self-supervised learning and synthetic data generation to enhance the performance of tyre defect detection models. In the first stage, known as the self-supervised pretraining stage, the authors utilize a standard Faster-RCNN architecture with a ResNet50 backbone. This stage aims to train the encoder without relying on labeled data. The chosen self-supervised learning method is SimCLR, a technique that takes input images and applies various data augmentations, resulting in two sets of augmented images. These images are then processed by the encoder to extract feature vectors, and a contrastive loss is employed to encourage similar images to be close in the embedding space. This pretraining stage is crucial for imparting a generic understanding of features to the encoder, enhancing its adaptability to downstream tasks with limited annotated data.
The second stage of the S3 framework involves fine-tuning the pretrained encoder using the limited annotated data available for tyre defect detection. The authors present the idea of synthesizing data to overcome the challenge of insufficient labeled defect data. An algorithm is proposed for synthesizing images by randomly pasting defects onto regular images. This approach ensures diversity in the dataset, improving the generalization capacity of the model to various failure circumstances. The defects' locations in the synthesized images are automatically labeled, reducing the manual effort typically associated with annotation.
The combination of self-supervised pretraining and synthetic data augmentation in the S3 framework aims to boost the tyre defect detection model's performance in scenarios with limited annotation. The authors conduct extensive experiments and ablation studies to validate the effectiveness of each component within the proposed framework. This innovative approach offers a practical solution for deploying defect detection models in industrial settings where labeled data is scarce, showcasing the adaptability and versatility of the S3 framework.
H. Dual-Domain Adaptation Model
The dual-domain adaptation model features a carefully crafted architecture aimed at addressing the intricacies of domain shifts in tyre defect detection. At the core of this architecture is the feature extractor, a pivotal element responsible for extracting relevant information from X-ray images. The unique fusion of ResNet50 & Vision Transformer (VIT) enhances the model's ability to capture intricate details from the intricate tyre X-ray background texturing.
To obtain features from the photos, ResNet50 is used. This includes specific enhancements such as standardized convolution and the replacement of batch normalization with group normalization. Concurrently, Vision Transformer processes the images by breaking them down into patches and utilizing a Multihead Self-Attention (MSA) mechanism. This dual approach ensures that the model can effectively focus on tyre defects, even when they are intricately mixed with background textures.
One of the key innovations in the architecture is the dual-domain adaptation strategy. This strategy is designed to simultaneously address global marginal distributions and fine-grained conditional distributions. For global distribution alignment, the model incorporates Multiple Kernel Maximum Mean Discrepancy (MK-MMD). MK-MMD optimally selects kernels to reduce domain discrepancy, contributing to a more consistent alignment of marginal distributions.
In contrast, the architecture leverages Local Maximum Mean Discrepancy (LMMD) for fine-grained subdomain alignment within the same defect category. This nuanced approach ensures that the model not only aligns overall distributions but also pays attention to the specific characteristics of defects within each category. To dynamically analyze the relative significance of conditional & marginal distributions., a dynamic adaptive factor (μ) is introduced. The A-distance, measuring distribution differences, is utilized to determine μ dynamically. This adaptive factor enables the model to flexibly adjust its focus based on the degree of domain shift, thereby enhancing its adaptability to varying conditions.
The defect classifier, represented by a fully connected layer (FC), is essential to the architecture. It receives features extracted from both domain samples of source & target, contributing significantly to the model's ability to distinguish between different defect types. Pseudo labels generated from the defect classifier's output further enhance the model's adaptability, particularly for unlabeled target domain samples. In terms of optimization, the architecture employs a combination of loss functions. Cross-Entropy Loss is utilized for source domain classification, ensuring effective learning from labeled source domain samples. Additionally, a dual-domain adaptation loss is introduced, where the dynamic adaptive factor μ controls the balance between contributions from MK-MMD and LMMD. The optimization process involves minimizing these losses, with a hyperparameter (α) governing the trade-off between source domain classification accuracy and dual-domain adaptation effectiveness.
I. Multi-scale Defect Detection Network (MDDN)
The Multi-scale Defect Detection Network (MDDN) represents a neural network architecture tailored for the specific task of identifying defects in tyre images within industrial inspection scenarios. Crafted by Wei et al., MDDN addresses the intricacies tied to the variety of tyre products and the intricate nature of textures, presenting a formidable challenge in the realm of automated defect detection.
In essence, MDDN comprises two parallel learning frameworks: the Semantic-Aware Network and the Texture-Aware Network. The Semantic-Aware Network, akin to a simplified FCN with VGG16 as its foundation, focuses on extracting high-level semantic features, encapsulating critical information about the shape and location of defects. Through convolutional and max-pooling layers, this network distills essential details and leverages a softmax layer for pixel-wise binary classification.
Complementing this, the Texture-Aware Network, comprising four groups of convolutional layers, strategically incorporates a shrinking layer to preserve spatial resolution and reduce redundant information. This network is purposefully designed to capture low-level clues often lost in deep networks with extensive pooling layers. The pivotal strength of MDDN lies in its integration approach. It combines features from both the Semantic-Aware and Texture-Aware Networks, capitalizing on the strengths of high-level semantics for precise location details and low-level textures for enhanced precision. This integration is facilitated by a learning scheme that concatenates feature maps from both networks, refining the final detection results through a convolution layer.
A. Integration of Hybrid Approaches
Numerous research papers underscore the efficacy of hybrid methodologies, seamlessly amalgamating conventional feature extraction techniques with cutting-edge deep learning models. A prime example is evident in the Hybrid-DBN system and the 3D laser scanning-based approach, both showcasing the synergistic potential of integrating established methods with state-of-the-art technologies. These hybrid approaches exemplify a holistic strategy, leveraging the strengths of traditional techniques alongside the capabilities of deep learning for elevated precision in tyre defect detection. This integration heralds a paradigm shift in the field, offering robust solutions that draw on the complementary aspects of diverse methodologies.
B. Addressing Imbalanced Datasets
Gao et al.'s pioneering work introduces a pivotal solution to the perennial challenge of imbalanced datasets in tyre defect detection. By employing Wasserstein GAN to generate balanced datasets, the study significantly contributes to overcoming the inherent biases in data distribution. This novel approach becomes particularly crucial in achieving heightened performance metrics, specifically in type and grade classification. The application of such methods not only rectifies data imbalances but also underscores their potential for seamless integration into real-world industrial applications, where accurate classification is imperative for quality control.
C. Innovation in Data Augmentation
In the pursuit of enhanced model accuracy, innovative data augmentation techniques, exemplified by the Multi-scale Defect Detection Network (MDDN), emerge as a cornerstone in the evolution of tyre defect detection methodologies. MDDN's unique multi-scale preprocessing strategy not only reflects a departure from traditional augmentation methods but also serves as a testament to the adaptability and versatility required in characterizing defects of varying sizes. This innovation transcends conventional augmentation practices, providing a comprehensive solution that augurs well for the intricate demands of tyre defect detection in industrial inspection applications.
D. Real-World Applicability
A notable trend in recent studies is the explicit emphasis on real-world applicability, with a focus on scenarios mirroring practical industrial conditions. Research endeavors such as real-road driving tyre tread wear detection and MEMS accelerometer-based tyre condition monitoring are strategically crafted to address the dynamic and diverse environments encountered in industrial settings. These studies prioritize not only accuracy but also the practicality and cost-effectiveness of implementing tyre quality control systems outside controlled laboratory conditions. This shift towards real-world relevance underscores the maturation of tyre defect detection research, aligning methodologies with the challenges of practical implementation.
E. Advancements in Domain Adaptation
The innovative dual-domain adaptation model proposed by Zhang et al. signifies a substantial leap forward in addressing the challenges posed by diverse X-ray image sensors. By concurrently aligning global marginal and fine-grained conditional distributions, the model showcases remarkable adaptability across different sensor types. This advancement is pivotal in enhancing the versatility and robustness of tyre defect detection systems, reducing the dependency on sensor-specific models. The model's success in aligning data distributions underscores its potential as a valuable solution for industrial applications where sensor variations are prevalent.
While the reviewed papers demonstrate significant progress, they collectively illuminate persisting challenges. Limited annotation, imbalanced datasets, and the absence of specialized datasets for specific defect types remain prominent hurdles. Addressing these challenges necessitates continuous innovation in data generation techniques, the refinement of domain adaptation strategies, and the development of benchmark datasets that mirror the complexities of real-world scenarios. Future research directions should prioritize the creation of standardized evaluation frameworks, fostering collaborative efforts in the tyre defect detection community to advance the reliability and generalizability of the proposed methodologies. Additionally, exploring novel avenues, such as transfer learning and federated learning, may pave the way for more adaptive and scalable solutions to the challenges at hand.
 L. Li and F.-Y. Wang, “Advanced tire friction modeling and monitoring,” in Proc. Adv. Motion Control Sens. Intell. Veh., 2007, pp. 33–76.  J. Xi, S. Wang, X. Chen, and A. Zhurongtao, “Accident cause analysis method based on traffic accident information system,” in Proc. IEEE Int. Conf. Comput. Appl. Syst. Model., vol. 13, 2010, pp. V13-230–V13-233.  J. Yi, “A piezo-sensor-based “smart tire” system for mobile robots and vehicles,” IEEE/ASME Trans. Mechatronics, vol. 13, no. 1, pp. 95–103, Feb. 2008.  K. Gui, L. Ye, J. Ge, F. A. Cheikh, and L. Huang, “Road surface condition detection utilizing resonance frequency and optical technologies,” Sensors Actuators A, Phys., vol. 297, 2019, Art. no. 111540.  S. Chandra, C. R. Sekhar, A. K. Bharti, and B. Kangadurai, “Relationship between pavement roughness and distress parameters for indian highways,” J. Transp. Eng., vol. 139, no. 5, pp. 467–475, 2013.  Al-juboori, A.M., Alsaeedi, A.H., Nuiaa, R.R., Alyasseri, Z.A.A., Sani, N.S., Hadi, S.M., Mohammed, H.J., Musawi, B.A. and Amin, M.M. A hybrid cracked tiers detection system based on adaptive correlation features selection and deep belief neural networks. Symmetry 15, no. 2 (2023): 358.  Ko, Dongbeom, Sungjoo Kang, Hyunsuk Kim, Wongok Lee, Yousuk Bae, and Jeongmin Park. \"Anomaly segmentation based on depth image for quality inspection processes in tire manufacturing.\" Applied Sciences 11, no. 21 (2021): 10376.  Gao, S., Dai, Y., Xu, Y., Chen, J., & Liu, Y. Generative adversarial network–assisted image classification for imbalanced tire X-ray defect detection. Transactions of the Institute of Measurement and Control, 45.8 (2023): 1492-1504.  Wang, Y., Zhang, Y., Jiang, Z., Zheng, L., Chen, J., & Lu, J. Prototype-based Supervised Contrastive Learning Method for Noisy Label Correction in Tire Defect Detection. IEEE Sensors Journal (2023).  Zheng L, Lou H, Xu X, Lu J. Tire Defect Detection via 3D Laser Scanning Technology. Applied Sciences. 2023; 13(20):11350. https://doi.org/10.3390/app132011350  Ho, C. H., Huang, Z., Kim, N., Bae, Y., & Vasconcelos, N. Tire defect detection with limited annotation. Electronic Imaging, 35. (2023), 1-6.  Vasan, V., Sridharan, N. V., Prabhakaranpillai Sreelatha, A., & Vaithiyanathan, S. (2023). Tire Condition Monitoring Using Transfer Learning-Based Deep Neural Network Approach. Sensors, 23(4), 2177.  Zhang, Y., Wang, Y., Jiang, Z., Zheng, L., Chen, J., & Lu, J. Tire defect detection by dual-domain adaptation-based transfer learning strategy. IEEE Sensors Journal, 22.19. (2022), 18804-18814.  Han, J. Y., Kwon, J. H., Lee, S., Lee, K. C., & Kim, H. J. Experimental Evaluation of Tire Tread Wear Detection Using Machine Learning in Real-Road Driving Conditions. IEEE Access, 11. (2023), 32996-33004.  Liu H, Jia X, Su C, Yang H and Li C. Tire appearance defect detection method via combining HOG and LBP features. Front. Phys. 10 (2023):1099261. doi: 10.3389/fphy.2022.1099261.  Wei, Minghua, Ren Wang, and Qiang Guo. \"Multi-scale Defect Detection Network for Tire Visual Inspection.\" Intelligent Computing: Proceedings of the 2021 Computing Conference, Volume 1. Springer International Publishing, 2022.  Ying Li, Binbin Fan, Weiping Zhang, and Zhiqiang Jiang. TireNet: A high recall rate method for practical application of tire defect type classification. Future Gener. Comput. Syst. 125, C (Dec 2021), 1–9. https://doi.org/10.1016/j.future.2021.06.009.
Copyright © 2024 Vamsi Sravanth Mahendrada, Murali Parameswaran. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.