This research presents a deep learning method for generating automatic captions from the segmentation of car part damage. It analyzes car images using a Unified Framework to accurately and quickly identify and describe the damage. The development is based on the research "GRiT: A Generative Region-to-text Transformer for Object Understanding," which has been adapted for car image analysis. The improvement aims to make the model generate precise descriptions for different areas of the car, from damaged parts to identifying various components.

The researchers focuses on developing deep learning techniques for automatic caption generation and damage segmentation in car damage analysis. The aim is to enable precise identification and description of damages on vehicles, there by increasing speed and reducing the work load of experts in damage assessment. Traditionally, damage assessment relies solely on expert evaluations, which are costly and time-consuming. To address this issue, we propose utilizing data generation for training, automatic caption creation, and damage segmentation using an integrated framework.

The researchers created a new dataset from CarDD, which is specifically designed for cardamage detection. This dataset includes labeled damages on vehicles, and the researchers have used it to feed into models for segmenting car parts and accurately labeling each part and damage category.

Preliminary results from the model demonstrate its capability in automatic caption generation and damage segmentation for car damage analysis to be satisfactory. With these results, the model serves as an essential foundation for future development. This advancement aims not only to enhance performance in damage segmentation and caption generation but also to improve the model’s adaptability to a diversity of damages occurring on various surfaces and parts of vehicles. This will allow the system to be applied more broadly to different vehicle types and conditions of damage inthe future