Across industries, innovators are looking to develop solutions that incorporate machine learning models automated by artificial intelligence. Machine learning models are files that have been trained to use algorithms to detect specific patterns. Using an original data set, you can teach a model to utilize an algorithm to reason and learn from the data. Once trained, when given new data, the model is able to analyze it and make predictions.
AI learning models are being implemented in hospitals and by manufacturers across industries to improve efficiency and increase speed. The automotive industry is one of the most prevalent in this endeavor. AI applications are currently used in self-driving cars, drones for delivery services, and other autonomous vehicles.
The following case study demonstrates a new use of automation in the automotive industry – the AI powered detection of vehicle damage.
The Importance of Annotation
A key element in attaining the target outcome of an AI learning model is the process of labeling data. The data must be labeled correctly so that your model can predict what you want it to. This process is called annotation, ie, taking the data and assigning a value. The learning model, in turn, creates the dataset with the features that teach your machine how to recognize the patterns that you desire.
Introducing SpinFrame
Founded in 2017, SpinFrame Technologies provides a single source of truth for the status of vehicles at every point along the supply chain. With SpinFrame’s technology, it is possible to monitor large volumes of vehicles without requiring any human intervention – from maritime ports to large transportation and logistic hubs, parking lots, dealerships, rental branches, and service centers.
SpinFrame Technologies is specifically focused on image annotation, a type of visual data labeling sometimes called tagging or transcribing. This is where Road2 brought in Point.AI to help SpinFrame enhance their technology of identifying & documenting visual damage on vehicles.
What were SpinFrame’s challenges?
SpinFrame approached the NVIDIA supported Road2 AI Lab and their experts to help advance their AI model to its fullest potential and to overcome three main challenges:
- Damage annotation is a complex task – unlike other kinds of annotation where the object is well defined, vehicle damages are versatile and can be easily confused with something else (e.g. reflection or dirt).
- The need to tag these damages as the right class (such as a dent, a scratch, etc.) when there is no clear, universal definition of damage classes, making the task subjective.
- SpinFrame’s machine learning model needs to be trained by immense quantities of annotated pictures of vehicles with damages.
Partnering with experts
Point.AI was brought in to build a strategy that would help overcome these challenges. The first few meetings between Point.AI and SpinFrame sparked new questions on how to approach teaching machine learning models the annotation of car damage via images. Point.AI uses a wide variety of data annotation tools to streamline a project’s completion. By offering these tools and the expertise of their specialists, the process was mapped out and SpinFrame received the guidance they needed to develop their technology. The processing of the first few batches of images then went underway. Point AI led the training for SpinFrame and continuously verified that the model was being developed correctly.
Tomer Gorovici, Founder of Point.AI, said “It was very difficult to annotate the damage data at the start, due to the vast language designated to cars across countries and models. Therefore, the starting point of guiding and mapping out the process through constant communication was vital to the success of the project.”
During the next phase of the project, Point.AI began the actual annotation execution. This work was done using an online annotation tool called (CVAT). CVAT allows for the constant monitoring of the quality of the work being done and assists in managing the ongoing comments on the project.
SpinFrame was able to quickly increase the speed and amount of images that need to be annotated, starting at hundreds of images and increasing to thousands in just a few weeks.
The key factor to this transition, states Avishag Yitzhak, Data Management & Annotation Expert at SpinFrame, is the constant communication and effort, “Once you put enough effort and precision in the process, you can achieve a smooth ride on the road to success. But first, you need to grind and put in the effort.”
The Annotation Process – a guide for start-ups
Before starting any annotation process, there are several necessary steps. The following steps could be used as a general guide for any startup working on a large annotation project.
- Create a detailed but easy-to-digest “guideline” presentation, including visual examples.
- Go over this presentation with the annotation team.
- Be ready to adapt or change the guidelines as many unanticipated questions will arise on-the-go.
- Adopt the right tool because there are several annotation platforms on the market (CVAT, v7, or Dataloop, to name a few).
- Define the review and tracking process, thereby leveraging the annotation platform’s management features.
- Set clear resources and KPIs such as number of annotated pictures, target time per picture and target reject rate.
Since creating damage classes involves defining very specific criteria, a detailed presentation with clear guidelines on identifying and defining various forms of vehicle damage was made.
During the project, the following practices were key:
- Communicating frequently with the team to understand their ongoing challenges and answering their questions
- Updating the guidelines and making sure the updates were understood, shared, and enforced.
- Monitoring of the KPIs on a daily, and then weekly basis
What was learned from the process?
Throughout the entire project, the various annotation teams filled out a spreadsheet that allowed for the close monitoring of the work being done. Once it was indicated that the work on a particular project was complete, it went into Review Mode where specific comments on incorrect annotation could be left.
In order to refine the procedures and answer frequently asked questions, Point.AI made sure to hold video conference meetings every two weeks, in addition to quickly responding to questions by email or text message.
What were the results/outcomes?
One of the most important outcomes was that after only a short period of time, SpinFrame reached an impressive supplier that shares their goals.
For now, we are eagerly awaiting the results from this amazing progress in data annotation as well as the partnership between Point.AI and SpiFframe. Point.AI will continue to monitor SpinFrame’s progress and support their needs until they produce the results needed to move on to the next stage. SpinFrame is very optimistic about the process and the speed of development. They will continue to be patient as they tweak the model.
Road2 continues to support start-ups and bring new business opportunities to Haifa by pairing Point.AI with promising visionaries. They plan to give Point.AI experts a permanent home in Haifa so that they can use their unique tools and services to help build more talented start-ups.
Contact us to find out more about our AI Lab supported by NVIDIA.
