Using Artificial Intelligence for road safety: a smarter way of maintenance

Road incidents are not simply the result of wrong behavior. There is a strong correlation between traffic safety and the deterioration of road infrastructure. Hence the importance of keeping our roads in top condition. In the Netherlands road authorities are performing regular asset inspections on roads and bridges as part of their asset maintenance and management responsibilities. This important and continuous process requires considerable effort and costs. The costs of road maintenance in the Netherlands add up to at least 1.6 billion on an annual basis. In this article we will firstly discuss the current process of defects detection. Secondly, we will take you through the optimization possibilities of this lengthy and costly process with the use of AI.

• 

• 

The current process: driving footage

How do defects currently get detected? A common method to keep track of road quality is the use of driving footage. Road maintenance companies first record footage of roads and highways with cameras mounted on vehicles. They then end up with hours of footage needed to be analyzed. This tedious job takes a lot of time and effort and requires a team of specialized technicians to go frame by frame through the footage in order to recognize anomalies. These defects are then manually categorized before they are reported for maintenance. This is a lengthy and costly process that is still error prone. How can we make this process smarter and more efficient? Together with Royal HaskoningDHV we came up with a way of using AI to optimize this process. 

Using AI to automatically inspect roads

The Transport & Planning division of Royal HaskoningDHV has come up with a more efficient way to solve this problem. Namely: using data science and AI to automatically inspect road segments to drastically optimize the road assessment workflow. Recently domain experts from RHDHV teamed up with data experts from Ynformed to work on the first mock-up of this solution. Our project consists of different milestones representing various detections to automate. We want to categorize types of defects as well as the severity of the damage.

Guardrails detection

In the first stage of this project we focused on the detection of damaged guardrails on highways. The footage collected with the mounted cameras, contains complex visual information. To automate the analysis of this information, we applied deep learning techniques. For this we built and trained neural networks.  Eventually we were able to create a computer vision pipeline that can automatically process the data to evaluate the condition of the roads on which the car was driving.

After training the neural networks to detect guardrails along the highways, our model can inspect the severity of rust on the guardrails if existent. It can also recognize anomalies in the height of the guardrails by detecting deviations outside the height norms defined by field experts (as shown in red in the figure below). 

Thanks to the GPS coordinates collected by the camera while on an inspection drive, the model is also able to report the exact location of the defects on a map. The visualized results on a Geographical Information System (or GIS) will give maintenance experts the possibility to zoom in and out to highlight sections that need intervention.

Let’s stay in touch

We’re currently working on optimizing and extending the capabilities of our model. We feel like we’re just scratching the surface of what’s possible. Especially because of the shared characteristics of roads globally, we believe that our work here can make a contribution to enhance roads safety worldwide. We will keep you posted on what is happening. If you take a special interest into this topic or want to see what we can do for you, don’t hesitate to reach out.