Civil engineering researchers develop data analysis technique to improve bridge inspection using drones

Brandon Perry pilots a drone for inspecting the Oxbow Bridge in Loveland, while Theresa O'Donnell-Sloan observes.
Brandon Perry pilots a drone for inspecting the Oxbow Bridge in Loveland, while Theresa O’Donnell-Sloan observes. The City of Loveland gave Yanlin Guo and Perry permission to test their inspection method on the bridge.

Colorado State University researchers have developed a process to streamline bridge inspections and make them more objective and quantitative. Their method could keep better tabs on deteriorating infrastructure and help bridge owners plan for maintenance.

Yanlin Guo, assistant professor of civil and environmental engineering, and Ph.D. student Brandon Perry used unmanned aerial vehicles and AI to devise an efficient data analysis and visualization technique for inspecting bridges.

Traditionally, bridges have been inspected by a person physically climbing on the structure, which can be costly and dangerous. These inspections generally are done once every two years, and they’re subjective; the inspector assigns a rating based on their individual perception.

Over the past five years, some transportation departments and inspection companies have begun using drones to collect images for inspections. Drones are incredibly efficient at collecting data; they can snap 500 photos an hour or shoot large video files. It can take a person one to two months to analyze all that imagery for bridge damage.

“The data analytics part can be tedious because of this large amount of data, and a lot of information might not be relevant,” Guo said. “Our process extracts the knowledge from this big data set efficiently.”

Guo and Perry developed tools to process the drone data in several hours, with minimal user input. Their process is largely automated and relies on machine learning and computer vision, removing subjectivity and potential for human error.

Guo and Perry’s data analysis and visualization technique generated this model of the Oxbow Bridge based on drone images.
Guo and Perry’s data analysis and visualization technique generated this model of the Oxbow Bridge based on drone images.

From the images collected by drone, Guo and Perry’s technique generates a three-dimensional bridge information model that includes the geometry of each bridge component, such as the pier, deck, etc., and structural condition information. An inspector could examine various angles of this bridge model and see damage as it appears on the different structural components, along with the corresponding data – the length of a crack, for example.

“The information [in the model] is much more comprehensive compared to the traditional inspection report,” Guo said. “The American Association of State Highway and Transportation Officials and the Federal Highway Administration require component-level assessment, so using AI to identify and assess the condition of structural components is critical.”

Instead of assigning a subjective rating of 0-9, Guo and Perry’s automated process measures damage extent precisely. They also developed automated algorithms to easily and effectively track damage growth over time.

Now the pair is working on techniques using machine learning and structural mechanics to predict how a structure will deteriorate in the future, so bridge owners can prepare for maintenance needs.

Their paper on this work, “Streamlined bridge inspection system using unmanned aerial vehicles (UAVs) and machine learning techniques,” was co-authored by Associate Professor Rebecca Atadero and Professor John van de Lindt.

Getting creative during COVID

While Guo and Perry were developing a portable, UAV-based technique to measure the dynamics of a vibrating structure, the pandemic interrupted their research. Perry brought home the equipment for the experiment, including a coffee table-sized shake table, filling the living room and dining room of his one-bedroom apartment.

“I’m sure my neighbors loved me for it,” he said.

Perry’s at-home experiments prepped the project for when it could move into a larger space at the Engineering Research Center.

To measure the vibration of their test structure, they attached a RealSense sensor – a technology often used to detect motion in video games – to a drone. This sensor has onboard optical and infrared cameras that allowed them to capture all three directions of movement.

Guo and Perry used a drone equipped with a RealSense sensor to measure the dynamics of a vibrating cantilever on a shake table at the Engineering Research Center. Chris Robertson piloted the drone.
Guo and Perry used a drone equipped with a RealSense sensor to measure the dynamics of a vibrating cantilever on a shake table at the Engineering Research Center. Chris Robertson piloted the drone.

The portable, 3D displacement measurement technique they developed could be used to conveniently and inexpensively monitor the health of bridges, without the need for sensors placed on the bridge or traffic control.

Despite pandemic-related complications – including Perry contracting COVID himself – the researchers were able to complete their project and publish a second paper, “A portable three-component displacement measurement technique using an unmanned aerial vehicle (UAV) and computer vision: A proof of concept.”

“There have definitely been new challenges, but we were able to overcome them,” Perry said. “Someone told me that Isaac Newton discovered the law of gravity while at home during the plague. I thought if he could make such an important discovery while in quarantine, then I can keep working too!”

Valuable resources for research

Having UAVs available for research made Guo and Perry’s work possible. The expense of purchasing these systems would have prevented Guo from pursuing this area of research when she started investigating it in 2017.

CSU researchers have access to several UAV platforms, as well as piloting expertise, through the Drone Center.

“This research represents a resourceful and excellent use of UAV technology in the field of infrastructure inspection and documentation,” said Chris Robertson, director of CSU’s Drone Center. “We were excited to be given the opportunity to support this project with UAV resources and piloting time.”