Augmenting Bridge Inspection with Augmented Reality and Haptics-based Aerial Manipulation (AS-7)


University of Nevada, Las Vegas

Missouri University of Science and Technology                          

Principal Investigator:

Dr. Paul Oh, University of Nevada, Las Vegas

PI Contact Information:

Phone: (702) 895-0168  |  Email:

Co-Principal Investigators:

Dr. Genda Chen, Missouri S&T

Dr. Hung La, University of Nevada, Reno                                      

Funding Sources and Amounts Provided:
University of Nevada Las Vegas: $50,499
INSPIRE UTC: $81,696

Total Project Cost: $132,195

Match Agencies ID or Contract Number:
UNLV: In-Kind Match   |    INSPIRE UTC: 00055082-05B

INSPIRE Grant Award Number: 69A3551747126

Start Date: January 1, 2020
End Date: June 20, 2021


Brief Description of Research Project:

Insight and leverage from prior project work identified the need for haptics; the inspector’s sense of touch is often used in bridge assessment. Haptics’ adoption in medical robotics shows potential for applications in remote inspection and maintenance. For example, surgeons employ haptic-based manipulators to both probe and operate minimally-invasive procedures. Such manipulators augment the surgeon by assessing areas that are difficult to see or reach with conventional tools. Likewise, envisioned is the augmentation of bridge personnel by providing a haptics-based aerial manipulator. Such a manipulator would reduce needs to ascend elevated structures like under bridge decks and cabling.

There is recent work that uses haptics in aerial manipulation. But these center on autonomous multi-drone flight formation for aerial shows or cooperative object delivery. The haptics research here is relatively straight-forward. The sense of touch is used to provide spatial distance between drones. By contrast, this proposal’s novelty and merit are centered on human-in-the-loop aerial manipulation. The notion is that the expert bridge worker remotely configures the aerial manipulator to “probe” areas of interest and “operate” maintenance procedures.

This Year 4 proposal integrates haptics, augmented reality (AR), and drone-mounted arm. The specific objective is to characterize haptics and AR for their applicability to perform remote probing (i.e. inspection) and operation (i.e. maintenance).

Approach and Methodology: This Year 4 proposal focuses on the design and realization of a testing-and-evaluation platform for human-in-the-loop aerial manipulation. This is a novel focus for this INSPIRE project and reflects an evolutionary, rather than incremental, advance.

Overall Objectives: The objectives of this study are to test, evaluate, verify and validate haptics-based aerial manipulation. Realizing such objectives would yield new capabilities for remote bridge inspection and maintenance. The tools and techniques would leverage bridge
personnel expertise and introduce a new paradigm of human-in-the-loop aerial
manipulation. This is akin to the advances and adoption found in medical robotics for
minimally-invasive surgery.

Scope of Year 1: (1) Spring Mount Design, Strain Gage Design, and Haptics Device Fixture, (2) Gantry-testing Motion Control, (3) Mo-cap Evaluation, (4) Fasteners and Drilling Benchmarks, (5) Assessment of Augmented Reality Information.


Describe Implementation of Research Outcomes:
Research outcomes and implementation plan will be described towards the end of this project.

Impacts/Benefits of Implementation:
Impact/Benefits of Implementation will be summarized at the end of this project.

Project Website:
Progress Reports: