2022 Webinar Series
Toward Dexterous Aerial Manipulation Using Embodied Human-Intelligence For Bridge Inspection and Maintenance
Presented: March 16, 2022, 10:00AM-11:00 AM Central Standard Time (US and Canada)
Speaker: Dr. Dongbin Kim, University of Nevada, Las Vegas
ABSTRACT
Current drones perform a wide variety of tasks in surveillance, photography, agriculture, package delivery, etc. However, these tasks are performed passively without the use of human interaction. Aerial manipulation shifts this paradigm and implements drones with robotic arms that allow interaction with the environment rather than simply sensing it. Many research groups have demonstrated aerially manipulated tasks for over a decade. For example, in construction, aerial manipulation in conjunction with human interaction could allow operators to perform several tasks, such as hosing decks, drill into surfaces, and sealing cracks via a drone. This integration with drones will henceforth be known as dexterous aerial manipulation. However, there are still no implementation and adoption in practical applications. To overcome this issue, we recently integrated the worker’s experience into aerial manipulation using haptic technology. The net effect was such a system could enable the worker to leverage drones and complete tasks while utilizing haptics on the task site remotely. However, the tasks were completed within the operator’s line-of-sight. Until now, immersive AR/VR frameworks has rarely been integrated in aerial manipulation. Yet, such a framework allows the drones to embody and transport the operator’s senses, actions, and presence to a remote location in real-time. Thus, AR/VR technology enables the operator to leverage their intelligence to collaboratively perform desired tasks anytime, anywhere with a drone that possesses great dexterity. Finally, the operator can both physically interact with the environment and socially interact with actual workers on the worksite. Toward this vision, this work presents aerial manipulation using embodied human intelligence.
SPEAKER
Dr. Dongbin Kim is currently an Adjunct Assistant Professor in Howard R. Hughes College of Engineering at the University of Nevada, Las Vegas. He is also a lab manager at Drones and Autonomous Systems Lab (Director, Dr. Paul Oh). In 2021, he received a PhD in Mechanical Engineering from the University of Nevada, Las Vegas, under Dr. Paul Oh, former National Science Foundation (NSF) Robotics Program Director. His PhD research project was to develop a Mobile Manipulating Unmanned Aerial Vehicle (MM-UAV) for bridge inspection and maintenance with support from the United States Department of Transportation (US-DOT) Inspecting and Preserving Through Robotic Exploration University Transportation Center (INSPIRE-UTC). He received the best presentation award from US-DOT INSPIRE-UTC annual meeting in 2021. He is an active volunteer of IEEE Robotics and Automation Society (RAS). He was the creator of IROS On-Demand, the innovative conference platform which hosted 25,000 international attendees during the pandemic. With this outstanding performance, he serves as one of the organizing committees for the flagship robotics conference, ICRA, and IROS. He also received the best student award and innovation finalist from the Society of Laboratory Automation and Screening in 2018. In 2016, He was selected as an honored student from the WEST program, the joint internship program by the United States Department of States and Korea Ministry of Education. His current research interests include Aerial Manipulation, Embodied Human Intelligence, Augmented/Virtual Reality (AR/VR), Haptics, Artificial Intelligence (AI), Human Behavior and Human Stiffness.
Education
PhD, Mechanical Engineering, University of Nevada, Las Vegas, USA.
MS, Mechanical Engineering, University of Nevada, Las Vegas, USA.
BS, Aircraft Systems Engineering, Korea Aerospace University, Republic of Korea.
Intelligent Human-Infrastructure Interfaces for Inspectors and Decision-makers
Presented: June 21, 2022, 10:00AM-11:00 AM Central Standard Time (US and Canada)
Speaker: Dr. Fernando Moreu, University of New Mexico
ABSTRACT
This seminar challenges the traditional conversation about digital twins and machine learning, by proposing a different paradigm for smart cities transformation centered in new human-infrastructure interfaces. This discussion explores the area of human decisions and cognition of the built environment enabling transformations of human interacting with structures in a new environment. To date, new technologies collecting data of the built environment are cheaper, more accurate, diverse, and more accessible than ever before. However, the use and implementation of these new technologies to structural engineers to assess, inspect, or inform actions have been very limited. Decision-makers, owners of infrastructure, policy makers, occupants, and inspectors of infrastructure are often not considered when developing new technologies to inform structural responses or condition. By empowering human-machine interfaces and fostering human involvement and participation (human-in-the-loop), this seminar will present specific practical implementations about how the collection of data, their analysis, and their interpretation can inform (and transform) human decisions. Specific applications include the connection of Augmented Reality with Wireless Sensors Networks, AI, ML, Structural Dynamics, and Inspections.
SPEAKER
Dr. Fernando Moreu, PE is an Assistant Professor in structural engineering at the Department of Civil, Construction and Environmental Engineering (CCEE) at the University of New Mexico (UNM) at Albuquerque, NM. He holds courtesy appointments in the Departments of Electrical and Computer Engineering, Mechanical Engineering, and Computer Science at UNM. He is the founder and director of the Smart Management of Infrastructure Laboratory (SMILab) at UNM. His industry experience includes ESCA Consultants, Inc. for over ten years, with experience in the design and construction of over thirty bridges in seven states in the US. His research interests include structural dynamics, structural health monitoring, wireless smart sensor networks, augmented reality, unmanned aerial systems, human-machine interfaces, and aerospace structures design, monitoring, and reusability. He received his MS and PhD degrees in structural engineering from the University of Illinois at Urbana-Champaign (2005 and 2015, respectively.)
Reliability of Bridge Inspection Technologies
Present: September 13, 2022, 11:00AM-12:00 PM Central Standard Time (US and Canada)
Speaker: Dr. Glenn Washer, University of Missouri
ABSTRACT
This presentation will discuss the reliability of inspection technologies used for the condition assessment of highway bridges. Data from condition assessments are critical to asset management and decision-making. The methodologies used to assess condition are usually subjective methods, such as visual inspections, that can vary as a result of human factors and differences in training and interpretation of inspection procedures. Nondestructive evaluation technologies (NDE) are implemented on a more limited basis and rely on indirect measurements to infer the presence of damage, and these assessments are affected by environmental factors, materials variability, and limitations in the interpretation of the resulting data. Field studies of inspection quality for element-level (CSs) and component-level (CRs) results will be discussed that illustrate the variability in data from routine inspections of highway bridges. Methods implemented in recent years to improve the quality of these inspections will be discussed. Variability in several common NDE techniques will also be discussed, such as Ground Penetrating Radar (GPR), Infrared Thermography (IRT), and Ultrasonic Testing (UT), based on results from field studies and performance tests. The impact of the reliability of these technologies of the future of bridge inspection, including risk-based inspection decision-making, will be addressed as a conclusion to the presentation.
SPEAKER
Dr. Glenn Washer is a Professor at the University of Missouri in Columbia. Before joining the University, Dr. Washer was with the Federal Highway Administration (FHWA) at the Turner Fairbank Research Center (TFHRC) where he served as the director of the FHWA Nondestructive Evaluation (NDE) program. His research interests are focused on condition assessment technology for civil infrastructure. This includes developing nondestructive evaluation (NDE) technologies for damage detection, reliability of inspection technologies, and risk-based inspection. Dr. Washer was the PI for NCHRP 12-82, Developing Reliability Based Bridge Inspection Practices. The results of this research are included in the most recent revision to the National Bridge Inspection Standards (NBIS). Dr. Washer was also the PI on NCHRP 12-104, Improving the Quality of Element-Level Bridge Inspection Data. The results of this research include the new AASHTO Manual for Bridge Element Inspection (MBEI) approved in 2018 and currently used for element-level bridge inspection throughout the US. Research in NDE technologies currently includes ultrasonic stress measurement, ultrasonic testing for steel bridge fabrication, thermal imaging technologies, and reliability of NDE technologies. He has published over 120 journal and conference papers related to the condition assessment of bridges, and is a Fellow of the American Society for Nondestructive Testing (ASNT). Dr. Washer received his Ph.D. in Materials Science and Engineering from the Center for Nondestructive Evaluation (CNDE) at the Johns Hopkins University in 2001.
Structural Inspection Automation: Research Challenges and 3D Machine Vision Techniques
Present: December 13, 2022, 11:00AM-12:00 PM Central Standard Time (US and Canada)
Speaker: Dr. ZhiQiang Chen, University of Missouri-Kansas City
ABSTRACT
This presentation will start by reviewing the history of vision or imaging-based structural (surface condition or damage) inspection and discuss the notion of structural inspection automation. Besides robotic platforms for operation, this conceived automation process must feature 3-dimensional (3D) integrated structural element and damage (or any anomaly) detection, quantification, and mapping amid complex scenes. Further, the results of such a process should be readily fused with existing lifecycle 3D BIM or digital-twin models (DTMs) for ultimate decision-making.
The presentation plans to discuss that machine vision with optical sensors is the most viable approach as a core component to realizing structural inspection automation. Using an analogy of Tesla cars that rely entirely on vision sensors without using active Radar or LiDAR sensors, the presentation will then elaborate on six automation levels for structural inspection. However, our civil infrastructure stakeholders are much different from those from private automobile sectors, which have poured tremendous investment into collecting and creating semantically rich datasets for developing machine learning algorithms and AI-based software frameworks. On the other hand, large-scale semantically annotated datasets from civil engineering sectors are not expected to be available in the near future. Considering such constraints and aiming at the goal of realizing cost-effective structural inspection automation, the presentation will introduce recent efforts in the following three topics:
1. Low-cost 3D structural element and damage data collection and deep learning based algorithmic benchmarking
2. Human-in-the-loop based structural data collection using augmented reality headsets.
3. Visual Simultaneous localization and mapping (SLAM) enabled optimal structural element and damage mapping using virtual reality-based robotic drones.
The presentation will conclude by sharing the vision and opportunities about the future of this research area.
SPEAKER
Dr. ZhiQiang Chen is an Associate Professor of Civil Engineering at the University of Missouri, Kansas City (UMKC). Prior to joining UMKC in 2010, he received his Ph.D. in Structural Engineering from the University of California, San Diego (UCSD). He was a visiting professor at the University of California, Berkeley, in 2020; and a visiting professor at the Saitama University, Japan, in 2022. Dr. Chen’s research interests, in general, focus on Civil Systems Intelligence and Resilience. Dr. Chen has been working on multi-hazard performance and resilience computing for civil structures, climate-change effects on structural loadings, and the application and development of AI, remote sensing, and human-infrastructure interfacing technologies for disaster response and infrastructure management. Dr. Chen has been a keynote speaker for the ASCE Engineering Mechanics Institute International Conference, received the Takuji Kobori Prize from the International Association of Structural Control and Monitoring (IASCM), and was awarded a JSPS Invitational Fellowship by the Japan Society for the Promotion of Science. Dr. Chen is an Associate Editor for the ASCE Journal of Natural Hazards Review and serves as a core member in many ASCE and TRB committees.
Follow INSPIRE