Developing a Robotic Simulator and Video Games for Professional and Public Training (WD-2)

Universities : University of Nevada, Reno (UNR)
                            Missouri University of Science and Technology

Principal Investigator : Dr. Sushil Louis, University of Nevada, Reno

PI Contact Information : phone: (775) 784-4315  |  email:

Co-Principal Investigators : Dr. Hung La, University of Nevada, Reno
                                                         Dr. Ruwen Qin, Missouri University of Science and Technology
                                                        Dr.  Suzanna Long, Missouri University of Science and Technology

Funding Sources and Amounts Provided :
UNR: $37,320
INSPIRE UTC: $37,320

Total Project Cost : $74,640

Match Agencies ID or Contract Number :
UNR: In-Kind Match   |   INSPIRE UTC: 00055082-04B

INSPIRE Grant Award Number : 69A3551747126

Start Date : March 31, 2017
End Date : July 31, 2018

Brief Description of Research Project:

Civil engineers are not educated with robotics. They need to be trained on the job with effective tools. The most recent simulation trainer that the PI has built is currently being used by the US Navy to train surface warfare officers in decision making under stress. In a crowded in-port environment, the crew on a ship’s bridge is trained to probe and identify suspicious boat behavior within the port’s traffic pattern. Officers in charge of the simulation training lesson use software for high-level control of dozens of other ships, boats, and aircrafts that quickly react and adapt to the crewed ship’s actions based on lower-level programmed autonomy and game-like user interaction. Without this virtual “experience,” improperly trained crews put lives in danger. Scenarios that would be catastrophic in reality can also be simulated and, without this training, especially for recovering from error states, operators may inadvertently lose valuable hardware, produce erroneous results, and compromise system and human safety.

Approach and Methodology. An iterative software engineering process will be used to investigate and build a series of Simulation Training And Control System (STACS) prototypes. These prototypes will run on Android/iOS devices, PCs, or MACs and will be used for operator training (in simulation) and operator control of a robot team in the real world during inspection. The tradeoff between operator control and robot autonomy in simulation will be investigated to gain insight into the design of efficient control software in robots and develop an effective robot team, including multiple UAVs and climbing robots.                                                                                                          

Overall Objectives: This project aims to build a STACS prototype within a 3D simulation game-like environment and develop a realistic training environment. Specific objectives include: (1) Investigate and optimize the design of user interaction and user interfaces within a full 3D game-like environment for training and control, (2) Investigate and optimize the tradeoff between manual and autonomous control of multi-robot teams for bridge inspection, (3) Train bridge inspectors in the use of the proposed multi-robot system, and (4) Provide human operators with complete situational awareness and operational control during an ongoing inspection.

Scope of Work in Year 1: The focus of the first year will be to: (1) Design, build, and test four STACS prototypes in a full 3D environment within the Unity3D game engine; (2) Design and build a simulation “game” with tele-operative, semi-autonomous, and fully autonomous control of the proposed multi-robot system for bridge inspection; and (3) Establish a communication link between the STACS and real robots to facilitate transition from simulation to real-robot monitoring, interaction, and control.

Describe Implementation of Research Outcomes:
Research outcomes and implementation plan will be described towards the end of this project.

Impacts/Benefits of Implementation: 
Impact/Benefits of Implementation will be summarized at the end of this project.

Project website:
Progress Reports: