[viewBag] title = "Targeting" url = "/project/targeting" layout = "project-template" is_hidden = 0 navigation_hidden = 0 ==

Computer vision allows our system to be able to see it’s environment. Devices such as compasses, gyroscopes, and acoustic sensors allow the robot to know where it is. However, computer vision allows Lionbot to determine whether or not the object in front of it is a lionfish.

As the robot is going through it’s designated navigation pattern it will constantly be seeking out lionfish using our current computer vision system which consists of two cameras feeding information to a deep neural network developed using TensorFlow, an open-source software library for dataflow programming. Once it identifies a lionfish it will then begin going into targeting mode. Once it targeting mode the robot will have a set of conditions it must meet to release the sling. It must ensure it is a lionfish, it is aimed properly at the critical point of the fish and ensure it is in range of the sling mechanism. 

In order to do this the 2017 MQP team developed a specific targeting program. The bounding boxes vision targeting program has been trained to identify four distinct categories: Front, Side, Critical, and Diver. The vision uses these categories to navigate around to the side of a Lionfish target and then hone in on the critical spot where the spear tip should be delivered. If the program sees anything it thinks to be a diver then it will not fire. With the current prototype the system does not have a way to navigate and so for the sake of testing, the team developed a program that would display the navigation commands on an LCD screen. This is in lieu of a submersible robotic navigation system, where rather than sending the commands to an LCD screen, they would be sent to the navigation system directly. 

Another critical aspect of our vision system is determining depth. This is currently done using two stereo cameras in order to determine how far away the fish is. Thus, once the robot recognizes the fish and begins to move closer, it will simultaneously need to determine the direction in which it needs to move in order to be properly aimed at the critical area on the fish.

The previous year’s team has designed and developed solutions for many of the problems associated to computer vision, however, there will still be plenty of work to do to improve upon these implement. As well as improving the vision system overall, by tackling problems that the previous year’s team did not have time to solve such as designing a protocol for handling when multiple lionfish are within the system’s field of view. Our current team looks forward to addressing these challenges.