Sea Machines Analyzes Diverse Visual Data To Deliver Safe Autonomous Boats
/
Fiona Hua, Lead Perception Research Scientist at Sea Machines Robotics (courtesy Fiona Hua)
The fast clip of innovation these days often translates to huge, massive changes. As industries mature, even with leading-edge technologies, changes come not in the form of exciting research but in tiny shifts. That was Fiona Hua’s experience as a research scientist, before joining Sea Machines, a marine- and maritime-focused technology company that recently closed a $10 million Series A. (LDV Capital invested in their seed and Series A alongside other top-tier funds including Accomplice, Eniac, Launch Capital, Geekdom, Toyota AI and others.)
“I have worked in biometrics for more than 10 years, developing algorithms for facial, iris and fingerprint recognition with products for millions of users all over the world from bank login system, law enforcement to border control applications. I have seen how new technologies helped to boost the system performance from good to superb, which makes the products much more reliable for daily use. But this area has been well researched and the accuracy could be 99.9999%. People now in this area really are working on improving that small tail of accuracy.” Interesting, but not interesting enough — Fiona knew she was ready for a new challenge.
Enter the prospect of autonomous vessels. “I am really interested in the autonomous industry, because it's happening, and it's changing the world. Comparing biometrics to autonomous perception work, the data is different, the operational situation is different, the problem is different, but the way to solve problems is similar and the fundamental technology is almost the same.”
In this article, Fiona talks about leading the perception team at Boston-based Sea Machines, one of our portfolio companies. Read on to learn about her journey from biometrics to autonomous vessels, how Sea Machines differentiates itself in the market, and more.
FROM SELF-DRIVING CARS TO SELF-DRIVING VESSELS
At first glance, biometrics might seem to have little in common with the operation of boats. That’s what Fiona thought until she got the phone call about Sea Machines.
After initial conversations, Fiona realized the as-yet-unfamiliar field included similar technologies but with unfamiliar data: how to collect and organize data, how to deal with incomplete data, and how to optimize your system to learn maximum information from the data.
“I started to think, wow this is a huge domain and a very important industry. Autonomous shipping is the future of the maritime industry. Similar to the autonomous car industry, autonomous vessels could affect everyone’s daily life. That's why I got into Sea Machines,” says Fiona. She immediately recognized that she could leverage her technical skills and experiences in a new way.
HOW SEA MACHINES’ PERCEPTION TEAM IS WORKING TO MAKE BOATS THINK
Sea Machines Robotics specializes in advanced control technology for workboats and other commercial surface vessels. (courtesy Sea Machines)
As the perception team lead at Sea Machines, Fiona is responsible for guiding the perception research and collaborating with other teams with the same goal in mind: “To make the vessels see more, sense more, and think more. We want to help the vessel to understand the situation around it, like locations of other vessels and markers, the sea status, the weather, etc. As long as the vessel knows what surround it, we can teach the vessel how to react to the environment appropriately.”
“We want to help the vessel to understand the situation around it, like locations of other vessels and markers, the sea status, the weather, etc. As long as the vessel knows what surround it, we can teach the vessel how to react to the environment appropriately.”
To help vessels sense their surrounding environment, the perception team uses multiple onboard sensors to gather as much info as possible: Radar, LiDAR, AIS, GPS, visible cameras and thermal cameras. From there, the team employs state-of-the-art technologies including deep learning, computer vision, image processing, and sensor fusion to process these different sensor data and translate them into information a human can easily understand.
It’s clear that the self-driving car industry is pioneering the algorithms and frameworks to provide autonomous technologies, says Fiona. But while the maritime use case is not quite at the maturity as the automotive industry, vessels have an advantage when it comes to sensors.
“It’s debatable if a car should have additional sensors other than cameras due to the cost,” says Fiona. On the contrary, cost is less of a concern for maritime use because most engine-powered ships may have already installed expensive sensors like Radar, AIS and GPS systems.
“Adding cameras, like wide dynamic range HD cameras and thermal cameras, is a very welcome thing for the captain/driver since they may “see” more from cameras, which is especially a huge convenience for big container ships. Adding LiDar is not a big cost when safety is the big concern for commercial vessels. All these sensor data are great resources that ultimately could make vessels safer, more efficient, and less expensive.”
This doesn’t mean it’s easier to develop technology for autonomous vessels, Fiona explains. “Sea surface vessels have much more complexity compared to cars. These vessels have different shapes, types, sizes, and they can be seen in different angles, speed, and distance. The weather conditions and sea states could have huge extreme conditions, which is hard to predict. The behaviors and controls of vessels for the collision avoidance are quite different for a thousand-foot cargo ship than a small buoy. All these variations add complications to our vision tasks. By looking at a busy harbor with many buoys, small crossing boats, and several thousand-feet container ships on the background of constructions, it is easy to understand the difficulties for perception work.”
LEADING THE PERCEPTION TEAM AT SEA MACHINES
So what’s a typical day for Fiona and the perception team? As might be expected, it’s a combination of a lot of research and time in the field.
“Thirty percent of the time we are talking about new ideas, new solutions, because I want to help my team finding the right direction and to optimize our work. Another forty percent of the time I'm actually doing the real work, the research, looking at the papers, coding, understanding the data.”
That doesn’t include the time spent collaborating with other colleagues around the world, or the data management itself.
“We are working closely with our Hamburg (Germany) team for integrating our perception system on Maersk’s thousand-feet container ship; we are sending data to an annotation company for labeling the ships in images; we go by boat with the testing team for understanding the real operational situation and what the harbor looks like.”
There’s also the experience of being out on the vessels themselves that’s unique to this role, says Fiona.”I like to find some time go with our lead testing captain. I want to know how she uses our developed autonomy control system, how to test and what the problems are. And I also like to know her experiences and future perspective of maritime industry.”
Being on the boat gives Fiona a chance to experience the traffic a vessel encounters — of many different types and sizes.
“Our testing team drives boats out in the harbor testing every day. All the way from the docker to the testing area, there are different vessels driving by, canoes, lobster boats, cruises, or even cargo ships. When you are really out there, you will learn how busy it can be in the harbor. It's much busier than you expect. If you have a chance to drive even further offshore, it's less busy than you expect.”
The perception team’s product has not integrated into the self-driving boat yet, but it will be testing on boat soon.
BEYOND TECH: SEA MACHINES STANDS OUT WITH DOMAIN EXPERTISE
There are several other companies doing similar things with autonomous maritime vessels, but according to investors, says Fiona, “Sea Machines has the most capability to bring this mission to the real product.”
What makes Sea Machines stand out is not only its technology and its big vision for the future, says Fiona. It’s also the deep understanding of the maritime industry from its leadership team. Sea Machines’ CEO Michael Johnson, COO Jim Daly and its Boston-based engineers and testing captain all have deep experience in the maritime industry. “They know deeply about the industry, where the needs of change come from, and what is possible to be changed. Everything that we are talking about, working on and aiming for are key parts for the future of the maritime industry.”
This specific expertise doesn’t limit their vision of the future. It makes them humble to absorb new concepts and open to new technology. They believe the vision, the autonomy and the combination of other new technologies could eventually lead the maritime industry into a new generation with smart, safer, efficient vessels.
Sea Machines’ vision, the autonomy and the combination of other new technologies could eventually lead the maritime industry into a new generation with smart, safer, efficient vessels.
If you’re building a unique visual tech company, we would love for you to join us. At LDV Capital, we’re focused on investing in deep tech people building visual technology businesses. Through our annual LDV Vision Summit and monthly community dinners, we bring together top technologists, researchers, startups, media/brand executives, creators and investors with the purpose of exploring how visual technologies leveraging computer vision, machine learning and artificial intelligence are revolutionizing how humans communicate and do business.