Startup’s Thermal Imaging and AR System for Firefighters Joins the COVID-19 Fight

Photo-illustraiton: Longan Vision An illustration of a firefighter wearing Longan Vision's AR visor with thermal imaging cameras over his helmet. The Fusion Vision System allows the first responder to see through smoke, locate victims, and find the source of the fire.

THE INSTITUTE Before the coronavirus pandemic hit Canada, Enzo Jia was busy developing the Fusion Vision System, an augmented-reality (AR) visor with thermal imaging to help firefighters see through smoke.

I always wanted to design something to help humans enhance their vision and also to see something they cannot with the naked eye," Jia told The Institute in March. He is chief executive of Longan Vision, a startup he helped found. I really want to help firefighters and first responders enhance their vision by using AR technologies."

The startup, based in Hamilton, Ont., was named a 2020 IEEE Entrepreneurship Star at this year's Consumer Electronics Show, held in January. The program recognizes early-stage companies that have the potential of bringing to market engineering-driven innovations in IEEE's field of interest. Along with the recognition, awardees get a free year of IEEE membership.

In the months before the pandemic, the company had developed a prototype of its Fusion Vision System and had demonstrated it to several fire departments.

When COVID-19 began spreading throughout Canada, Jia and his colleagues realized they could use some of the same technologies to combat the spread of the virus, so they began a side project. To detect a high body temperature, which is a common symptom of COVID-19, the startup used components from the visor to build Gatekeeper, a thermal-imaging system. Gatekeeper can be mounted on a wall or tripod to measure body temperature of up to five people at once.

Several units have been installed in long-term-care facilities, grocery stores, and universities, Jia says.

AR VISORS

Jia, a mechanical engineer, says he has been a fan of AR technology for some time. His undergraduate capstone project at McMaster University, in Hamilton, was about how AR could be used in vehicle head-up displays. Such displays, which already exist in some vehicles, can project information on the windshield, including navigation instructions, speed limit, and mileage.

Jia, who earned a bachelor's degree in automotive engineering technology, was a member of the university's IEEE student branch. He later earned a master's degree in mechanical engineering from the school.

He launched Longan in 2018 with five colleagues shortly after graduating. At first, the job wasn't full time. To get business experience under his belt as well as an understanding of how to manufacture products, he worked as a mechanical engineer for material-handling-equipment company Skyjack and automotive supplier Magna.

Today he works full time for Longan, which has five other full-time employees and two interns.

His initial idea was to develop AR glasses that integrated thermal imaging for industrial applications, such as Google Glass and Microsoft HoloLens. The company changed direction after several fires in Ontario caused major losses of life and property.

In some of these incidents, firefighters lost their lives saving people while the building was collapsing around them," Jia says. They needed to fight not only the fire but also [a lack of] time. Additionally, they needed to overcome obstacles, like lack of communication and terrible visibility. Their bravery inspired me."

Today's firefighters use outdated technology, he says. He compares their equipment to cellphones of the past-which offered only basic features such as calling and texting. He wants to give first responders commercially available smart technologies to jump-start them to the next generation of technology," he says.

We are developing a solution," he says, to try and address all three of these pain points: poor visibility, communication, and data."

Photo: Longan Vision

The visor [above] uses mixed-reality technology: AR and thermal imaging. A temperature sensor identifies the location of the fire. The headset allows firefighters to stream live video to a central command terminal so other responders can learn details about the structure and determine how to fight the blaze.

In a smoky building, you can't see with the human eye," Jia says. Our technology can actually see through the smoke. It can give firefighters super vision, tell them where there's a victim and where they can go past an object without bumping into it.

I believe our product with AR technology could help them prevent casualties, boost their efficiency, and send our heroes back to their families."

The images can be saved and reviewed later for training or for investigations by insurance companies. Longan Vision also plans to collect the data to create predictive models for firefighters, Jia says.

The visor is compatible with many styles of firefighter helmets including models used in Asia, Europe, and the United States. Jia says there currently aren't any other smart visors for firefighter helmets on the market.

OVERCOMING OBSTACLES

As with any startup, the biggest hurdle has been a lack of funding for product research, design, development, and hiring experienced workers, Jia says.

In general," he says, no budget means no progress, and no progress means death to a startup."

The company began presenting its work at conferences and participating in startup competitions including those held by the IEEE Entrepreneurship initiative.

Jia says those activities helped bring visibility to the company and attract investors. The founders also developed a business plan and began hiring experienced engineers who have the passion" and are willing to grow with the company," he says.

The first round of seed funding closed in July 2019. The company won contracts worth more than US $300,000, which went toward development, Jia says. One investor is Innovation Solutions Canada, which helps fund the country's startups.

MULTIPERSON TEMPERATURE CHECK

When The Institute spoke with Jia in July to see how the company was faring during the pandemic, he told us about Gatekeeper. In addition to AR and thermal-imaging technology, it also uses a facial-recognition system.

Photos: Longan Vision The Gatekeeper measurement component [left] monitors the calibrated temperature of an area. The Gatekeeper camera component [right] scans individuals who cross its path and produces a visual image of each person and displays the individual's temperature.

Jia worked with McMaster's business incubator program, The Forge, on that detection system, which uses two thermal-imaging tools to get accurate temperature readings. The Gatekeeper measurement component monitors the calibrated temperature of an area. The Gatekeeper camera component (GCC) scans individuals who cross its path.

The gathered data is sent to a computer that has a built-in image-processing program, which looks for variances between the two feeds. Information from the GCC produces a visual image of each person and displays a message on the screen identifying each individual's temperature. Should an elevated temperature be detected, the individual can be asked to return home or seek medical attention.

Jia says his system is less expensive than others on the market because Gatekeeper uses off-the-shelf components.

STAR TREATMENT

Jia says he was humbled to receive the IEEE entrepreneurship recognition, which he says has opened up many doors.

It's a big honor to be named an IEEE Entrepreneurship Star. It gives us a really big thumbs-up," he says. Receiving the award is also helping us on the development side. Being part of IEEE's communities means we can actually expand our network and access potential investors and technologies that we can integrate into our [visor] system."

Another benefit that comes with the recognition is a free subscription to the IEEE Xplore Digital Library-which Jia says he is thrilled about. He says much of his master's thesis was based on research he found in the library.

We need access to papers," he says. That means a lot to us, to be honest. We can use the IEEE community to share our research findings on thermal imaging, AR and how it helps first responders."

< Back to IEEE COVID-19 Resources