AZoSensors speaks with Mr. Takeshi Konno from Fujitsu about the release of their new millimeter-wave sensor system that aims to improve patient privacy and safety in healthcare settings. We discuss how this technology monitors body motion, as well as the impact this sensor could have on the medical sector.
Please could you introduce yourself and your role at Fujitsu?
My name is Takeshi Konno, and I am researching behavior detection technologies at Fujitsu Research. Behavior detection technologies fuse image analysis technologies with knowledge of human behavior and fall under the "converging technologies” category. Converging technologies combine two or more different social sciences and technology fields to achieve a specific goal and represent a major pillar of Fujitsu's research and development.
Fujitsu has developed a new technology to monitor body posture in healthcare settings. Could you describe this advancement and why it was created?
In 2019, Fujitsu developed a human sensing technology called "Actlyzer," an AI technology for video-based behavioral analysis. The challenge with traditional behavioral analysis technologies has been to prepare large amounts of learning data for the AI model.
The data set for Actlyzer consists of a combination of basic human movements, such as walking, head shaking, and hand reaching. The Actlyzer AI model has been trained with data to recognize about 100 types of basic human movements. By combining them, Actlyzer can recognize complex human behavior without requiring a large amount of learning data.
Currently, Actlyzer is used for real-world applications including usage in open spaces such as cities and retail stores, where it is often used in combination with cameras.
We started to utilize millimeter-wave sensors to offer a technology that can also be used in privacy-conscious locations such as hospitals and nursing homes where cameras cannot be installed.
© Fujitsu
What are millimeter-wave sensors, and why were they utilized in this new technology?
Millimeter-wave sensors emit radio waves and can acquire three-dimensional point cloud data. We focused on millimeter-wave sensors as point cloud data is a privacy-conscious way of monitoring information such as a person’s gender, age and body shape that are not detected by the sensors.
Instead of course-gained point cloud data, which is typical of cheap millimeter-wave sensors, this technology provides fine-grained point cloud data. Why was it important to expand this data type, and what impact will it have on the accuracy of posture estimation?
Coarse-grained point cloud data obtained by a conventional millimeter-wave sensor is not suited to deliver accurate estimations of postures of the human body. Fujitsu was trying to find ways to make point cloud data denser and noticed that human body postures could be represented as time-series point cloud data.
We have thus developed a new technology that can estimate human body postures with high accuracy. The technology works by efficiently selecting point group data suitable for estimating body postures from a large amount of point group data that multiple irradiations can acquire.
In this sense, our new technology is a converging technology as it combines technology with knowledge of human behavior.
© Fujitsu
A significant benefit to this technology is that it irradicates the need for intrusive cameras. Why is it important to develop technologies that not only improve patient safety, but also respect their privacy?
In February 2022, we conducted field trials with Tottori City (Tottori Prefecture, Japan) and in June 2022, trials with Kawasaki City (Kanagawa Prefecture, Japan). We are currently also conducting several field trials. When we talked to different companies and elderly adults, we learned that the barriers to placing cameras in privacy-conscious spaces are high.
We think that millimeter-wave sensors can be deployed safely in such spaces. As such, they are a necessary technology development to increase the reliability of healthcare systems and accelerate their deployment.
Fujitsu’s new technology can also help alleviate healthcare workers' workload in hospitals and care settings. Could you discuss this further?
Previous millimeter-wave sensors could only detect when patients fell and did not offer further information on what caused the fall.
Fujitsu’s new technology uses millimeter-wave sensors to estimate the postures of the human body, making it possible to perform a detailed analysis of human actions, including the behavior before and after a specific action.
In the case of elderly adults falling, for example, the new technology can detect if the fall happened when they got up from bed or when they were walking, making it easier for doctors, nurses and caregivers to judge the urgency of a situation.
Fujitsu’s new technology thus helps to enhance work processes at hospitals and nursing care facilities and ultimately reduce the workload of nurses and caregivers
Fujitsu: estimating body postures from point cloud data using millimeter-wave sensor (image video)
Fujitsu: estimating body postures from point cloud data using millimeter-wave sensor (image video). © FujitsuJpPR
This technology utilizes an AI model alongside the millimeter-wave sensor. With the rise of intelligent systems, we can expect more partnerships between sensor technologies and AI in the future. How is Fujitsu approaching the rise of these new interfaces?
Actlyzer, Fujitsu’s AI technology to analyze people's complex behaviors, originally utilized input from cameras.
With Actlyzer, we achieved accurate analyses of complex behavior, even when we replaced the cameras with another sensor technology (millimeter-wave sensors) as we combined technology with human behavior knowledge. By fusing the respective technologies and sciences in the same way, we believe that we will be able to adapt Actlyzer to various areas and scenes where AI technology can be applied even with new interfaces.
Fujitsu will conduct field trials in healthcare settings to verify this technology’s effectiveness. What do you think are the main challenges regarding this device’s expansion to market?
Problems with the accuracy of the posture estimation still remain. In collaboration with doctors, we have built a data set by narrowing down the fall scenarios that hospitals want to detect. The data set includes data from about 140 people of different body types, genders and ages and 50 different scenes. However, we think that it is necessary to create more data sets to accurately estimate all actions.
Other than posture monitoring in healthcare settings, could this technology be utilized in any other applications?
Unlike cameras, millimeter waves can be used in the dark. Millimeter-waves can also detect vital signs of humans, so they could also be used for crime prevention. For example, a robot equipped with a millimeter-wave sensor could detect suspicious persons in dark office rooms at night.
About Takeshi Konno
Takeshi Konno is a Project Manager at Fujitsu Research, overseeing the research and development group for behavior detection technologies. Mr. Konno joined Fujitsu Limited in 2003 and started work at Fujitsu Research in 2010. Mr. Konno's specialty is research on data analysis, prediction optimization, and image processing. In recent years he has been engaged in the development of behavior analysis technology using AI technology
Disclaimer: The views expressed here are those of the interviewee and do not necessarily represent the views of AZoM.com Limited (T/A) AZoNetwork, the owner and operator of this website. This disclaimer forms part of the Terms and Conditions of use of this website.