Fujitsu Develops New Technology for Accurately Estimating Postures of the Human Body from Point Cloud Data Using Millimeter-wave Sensor

Jul 06, 2022 10:38 JST

Source: Fujitsu Ltd

Detailed monitoring in hospitals and nursing homes without cameras to ensure patients' privacy

TOKYO, Jul 06, 2022 - (JCN Newswire) - Fujitsu has successfully developed a new technology to accurately estimate postures of the human body from coarse-grained point cloud data obtained with a conventional millimeter-wave sensor. Fujitsu plans to deploy the newly developed technology at hospitals and nursing care facilities to support nurses and caregivers in visually monitoring patients and reacting to emergency situations such as falls of patients.

Figure: Overview of the newly developed technology

The technology leverages an AI model for highly accurate estimations of postures of the human body and Fujitsu's "Actlyzer"(1) AI technology for analysis of complex human behaviors to analyze the movement of patients in detail before and after a fall without installing cameras in highly private spaces such as hospitals and nursing homes.

Fujitsu will conduct field trials with hospitals and nursing care facilities to verify the effectiveness and improve the accuracy of the new technology with the aim to offer it as a service to the Japanese market by the end of FY 2023.

Innovation to improve independence and safety for the elderly and those with impaired mobility
Securing the safety of patients and elderly people while reducing the workload of nurses and caregivers represents an increasingly urgent task at hospitals and nursing homes.

While monitoring technology using sensors has been attracting attention in recent years, the demand for solutions using millimeter wave sensors is expected to grow as it can be deployed at low cost and also helps preserve the privacy of patients.

However, low-priced millimeter-wave sensors, which are widely available, provide only coarse-grained point cloud data and are thus often not accurate enough to detect falls of patients and the elderly and to perform detailed analysis of their behavior before and after falls.

To address this issue, Fujitsu has developed a technology that enables expansion from coarse-grained point cloud data to fine-grained point cloud data necessary for an accurate posture estimation by fusing time series information of point cloud data in a series of movements of the human body. The new technology that does not utilize cameras will help to support nurses and caregivers to monitor patients and achieve a faster response to emergency situations while ensuring patients' privacy.

About the newly developed technology

Fujitsu has developed a new technology that enables a detailed analysis of human behavior using a common 79 GHz millimeter-wave sensor compliant with the Radio Law of Japan.
In February 2022, Tottori City (Tottori Prefecture, Japan) and Fujitsu Japan Co., Ltd.(2), conducted field trials with elderly people living alone in municipal housing units using a prototype of the newly developed technology and confirmed the effectiveness of the technology in privacy-conscious monitoring of the residents.

In June 2022, Fujitsu and Kawasaki City (Kanagawa prefecture, Japan) conducted trials at the simulation laboratory "Kawasaki Welfare Technology Lab" ("Weltech", a facility to support the development and improvement of welfare products and services), to test the response of the prototype of the new technology and notifications sent by the system in various behavior situations such as elderly people getting out of bed. Based on the results of these tests, Fujitsu and members of Weltech plan to conduct field trials of the prototype at an actual facility for the elderly from August 2022 onwards.

Features of the new technology:

1. Point group data extension technology for generating input data suitable for measurements of postures of the human body
Development of a point cloud data extension technology that can be extended to fine-grained point cloud data by selecting point cloud data suitable for the estimation of postures from a large amount of point cloud data that can be acquired by multiple radio waves(3)
2. Large-scale dataset and AI model to measure postures of the human body
Construction of a large-scale data set that combines point cloud data with three dimensional coordinates of human joints based on point cloud data extended to sufficient granularity to estimate the position of the human body to achieve estimations with higher accuracy
Development of an AI model for highly accurate estimations of the position of the human body based on a dataset consisting of behavioral data from about 140 people in about 50 different scenes
3. Actlyzer AI technology for complex and detailed analysis of human behavior
Detailed analysis of various human behaviors (falling after standing up from bed or during walking as well as behavior before and after falling etc.) through utilizing Actlyzer, an AI technology able to analyze complex human behavior by combining approximately 100 types of basic movement data to support nurses and caregivers in visually monitoring patients and achieving a faster response time to emergency situations

(1) Behavior analytics technology Actlyzer: technology commercialized as Fujitsu's AI technology for video-based behavioral analysis solution "FUJITSU Technical Computing Solution GREENAGES Citywide Surveillance" to recognize human actions.
(2) Fujitsu Japan Co., Ltd. : Headquarters: Minato-ku, Tokyo; COO and Representative Director of the Board: Takayuki Sunada.
(3) Postures of the human body can be expressed as time-series point cloud data as even a general millimeter-wave sensor with a coarse granularity of point cloud data that can be acquired by radio wave irradiation at one time can achieve highly accurate estimation.

The Sustainable Development Goals (SDGs) adopted by the United Nations in 2015 represent a set of common goals to be achieved worldwide by 2030. Fujitsu?s purpose -- "to make the world more sustainable by building trust in society through innovation" -- is a promise to contribute to the vision of a better future empowered by the SDGs.

About Fujitsu

Fujitsu's purpose is to make the world more sustainable by building trust in society through innovation. As the digital transformation partner of choice for customers in over 100 countries, our 124,000 employees work to resolve some of the greatest challenges facing humanity. Our range of services and solutions draw on five key technologies: Computing, Networks, AI, Data & Security, and Converging Technologies, which we bring together to deliver sustainability transformation. Fujitsu Limited (TSE:6702) reported consolidated revenues of 3.6 trillion yen (US$32 billion) for the fiscal year ended March 31, 2022 and remains the top digital services company in Japan by market share. Find out more: www.fujitsu.com.

Source: Fujitsu Ltd
Sectors: Enterprise IT