On the evening of March 4, the Beijing Winter Paralympics opened, and athletes from various countries (regions) competed fiercely with the Olympic spirit. There is always a game or a moment that touches your heart. Over the years, ZJUIers have also been paying attention to such groups in need, hoping to bring some convenience to their lives.

Research on wearable devices based on surface EMG signals

by XIAO Shuhong, YU Yue, ZHENG Xiuwen (22'); Instructor: WANG Hongwei

The students of this project group have paid attention to the increasing number of stroke hemiplegia patients in China, aiming to design a wearable device based on the potential decoding information of the epidermal motor nerve to make a strengthened response, so as to enable the patients with hand hemiplegia to carry out effective rehabilitation training. According to reaction behavior intention of the cerebral cortex motor neuron specific information and neuromuscular electrical signals, the connection between the hand surface by decoding the hand more specific parts of the surface electromyographic signal to obtain the corresponding action intention, so as to establish a relatively accurate corresponding relation, and borrow from the device to help strengthen and ultimately enable patients to use this device independent relatively difficult finish treatment rehabilitation and normal life in order to achieve purpose.

Camera-based Sensing and Visual Analysis for Robot-Simulated Neurological Examination Training 

by ZHENG Tianxiang, ZHOU Pengyang, SU Haoyu(22'); Instructor: YANG Liangjing   

The project includes the development of a camera-based marker-ess motion capture system to perform visual analysis. Its purpose is to develop the sensing and analysis capabilities of a robotic simulator for neurological examination training. After recording and analyzing the interaction between the clinician learner and the training simulator, the project team provides assessment for the clinician learner based on a large amount of clinical video data, making the test results more efficient and accurate, and helping to complete the robotic simulated neurological examination training.

They completed the calibration and correction of the camera parameters of the binocular camera, and realized the preliminary restoration of three-dimensional information through the binocular camera; and realized the effective identification and tracking of the upper arm of the robotic arm. And it has relatively good robustness in the whole movement process; successfully extracts the human skeleton information of the experimenter from the video image of a single camera through various methods, and also completes the extraction of the researcher’s hand movements.

Biomechanical Design and Clinical Evaluation of an Ankle-Foot Simulator for Ankle Clonus Assessment Training

Mechanical Design by HAN Tianyi (21'); 韩天屹 （17级）；Instructor: YANG Liangjing, Elizabeth T. Hsiao-Wecksler    

In this project, Han Tianyi undertook the design and production of a mechanical leg simulating clinical diagnostic training. The simulator has been effectively verified in both bench-top tests and clinician evaluations. It can provide medical students with opportunities for repeated training and effectively help the improving of diagnostic skills of ankle clonus. At the same time, relying on this project, he published the paper "Biomechanical Design and Clinical Evaluation of an Ankle-Foot Simulator for Ankle Clonus Assessment Training" at IEEE ICRA, a top international conference in the field of robotics, as the second author. The paper was also approved by IEEE RA- L journals included.

Design Optimization of a Mechanically Constrained Re-Localizable Remote-Center-of-Motion

by TAN Anqi, CHEN Shengwei, GAN Hanyu(22'); Instructor: YANG Liangjing

Minimally invasive surgeries have advantages in the reduction of infection risk, genesis of complications and recovery time. Especially for the surgeries operated by a robot manipulator controlled by the doctor through the computer system, it greatly reduces human risks and improves the efficiency. Such robot manipulator system has continuously developed in recent decades. During the Covid-19 pandemic, this remote manipulator system was further recommended to reduce the risks of infection.

Mechanical engineering students in ZJUI institute designed a mechanism that successfully realized RCM (Remote-center of motion, which means the mechanical constraint to limit the manipulator end effector at the incision point of the minimally invasive surgeries). The research team developed the model in the computer and simulated the motion to verify its functionality. Moreover, genetic algorithm was applied to optimize the mechanical parameters, leading to a relatively appropriate solution to satisfy the requirements of workspace during the surgeries.

Research on wearable devices based on surface EMG signals

by ZONG Zhenyu, WANG Tengyue (21'), Instructor: YANG Liangjing

A new surgery assisted system, combining with mixed reality technologies and robot manipulator control, is proposed to solve the limited field-of-view (FOV) in minimally invasive surgery (MIS). According to the preoperative information, a 3D scene is constructed in a development platform. Based on advanced MR technologies, including human-cobot interaction, imaging augmentation and immersive visualization, the system facilitates intuitive surgical procedures for surgeons. In addition, by utilizing robot manipulator to control the pose of the endoscope, the system provides a more stable and reliable control algorithm while satisfying the certain constraints, such as RCM and limited workspace.

Visual Optimization of Ultrasound-Guided Robot-Assisted Procedures Using Variable Impedance Control

by XIAO Songjie (PhD 3rd year)；Instructor: YANG Liangjing

Ultrasonography is one of the most common diagnostic and intraoperative imaging modalities in our daily life, due to the advantages of portability, non-invasiveness, low cost and real-time capabilities over other imaging techniques. The quality of ultrasound image depends largely on the experience of the surgeon. However, the excessive workload has been exposing the sonographers to health risks such as regional pain and musculoskeletal disorders, since they have to continuously exert a significant force onto the patient during imaging, in order to obtain a clear image.

To this end, we proposed a robotic control system to optimize visual quality of the ultrasound image using variable impedance control. This human-robot-environment interaction (HREI) system can control the robot in collaboration with the human while maintaining a constant contact force with the environment.

This kind of robotic solution can not only relieve the surgeon from the burden of intensive multi-tasking (like in ultrasound-guided percutaneous interventional procedures), but also improve repeatability in the acquisition state variables, which makes dimensional measurements possible to precisely reproduce at a later date. Additionally, we are working on adding haptic feedback into robotized tele-echography, which will provide the surgeon a more comprehensive perception with both visual and haptic information. And the teleoperation ability allows specialists to remotely examine patients who are not convenient to hospital or located in medically isolated sites. Moreover, it helps reduce the risk of infection for medical staff by avoiding direct patient contact, especially during a pandemic such as COVID-19.

Clinically Applicable 3D Tooth Semantic Segmentation in Intra-Oral Scans with Deep Learning

by XIONG Huimin (PhD 1st year), HE Xiaoxuan, (Graduate 1st year), CHEN Ruizhe (Graduate 1st year)); Instructor: LIU Zuozhu

Intra-Oral Scan (IOS) data refers to the three-dimensional topography and texture structure information of the soft and hard tissue surfaces such as teeth, gums, mucous membranes, etc. obtained by scanning the inside of the patient's oral cavity. Accurate classification of each face into corresponding teeth or gums is a key step in many digital oral diagnostic treatments (eg, malocclusion, etc.). However, in the actual diagnosis and treatment process, the IOS data of patient varies from individual to individual, with diversity and heterogeneity; and often accompanied by different clinical symptoms, such as crowded dentition, missing teeth, unclear boundaries, etc.; the model scanning process noise is often cropped up in the system, which brings great challenges to this task.

To this end, based on the deep learning algorithm of graph neural network and Bayesian statistical model, we propose a clinically applicable semantic segmentation framework for large-scale, high-precision, and heterogeneous IOS data, which can correct the shortcomings of previous algorithms, such as, it is impossible to distinguish adjacent teeth with similar characteristics, identification of erupting teeth, incomplete segmentation, etc., which greatly improves the accuracy and efficiency of segmentation. At the same time, we further propose a new 3D point cloud self-supervised learning framework, which exceeds the performance of the original supervised algorithm while using a very small amount of labeled data, which can greatly reduce the need for time-consuming manual annotation. Our algorithm has been unanimously recognized by doctors in the clinical applicability test, and has been successfully deployed in clinical diagnosis and treatment software to provide intelligent medical services for oral patients at home and abroad. Relevant results were published in Journal of Dental Research, a top journal of stomatology, and in top conference on medical image processing, e.g. Medical Imaging with Deep Learning.

Self-supervised Learning for Cone Beam CT Semantic Segmentation

by MA Jianhan (21'), ZHU Haoqiang(21'); Instructor: LIU Zuozhu

Accurate identification of tooth images in oral Cone Beam CT (CBCT) images is crucial for 3D image reconstruction of oral models and subsequent diagnosis and treatment plan design. But even for well-trained dentists, labeling CBCT images still take a lot of time. Students in this project are committed to developing a semantic segmentation model framework for CBCT images to improve the efficiency of dentists. They proposed a new self-supervised deep learning framework, which first unsupervised pre-training with a large number of unlabeled samples, and then fine-tuned with a small number of labeled samples, which is superior to the model that uses the same number of labeled samples for supervised learning. The segmentation effect of CBCT provides a feasible solution for reducing the human annotation required to develop CBCT segmentation models.

Impact the future with engineering and warm the world with sincerity

In the future, ZJUIer will continue to uphold humanistic care, and committed to create a better and more harmonious community with a shared future for mankind.