Associated-Team: @WALK / ArTificial WALKing
@WALK (ArTificial WALKing) is a collaborative project supported by INRIA for researchers and students exchanges between DEMAR Project-Team and the Robotics Lab of Stanford University. French Coordinator: Pr. P. Fraisse (LIRMM), Université Montpellier 2, France, Partner Coordinator: Pr. O. Khatib (Robotics Lab), Stanford University, USA
Scientific goals of the proposal
Overall Objectives
The motivation approach is the complementary research works of these teams. Indeed, a collaborative project should give an additional value to their research results. On one hand, the DEMAR Project Team has experience in Functional Electrical Stimulation to restore or modulate movements on spinal cord injured patients and post stroke patients. In both pathologies researches on assisted gait using FES (for paraplegics with a walker and hemiplegics) are carried out in the team (cf. Figure 1).
Figure 1: (a) Paraplegic patient walking under FES
On the other hand, the Robotics research group (Stanford) carries out manipulation tasks with a humanoid robot under equilibrium constraints (cf. Figure 2).
Figure 2: Stanbot manipulating an object on SAI
Collaborative projects
Within the framework of the previous collaboration, the crossed visits and seminars last year led us to identify several research topics of feasible collaborations. We have selected four research activities, two from each side. We describe briefly each of them and make a conclusion on the topics we want to focus our efforts.
Biomechanical Analysis of Robotics Optimal Motion (Stanford)
Human motion analysis is used to simulate human movement and study human skills, which can then be implemented on humanoid robots. Human motor development depends on motor coordination and posture control as well as the development of strength, and perception. Development of such skillful dynamic behaviors for humanoid robots is possible by drawing inspiration from their biological counterparts. Acquisition of motion capture data from a Tai chi master offers the opportunity to study motions that have been honed to near optimality with respect to skill performance (cf. Figure 3). Once we better understand how humans move and interact with the world, we can transfer these skills to robots using similar energy minimization criteria which have evolved and which humans have learned through experience. The ultimate purpose of this with respect to robotics is to implement these findings into humanoid robots to enhance their agility and efficiency.
Figure 3: The Tai chi master’s movements are recorded using markers in three-dimensional space. These markers are then mapped to joint angles that will be the basis for comparison between the taiji master’s movements and optimally efficient movements simulated by a humanoid model robot (Stanbot).
Preliminary Findings: The patterns of motion of a Tai chi master are a source of inspiration for creating humanoid simulation of ideal skilled human movement. For this purpose, we started by implementing the direct control of marker trajectory data obtained from motion capture. Our controller SAI is used to map the cloud of Cartesian space marker positions into a humanoid model, Stanbot. This is done based on task-level control strategies. We started by scaling the Tai Chi master anthropometry to fit Stanbot. Then, we tracked the right arm (shoulder, elbow, and wrist) and hip by direct marker control. Finally, we obtained corresponding joint angles without referring to inverse kinematics. This approach produced good results. The tracked trajectories of the shoulder, elbow and wrist match almost exactly the corresponding goal trajectories. Once the joint angles are determined, the next step is to use them to analyze the energetic of high performance Tai chi motion patterns.
Modeling Muscles (Demar)
The scientific approach is to develop multi scale models based on the physiological microscopic reality up to a macroscopic behavior of the main parts of the sensory motor system: muscles, natural sensors and neural structures (cf. Figure 4). We also aim at describing multi scale time models to determine impulse-synchronized responses that occur in a reflex or with FES, up to a long-term fatigue phenomenon. All these models have a control input that allows them to be linked as different blocks of the sensory motor system.
Figure 4: Muscle model developed by DEMAR
Besides, we have to deal with problems related to the identification protocols. Identification is then based on the observation of signals such as EMG, output forces, and movement kinematics, while medical imaging gives the geometrical parameters and mass distributions. The success of the identification process is highly sensitive to the quality of the experimental protocols on humans.
Movement synthesis (Demar)
As regards synthesis, generating a useful and efficient movement means that criteria can be defined and evaluated through an accurate numeric simulation. Optimization methods are then used to process the data in order to obtain stimulation patterns for a given movement. Two problems occur, firstly the complexity of the models may provoke the failure of the optimization process, secondly the criteria that have to be optimized are not always known. For instance, we have to define what is a “normal” gait for a paraplegic patient under FES; are the global energy, the joint torques, the estimated fatigue for each muscle related to the appropriate criteria.
Simulation and Active Interface (SAI) (Stanford)
SAI was designed as a graphics and dynamics engine to simulate multi degrees of freedom robots (cf. Figure 5). The environment offers functionalities to connect active interfaces such as force feedback devices for real time interactions with the simulated bodies.
Figure 5: Simulation and Active Interface
The environment contains a Graphics Engine based on Open GL and a Dynamics Engine developed by Arachi Company for rigid body simulation. The libraries have been used so far for robot simulation where multi bodies can interact with each other. Complex controllers are currently under development and will be integrated into SAI to pilot multi degrees of freedoms mechanisms.
2. Partners presentation
a) DEMAR is a joint project between INRIA, CNRS, Universities of Montpellier 1 and 2. DEMAR is located at LIRMM (joint CNRS and University laboratory working on Computer sciences, Micro electronics, and Robotics) in Montpellier. DEMAR works in close relationship with rehabilitation centers among them the Centre Bouffard Vercelli in Cerbère, Propara in Montpellier, and CHU in Nîmes. DEMAR research interests are centered on the human sensory motor system, including muscles, sensory feedbacks, and neural motor networks. Indeed, DEMAR focuses on three global topics of research:
– Models and identification of the human sensory motor system.
– Functions’ synthesis and control.
– Interfaces between artificial and natural parts through implanted / external neuroprosthetic devices.
The main applied research fields are then:
– Quantitative characterization of the human sensory motor system firstly for motor disorders diagnosis and objective quantification, and secondly in order to help the design of neuroprosthetic devices.
– Restoring motor and sensitive functions through implanted / external functional electrical stimulation (FES) and neural signals sensing.
– Improving surface stimulation for therapy (verticalization of paraplegic patients, reduction of tremor, reeducation of hemiplegic post-stroke patients…)
The researchers involved in this project are Philippe Fraisse (Pr, UM2), Mitsuhiro Hayashibe (CR INRIA), David Guiraud (DR INRIA), Michele Vanoncini (Post-doc), Vincent Bonnet (Post-doc), Qin Zhang (PhD Student).
Web links: http://www.lirmm.fr/DEMAR/, http://www.lirmm.fr/~fraisse/ ,http://www.lirmm.fr/~hayashibe/ .
b) The Robotics research group is located at the Stanford University in the Artificial Intelligence Laboratory and leaded by Professor O. Khatib. The research activities are mainly dedicated to Humanoid Robotics. The team has developed a whole body control approach carrying out complex tasks including manipulation and walking. The robotics lab is also involved in the Bio-X program from Stanford University bringing the knowledge of biomechanic modeling and anthropomorphic control structure (humanoid robot). The main research fields are:
– Methodologies and technologies of autonomous robots,
– Cooperative robots,
– Human-centered robotics,
– Haptic interaction,
– Dynamic simulation,
– Virtual environments,
– Augmented teleoperation,
– Human-friendly robot design
The researchers involved in this project are Oussama Khatib (Pr.), Luis Sentis (Post-doc), Emel Demircan (PhD Student), Samir Menon (PhD Student).
Web links: http://robotics.stanford.edu/~ok/, http://robotics.stanford.edu/~lsentis/,http://stanford.edu/~emeld/, http://ai.stanford.edu/~smenon/
Background of the collaboration between the teams:
We have started our discussion about a potential collaboration 2 years ago by pointing out our complementarities in term of research activities for rehabilitation robotics. Philippe Fraisse has visited the AI Lab in 2008 for elaborating the objectives of this collaboration while analyzing the potentialities of the interactions between the project-team DEMAR and the Robotics Lab. Emel Demircan came to the LIRMM in April 2009 for starting a research collaboration with M. Hayashibe, M. Vanoncini and P. Fraisse. These visits have been fruitful and very promising. At this moment, we have planned some experimentation about EMG measurements and Sit-to-Stand transition for October 2009. These experimentations are currently in progress and should be done end of November. We planned to publish the results of these collaborative studies in 2010. This is why this period of time is critical in term of funding because of travel expenses needed to complete the first phase of this work within the next year. We have to mention the fact that a part of the previous travel expenses have been supported by DEMAR and Robotics Lab teams on own budgets and the other part was on a grant of 14000$ from France-Stanford Interdisciplinary Studies.
3. Impact
The impact of the collaboration between our teams concerns the following topics:
– Simulation Tools: DEMAR team needs for advanced simulation software that may be used to test movement synthesis and control strategies. We want to use our model but we have not the task force to develop our own simulator of a whole body system. On the other hand The Stanford university wants to test more accurate models of muscles in conditions where the non linearities becomes important such as isometric force controlled action, reflex actions, very slow to very fast movements with the same set of parameters.
– Experimental Platform: Although DEMAR has access to a humanoid robot (HOAP3) it is less realistic than the ASIMO platform. Moreover Stanford has permanent engineers to develop software for the robot. Humanoid robotics could be of interest for DEMAR but, as for simulation, our task force is not so important to carry out deep researches on this area. On the other hand, AI lab, has very few experience with patients and FES and they are interested in the control of such systems using FES. They haven’t the technology nor the skill to perform advanced clinical trials on paraplegics.
– Student’s exchanges have already begun but are very limited due to the lack of financial support. We aim at providing regular exchanges about 1 to 2 months a year to increase the scientific co-researches and ensure a greater success.
Our teams are really complementary so that synergies may be beneficial to each other. This formal collaboration would be the first international one in the field of movement control and synthesis that represents the second, among three, main objectives of the team for the next 4 years.