[TMP-099] Prediction of static and perturbed reach goals from movement kinematics

Contact person: Patrizia Fattori 

Internal Partners:

1. Università di Bologna (UNIBO), Patrizia Fattori
2. German Research Centre for Artificial Intelligence (DFKI), Elsa Kirchner

 

Reaching movements towards targets located in the 3-dimensional space are fast and accurate. Although they may seem simple and natural movements, they imply the integration of different sensory information that is carried in real time by our brain. We apply machine learning techniques to address different questions as follows: i) at which point of the movement is it accurately possible to predict the final target goal in static and dynamic conditions? ii) as at behavioural level it was hypothesized that direction and depth dimension do not rely on shared networks in the brain during the execution of movement but they are processed separately, can the targets located along the horizontal or sagittal dimension be predicted with the same or different accuracy? Finally, we frame our result in the context of improving user-agent interactions, moving from a description of human movement to a possible implementation in social/collaborative AI.

Results Summary

We measured the kinematics of reaching movement in 12 participants towards visual targets located in the 3D-space. The targets could remain static or be perturbed at the movement onset. Experiment 1: by a supervised recurrent neural network, we tested at what point, during the movement, it was possible to accurately detect the reaching endpoints given the instantaneous x, y, z coordinates of the index and wrist. The classifier successfully predicted static and perturbed reaching endpoints with progressive increasing accuracy across movement execution (mean accuracy = 0.560.19, chance level = 0.16). Experiment 2: using the same network architecture, we trained a regressor to predict the future x, y, z position of the index and wrist given the actual x, y, z positions. X, y and z components of index and wrist showed an average Rsquared higher than 0.9 suggesting an optimal reconstruction of future trajectory given the actual one.

Tangible Outcomes

1. Individual subject trajectories – Annalisa Bosco
   https://drive.google.com/drive/folders/1FdDXKjhCupDdyLlyvCdUwfxmyGfoRZDE
2. Program/code: Recurrent neural network codes – Annalisa Bosco
   https://drive.google.com/drive/folders/1FdDXKjhCupDdyLlyvCdUwfxmyGfoRZDE
3. Video presentation summarizing the project