Muscleless motor synergies and actions without movements: From motor neuroscience to cognitive robotics

Highlights

• In both humans and cognitive robots, real and imagined actions must seamlessly alternate during any goal directed behavior and social interaction.

• Action generation/prediction must represent a continuum and hence ‘recycle’ some underlying computational building blocks.

• A shared cortical/computational basis for action “generation, imagination” in humans and robots is presented.

• Goal oriented animation of the internal body model facilitates both action generation and prediction of consequences of potential actions (of oneself, others).

• Even real movements are consequences of an internal simulation.

Abstract

Emerging trends in neurosciences are providing converging evidence that cortical networks in predominantly motor areas are activated in several contexts related to ‘action’ that do not cause any overt movement. Indeed for any complex body, human or embodied robot inhabiting unstructured environments, the dual processes of shaping motor output during action execution and providing the self with information related to feasibility, consequence and understanding of potential actions (of oneself/others) must seamlessly alternate during goal-oriented behaviors, social interactions. While prominent approaches like Optimal Control, Active Inference converge on the role of forward models, they diverge on the underlying computational basis. In this context, revisiting older ideas from motor control like the Equilibrium Point Hypothesis and synergy formation, this article offers an alternative perspective emphasizing the functional role of a ‘plastic, configurable’ internal representation of the body (body-schema) as a critical link enabling the seamless continuum between motor control and imagery. With the central proposition that both “real and imagined” actions are consequences of an internal simulation process achieved though passive goal-oriented animation of the body schema, the computational/neural basis of muscleless motor synergies (and ensuing simulated actions without movements) is explored. The rationale behind this perspective is articulated in the context of several interdisciplinary studies in motor neurosciences (for example, intracranial depth recordings from the parietal cortex, FMRI studies highlighting a shared cortical basis for action ‘execution, imagination and understanding’), animal cognition (in particular, tool-use and neuro-rehabilitation experiments, revealing how coordinated tools are incorporated as an extension to the body schema) and pertinent challenges towards building cognitive robots that can seamlessly “act, interact, anticipate and understand” in unstructured natural living spaces.

Introduction

Even during common daily activities like dining together, playing a game, using a tool, assembling an object from constituent parts etc., we effortlessly generate dexterous actions, predict potential consequences' of possible actions of oneself (and others). Emerging trends in motor neurosciences now provide converging evidence that seemingly disparate functions of action ‘generation, simulation, and observation’ consistently engage an overlapping network of cortical areas in the predominantly motor region of the brain (see Ptak et al. [86], Pickering and Clark [78], Grafton et al. [42], Pulvenmuller [77], Gallese and Sinigaglia [41], Kranczioch et al. [60] for reviews). The general insight emerging is that the fundamental problems of shaping motor output during action execution and providing the self with critical information related to feasibility, consequence and understanding of potential actions are closely intertwined than previously conceived. From an evolutionary perspective too, in any organism with complex body (human or embodied robot) inhabiting unstructured environments, actions are ‘goal oriented’ and not just stimulus oriented. This fundamentally requires ‘covert simulation and overt execution of action’ to incessantly alternate during the evolution of purposive behaviors and social interactions. In this sense, real and imagined actions are like Siamese twins, inseparable but to some extent independent. How is this delicate balance realized in the brain? How are cortical substrates basically involved in organization of overt action functionally ‘recycled’ in other contexts (i.e. actions without movements)? While the prevalent computational modeling approaches generally converge on the role of internal models, they diverge on the perspective of how they might be realized in the brain (Pickering and Clark [78]) or modeled computationally (Friston [29]).

Revisiting older ideas from motor control like the Equilibrium Point Hypothesis (EPH) and synergy formation (Bernstein [8], Asatryan and Feldman [2], Bizzi et al. [13], Abend et al. [1]) in the context of emerging trends in motor neuroscience, this article offers an alternative perspective emphasizing the functional role of a ‘plastic, expandable’ internal representation of the body i.e. the ‘body schema’, as a fundamental connecting link to facilitate the seamless continuum between motor control and motor imagery. In general, synergies are often associated with muscles and actions with movements. Instead, we posit that muscleless motor synergies emerging from the goal-oriented animation of the body schema (and expandable to coupled tools) is the basic mechanism to unify the computational basis of actions with and without movements while we ‘act, anticipate and understand’. The underlying rationale is put in the context of several recent studies in motor cognition in both humans and embodied robots in particular: a) Recording from the parietal cortex in patients undergoing awake brain surgery suggesting the coupling between motor imagery and internal representation of the body; b) Functional imaging studies emphasizing the shared cortical basis for ‘execution, imagination and understanding’ of action; c) Studies related to tool-use learning, revealing how tools are incorporated as an extension to the body schema during coordination, highlighting the blurred distinction between tool and the body (other bodies); d) The still pertinent problems in making complex redundant robots more dexterous, cognitive and social: opening up practical issues like need for computational simplicity, task specific configurability, effective human robot interaction; e) How all of this influences our understanding of synthesis of overt movements itself, a pertinent topic that has also been in recent debate. The next subsections review the literature and various issues related to coordination of a complex body (human/robot) inhabiting unstructured environments and their link to muscleless motor synergies and simulation of action to support goal directed reasoning and social interaction.

Unlike the range of direct problems in conventional physics (like computing effects of forces on objects), during the production of common day to day movements our brains have to deal exactly with the inverse problems of determining muscle activations, joint rotations, movement trajectories, speed profiles that would allow the desired goal-directed interaction with the environment. Strikingly, many of the inverse problems faced by the brain to control movements are indeed similar to the ones roboticians must solve to make their robots move dexterously. Note that, while coordinating any complex body (human or robot), the underlying control system has to deal with two typically contrasting requirements: the need to choose ‘one’ from infinite possible solutions (Bernstein's Degrees of Freedom problem) and the need to produce ‘one solution’ in an infinite number of ways (Lashley's Principle of Motor equivalence). As a simple example, even the task of aimlessly moving the finger from one point to another can be achieved in an infinite number of ways (motion trajectories, joint rotations, speed profiles, muscle activations). Further in most manipulation tasks, the solution must be compatible with a combination of multiple bodily (joint limits, torque limits etc.) and task specific (desired end-effector pose, obstacles etc.) constraints. How does the brain deal with this ‘problem of plenty’ and how can embodied robots efficiently coordinate their complex bodies to generate dexterous goal oriented movements? The present understanding of the plausible underlying computational basis is broadly based on three interrelated yet diverging frameworks i.e. Optimal Control (OCT: Diedrichsen et al. [34], Todorov [88]), Active Inference (AI: Friston [29]) and Equilibrium Point hypothesis (EPH: Asatryan and Feldman [2], Bizzi et al. [13]), see Pickering and Clark [78], Mohan and Morasso [66] for reviews on the pros and cons and interrelations between these approaches.

While OCT has emerged as a dominant theory for interpreting a range of motor behaviors (Scott [91], Li [92], Chhabra et al. [93], Shadmehr et al. [34], Harris and Wolpert [49], Kording et al. [59]), online movement corrections (Saunders and Knill [111]; Liu and Todorov [112]), structure of motor variability (Guigon et al. [94]), Fitts' law and control of precision, and coordinating anthropomorphic robots (Nori et al. [95], Kumar et al. [96], Ivaldi et al. [55], Parmiggiani et al. [74], Fumagalli et al. [38]), several open questions like the massive computational cost to perform the necessary complex optimizations especially in highly redundant systems like robots and humans (Doya [98], Friston [29]), hence the neuro-biological plausibility (Todorov [97]), the nature of the cost function itself given that multiple cost functions (minimum jerk, torque change, end point variance, object crackle etc.) make similar predictions on basic qualitative characteristics of movement and the issue of sub-optimality (Kodl et al. [99]) have been in recent debate.

In this context, multiple authors (Friston [29], Mohan and Morasso [66], Herbort and Butz [101], Pickering and Clark [78]) have raised an even deeper pertinent question i.e. do muscle activations cause joint rotations that cause the end effector motion ‘or’ is it the other way round? This perspective sounding like the classical ‘chicken vs. egg’ problem draws upon ideas from different disciplines like active inference and predictive coding (Friston [45], Kilner [48]), Ideomotor theory (Herbort and Butz [101]), the EPH (Feldman and Levin [37]), and has at least three following ramifications towards shaping our understanding of coordination of action in humans/robots:

(a) Computational cost/simplicity: this line of thinking converts an ‘ill posed’ problem of motor control (one from many) into a ‘well posed’ problem (one to one) thus circumventing the need for explicit kinematic inversions (Mohan and Morasso [66], Pickering and Clark [78]) and cost function computations that can be prohibitive for example while coordinating a 53-DoF humanoid;

(b) Real and imagined actions: the idea of action being understood as consequences of our own predictions concerning the flow of sensation i.e. a version of the Ideomotor theory of James [46] and Lotze [47], resonates with emerging studies form neurosciences suggesting that action ‘generation, perception, simulation’ share cortical substrates (thus complementing (a));

(c) Embodiment: emphasizing that it might be possible to simplify the computational process of coordination of action by actively exploiting properties and constraints of the physical system that is being coordinated (like, stiffness, compliance, reflex, local-to-global distributed processing) drawing upon ideas from EPH and Embodied simulation (hence complementing (a) and (b));

These topics (a)–(c) will be connected gradually with both empirical studies from humans and experiments with robots as we progress.

A general concept that was in the background of many studies to explain neural control of movement during the mid-60s to mid-80s was the Equilibrium Point Hypothesis (EPH: Asatryan and Feldman [2], Feldman [36], Bizzi et al. [10], [13], Feldman and Levin [37]). Innovative aspects of the EPH was its strong grounding in the biomechanics of the body and the apparent computational simplicity in solving the degrees of freedom problem. The basic Idea was that posture is not directly controlled by the brain in a detailed way but is a ‘biomechanical consequence’ of equilibrium among a large set of muscular and environmental forces. In other words, complex actions can indeed be achieved (i.e. choosing of one from many) without a complex, high dimensional optimization process by simply allowing the intrinsic dynamics of the neuromuscular system to seek its equilibrium state when trigged by intended goals.

The EPH idea exploited two beneficial properties of the neuromuscular apparatus of the body: 1) to induce, locally (in a muscle-wise manner), an instantaneous disturbance compensation action, and 2) to induce, globally (in a total body-wise manner), a multi-dimensional force field with attractor dynamics. Numerous studies carried out with intact and spinalized animals (Bizzi et al. [12], Mussa-Ivaldi and Bizzi [102], d'Avella and Bizzi [6], Bizzi and Cheung [11], Roh et al. [103], Berniker et al. [7]) demonstrated that complex motor behaviors can be constructed by muscle synergies, with the associated force fields organized within the brain stem and spinal cord, and activated by descending commands from supraspinal areas. Muscle synergies were also shown to be correlated to the control of task-related variables (e.g. endpoint kinematics, displacement of the center of pressure; Ivanenko et al. [104], Torres-Oviedo et al. [87]).

On the other hand, it is still an open question whether or not the motor system represents equilibrium trajectories (Karniel [58]). Motor adaptation studies, starting with the seminal paper by Shadmehr and Mussa-Ivaldi [81], demonstrate that equilibrium points or equilibrium trajectories per se are not sufficient to account for adaptive motor behavior, but this is not sufficient to rule out the existence of neural mechanisms or internal models capable of generating equilibrium trajectories. Rather, as suggested by Karniel [58], such findings should induce the research to shift from the lower level analysis of reflex loops and muscle properties to the level of internal representations and the structure of internal models. This viewpoint is also supported by recent electrophysiological experiments in the lower vertebrates, cat, and monkey that provide evidence that the temporal activations of muscle synergies identified by computational algorithms are ultimately expressions of neural activities.

In this context, only recently advanced brain imaging techniques have allowed to gain direct access to cognitive/mental states in absence of movement, thus making clear that actions involve a covert stage. In this renewed context, it is worth pondering how the computational principles captured by the EPH idea proposed for coordination of overt movement could be recycled to explain actions without movements and without muscle contractions. Here, a problem with EPH is that, given neural circuits in motor areas are activated in other contexts related to ‘action’ that do not cause any overt movement, attributing only the intrinsic properties of the musculoskeletal system to explain movement might be a fallacy. Otherwise, as pointed out by Martin [64], reading words like “lick, pick, and kick” would result in licking, picking and kicking. While motor synergies have traditionally been associated with muscles, a way to resolve this conundrum is to take the EPH idea beyond its manifestation ‘in flesh’ and look instead at ‘muscleless’ motor synergies and how they could be realized computationally and in the neocortex, so as to support a diverse set of cognitive functions related to action generation, perception, simulation and understanding. This is indeed the motivation of our proposal connecting emerging results from functional imaging, neuro-rehabilitation, tool-use in animals and cognitive robotics.

Presently, there is growing consensus that cortical networks in the predominantly motor areas are activated in other contexts related to ‘action’ that do not cause any overt movement. Emerging studies (see Ptak et al. [86] for recent review) suggests that the dorsal frontoparietal network forms a core system for action emulation, internal representation/manipulation of movement kinematics to support inference in diverse cognitive/social tasks. Distributed multi-center neural activation in the parietal and premotor areas are consistently detected not only during the production of overt movements but also during disparate cognitive functions like observation and imitation of others actions (Frey and Gerry [39], Grafton et al. [82], Iacoboni [52], Rizzolatti et al. [80]), comprehension of language namely action related verbs and nouns (Pulvermüller [77], Pulvermüller and Fadiga [76], Marino et al. [62], Glenberg and Gallese [43], Andres et al. [5]) and action interpretation/perspective taking during social interactions (Decety and Stevens [21], Decety et al. [33], Gallese and Cuccio [40], Koster-Hale and Saxe [57], [100]). The central insight that emerges out of these results is that action simulation and action execution draw on a shared set of cortical networks in the parietal-premotor areas of the brain. Further when observing others actions, people recruit motor representations as if they were themselves acting (Gallese and Sinigaglia [41]). Simply put, understanding may be conceived as an internal simulation that entails the reuse of our own ability to act with our bodily resources in order to functionally attribute meaning to others' actions, importantly recycling some of the same cortical substrates the enable our own selves to act.

A provocative proposal to explain a shared/recycled ‘cortical and computational basis’ for covert simulation and overt generation of action is to posit that even real movements are consequences of a ‘covert internal simulation’. This idea, formulated in its basic essence in the Mental Simulation Theory of Marc Jeannerod [56] is relevant presently given the trends in motor neurosciences. Undoubtedly, there must be a continuum with the scope of similar computational principles applied at different levels: physical and mental. Even during the generation of overt actions as posited by EPH, the ‘compositionality’ of the muscle synergies is ultimately made possible by the ‘compositionality’ of the underlying force fields and attractor dynamics. A plausible extension to EPH while retaining its beneficial properties (computational simplicity, biomechanical grounding) and at the same time connect to emerging trends in motor neuroscience is to consider that what occurs in the brain during both mental simulation and overt execution of action reflects an endogenous cortical dynamics very similar to the physical dynamics implicit in EPH, but realized through ‘animation’ of a ‘flexible, plastic and configurable’ internal representation of the body, with the attractor dynamics of force fields induced by the intended goal. This line of thought is not new but emerged infact during the 80s formally extending the EPH idea from muscles to an internal representation of the body in the brain: body schema (Mussa-Ivaldi et al. [73], Hogan [50], Mohan and Morasso [66], Mohan et al. [65]).

That humans have an integrated, internal representation of their body is strongly suggested by the variety of pathological conditions which can only be explained by a deficient internal representation (Haggard and Wolpert [44], Ramachandran [79], Caeyenberghs et al. [17]) or by sensory illusions (Botvinick [15], Ehrsson et al. [16], [28], Lewis and Lloyd [30], [105]). Modern neuroscience has greatly enriched the concept, with numerous studies identifying cortical areas in parietal cortex (Buccino et al. [14]) with multimodal neurons integrating proprioceptive and exteroceptive sensory information to maintain a coherent/updated internal representation of the spatio-temporal organization of the body, see Berlucchi et al. [3], Chiel and Beer [18], Blanke [4] for recent reviews. However, the functional role of the body schema in synergy formation and coordination/simulation of movements has not been investigated in depth. Intriguing insights are now emerging in this direction, particularly in support of body schema being the connecting link between overt and covert action. As Desmurget et al. [31] and Desmurget and Sirigu [32] demonstrate, stimulating right inferior parietal regions (in patients undergoing awake brain surgery) triggers a strong intention to move the contralateral hand, arm, or foot with participants believing that they performed the movements, although no movement was performed and even no electromyography activity was detected. Conversely, stimulating the premotor region triggered overt limb movements, though the patients firmly denied that they moved. Such results from direct intracranial recordings from humans highlight, on one hand, the coupling between motor imagery and the internal representation of the body, and, on the other hand, the link between actions with and without movements. Further, the cortical representation of the body is susceptible to plasticity as has been demonstrated from several experiments related to coordinating tools coupled to the body in primates (Iriki and Sakura [53], Maravita and Iriki [61], Umiltà et al. [89]), virtual reality experiments and neuro-rehabilitation studies (Blanke [4], Shokur et al. [83]) where coordinating common tools, virtual avatars and neuro-prosthetic limbs result in task-specific assimilation of such additional Tool DoF into the body schema. During movement generation, the body and the tool act as one cohesive unit, tool effector assuming the role of end effector (like pliers becoming fingers: Umiltà et al. [89]). This analogy can be extended from coordinated tools to other bodies (conspecifics). Recent studies on infants (Marshall and Meltzoff [63]), are also pointing out that body maps in infants facilitate early registration of the similarity between self and the other, a foundation to developing social cognition.

In sum, numerous studies from different disciplines are pointing towards the central function of the body schema in synergy formation, motor imagery, tool use and social cognition. However, the underlying computational basis is still blurred. This issue is highly relevant also in the context of embodied cognitive robotics given that dexterity in overt movement, purposive behavior with anticipation of the consequences of one's actions, other's actions are critical desirable features if robots are to become commonplace assistants in numerous application domains: domestic, industrial, elderly care to mention a few. In the following sections, we review both how emerging results from diverse empirical studies summarized so far can guide development of biomimetic architectures for action generation/simulation in cognitive robots and what the interrelations imply on our understanding of neural control of movement in general.