Introduction

Skill is Action Requiring Intelligence

Skill is a sort of knowledge acquired through deliberate practices and is observed as a set of sequences of motions to achieve a goal. Skill enables for an action and can be understood in terms of cause and effect, but the change due to an action often surprises us owing to its rarity. Sports performance is a skill and we are all amazed when we see someone run extremely fast or jump up very high. Musical performance is also included in the list of actions called skill. We are fascinated seeing extreme performances because we sense a vast amount of efforts spent for acquiring the capability. We believe that they learnt something important and essential in acquiring the skill. The gap between the skilled and normal indicates an involvement of intelligence because we know that no one can achieve that mastery without thinking about something. Skill is not a matter of whether you do a physical exercise daily, but how carefully you analyse what you are doing and think how to improve it.

The concept of action enabled by intelligence sets a boundary for our research of skills. Skill is not the matter of power and speed, but the matter of dexterity [1]. See for example what you need to think when you play Bach’s Goldberg variation on the piano. Some passages require a quick fingering and you might try to physically train your fingers so that they respond to your intention of moving them. There is, however, a limitation due to the muscular skeletal and neurological systems. A different fingering may solve the problem. You are thinking of motions when you look for an appropriate fingering. An action is skill when some intelligence is involved in enabling it.

Skill is Action Without Belief

There is a class of actions called skill, whose causes cannot be reduced to conditioned reflex responses. There are a number of actions involving thoughts. Speech and communication are among them. Actions conceived as skill are different from those requiring high level cognitive functions, though. An utterance in dialogue updates believes of participants, whose contents are supposed to be propositional. A beliefs is thus either be supported or rejected depending on whether the proposition is true under the circumstance. But the same seems not to hold for the items of knowledge enabling for skill.

It is often difficult for the skilled person to explain how he is performing the elaborate action. He is often ended up with demonstrating it to the apprentice. The difficulty of verbalisation indicates that the items of knowledge involved in skill are not propositional.

The distinction between the declarative and procedural knowledge has long dominated our conception of knowledge. One is led to believe that items of knowledge are procedural if they are not declarative. Knowledge of skill is thus usually thought to be procedural and to describe how to activate particular muscles under specific circumstances. Knowledge of skill is in other words regarded as a set of instructions and is conceived in terms of executions. The naive view of skill thus assumes a simple two step computation, that is, the agent infers what to do in the situation, which may be represented in some form, then it is handed to another component that executes each instruction contained in the plan.

Execution Requires a Thought

Execution cannot be a target of study if everything is predefined and is executed blindly without any thinking. Execution may, however, involve thinking although we need to modify the concept of thinking for claiming it. One can change the course of actions instantly when he finds that he is going to fail to achieve the goal unless he changes the way he is executing it. The pianist for example can skip pressing a key if he finds that he is hitting a wrong note. That sort of error recovery hints that the motion is monitored by some component and can be reprogrammed in the midst of execution to avoid mistakes. The high level of micro-motor control suggests that the execution requires some form of thinking although it may be different mode of thinking from the one for thinking what to say in a conversation.

Micro-term Memory for Motor Control

We also need to expand our conception of memory when we look for a new concept between declarative and procedural knowledge. Two types of memory are assumed for studying inference and execution, namely, long-term and short-term memory. Long-term memory stores items of knowledge acquired through learning while short-term memory items of information representing things to be considered now. Extending that two-story architecture may result in a three-story architecture with a micro-term memory beneath the short-term memory.

Motor Representation

We have so far identified a sort of knowledge to control execution and add a new layer, the micro-term memory, as a seat of thinking for skill to the conventional two story architecture. The next question is what form the skill knowledge would take if it is not propositional. We hypothesise that the knowledge of execution is represented in some form so that it can be stored into and remembered from a memory. This marks our departure from the subsumption architecture [2], which assumes no representation in controlling motions.

A symbolic approach to motor control raises, however, several issues. What are the basic items of skill knowledge and how are they combined to result in a representation? How is it interpreted against what? We assume that an action is initialised by reading a sort of musical score, a sequence of abstract instructions of actions, and is instantiated when it is played under some circumstance. Our metaphor for the skill knowledge is the musical score. You have to observe a grammar when you write a piece of music, which enables others to understand how the piece should be played. It is also largely left for the performer how to articulate it. The abstract instruction and the flexibility allowed for the performer are the keys to an effective encoding of skill knowledge.

The concepts of form and process are useful for illustrating the symbolic aspect of skill knowledge. The form to be shaped is specified by the score. A process is instantiated and maintained as the score is read by the performer for execution.

Organisation

The article elaborates the concept of skill knowledge by demonstrating how it is formed in our mind and is referred to in execution. Our conception of skill knowledge is in between declarative and procedural knowledge. The skill knowledge is declarative in that it takes a form in encoding it. It is also procedural in that it can be instantiated into action. The nearest example satisfying both requirements is the musical score. We will see in the following how the skill knowledge can be understood in a similar vein. The next section reports how a professional musician explained of the samba rhythm. The shape he drew attracted our attention because it encodes his knowledge of samba. The following section establishes the shape as a representation of rhythmic motions by presenting the results of an experiment, where we employed five musicians to collect motion data. The last section suggests that the representation bridges the gap between the declarative and procedural knowledge. We propose through the article a multi-layered architecture of knowledge to unify thought and execution.

Representation of Rhythms

Playing Samba

We studied the skill of samba players to see what the mastery is. Our interest to rhythm is traced back to our previous study, where we analysed how craftsmen kneaded the clay to make ceramics [10]. The mastery can be felt as a rhythm of kneading according to one of the masters. The reference to rhythm indicates several properties pertaining to the motions of the skilled persons, namely speed, stability, regularity, and complexity. A master can quickly knead three kilograms of heavy clay for several minutes because he does it effectively. He can do it anytime for the same quality. The way he does it is always the same. His mastery is built upon a coordination between body parts. We thought that studying motions for playing rhythms is direct approach to elaborating the idea of mastery if it is evaluated in terms of rhythm felt of motions. Samba was our choice as we had already been playing it for our spare time.

The Back Beat

Playing samba is difficult for us because the beat, often called back beat, is counted in the opposite phase to the one we are familiar with. Western music marks the start of rhythm with the down beat and the end with the up beat as a preparation for the new cycle of the rhythm. The way of articulating rhythms can be observed in the conductor’s motion during Baroque era, who used a stick to knock the flour, marking the starting beat. Starting a rhythm by a downward motion is so deeply inherited in Western music that musicians often take the rhythm by tapping his foot on the flour for each beat, resulting in repeating the down and up motion. Playing the back beat such as samba, however, requires the opposite, that is, the musicians take the rhythm by toeing up for each beat, resulting in the up and down motion.

We played samba in fact wrongly for months until a group of local samba players came to correct our misunderstanding. Taking rhythms in the opposite phase was, however, difficult to master. Some experienced musicians advised us to practice dancing to the rhythm because doing samba means primarily to dance to the rhythm and the music produced by percussive instruments are to accompany the dancer. We thus took part in a practice session regularly instructed by an experienced dancer for half a year. Two of us volunteered for data collection using a motion capture system and we studied how they had improved their dances through the period [11].

Conception of Rhythms

We also consulted a professional musician to master samba in addition to practice dancing. He drew a torus with two knots to illustrate how the rhythm was structured (Fig. 1). Each knot corresponds to a beat according to him, which is natural because one meter consists of two beats for samba. The shape can be thought to represent an abstract trajectory producing the rhythm because you experience the rhythm when you trace it with a finger. Not evident of the figure is the speed in which you trace the torus. He moved his finger quicker when he traced down the torus towards the bottom than when he traced it up towards the top. Assuming the bottom to be near to the ground, it indicates that the down motion is accelerated towards the hitting point on the ground, equivalent with the beat. The up motion is slowed down to compensate the time saved.

Fig. 1
figure 1

An expert explaining the samba rhythm

Full size image

He was the only person who explained how to conceive the rhythm. It is rare, generally speaking, for a professional to explain what he does in detail, especially using figures. He was able to do so because he was trained at a leading school of jazz in the western coast of the United States. Conductors learn how to represent rhythms to musicians in the tradition of classical music. The same may go for those working in jazz.

Interesting point to us is that the knowledge of rhythms is represented as shape. Different rhythms take different shapes. Schoolchildren are often instructed to draw triangles when they take the rhythm of waltz, for example. For complex rhythms such as samba, you need extra items of knowledge on how to trace the shape. You need to know where to start, where you accentuate the motion, where the acceleration reaches to the maximum, etc. Controlling the speed introduces another layer of knowledge on top of the knowledge of shapes.

We are led to formulating the skill knowledge in two components, one of which defines shapes and the other how to trace it. The structure is analogous to that found in logic, namely the language allows to represent knowledge in certain manners and it is well controlled by the inference rules how to deduce a conclusion from given formulae. Constructing proofs corresponds to tracing shapes in this analogy. We further examine this parallelism to see what language is required to represent shapes and how the skill representations in the language are proved through the course of actions.

Information of Actions

The Power of Shapes

The skill knowledge for playing samba was represented as a torus with two knots by a professional musician. The shape somehow enables one to reproduce the same rhythm by tracing it in a particular manner. We are concerned with how informative those shapes are. We show below some figures we extracted of the motions for playing samba to see how expressive the shapes are. We prove that they are expressive enough to reproduce rhythms if we confirm that the same shape can be found across subjects.

Data Collection and Preprocess

We employed five musicians for our study, one of who is the author himself. The data collection was carried out in March 2005. Each subject was asked to shake a percussion instrument to produce samba rhythm in different tempos, namely 60, 75, 90, 105, and 120 beat per minute (bpm). The subjects were informed of these tempos by metronome. Each session lasted for 90 s so that we could extract a good part of stable time series by trimming out 7 s of data in the beginning and ending. Sixteen markers were put on the body and two markers on the instrument (5 and 6) as depicted in Fig. 2. The markers on the body were put to the left and right sides of the head, to the left and right shoulders, to the left and right elbows, to both sides of the wrist, to the left and right side of hip, to the left and right knees, to the left and right feet, and to both ends of the instrument. The sampling rate is 86.1 Hz and the data were down-sampled to 43.05 Hz to remove noises.

Fig. 2
figure 2

The body model holding an instrument

Full size image

Analysis

We turn to dynamical view  [6, 9] for capturing features of movements and examine actions as a trajectory in the phase space, a higher dimensional state space. Conventional approaches assume a compositional theory of actions, namely parts of body are controlled independently with each other to result in an action when combined. Complex tasks requiring a high degree of coordination make such approaches implausible in that the goal can be achieved by different ways. Our study of coordinated movements thus abstracts away from particular motions of body parts.

We focus on the five markers, that is, two at both ends of instrument, one at the right elbow, another at the right wrist, and the other at the right hip, showing stable moves to compare motions between the five subjects. We analyse the correlation dimensions [4] between the five time series to visualise the result as a phase space. Figure 3 shows such a phase space constructed based on the data of Subject 1, the professional musician mentioned above. This attractor is reconstructed by the time delay embedding method [8] to 30 delayed copies of the three dimensions of five bodily markers (450 dimensions in total) and we visualise it in the three-dimensional subspace by extracting the first three principal components. The attractor constructed this way is depicted as a two-dimensional manifold similar to the figure that the professional musician drew to explain the samba rhythm. The belt-like shape indicates that the deviation of motions is constrained within a certain range of directions. The constraint is weak where the lines are sparsely arranged and is strong if they are densely arranged. The colour reflects the velocity, that is, the red indicates that the motions are quick in those phases.

Fig. 3
figure 3

The attractor identified of the professional musician

Full size image

Results

We examine how motions are visualised for each subject when they are turned into phase space. The different shapes may suggest different qualities of motions between subjects. Subject 1 is the professional musician mentioned above. Subject 2 is an experienced musician born and grown in Brazil. Subject 3 is one of the authors, who was still unfamiliar with samba. Subject 4 is a pupil of Subject 1, who was reasonably familiar with samba. Subject 5 is also an amateur musician, who played the jazz clarinet.

Figure 4 shows for each subject the reconstructed phase spaces on different tempos, 60, 75, 90, 105, and 120 bpm, from top to bottom. Depicted at top left is the same figure as shown in Fig. 3, scaled down to fit in the figure. Most phase spaces of five subjects show similar two knot belts. Every subject shows similar shapes when their motions are turned into the phase space despite the fact that their levels of mastery vary. Those shapes reflect the characteristics of samba rhythm, that is, the back beat in duple time. The red parts also correspond to the timings where the motion is accelerated for articulating the rhythm.

Fig. 4
figure 4

The reconstructed phase space for five subjects in five trials. The trajectories were colored based on the normalised velocity in the x, y, and z axis for visibility

Full size image

The shapes constructed of Subject 1 all follow what he explained of samba rhythm. The belt becomes thin as the tempo gets faster from Trial 1–5. The result is reasonable because the possible range of motions is more constrained as the subject has to move quicker to catch up with the tempo. Subject 2 exhibits a similar tendency although the width of belts are wider than that of Subject 1, the professional musician. The difference reflects the fact that the control of Subject 1 is stricter than that of Subject 2. No wonder he shows an evidence of better control over performance as he is professional. Subject 4 exhibits a similar tendency, too, but the widths of belts are wider than those of Subject 1. He also seems to have a trouble in playing the rhythm to faster tempos, 105 and 120 bpm (Trial 4 and 5). Subject 5 shows thin belts for all the shapes. We learnt that his motion was smaller than others, which can be the reason. Subject 3, one of the authors, was disappointed by the poor performance indicated with the result although the two-knot torus is observed for four tempos. It is also observable that the timings of acceleration are not stable as the colours are not well separated.

Different levels of mastery can be observed by looking into the sound produced by them. Figure 5 Left shows the amplitude of sound, where the sound for each two beat is overlapped with each other, drawn in the interval. The image produced of Subject 1 is clear, suggesting that he produced repeatedly the same pattern of sound through the session. It is more clearly observed when the peaks are plotted as seen in Fig. 5 Right. Subject 2 shows a good shape, but the deviation is wider than Subject 1. He also produces four sounds, the third of which is weaker than others. The third position corresponds to the timing where the body sinks. Subject 1 contrasted that point with other points by subtly moving the instrument. The sound patterns produced by other three subjects are more or less blurred. Subject 5, who showed thin belts for the analysis above, shows a large deviation in producing sounds. The evidence is enough for us to categorise his performance to be average quality. It is thus appropriate to regard shapes of Subject 1 and 2 to be correct while those of other subjects to be deformed.

Fig. 5
figure 5

The sounds produced by the five subjects in amplitude (left) and peak (right)

Full size image

Discussion

We presented that a professional musician conceives the samba rhythm as a two-knot torus and showed that the structure can be retrieved of his motion as a phase space constructed by referring to dimensions of motions. We discuss what we can conclude from the coincidence between his perceived structure of rhythm and the identified structures of his motion data. We are sure that he has no knowledge of dynamical systems, fractals, etc., thus the origin of the two-knot torus has to be sought into his perception of his own motion. How did he find it?

The structure retrieved of his motion through our analysis is abstract in that it does not reflect a particular part of motion. The phase space is established based on the motion data of five parts of his body. The items of information resulting in the structure are distributed among all the five streams of data. This suggests that his knowledge of playing samba rhythm is distributed among several places. His knowledge is in other words not modular to each part of body. The independence of particulars indicates that his knowledge is about rhythm, not about motions. The separate component storing items of knowledge of samba rhythm allow him to produce the same rhythm in several different ways. That is, he can still play the rhythm when he uses some other instruments. Inversely, he may have learnt of the rhythm through different types of experiences. He is actually a guitarist and usually plays Brazilian music on the instrument. He can use percussions to express the same rhythm and can dance to it, too. These multi-talents may have helped him to abstract over experiences he acquired with each instrument and dancing.

But why does he conceive the abstract representation of skill as two-knot torus? We look into the distributed knowledge by examining how each part of his body moves. Figure 6 shows phase spaces constructed of the data taken from the professional musician, who played the instrument to the tempo for 60 bpm. Each shape represents the characteristics of motion at particular part. The shape in the top left is constructed of the data collected at the marker 1, namely his left side of head (Fig. 2). Shapes constructed of the data are arranged in order from 1 to 18, each corresponding to the marker as depicted in Fig. 2.

Fig. 6
figure 6

Phase spaces constructed of the data taken from the professional musician, who played the instrument to the tempo for 60 bpm

Full size image

The shapes constructed of the data collected at the both sides of instrument (Marker 5 and 6) exhibit a feature characteristic to samba, namely, double circle. The same applies to other shapes constructed of the data collected at the markers, 7, 8, 10, and 12. These places correspond to his right shoulder, right elbow, right hand, and the right side of his waist. All these parts are directly connected to the instrument in his right hand. Other parts are more like single circle than double. The shapes of Marker 16 and 17 are circle, whose data are taken from his right leg. The shapes at other parts are blurred. The evidence suggests that two control points may exist somewhere in his right arm and leg.

Another set of shapes reveals the nature of control further. Figure 7 shows phase spaces constructed of the data taken from the professional musician, who played the instrument to the tempo for 120 bpm. The shapes of the instrument remain the same, but other parts are affected by the quicker tempo. The shapes of Marker 7, 8, 10, and 12 are no longer double circle, which correspond to the parts adjacent to the instrument in his right hand. They are single circle. Parts below his waist are single circle, namely his left leg (Marker 13 and 14) and right leg (Maker 16, 17, and 18). The right side of his waist is now single circle (Marker 12).

Fig. 7
figure 7

Phase spaces constructed of the data taken from the professional musician, who played the instrument to the tempo for 120 bpm

Full size image

Those two sets of shapes taken of body motions responding to different tempos tell us that the motions to play the instrument to samba require two control points or two oscillators in his body. One of them roots in his legs, producing a single, base beat. The rhythm produced by the oscillator is not yet samba; thus, it only shows single circle. The simple metronome like pulse is modulated by another motion around his right arm, resulting in a samba rhythm. The motion at the instrument (Marker 5 and 6) is depicted as the double circle because it is a synthesised motion by the two oscillators. The degree of contribution from the oscillator around legs is increased as the tempo becomes quicker because the larger parts of his body are engaged in catching the faster tempo. The different shapes for different tempos as shown in Fig. 4 may reflect the increased contributions of the base oscillator around legs as the tempo gets faster.

The existence of two oscillators may explain why he conceives the skill for playing the instrument to samba as two-knot torus. He is aware of two oscillators, one in his legs and the other in his right arm. Controlling his right arm in response to the motion from his legs results in the shape. As evidence to our hypothesis, we show below how legs are controlled by musicians.

Prior to our analysis of phase spaces, we calculated correlations between body parts for each subject by Hilbert transformation (Fig. 8). The points in the same color, black or gray, indicate that they move in the same timing. The points in different colors, black and gray, indicate that they move in the opposite timing. That is, the point colored gray starts moving when another point colored black is in the middle of motion.

We identified a feature peculiar to Subjects 1 and 2. The figure shows that the left hip joint and the right knee are synchronised for them as those points are colored gray. The right hip joint and the left knee are also synchronised as they are colored black. The two pairs of synchronised motions move exactly in the anti phase, that is, a pair of synchronised motions is followed by another and vice versa for the same time interval. Two legs were moved in turn as if he walked in the same pace although you never notice it when you observe his performance because the motions are too subtle.

Fig. 8
figure 8

The synchronised pairs of joints among subjects

Full size image

He is likely to have perceived his motion of pelvis pushed up by each leg in turn. We believe that he firstly leant to move his body that way around his pelvis to move his torso to the base tempo. The base tempo was refereed to when he plays Brazilian music with instruments. We investigated through another project how high school students learnt to play percussions to samba rhythm [7]. We found that they firstly learnt to move their bodies around the pelvis, then proceed to moving their arms to play the instrument to the rhythm. Our finding suggests that the image of rhythm is formed while they practice dancing to the rhythm. We would find similar forms of motions for other parts than the hip and knees as they abstract away from the motion around pelvis to apply the representation for moving other parts of body.

It is interesting to see that the professional musician found by himself the image of rhythms abstracted away from particular motions although we can only guess how it came into his mind. The degree of abstraction was identified by constructing the same shape from his motion data. It abstracts over velocity, direction, and position. It is in fact independent of physical factors. The shape just shows where the motion has to be strictly controlled in a process of executing motion. This largely reflects the nature of performance, that is, playing the rhythm only requires moving the instrument with proper accelerations for producing accented sounds heard as samba. It does not matter, for example, how you grab the instrument, where you hold it, which directions you move it in, etc. The necessary items of information are all encoded in the shape for playing the samba rhythm. One can thus immediately feel the rhythm when he traces the trajectory with a finger.

The shape can represent the knowledge of movement. The items of information contained in the shape can be retrieved with any parts of body or any combinations of parts. The validity of the shape is confirmed when one traces it to produce the desired result. Tracing the shape results in proving it. Motions are proof of the skill knowledge. Motions are constrained anatomically and, thus, the validation requires a certain form of search for solutions through actions.

Conclusion

Summary

We reported how a professional musician illustrated his conception of samba rhythm as a movement tracing a two-knot torus and showed that the same structure can be extracted as a phase space by abstracting over particulars such as velocity, position, direction, and body parts. We could identify similar structures of the motion data collected from experienced musicians but not of those collected from learners. We conclude from the evidence that the structure indicates a mastery of samba rhythm. One can attain such a representation through experiences under various conditions. He can dance and play several instruments to the same rhythm and may have generalised his motions across different performances to find the shape. The case of the professional musician suggests that one can acquire some representation of motions abstracted over physical particulars.

Significance

We report of the result of our experiment although only the single person taught us how he conceived the rhythm because his case casts a new light on skills. The skill knowledge has been thought to include the procedural knowledge, that is, how to move a particular part of body in coordination with other parts. The skill knowledge then has to accommodate plenty items of information concerning muscles, joints, tools, etc. Optimising motions is impossible if one has to consider so many factors in planning actions. We normally separate factors involved in skills into thought and execution. The dichotomy also raises another issues, namely how thought intervenes in execution and how executions are taken into consideration in thinking actions.

We identified a middle layer between thought and execution to bridge the gap. The representation of motions as reported here abstracts over physical particulars, thus, it is easily accessible while one is thinking of actions. Thought can alter the representation, resulting in different motion, without taking into so many physical particulars into consideration. The hierarchical structure unifying thoughts and executions looks promising for elaborating our theory of skills. We expect more layers to be identified as we look into other types of skills. The representation we found through our research of samba only encodes items of information necessary for controlling timing. More complex tasks than playing samba should include other types of information in different shapes.

Tribute

The study of skills has been initiated by the late Prof. Furukawa, who has extensively focused on playing the cello. He took a logical approach to investigating the skill and defined it as a set of formulae [3]. You can introduce logical methods to studying skills once you have defined them in logic language. Logical approach to skill, however, suffers problems of description. You need a systematic approach to describing factors involved in skills as logical formulae. It is unclear in what level one should describe elements of actions. The terms employed are also restricted to those noticed by the encoder himself. Motions observed need to be interpreted into the language of logic. The mapping requires a certain amount of work if one has to describe what are involved in motions such as muscles, joints, velocity, acceleration, etc. His logic approach to skills faced those problems and he was certainly aware of the necessity for intermediate languages to ground theories of skills described in logic to motions.

We believe that our report will encourage researchers to turn into logical approaches to skills. Knots are studied in discrete mathematics and some have already studied them using logic [5]. Skill science will motivate researchers to look into those intersections between logic and discrete mathematics.