A Systematic Analysis of Hand Movement Functionality: Qualitative Classification and Quantitative Investigation of Hand Grasp Behavior

Introduction

The ability to perform various and skillful tasks using the hand is one of the critical characteristics of humans as a most refined primate, discriminating with other mammals (Cartmill, 1974). In daily life, people may grasp objects securely, such as eating a meal with a knife and fork, perhaps changing the position and orientation of the manipulated objects, such as writing with a pen or touching an object such as an antenna, etc. The human hand is versatile in interactions with the surrounding environment, showing a tremendous functionality (Iberall, 1986). A systematic description of human hand functionality is needed and contributes to the research in many domains around the human hand. In neuroscience, it is not only important to understand the human hand movement behavior, but also to develop the experimental protocol of a cognitive experiment to explore the elaborate mechanisms of motor control (Castiello, 2005). In human–computer interaction in which the hand is used to interact with the computer, specific grasp types selected in advance will serve as a friendly interaction between human and machine (Loclair et al., 2010). In packaging design of necessaries and products on which hands act, it is important to understand the grasp habit during normal use (DiSalvo and Gemperle, 2003). In robotics, understanding hand functionality is essential to mechanical implementation (Catalano et al., 2014; Xiong et al., 2016). In rehabilitation, the extracted critical grasp types can be used to evaluate the remaining capabilities after accidents, disease, or surgery (Light et al., 2002).

However, the complexity of the motor control system makes understanding human hand movement functionality challenging. It is difficult to directly model hand movement functionality from the motor control system. Some key issues are still in the subject of hot debate, such as the brain activity in the grasping process (Sereno and Maunsell, 1998; Castiello, 2005; Goodale, 2010), cortical representations of hand movement–related muscles (Schieber and Hibbard, 1993; Sanes et al., 1995; Meier et al., 2008), biomechanical constraints of hand grasping by tendons and ligaments (Kapandji, 1971; Santello et al., 2013), an accurate kinematic model of the human hand (Stillfried and van der Smagt, 2010; Bullock et al., 2012), etc. In addition, for ethical reasons, substantial investigations about elaborate mechanisms of grasping come from non-human primates (Castiello, 2005), such as macaque monkeys. The substantial differences in hand morphology between human and non-human primates are still in debate (Preuschoft and Chivers, 2012).

Compared with directly modeling the hand movement functionality from the motor control system, hand movement patterns can be seen and measured directly and are also the direct reflections of human hand movement functionality. Meanwhile, for the same given task, movement patterns performed by the motor system are highly stereotyped, between both repetitions and individuals (Wolpert and Ghahramani, 2000). Systematically analyzing hand movement patterns is a feasible way to understand hand movement functionality. As we show the summary of classifying hand movement functionality studies, there is little systematic description of hand movement functionality from both qualitative and quantitative perspectives. There is a lack of a comprehensive view for understanding hand movement functionality systematically.

The contribution of this paper is to provide a systematic framework to help understand hand movement functionality from both qualitative and quantitative perspectives in a logical manner. We qualitatively classify hand movement functionality, build prehensile taxonomy to explore the general influence factors of human prehension, and accordingly design a behavioral experiment to quantitatively understand the way of hand grasp.

In qualitative analysis, we built a hierarchical tree of hand action to classify hand movement functionality to eight action classes. The arm's and wrist's important contributions to hand movement are clarified. Moreover, a prehensile taxonomy containing 52 types is constructed to understand the human prehensile functionality contained stable hold and within hand manipulation in detail. For further exploring the general influence factors of human prehension, the grasp types of prehensile taxonomy are rearranged into object prehensile taxonomy. Two facts are found: (1) the arm and wrist make an important contribution to hand movement functionality; (2) the relative position is a general influence factor significantly impacting human prehension except for the object shape and size. Accordingly, a human grasping experiment is designed to quantitatively understand the way of hand grasping and demonstrate the significant impacts of the general influence factors. Simultaneously considering the impact of relative position, object shape and size, prehensile taxonomy, and the behavioral experiment results presented here should be more representative and complete.

The paper is organized as follows. Section Background presents a systematic discussion of the previous investigations about human hand movement classification. Section Classifying Hand Movement Functionality presents a qualitative analysis of hand movement functionality, including the hand action hierarchical tree and prehensile taxonomy. In section Behavioral Experiment to Investigate Human Grasp Functionality, a behavioral experiment simultaneously considering the impact of relative position, object shape, and size is implemented to quantitatively demonstrate the significant influence and understand the way of hand grasp driven by influence factors. Finally, section Discussion presents some potential uses and discusses the limitations of the presented work.

Background

In this section, we investigated the previous research about human hand movement classification. Some significant definitions are introduced and arranged in Table 1, which contributes to the later classification. Although substantial efforts have been made to explore hand movements, previous efforts have been mostly made from one particular aspect. To the best of our knowledge, there has been little research on building a systematic framework to understand hand movement functions from both qualitative and quantitative perspectives in a logical manner.

Table 1. Some significant definitions of human hand movement.

Table 1 shows some significant definitions of human hand movement classification. Schlesinger first categorized grasp into six types (see Table 1)—cylindrical, spherical, tip, palmar, lateral, and hook—based on the object shape, hand surfaces, and hand shape (Schlesinger, 1919). Three critical notions (object shape, hand surfaces, and hand shape) were proposed to categorize an enormous variety of grasp types. Napier (1956) divided hand movements into two main groups, prehensile and non-prehensile, based on whether the object is seized and held by the hand or not. For prehensile movement, Napier believes that the grasp can be categorized by task action goals, power, or precision grasps (Napier, 1956), based on whether the task requires large force or precision movement. According to the contact areas on the hand, Kamakura divided the static grasps into four categories (see Table 1): power, intermediate, precision, and no thumb involvement grip categories (Kamakura et al., 1980). Iberall (1986, 1987, 1997) described the hand grasp with a detailed discussion of virtual finger (VF) and opposition spaces (see Table 1). The VF is defined as an abstract representation of fingers applying an oppositional force. Based on the direction of applied force between the hand and object, hand grasps can be divided into three categories: palm, pad and side opposition, which Table 1 shows, recently has been used to analyze the grasp functionality of the bionic hand (Zhan and Liu, 2013). Cutkosky (1989) proposes a hierarchical tree of grasps, which begins with the two basic action goals suggested by Napier and then moves down the tree to VF and object shape, in total listing 16 different grasps (containing one non-prehensile platform type). For exploring human grasping skills in more detail, Feix et al. (2015) constructed a grasp taxonomy containing 33 grasp types. The definitions of action goal, object shape, VF, and thumb position are considered. Feix's GRASP taxonomy is well-structured and has been widely used, such as assisting in determining anthropomorphic hand capabilities (Feix et al., 2012), robotic hand design (Xiong et al., 2016), and experiments analyzing human hand functionality (Juravle et al., 2011; Tessitore et al., 2013). Pollard attempted to refine the previous taxonomies (Abbasi et al., 2016) and used high frame rate handheld cameras to encode for shelf picking and placing actions (Nakamura et al., 2017).

Compared with the previous extensive investigations of human static grasp functionality, there have been relatively few efforts devoted to understanding human manipulative functionality. Elliott and Connolly (1984) believe that, within hand manipulation, movements can be reduced to three basic classes, defined as simple synergies, reciprocal synergies, and sequential patterns (see Table 1). Bullock et al. (2013a) provides a hand-centric classification that contributes to the development of a common framework for describing hand dexterous manipulation; motion within hand and motion at contact are defined as shown in Table 1. Recently, Feix studied the effect of the number of fingers on human precision manipulation workspaces (Feix et al., 2020).

Classifying Hand Movement Functionality

In this section, hand movement functionality is refined with a logic classification, and then a hierarchical tree of hand action is built for a systematic description of hand movement functionality containing prehension and non-prehension under the consideration of arm and wrist contributions (section Hierarchical Tree of Hand Action), followed by a detailed hand prehensile taxonomy containing 52 distinct grasp types to provide a comprehensive and detailed understanding of human prehensile functionality contained in stable hold and within-hand manipulation (section Hand Prehensile Taxonomy) and finished by decomposing prehensile taxonomy into object prehensile taxonomies to further explore the human grasping general influence factors.

Analysis of Hand Movement Functionality

Figure 1 presents the logical refining analysis of human hand movement functionality. The relationship between hand movement functionality, task, subtask, and action is provided. The human hand movement functionality is made up of enormous numbers of tasks able to be performed by hand. Each task can be decomposed into subtasks driven by more specific action goals. For accomplishing each subtask, the distinct action should be chosen from the set of possible actions corresponding to the specific action goal. The set of possible actions are termed the action class. Therefore, compared with task and subtask, the action is the smallest composition unit of human movement functionality. Some terms should be clarified:

• Hand movement functionality-The sum of the variety of tasks able to be performed by human hands.

• Task-Related to human hands, generally specified at a high, often symbolic level, such as drinking water.

• Subtask-Decomposed from a task and driven by a more specific action goal.

• Action class-Set of possible actions for the specific action goal.

• Action-The distinct action chosen from the action class to accomplish the subtask.

Figure 1. Relationship between hand movement functionality, task, subtask, and action.

Take a simple task such as grasping a cup on the table as an example. This task is one part of hand movement functionality and can be decomposed into two subtasks in sequence: reaching for the cup and grasping it. Whether for reaching or grasping, there are several possible implemented ways that constitute the reaching or grasping action class. While implementing the task, people choose the distinct action from the corresponding action class to implement each subtask in sequence for accomplishing the whole task.

As each action is driven by an action goal, the action goal is important and needs to be clarified. Inspired by Napier's action goals shown in Table 1, we believe the action goals can be summarized as shown in Figure 2: (1) apply forces to keep a stable relationship between hand and object; (2) impart object dexterous motion with respect to the hand, such as transporting an object or in-hand manipulation; and (3) impart hand motion for other functional demands, such as gathering sensory information like an antenna, transmitting information by sign, reaching to an object, etc.

Figure 2. A systematic description of human hand action.

Based on the analysis, we can see that the action of the hand is not only a product of the internal degrees of freedom of the hand, but also the movement of the wrists and arms (Bullock et al., 2013a; Feix et al., 2015). When performing a task using our hands, we reach to objects, grasp and lift them, manipulate them, use them to act on other objects, and finally set them down. In the whole process, the hand, wrist, and arm should be adjusted to various configurations for matching the task requirements. They all need to be considered when describing human hand movement functionality.

Hierarchical Tree of Hand Action

Figure 2 presents a hierarchical tree to systematically classify human hand actions. Due to the diversity of human hand movements, it is difficult to describe entire hand movements on a very detailed level. To describe them comprehensively and systematically, the action class is set as the smallest unit in the hierarchical tree for classifying the human hand action. For hand prehensile functionality (classes 1 and 3 in Figure 2) merely related to the hand itself, the detailed description is presented in section Hand Prehensile Taxonomy. For a systematic and clear description, the definitions of some terms are clarified and applied in our hierarchical tree.

• Prehensile: Object is seized and held partly or wholly, including grasp and manipulation.

• Nonprehensile: Object is not seized and held but acts on objects, including a platform (class 5) to resist gravity; contact motion (class 7), such as flipping a light switch; or free motion (class 8) (no objects included) and rest (class 6).

• Grasp: Keeping a stable relationship between the hand and object, including hold (class 1) and transfer (class 2).

• Manipulation: Motion occurring within the hand is necessary, including within-hand [Table 1, as shown in the figure (f) of Table 1] manipulation (class 3) and hybrid (class 4), accompanying the finger precision motion and hand motion driven by the wrist and arm [motion definition in Table 1 as shown in figure (e) of Table 1].

• Motion: Hand is driven by the wrist and arm and moves with respect to a body coordinate frame as shown in figure (e) of Table 1.

• Contact: The hand acts on the object, and there are contact areas between the hand and object.

First of all, one basic choice between prehensile and non-prehensile actions should be made, starting at the top of Figure 2. For a task in which we need to seize and hold the object partly or wholly, the prehensile action should be chosen. Then, the first question is does the object need to be clamped stably (keeping a stable relationship between hand and object) or impart dexterous and precision motion with respect to the hand? If the stable relationship should be kept, then the grasp is chosen; hold (class 1) or transfer (class 2) is performed according to the task requirement of holding the object stably or transferring object from one place to another. If a dexterous and precision relative motion is needed, then manipulation is chosen, and within-hand (class 3) or hybrid (class 4) manipulation of the object is performed, such as spinning a top using within-hand manipulation or writing using the hybrid manipulation accompanying the finger precision motion and hand motion driven by wrist and arm. For the non-prehensile action, the first question concerns whether contact between the hand and object is needed? If the task merely needs to support or contact the object, the platform (class 5) and contact motion (class 7) is performed, such as a platform push, flipping a light switch. If it does not need contact, the action can be divided into rest (class 6) and free motion (class 8).

Consequently, hand movement action is divided into eight classes. In classification, we find that the contribution of the wrist and arm for hand movement functionality is important. The hand action class related to the arm and wrist is marked by a blue dotted line in Figure 2 to further clarify the arm's and wrist's important role. In particular, the differences between classes 1, 2, 3, and 4 are mainly reflected in whether hand relative motion imparted by the arm and wrist occurs or not. In addition, the arm and wrist movements are also related to the integrity of a task. No matter which kind of prehensile movement, the hand should be transported to the right position to contact the object in the beginning of the task. When the task is finished, the object should be placed at the target location. These actions all need the participation of the arm and wrist and can be described by reaching (class 8) and transferring the object to the end point (class 2). The arm and wrist contribution is also widely reflected in our activities of daily living (ADL), such as sign language to transmit information (class 8), tactile surface exploration to gather sensory information (class 7), etc. Therefore, the arm and wrist are important and make a vital contribution to hand movement functionality.

Hand Prehensile Taxonomy

Figure 3 presents the prehensile taxonomy, providing a detailed summary of hand prehensile function merely related to the hand itself, incorporating hold (class 1) and within-hand manipulation (class 3). The taxonomy contains 52 types. This is the first hand prehensile taxonomy for describing human prehensile functionality containing stable hold and within-hand manipulation from a comprehensive view. To guarantee the comprehensiveness, we compared 29 human hand prehensile investigations, covering neuroscience (Jakobson and Goodale, 1991; Rosenbaum et al., 1996; Santello et al., 1998; Smeets and Brenner, 1999; Cohen and Rosenbaum, 2004; Schieber and Santello, 2004; Ansuini et al., 2008; Bullock et al., 2012; Touvet et al., 2014), hand surgery and rehabilitation (Schlesinger, 1919; Napier, 1956; Kamakura et al., 1980; Light et al., 2002), anatomy (Preuschoft and Chivers, 2012; Santello et al., 2013), biomechanics (Kamper et al., 2003), and robotics (Elliott and Connolly, 1984; Iberall, 1987, 1997; Cutkosky, 1989; Juravle et al., 2011; Feix et al., 2012, 2014a,b, 2015, 2020; Bullock et al., 2013a,b; Tessitore et al., 2013; Zhan and Liu, 2013; Abbasi et al., 2016; Nakamura et al., 2017). For understanding human static grasp ability, Feix provided a good foundation (Feix et al., 2015) and built a grasp taxonomy to cover human static grasping functionality, which has been widely used related to multiple domains. To construct a comprehensive human prehensile taxonomy for covering not only the static hold, but also the within-hand manipulation, we selected two highly cited papers on the classification of within-hand manipulation behavior (Elliott and Connolly, 1984; Bullock et al., 2013a) to help efficiently summarize within-hand manipulation ability in our taxonomy. These two papers classify within-hand manipulation behavior from the view of the manipulative patterns and object motion with respect to the hand coordinate frame, respectively. We unify these two views and arrange them in our prehensile taxonomy. The Electronic Supplementary Material provides a detailed explanation for the selection of the manipulative postures in our taxonomy. Therefore, manipulation behaviors classified in these two representative papers (Elliott and Connolly, 1984; Bullock et al., 2013a) are all covered and presented by finger gaits in our taxonomy.

Figure 3. Prehensile taxonomy that incorporates both human grasp and manipulative movements. The grasp types within the each color bracket represent a within hand manipulative movement in sequence.

To provide a systematic and efficient description, 52 types are arranged as shown in Figure 3 based on the terms (action goal, VF, thumb position, opposition, and within-hand) presented in Table 1. For understanding human prehensile behavior or being applied to robotic hand manipulation, it is perhaps the most natural that the within-hand manipulation behavior is classified based on the direction of motion with respect to the hand coordinate frame shown in figure (d) of Table 1. The human hand, radial-ulnar, distal-proximal, and dorsal-palmar axes can be substituted for the x-, y-, and z-axes, respectively. Movements can be described by the motion along hand axes or the combination involving the principal axes: Δz means z-axis translation, ϑz means rotation about the z-axis. Only x-axis translation is not covered as the dexterous motion of x-axis translation (Δx) is difficult for the human hand to implement (Bullock et al., 2013a), which is compared with actual within-hand manipulation functionality. Therefore, as the relative position impact on grasp types are concerned, our taxonomy is expected to provide a more complete understanding of hand prehensile functionality. To further clarify the general influence factors of hand prehension, the impact interference of object (shape and size) is abandoned by dividing the 52 types of prehensile taxonomy to object prehensile taxonomy containing cylinder, sphere, and flat object.

Object Prehensile Taxonomy

All 52 prehensile types in Figure 3, except for the using scissor type, are rearranged into cylinder, sphere, and flat object prehensile taxonomy (Figures 4–6) based on the object shape, respectively. Many grasps have similar VF; therefore, some cells are populated with more than one grasp. Within each cell and for the same thumb position, the difference between the grasps is mainly the relative position between object and human hand. In this case, the grasps can be sorted according to the relative distance from distal to proximal as shown in Figures 4–6. Then, a striking observation is that both precision and power grasps can be sorted into within-hand manipulation tasks, mostly showing an object pickup process from distal to proximal to the palm.

Figure 4. Cylinder object prehensile taxonomy. The grasp types within the golden brackets represent the picking up movement in sequence. The red arrow means the direction of movement execution in sequence. The grasp types within the green box or bracket represent the within-hand manipulation included in Figure 3.

Figure 5. Sphere object prehensile taxonomy. The grasp types within the golden brackets represent the picking up movement in sequence. The red arrow means the direction of movement execution in sequence. The grasp types within the green box or bracket represent the within-hand manipulation included in Figure 3.

Figure 6. Flat object prehensile taxonomy. The grasp types within the golden brackets represent the picking up movement in sequence. The red arrow means the direction of movement execution in sequence. The grasp types within the green box or bracket represent the within-hand manipulation included in Figure 3.

From Figures 4–6, we can observe the following:

1) The relative positions significantly impact human prehension. The relative position between the human hand and objects is an important factor leading to the diversity of prehensile types. As shown in Figures 4–6, the grasp types in the same golden brackets, green boxes, and brackets vary significantly because of the impact of changing relative position and orientation between hand and objects.

2) The complex and diverse human precision and power grasp types mostly can be concisely described by human pickup motion. Human precision and power grasp types can be both arranged and reflected in the object pickup sequential process from distal position to the palm as shown in the golden brackets of Figures 4–6. This kind of pickup behavior is commonly used in our daily life, especially for grasping a small object. For instance, human hands pick up a thin stick on the table (shown as the prehensile types within the left upper golden brackets of Figure 4). At the beginning, the precision grasps are used to pinch the stick in the distal position. Subsequently, the stick is pressed to the palm by flexing the finger proximal interphalangeal (PIP) joints. Hereafter, finger MCP joints largely flex, and then the stick is stably grasped. By now, the object is picked up to the palm and the whole pickup motion is finished.

Behavioral Experiment to Investigate Human Grasp Functionality

According to the observation obtained from object prehensile taxonomy analysis, we know that, except object (shape and size), relative position significantly impacts human prehension. Therefore, after simultaneously considering the impact of relative position, object shape, and size, we designed a laboratory-based unstructured behavioral experiment to comprehensively understand the human grasp and qualitatively demonstrate the significant impact factors on human prehension.

The experimental setup, protocol (contained hand, wrist and arm starting position, experimental paradigm, etc.), data record, and calibration are reported in our recent study (Mason et al., 2001). Ten healthy, right-handed subjects (24–27 years old, eight men and two women) were asked to grasp six objects (three shape × two size) in 27 different relative positions (3 X deviation × 3 Y deviation × 3 Z deviation) between human hand and objects as shown in Figure 7. The object shape, size, and weight (Table 2) were chosen based on the Yale human grasping data set (Feix et al., 2014a,b) for efficiently representing the objects we grasp in daily life. Three typical shapes—sphere, cylinder, and prism—are selected. To embody the effect of object size on grasp, each shape is divided into large and small sizes. Therefore, in total, six objects are selected in our experiment. As 96% of grasp locations are 70 mm or less in width in ADL (Feix et al., 2014a), the smallest sizes of five objects are lower than 70 mm. The large sphere size is larger than 70 mm to cover the large size type. As 92% of objects have a mass of 500 g or less in ADL (Feix et al., 2014a), five objects' weight is lower than 500 g. The large cylinder weight is larger than 500 g to cover the large weight type. As the object prehensile taxonomy shows that the relative position is a significant and important impact factor, we give it particular attention. The 27 different relative positions (3 X deviation × 3 Y deviation × 3 Z deviation) are set in the experiment.

Figure 7. Experimental setup. (A) Top view of the experimental setup. (B) Schema of the Cyberglove kinematic model contained 15 active joints.

Table 2. Shape, size, and weight of the six grasping objects.

The static grasping postures that can lift up the object are recorded. We hope to obtain a common joint angle feature for the same task rather than the individual cases. Therefore, two repetitions are performed. Thus, in total, 3,240 postures (10 subjects × 6 objects × 27 relative distances × 2 repetitions) are collected. The mean of two repeated trials was used in all statistical analyses. Hand grasp postures containing 15 joint angles were recorded by Cyberglove III (Virtual Technologies, Palo Alto, CA) at a resolution of < 0.1° and sampled at 100 Hz each. The following joint angles were measured: PIP joints and metacarpo-phalangeal (MCP) joints of four fingers except for the thumb as well as the interphalangeal (IP) and MCP joints of the thumb: opposition rotation (Rot) of the thumb, abduction/adduction (ABD) of thumb carpometacarpal (CMC) joint and MCP joints between four fingers except for the thumb.

Principal component analysis (PCA) is used to reduce the dimensionality of hand grasp postures and efficiently represent hand movement characteristics to help clearly understand human grasp behavior influenced by relative position, object shape, and size. In this paper, a five-factor ANOVA (object shape, object size, X deviation, Y deviation, and Z deviation) was performed to investigate the effects on each PC score. Independent factors were object shape (1–3), object size (1–2), X deviation (1–3), Y deviation (1–3), and Z deviation (1–3) between human hand and object. The dependent variables were the first four PC scores.

Figure 8A shows the explained variance of each PC. We chose the first four postural synergies (PC1–PC4) to represent hand movement functionality as PC1–PC4 can explain much of total posture variance (82.8%). Figure 8B shows the joint movement characteristics of each PC. The PC1 mainly reflects the major flexion of four-finger MCP and thumb IP joints, major thumb inverse opposition, and minor flexion of four-finger PIP joints. For ADL grasping, this could be characterized by cylinder grasping (such as the pen, cup, handle, mug, et al.) and flat thin object grasping (such as spoon, wrench, card, dinner plate, et al.), accounting for 45.3% of variance. The PC2 explains 18.7% of the variance and was characterized by converse motion between four-finger MCP joints and all PIP joints; this could be used to grasp flat objects (such as book, plate, brick, disk, et al.) in the palm-opposability or pad-opposability posture. PC3 and PC4 explain 10.3 and 8.4% of the variance, respectively. PC3 was characterized by major extension of four-finger MCP and thumb inverse opposition, minor extension of four-finger PIP, ABD of index, ring, and pinky. It shows an opposability grasp of sphere objects (like apple, tennis ball, et al.). The PC4 was mainly characterized by thumb motion containing IP flexion and opposition and the ABD of index finger, ring finger, and pinky.

Figure 8. PCA on hand grasp with different shape, size objects in different relative positions. (A) Information transmitted by PC1-PC4. (B) Hand posture reconstruction by PC1-PC4.

More importantly, the PCA analysis can estimate the amount of information in grasp postures by comparing the explained variance of postural synergies (PCs). The results show that the grasp postures in our experiment include more information when relative position is considered. The explained variance by the same number of PCs is much less than in other research. The first two PCs of tolerance grasping can only explain <65% (64.1%) of the information. This is obviously lower than other studies of hand kinematic synergies in related studies [~80% in grasp imagined objects (Santello et al., 1998), ~99% in reach-to-grasp for columnar objects (Mason et al., 2001), 70% in biometrics for secure identity verification (Vrajeshri et al., 2017), ~99% in precision grasping for cylinders of different size (Park et al., 2014), ~70% in haptic exploration (Thakur et al., 2008), ~80% in rapid grasping (Mao et al., 2010), ~88% in bimanual manipulation (Jarrassé et al., 2014)]. This indicates that the amount and dimension of information in tolerance grasping is increased with the simultaneous consideration of relative position, object shape, and size. These results quantitatively demonstrate that, after considering the relative position impact on hand grasp, the tolerance grasping and prehensile taxonomy presented here can represent human grasp functionality more comprehensively.

The five-factor ANOVA results are shown in Table 3 for quantifying the factor effects on hand grasping. It can be seen from Table 3 that Y deviation significantly affects PC1, PC2, and PC4. From the F-value, it has the most significant effect on PC1. Object size significantly affects PC2–PC4. From the F-value, the size has the most significant effect on PC3. For object shape, it significantly affects PC1–PC4, and the impact is more obvious to PC1. For X and Z deviation, the impact is smaller than Y deviation, object shape, and size. They both have the most significant effect on PC1. There were also some significant interactions (from PC1 to PC4, the most significant interaction impact is shape-Y deviation, shape-size-Y deviation, shape-size, and shape-Y deviation, respectively). The Electronic Supplementary Material provides a parameterized understanding of human grasp behavior driven by object shape, size, and relative position in detail.

Table 3. F-values of the ANOVA on the factor scores of each PC.

Consequently, we mainly observed that (1) Y deviation is the most significant impact factor for PC1 and PC2 due to the maximum value of F in Table 3, PC1 and PC2 performing a three (Supplementary Figure 1) and two ladder-like (Supplementary Figures 2A–C) distribution under the significant impact of Y deviation changing from distal to proximal. (2) Object sizes largely impact the higher order PC (PC2–PC4) as shown in Table 3, Supplementary Figures 2A-C, 3, 4A–C. Different from PC1, higher order PC mainly controls the hand to precisely contact and grasp the object. (3) Object shapes significantly impact PC1–PC4 scores as shown in Table 3. The impact on PC1 is most obvious across PC1–PC4 as shown in Table 3, Supplementary Figure 1. Especially when the object is in the distal and middle Y deviation, PC1 varies across different shapes; the largest is the prism, followed by the cylinder and sphere, which means that the flexion of four-finger MCP joints increases in the order of sphere, cylinder, and prism. (4) The X and Z deviation impact on hand grasping is obviously lower than the Y deviation, object sizes, and shapes as shown in Table 3, Supplementary Figures 1, 2A-C, 3, 4A–C.

Discussion

This paper provides a systematic framework to help understand hand movement functionality more comprehensively with both qualitative and quantitative analysis. Two novel observations are found: (1) The arm and wrist make a vital contribution to hand movement functionality, and (2) the relative position is a general influence factor significantly impacting human prehension. Finally, a laboratory-based unstructured behavioral experiment is implemented to quantitatively demonstrate the significant influence and understand the human grasp parameterized as driven by object shape, size, and relative positions. The results presented here should be more representative and complete to understand human grasp functionality.

The hierarchical tree presented in section Analysis of Hand Movement Functionality can be used to help decompose the complex task and help understand hand movement skills in ADLs. For the task-contained complex time-separated sequence, the complex task can be decomposed into multiple more specific subtasks according to the hierarchical tree. For instance, drinking hot water might be decomposed into six subtasks: reaching for the cup (class 8), exploring the cup temperature whether hot or not as an antenna (class 7), choosing one distinct grasp type from prehensile taxonomy to grasp the cup stably (class 1) if the cup is not very hot, the arm and wrist transferring the hand to lift the cup to the mouth, and pouring the water into the mouth (class 2), putting the cup back on the table with a stable grasp (class 2), letting the cup, arm, and hand go back to the initial position (class 8), and finally finishing the whole task with a rest position (class 6).

For the bimanual task, both hand movements can be clearly described. For instance, taking pills can be decomposed into 12 subtasks: reaching for the pill bottle with the left hand (class 8), choosing one distinct grasp type from the prehensile taxonomy to grasp the bottle stably by the left hand (class 1), the left arm and wrist transferring the left hand to the front of the chest (class 2), reaching for the bottle cap with the right hand (class 8), both hands keeping a stable grasp and loosening the bottle cap with both arms and wrists helping if the bottle cap is tight enough (class 2), the left hand keeping the stable grasp (class 1) and right hand screwing off the bottle cap by the within-hand rotation about the z-axis from the prehensile taxonomy (class 3), if the bottle cap is loose, putting the bottle cap on the table with the right hand (class 2), the left hand imparting the bottle motion to pour the pills into the right hand (class 2) while the right hand with a platform push type gets the pills (class 5), the left hand putting the bottle on the table (class 2), the right hand putting the pills in the mouth (class 2), and finally finishing the pill-taking task.

The systematic analysis of hand movement functionality also can be used to explore the manner of neuroscience control. Human hands, having more than 24° of freedom (DoF), are actuated by multiple intrinsic and extrinsic muscles by means of a complex web of tendons under the control of the central nervous system. Although extensive investigations have been carried out in biomechanics and the neuroscience of the human hand, there are still many difficulties needing further exploration, such as human hands having how many DoF (especially for the thumb and finger metacarpus), grasp kinematics and impedance properties for different tasks, movement correlations between hand joints under biomechanical constraints, muscle synergies and neural synergies of hand control, brain activity in the grasping process, etc. We believe our systematic analysis of human hand movement functionality can serve this research. Based on the hierarchical tree of hand action and hand prehensile taxonomy, researchers can develop different experimental protocols contributing to their research.

Because the hand prehensile taxonomy presents a descriptive vocabulary of human prehensile behaviors, it can be used to inform design related to the hand, such as product packages (DiSalvo and Gemperle, 2003) on which hands act and assistive devices. As a representative instance of assistive devices, the prosthetic hand is a reconstruction device of human hand functionality (Catalano et al., 2014; Xiong et al., 2016). Amputees wear it to accomplish lots of versatile grasp tasks in their daily lives. Under the limitations of being anthropomorphic, lightweight, robust to uncertainty, and durable, a prosthetic hand should be actuated by a small number of actuators but still have versatile grasp functionality. It is important to build the actuation strategies contributing to the reconstruction of versatile grasp functionality. Based on our proposed prehensile taxonomy, the contribution of each hand DoF to the distinct grasp and manipulative types can be evaluated. In this case, the actuation strategies can be constructed. For reconstructing the move characteristics of the hand joints, the joint angles or positions of each type in prehensile taxonomy can be analyzed and implemented by a mechanism. Moreover, the impedance properties for each grasp type in different tasks can be explored and also can be mechanically implemented by embedding some elastic elements.

The systematic analysis of hand movement functionality also can be used in the clinical rehabilitation and assessment of hand movement capabilities (Light et al., 2002). The hierarchical tree of hand action helps therapists to develop the standardized structure of assessment protocols. The prehensile taxonomy provides a detailed, wide-ranging congregation for developing the protocols on a detailed level. For assistive devices, they also can be used to evaluate and improve the performance contained in mechanical implementation and human–computer interaction.

Even though hand movement functionality has been studied for many years, people are still used to analyze qualitative and quantitative perspectives separately. In this paper, we provide an integral framework to understand human hand movement functions both from qualitative and quantitative perspectives. In qualitative analysis, the general term “hand movement function” is specific to the action class and the detailed postures of prehensile taxonomy. Therefore, we find that the relative position is a significant impact factor except for object shape and size. After simultaneously considering these three impact factors, we quantitatively investigated the posture changing characteristics impacted by these factors through the behavior experiment. Therefore, we hope the qualitative analysis can provide a comprehensive consideration of hand movement functionality and the quantitative analysis can reveal the characteristics of hand movement driven by significant influencing factors. However, several limitations can be identified.

First, the hierarchical tree is a top-level consideration. The aim is to provide a systematic framework about hand movement functionality. However, it does not capture many of the low-level details about hand posture. We pay particular attention to prehensile functionality. The representative postures of two action classes about hold and within-hand manipulation are collected in prehensile taxonomy. The non-prehensile movement might be given additional detailed attention through further subclassification of our hierarchical tree. Second, according to the hierarchical tree, we find that contribution of the wrist and arm for hand movement functionality is important. Although it is easy to accept, the quantitative evidence of arm and wrist contribution should be further investigated, such as the contribution ratio of each hand action, etc. Third, the behavioral experiments considered the significant influencing factors obtained from the analysis of prehensile taxonomy, but the collected postures are not the specific postures of the prehensile taxonomy. It is very interesting that we provide a quantitative evaluation of the posture diversity construction ability to prehensile taxonomy by PCs of behavioral experiment (see Supplementary Material). As the mean construction errors to each posture are 5°-10° (Supplementary Figure 6), the PCs perform diverse posture construction ability. However, the maximum construction error joints to each posture are mainly distributed in the thumb, pinky, and index finger as shown in Supplementary Figure 7. Therefore, we know that the independent movements of these fingers are relatively difficult to accurately construct by the PCs.

Conclusion

In this paper, we provide a systematic framework to help understand human hand movement functionality from an integral consideration: (1) Hand movement functionality is decomposed and specific to the action as the smallest composition unit of movement functionality. (2) A hierarchical tree is built to classify the action to eight classes based on the hitherto significant definitions of hand movement. (3) A hand prehensile taxonomy containing 52 grasp types is built to provide a macroscopic understanding of hand prehensile function containing both static hold and within-hand manipulations. (4) Prehensile taxonomy is rearranged into object prehensile taxonomies to further explore the general influence factors of hand prehension. (5) Based on the influence factor exploration result, a laboratory-based unstructured behavioral experiment is implemented to comprehensively understand the human grasp and qualitatively demonstrate the significant impact factors.

Consequently, a general analysis method of human movement functionality is presented in this paper and applicable to other body parts, such as wrist, arm, etc. We believe the novel and integral framework of hand movement functionality presented here should help understand hand movement functionality more comprehensively and precisely.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by Institutional Review Board (IRB) of Harbin Institute of Technology. The patients/participants provided their written informed consent to participate in this study.

Author's Note

In 2019 YL was awarded the Peiyang Elite Scholar Program of Tianjin University.

Author Contributions

YL developed the original concept, drafted this paper, and analyzed the resulting data. LJ gave advice on the research methods. HL provided the necessary testing facilities and equipment essential for this research. DM gave some advice on the research methods and helped review the content of the paper. All authors contributed to the article and approved the submitted version.

Funding

This work was supported by the National Natural Science Foundation of China (51905375), the China Post-doctoral Science Foundation Funded Project (2019M651033), Foundation of State Key Laboratory of Robotics and System (HIT) (SKLRS-2019-KF-06), Peiyang Elite Scholar Program of Tianjin University (2020XRG-0023), and National Natural Science Foundation of China-Shenzhen Robot Research Center Key Support Project (U1813209).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnbot.2021.658075/full#supplementary-material

References