1. Introduction

Consider a metacommunity as a network of communities. From the network perspective, the units are the communities, and the communities are connected by relations that compare structural characteristics of the communities’ compositions. Communities themselves are conceived in a wide sense as collections of hypothetical or real biological objects that have some defined feature in common. For individuals as real objects, such features could include their existence in a specified spatial region during a specified period of time (see, e.g. Palmer and White (1994)). The same generalizing conception is applied to metacommunities as communities of communities. For this reason, the associated structural characteristics of metacommunity states are not limited to the outcome of colonization-competition dynamics that conform with the traditional concept of a metacommunity.

This network perspective differs from common approaches that are directed towards characteristics of the distribution of type variation (species, genotypes, etc.) within and among communities (such as partitioning diversity into \(\alpha\)-, \(\beta\)-, \(\gamma\)-components, or differentiation among communities or populations, see, e.g. Sokol et al. (2017)). Differentiation studies, in particular, are usually based on differences among species or the genetic compositions of communities or populations. They are, however, also performed with the aim of discovering spatial gradients among communities with respect to their species compositions. The absence of such gradients is then understood to be an indication that the communities lack coherence and thus do not function as a metacommunity (see, e.g. Presley et al. (2010)).

Especially in the context of discovering spatial gradients in species composition, existing methods essentially rest on verification of special (idealized) patterns of community-species (or site-species) associations that result from various kinds of interspecific or species-environment interactions (represented by site-by-species incidence matrices). In recent overviews, Schmera et al. (2018) and Presley et al. (2019) take a critical position as to the validity of those methods when special emphasis is placed on details of the underlying concepts and measures of coherence and nestedness among communities. Most of their criticism refers to the framework within which coherence is based on species ranges (developed by Leibold and Mikkelson (2002)), whereas network perspectives are actually viewed from a community perspective (see, e.g. Shevtsov et al. (1992)).

Transitions between community compositions (including ecotone characteristics), as are typical of gradients, play a particular role, since they cannot be detected with the help of summarizing differentiation statistics (such as the above-mentioned measures of \(\beta\)-diversity, or measures such as \(G_{ST}\) that are used in population genetics to quantify meta-population structure). This calls for a generalizing perspective that allows aspects of coherence to be viewed within an established scheme, such as within networks with their intuitively straightforward connotations of connectedness and thus coherence.

Community networks can be conceived as graphs with nodes (vertices) in place of the single communities and edges specifying the relations between the communities (cf., e.g. Urban and Keitt (2001) or Thompson et al. (2017)). Edges can be endowed with weights that are defined by the kind of relations between the nodes (communities). In metacommunities, relations are determined by the degrees to which the single communities share or differ in characteristics of their members. The representation of these characteristics across communities sets the limits within which metacommunity coherence is to be assessed.

A minimum requirement for coherence in a collection of communities is the existence of a connection between any two communities via a sequence of communities, each of which is related to its adjacent communities in the sequence by shared characteristics. This is known in graph theory as a path. Paths that return to their initial nodes (cycles) do not add to this minimum requirement of coherence, nor do multiple paths between nodes. The set of such unique paths establishes a tree graph that spans all nodes (i.e. connects all communities in the collection). The existence of a spanning tree therefore is a prerequisite for coherence of the collection and thus for its assessment as a metacommunity.

Since a connected graph may well have several spanning trees that may vary in the sums of their edge weights, those trees that have the minimum weight sum (known as minimum spanning trees or MSTs) would be of primary interest. The reasoning is that the minimum weight sum specifies a tree with the minimum sum of differences between adjacent communities and thus with the maximum sum of shared characteristics within the tree. Consequently, the strength of coherence realized in a graph becomes most clearly evident when such trees are viewed. MSTs and their properties will therefore have an important part in the following analyses. Recall that according to the general concept of metacommunity applied here, connections between communities are not limited to the outcome of migrational events.

A distinction is made between connectedness and coherence, in that the former refers to the plain numerical values of the edge weights, while the latter additionally includes qualitative aspects. Whereas edge weights are quantified in terms of amounts of shared characteristics of communities (or differences between species compositions), coherence adds properties to these values that refer to aspects such as complete distinctness in species composition or nestedness of one community in another. While determination of an MST simply rests on the edge weights, the assessment of metacommunity structure additionally considers coherence properties as well as their strength as reflected in the MST.

The present paper treats these structural aspects with special reference to complex differences among community members that go beyond simple qualitative traits (characters) such as species affiliation or genetic type (for more details on the concept of trait, see, e.g. Violle et al. (2007)). To provide a proper basis for the subsequent analysis of coherence and its implications, the following section (Section 2) begins by arguing elementary features of difference measurement between communities or more general sets of objects (or individuals), then recalls how these features enter common perceptions of differentiation among community compositions (Section 2.1), and introduces a method that allows variably defined object differences (variable object differences, for short) to be considered within this framework of difference measurement between communities (Section 2.2). The graph theoretical foundation of coherence characteristics based on the introduced measures of community difference is then specified (Section 3), where MSTs are shown to provide the conceptual means for determining coherence components (Section 3.1). It is reasoned that methods for quantifying strength of coherence are meaningfully referred to total MST length (Section 4). In conclusion, several applications are suggested, such as the partitioning of community collections into one or more metacommunities based on a given level of distinction, assessment of their degree of fragmentation, or determination of coherence gradients (Section 5). The summary in Table 1 provides a stepwise reference to accompany the development of these concepts and methods.

In order to address a wider range of questions relating to metacommunity coherence, we rely on a more general definition of metacommunity that allows not only for sets of interacting communities that are linked by the dispersal of multiple, potentially interacting species and are spatially separated (Leibold et al. (2004)). Interaction within and between communities may also be defined by reproduction, which turns communities into populations and metacommunities into metapopulations. Moreover, factors other than space could have effectively separated communities (populations), examples of which are non-overlapping flowering periods among plant species (e.g. Degen et al. (2006)) or positive assortative mating for differential characters among animal species (e.g. Hinton et al. (2018)). Different communities may exist here in sympatry, and this can be extended to many other forms of interaction within and among species.

More basically, however, a clear distinction must be made between the forces that give rise to certain structures (forces that are rarely, if ever, amenable to precise observation) and the observable structural characteristics themselves, which are usually identified via characteristics that are or are not shared by members of different communities. While in a conceptual context, metacommunities are defined by abstract relations (dispersal, etc.), the realization of these relations can be determined only by viewing the observable consequences of these relations. In this context, the methods developed in the following are to be classified as descriptive, with the focus on detecting signs of coherence as they can be caused by the forces forming metacommunities. Among the numerous examples of the application of descriptors relating to metacommunity structure that are comparable to those introduced in the following and that specifically treat problems of stability in metacommunity networks, the paper by Thompson et al. (2017) may be a useful guide.

Table 1 Stepwise reference to accompany the development of concepts and methods

2. Basic features of representation or nestedness among sets of objects

As indicated above, any statement of coherence between communities rests on the possibility to draw connections between their compositions. Most basically, this requires the detection of characteristics of members of one community that can also be found among the members of another. The implied direction of perspective is commonly associated with connotations of inclusion and thus with the nestedness of one community in another. Inclusion relationships between two sets A and B of objects are primarily determined by the degree to which the objects in one set can be considered to be represented in the other set with respect to a specified trait. Whether or not the communities actually overlap is irrelevant here. Depending on the definition of community, the same individual can in effect be a member of different communities. Even though set inclusion is implied, the directedness aspect seems to justify our preference for the term “nestedness” over “overlap”.

For a discrete trait, the inclusion relationships can be obtained from application of the following recursive assignment algorithm which corresponds to sampling pairs of objects without replacement:

  1. (1)

    take any object from A and look for an object from B with the same trait state,

  2. (2)

    if there exists such a pair, remove the two objects from their respective sets,

  3. (3)

    continue step (1) and (2) for the respective remainders of A and B until no further such pairs can be formed.

The objects remaining in A (if any) after completion of the assignment constitute the difference of A from B. There may be several such assignments, but all of them necessarily yield the same number of non-assigned objects from A.

If the number of non-assigned objects from A is zero, A is understood to be (completely) nested in B. The number of objects in A minus the number of non-assigned objects from A then specifies the number R(AB) of objects from A that are “represented” in B. If \(a_i\) and \(b_i\) denote the number of objects with the i-th trait state in set A and in set B, respectively, then the number of objects from A represented in B amounts to the well-known quantity \(R(A,B):=\sum _i \min \{a_i,b_i\}\). In the same way, one obtains the number of objects from B represented in A as \(R(B,A)=\sum _i \min \{b_i,a_i\}\), from which \(R(A,B)=R(B,A)\) follows. Accordingly, with \(N_A\) and \(N_B\) as numbers of objects in A and B, respectively, one obtains the absolute differences \(d_a(A,B)=N_A-R(A,B)\) and \(d_a(B,A)=N_B-R(B,A)\) as the number of objects from A not represented in B and the number from B not represented in A, respectively (compare, e.g. equations 1b and 1c in Baselga (2017)).

The corresponding relative differences are then obtained as \(d_r(A,B)=[N_A-R(A,B)]/N_A=1-R(A,B)/N_A\) and \(d_r(B,A)=1-R(B,A)/N_B\), for which \(d_r(A,B)<d_r(B,A)\) if and only if \(N_A<N_B\). Complete nestedness of A in B is now indicated by \(d_a(A,B)=0\) (and thus, \(d_r(A,B)=0\)), while complete distinctness of A from B is reached for \(R(A,B)=0\) and thus, \(d_r(A,B)=1\). Clearly, \(d_r(A,B)=1\) is equivalent to \(d_r(B,A)=1\).

Note that \(R(A,B)=\sum _i \min \{a_i,b_i\}= {1\over 2}\sum _i(a_i+b_i)-{1\over 2}\sum _i|a_i-b_i|= {1\over 2}(N_A+N_B)-{1\over 2}\sum _i|a_i-b_i|\) so that \(d_a(A,B)+d_a(B,A)=\sum _i|a_i-b_i|\), which is known as the “Manhattan distance” between A and B. While this metric and its statistics are well known, its significance in quantifying nestedness via the absolute and relative directed differences \(d_a\) and \(d_r\) seems to have escaped notice so far. Other well-established indices such as Jaccard’s index (which is identical to the Tanimoto index) can also be derived from the Manhattan distance (Gregorius, 1996) but have likewise not been applied to measuring nestedness in the above-specified sense.

However, when the abundances of types are ignored, so that only the presence of types is counted in communities (as is done in the site-by-species incidence matrices), then all \(a_i\)’s and \(b_i\)’s are either 1 or 0. If \(N_A\le N_B\) holds in addition, then \(d_r(A,B)\) equals the measure of nestedness suggested by Almeida-Neto et al. (2008).

2.1 Replacement and nestedness aspects of community difference

The absolute or relative set differences are particularly suited for qualifying and quantifying the common notions of nestedness and replacement, where the latter term is frequently used when addressing turnover events (for a review of the terms see, e.g. Legendre (2014)). In common perception, replacement refers to individual exchanges with the characteristic result that the total set size remains the same. Hence, replacements within A to yield B require that \(N_A=N_B\) and therefore \(d_a(A,B)=d_a(B,A)\) hold (or equivalently \(d_r(A,B)=d_r(B,A)\)). The number of objects to be replaced in A in order to arrive at B then equals \(d_a(A,B)\).

On the other hand, if \(N_A\ne N_B\), it is helpful to return to the above assignment method, which by definition specifies the number of objects from A that need not be replaced when trying to mirror the composition of B. Consequently, if \(N_A<N_B\), then \(d_a(A,B)\) reports the number of objects that have to be replaced in A so that it becomes completely nested in B. Therefore, \(d_a(A,B)\) and especially its relative version \(d_r(A,B)\) can be interpreted alternatively as specification of the deviation from complete nestedness. This reveals more explicitly the relationship between the terms nestedness, replacement, and (set) difference. It provides justification for generally considering differences in set compositions from a nestedness perspective and gives meaning to ideas of “partial nestedness”.

In fact, in the case \(N_A>N_B\), it is not possible to achieve complete nestedness of A in B by making replacements in A. Yet, reversing the direction of view reveals that \(d_a(B,A)\) is the appropriate measure of deviation from complete nestedness. The observation of \(d_a(A,B)-d_a(B,A)=N_A-N_B\) emphasizes the fact that the asymmetry in the absolute set differences is solely due to the difference in the set sizes. Compositional characteristics of the sets have no effect. When applied to the relative set differences \(d_r\), one obtains \((1-d_r(A,B))/(1-d_r(B,A))=N_B/N_A\), in which case the effect of set sizes on set differences is expressed in terms of ratios with the relative differences replaced by their corresponding similarities (\(1-d_r\)).

Especially when allowing for nestedness to be partial, its affinity to the concept of replacement and set difference becomes apparent. In effect, all three terms refer to the same initial concept of individual assignments and are summarized by the same measure of set difference with \(d(A,B)=0\) indicating complete nestedness of A in B, \(d(A,B)=d(B,A)\) indicating that the difference between A and B is solely due to replacement, and \(d_r(A,B)=1\) indicating complete distinctness between the sets (d without an index refers to absolute as well as relative difference). Since \(d(A,B)=d(B,A)\) can also be conceived of as mutual partial nestedness between the two sets, nestedness presents itself as an overarching idea of set difference. Numbers of trait states (frequently referred to as richness or diversity) may indirectly affect the d-measures. Richness differences are occasionally treated as a separate component of dissimilarity measurement (cf., e.g. Legendre (2014); Baselga (2017)).

The notions of representation, replacement, and nestedness become indistinguishable when set sizes are equal. The relative set differences \(d_r\) are symmetric in this case and do not depend on set size, since relative type frequencies take the place of (absolute) type abundances together with set sizes. Hence, whenever information on community compositions is restricted to relative type frequencies, analyses of metacommunity coherence cannot consider directions of difference or change.

2.2 Variable object differences

When species are distinguished by multiple traits or phylogenetic relationships and genotypes are distinguished by their gene differences, the assessment of differences between the individuals with respect to this characterization varies beyond simple binary statements of identity. In this case, the above idea of representation or nestedness among communities requires definition of a threshold difference \(\lambda\) between objects (individuals), above which two objects are considered not to be sufficiently similar to represent each other. Alternatively, \(\lambda\) could be referred to as the resolution with the understanding that only objects that differ by more than \(\lambda\) are considered to be distinguishable in a specified context. Application of the above assignment algorithm then reads for the first step: (1) take any object from A and look for an object from B that differs by not more than \(\lambda\) from the former. The other two steps are the same, so that the objects remaining in A after completion of the assignment determine the degree to which A is represented in B. Yet, this would make sense only if all admissible assignments lead to the same number of remaining objects.

The approach for discrete traits can be transferred to the situation of variable object differences when considering ultrametrics as difference measures. With ultrametrics, a relation of ‘identity’ between objects can be defined in the sense that any two objects are considered ‘identical’, if their ultrametric distance does not exceed a selected threshold difference. The thus defined relation among objects establishes an equivalence relation, which, in turn, specifies a partition of the set into equivalence classes. Within each class, the objects differ by not more than the threshold, while among classes the difference exceeds the threshold.

The approach to characterizing nestedness relationships for discrete traits can then be applied identically to these equivalence classes when they are conceived of as states of a discrete trait. This requires that the ultrametric be defined on the totality of objects under consideration. Step (1) of the assignments is then realized, since differences between objects do not exceed the threshold only when they belong to the same class. Consequently, when abundance of objects with a given trait state is replaced by abundance of objects that belong to a predetermined class, all of the above statements on measuring differences between sets apply identically.

This suggests transformation of an arbitrary difference measure to the ultrametric cophenetic distance that results from application of a hierarchical clustering method, such as single-linkage. Cutting stems in the resulting tree (or dendrogram) at a height corresponding to the desired threshold difference (or resolution) produces a partition of the set of objects into the above-mentioned equivalence classes. Herewith, it must be taken into account that the cophenetic differences may not directly reflect the original difference measures, since the cophenetic differences depend on the clustering method as well as on the totality of objects under consideration. Nevertheless, the special problem at hand may justify replacement of the original differences by the cophenetic differences. The closeness to the original differences can be assessed with the help of cophenetic correlations, for example.

There exists a clustering method, however, that realizes step (1) as stated at the beginning of the present subsection without any transformation of the original difference measure. It rests on the definition of G-clusters as groups of objects within which the maximum difference (the group diameter) is less than the minimum difference from objects outside the group. G-clusters are either mutually distinct or one is contained in the other (hierarchical ordering). Denote a G-cluster as a G-cluster at level \(\lambda\), if it has diameter \(\le \lambda\) and if any other G-cluster properly containing it has diameter \(>\lambda\). The set of all G-clusters at level \(\lambda\) can be shown to form a partition of the set of all objects that is analogous to the above partition into equivalence classes (Gregorius 2012). See Box 1 for an example.

figure a

Recall that G-clusters are to be formed on the totality of objects under consideration. For a collection of communities (a presumed metacommunity), this totality amounts to the union of all constituent communities. As opposed to other clustering methods, G-clusters obey the requirement that objects from the same cluster differ by not more than the threshold level determined for the original scale of difference measurement. Transformation of cophenetic distances into directly interpretable differences is therefore unnecessary. All of the above statements on set differences apply identically. One thus arrives at the desired result that the assignments of objects between communities do not exceed the threshold level and the number of objects remaining after the assignment does not depend on the particular assignment.

3. Graph theoretical treatment of nestedness and coherence

In this section, the above considerations are cast into a graph theoretical framework. For an arbitrary number of sets, asymmetric measures of pairwise difference define a directed weighted graph in which each set is a node. Each pair of nodes is connected by two edges pointing in opposite directions and weighted by the respective set difference. The nodes themselves could be weighted by the set sizes, but these are already incorporated in the set differences, as implied by the above assignment method.

When the sets are communities, such graphs help to detect structural relationships among the communities based on their species compositions, for example. Since structural relationships are determined primarily by the degrees to which the communities share species or show more general conformity in their species characteristics, they are described most basically by nestedness relations among the communities. Since coherence is mediated by nestedness, structural relationships among communities can be assessed by coherence. The translation of aspects of metacommunity coherence into type of graph is then guided by the connectedness among nodes.

Though two nodes can be connected in many ways in the original graph, coherence is chiefly determined by the shortest connections between the nodes. Since shortest connections appear as paths within spanning trees, only spanning trees of the graph are relevant for consideration of presumed metacommunity coherence. In particular, minimum spanning trees (MST) minimize the totality of one-step connections (edges) across the whole tree. Box 2 shows MSTs for the example in Box 1.

figure b

3.1 MST characteristics and coherence

When applying set differences as edge weights in a graph, MSTs have several properties that accord well with the idea of connectedness via nestedness. To see this, recall that for any tree, when adding an edge connecting two nodes of the tree, a cycle results. Removing another edge (not its incident nodes) of this cycle then results in a new tree that connects the same nodes. If the removed edge has a higher weight than the added edge, then the sum of edge weights in the new tree is smaller than in the old tree. Therefore, the possibility of any such transformation in an MST would corrupt the minimum property of the resulting tree (occasionally called the cycle property). Hence, only the smaller of the reciprocal differences d(AB) and d(BA) (whether absolute or relative) enters the construction of an MST. The smaller of the two is, however, exactly the one that determines the degree of nestedness.

The cycle property also implies that none of the edges of an MST-path that connects two nodes can have more weight than the edge connecting the two nodes. This very basic and important MST characteristic will repeatedly be made use of in the following. It conforms with the intuitive understanding of communities that spread and differentiate in space and time without losing coherence.

Loss of coherence in a presumed metacommunity is impending if there are communities A and B with \(d_r(A,B)=1\). Loss of coherence of a metacommunity is, however, not tantamount to loss of connectedness in the general graph theoretical sense (i.e. \(d_r(A,B)=1\) does not indicate disconnectedness of the “nodes” A and B). To understand the implications of this situation within an MST, suppose that the MST contains an edge of weight 1. Removing this edge creates two subtrees for which it holds that any edge connecting two nodes, one from each subtree, has weight not less than 1 according to the cycle property. Hence, between the two sets of nodes present in the two subtrees, all differences equal \(d_r=1\).

In terms of coherence, this graph theoretical situation corresponds to two sets of communities with complete distinctness between the sets. Hence, coherence is completely absent between these sets of communities. Moreover, the subtrees in the two sets of nodes are again MSTs, since otherwise there would exist subtrees with lower edge sum and reconnection of these would yield a spanning tree of the entire graph with reduced total edge sum. This would contradict the MST property of the original tree. Consequently, additional edges of weight 1 could be identified in each subtree, implying a further decomposition of the nodes. At the end, one arrives at a decomposition into disjoint sets of nodes, such that the edge between two nodes from different sets has weight 1 and an MST-edge between two nodes from the same set has weight less than 1.

These sets will be termed coherence components of the network, since they correspond to the connected components in general graphs. Within a coherence component, each pair of communities is connected by a sequence of non-distinct communities that follows an MST-path. Hence, in order for all communities in a collection of communities to form a single coherence component, all MST-edges must have \(d_r\)-weight less than 1. In terms of tree graphs, the components result from removing (cutting) MST-edges of weight 1, with the result that each component is provided with an MST that is a subtree of the overall MST. Note that within a coherence component, each community need not be directly connected to every other.

If several edges are of equal weight, more than one MST can possibly be formed. Accordingly, several alternative structures of network coherence among the communities are possible. Again, the above cycle property can be called on to characterize the alternatives. For example, when the weight of an edge not yet used in a particular MST equals that of another edge which belongs to an MST-path connecting the adjacent nodes of the first edge, a new MST can be formed by adding the new edge and removing the old one. It therefore is of central importance to know whether alternative MSTs may lead to different decompositions into coherence components. The proof provided below confirms the important insight that indeed the formation of coherence components is not affected by the existence of alternative MSTs (for a generalization see Section 5.2). This also demonstrates the appropriateness of relating coherence components to functional metacommunities. Footnote 1

Complete nestedness of all communities, meaning that \(\min \{d(A,B),d(B,A)\}=0\) for all A and B, marks the extreme state of coherence. In this case, the MST degenerates to a single path and indicates that the total collection effectively consists of a single community. This follows from the fact that \(d(A,B)=d(B,C)=0\) implies \(d(A,C)=0\), so that \(d(A,B)=0\) defines a relation between two communities that is transitive. While this is obvious for discrete traits, it is seen to extend to variable object differences when considering G-cluster partitions of the totality of objects under consideration at specified threshold differences, as explained above.

For variable object differences, the community differences \(d_r\) depend on the respective threshold difference \(\lambda\). In particular, as \(\lambda\) approaches zero the corresponding G-cluster partition becomes finer, with the result that fewer objects belong to the same G-cluster at level \(\lambda\). The opportunities to find objects (individuals) in the same cluster between two communities therefore decrease and \(d_r\) increases. This in turn increases the chances for communities to be completely distinct and thus of lower coherence.

Conversely, as \(\lambda\) becomes large in relation to the maximum object difference, G-clusters at this level become fewer and larger. Opportunities for object assignments within the same cluster increase between communities, and this implies smaller \(d_r\)-values indicating stronger coherence.

To prevent possible misinterpretation, it seems apt to mention that it is possible that two nodes in a coherence path (with all of its edge weights less than 1) appear as distinct communities. This does not contradict the general idea of coherence, even though it may happen that one community shares types with a second, and the second shares types with a third, but the latter types are not shared between the first two communities. Hence, one community, in this case the second, would comprise types from two distinct communities. A typical example would be selective colonization, where two distinct species groups of a community colonize different habitats. Coherence is still realized in a stepwise fashion along coherence paths, but the directions of the steps vary along a path.

Another example is provided by community collections extending along riverbanks. In such arrangements, it is conceivable that complete distinctness is reached between the communities at either end via stepwise replacement of species or genetic types. The MST would ideally mirror this situation by consisting of a single path.

4. Measuring degrees of coherence

It is argued in the previous explanations that coherence in a collection of communities (a presumed metacommunity) is defined from an MST perspective by coherence paths and coherence components in particular. Therefore, total coherence is basically invalidated if several coherence components exist (all communities are divided into several non-overlapping coherence components). Consequently, it is meaningful to consider degrees of coherence for each coherence component separately. Recall that an MST of a coherence component is a subgraph of an MST of the total graph (with weights of all subgraph edges less than 1).

An intuitively obvious measure of the strength of coherence is the sum of edge weights of an MST of the coherence component, where coherence increases with decreasing sum. At the extreme, complete coherence is realized if all communities in the component are completely nested, which makes the sum zero and reveals that the coherence component effectively consists of a single community. Given the number of communities in the component, the sum of MST-edge weights \(d_r\) always remains below the number of nodes minus 1 as its supremum. This suggests normalization of the sum by the number of component nodes minus 1, which yields the average edge weight of the MST of the coherence component. The closer this average approaches 1, the less coherent the component becomes and the more it successively disintegrates into independent communities. The separation from the other components, if there are any, is not lost.

It was shown above (footnote in Section 3.1) that the decomposition of the collection of communities into coherence components is independent of the existence of alternative MSTs. Since the subtrees within each component are again MSTs, the average MST-edge weight is well-defined in each coherence component. This fact suggests definition of a coherence index as 1 minus the average MST-edge weight in the respective coherence component. The index equals 1 for complete coherence (complete nestedness), and it approaches 0 as coherence declines towards complete distinctness of all communities in the component.

The extreme values of the average MST-edge weight are reminiscent of ordinary differentiation measurement in that a value of 0 is obtained in the absence of differentiation among communities, i.e. if all communities are identical for their type distributions, and a value of 1 is realized for complete differentiation, i.e. where communities share no types. Yet, since differentiation measures are symmetric by definition and apply to relative frequencies (independently of sample sizes) or to absolute frequencies for equal sample sizes, they cannot be used to explicitly distinguish between aspects of nestedness or replacement (turnover) in the assessment of community differences. Moreover, measures of differentiation summarize the differences among all communities under consideration and are therefore not qualified for the detection of metacommunity structure, as is inherent in the coherence components of MSTs. The latter becomes particularly evident from the above examples, in which coherence can be reached despite the fact that some communities are completely distinct.

5. Concluding remarks

Because the specification of coherence relationships among communities could involve multiple traits, the probably most relevant characterization of community members (as objects) is by object differences that summarize the differences in the contributions of the relevant traits in a meaningful way (see, e.g. Gower (1971); Gregorius (2006); Pavoine et al. (2009); Scheiner et al. (2017)). Besides species affiliation or genetic type, traits that are related to coherence aspects include, for example, geographical, spatial or ecological conditions. In fact, a common view maintains that “species should occupy a coherent range of sites along environmental gradients (i.e. no gaps or discontinuities should exist in the fundamental niche of a species)” (Presley et al., 2019). Difference measures that jointly cover these aspects of coherence are therefore needed, and this confirms that variable object differences are likely to be the most relevant differences in the analysis of coherence in community networks.

Since variable object differences require specification of threshold differences (or resolutions) in coherence analysis, the first question to be answered is how an increase in the threshold (decrease in resolution) affects community differences. As indicated above (also see Box 2), community differences are expected to decline with increasing threshold difference (or decreasing resolution). An explicit and straightforward proof is obtained by recalling that the G-clusters at higher threshold levels result from union of G-clusters at lower threshold levels according to the hierarchical nature of G-clustering. The assignment method for obtaining the community differences then implies that all of the assignments at the lower level also apply to the higher level, so that the number of assignments for the higher level cluster is at least as large as the sum of assignments realized for the included lower level clusters. The number of non-assignable objects and thus the community difference can therefore not increase with increasing threshold level.

The community differences reach their extremes for a threshold level of zero and for a threshold level equal to the maximum object difference. The first case includes the situation of discrete traits, and in the second case the smaller in each pair of communities is completely nested in the larger because all objects are considered “identical”. Hence, as the threshold level increases from its lower to its upper value, the assessment of the collection of communities approaches the situation of complete coherence (also see Box 2).

During this passage, it might happen that for particular threshold levels, several communities are united to form a coherence component within the community collection. At the latest, when all relative community differences \(d_r\) are less than 1, the collection of communities forms a single coherence component. The minimum threshold level \(\lambda\) above which this happens therefore identifies the resolution below which the collection is recognized as a single and proper metacommunity (“proper” meaning that no two communities are completely distinct). Since it may occur that at level \(\lambda =0\) all relative community differences \(d_r\) are already less than 1, the community collection could be classified as a proper metacommunity for all resolutions (threshold levels). It should be recognized, however, that complete coherence at low levels of resolution is ecologically meaningless. One could argue that an increase in coherence that is solely the result of a decrease in resolution is a methodological artefact that fails to reflect the true relations among the objects within the communities.

5.1 Coherence gradients

As mentioned above, coherence is frequently associated with environmental or spatial gradients of the species compositions of communities. The community-network analogue of this approach requires inclusion of environmental conditions or spatial traits into the measurement of differences between communities as network nodes. Here, it has to be considered that communities may overlap in their species as well as in their environmental conditions and possibly also with respect to their spatial extensions. Hence, community differences can be assessed jointly by the differences of their members for species affiliation and for environmental conditions (or spatial location). Once a joint object difference is defined, determination of community differences is again provided by formation of G-cluster partitions at specified threshold levels. Coherence gradients can then be identified by the corresponding coherence paths in the MSTs of the community network. Note that this concept of gradient differs from what is called “abundance gradient” by Baselga (2017), which is simply a series of completely nested communities.

Before several object differences can be combined into a single joint measure, their commensurability has to be established. This can be achieved in many ways (see, e.g. Gregorius (2006)) but is most frequently done by normalization via the maximum of the observed differences (see, e.g. Pavoine et al. (2009)). The former author suggested a method of combining commensurable difference measures into a single joint measure, where the measures can be given variable weights and their asymmetries can be taken into consideration. Differences in species characteristics and spatial distances, for example, may then be weighted according to the desired significance of their effects on coherence issues. In the same way, adverse effects of the two difference categories can be taken into consideration (for more details see Gregorius (2006)). Thus, if similarity in species characteristics, for example, is considered to be more significant in the assessment of coherence than spatial distance, the former receives higher weight.

To affirm that the joint object differences capture coherence relations among communities at a specified threshold level, recall that assignment between communities takes place only among those individuals that are sufficiently similar in their species characteristics and are spatially sufficiently close (according to the joint threshold difference). The resulting community differences thus count only those individuals for which this correspondence is not realized and which therefore do not add to coherence at the specified level. Complete distinctness is now realized between communities, the members of which are either spatially too far apart or belong to species with characteristics whose joint differences exceed the threshold level.

Coherence in spatial gradients of species composition is usually determined by continuity (or the absence of gaps) in the occurrence of species (see, e.g. the above citation of Presley et al. (2019)). The detection of gradients therefore depends on prior specification of the spatial transects. In the present approach, in contrast, such transects are revealed in the coherence paths of the community network and are therefore a possible outcome rather than a precondition of the analysis. Coherence may therefore be observable even though similar species exist at higher (spatial) distances or in reverse. Gradients of coherence are then determined by the coherence paths in the MST.

figure c

5.2 Coherence at various levels of distinctness

The above specification of coherence components can be extended so as to include arbitrary thresholds for the assessment of distinctness or coherence between communities. For example, migration below well-argued rates is not expected to have a significant effect on community differentiation, since the rates are not sufficient to guarantee establishment (as is considered in the species-sorting and mass-effect paradigms of the traditional metacommunity concept, see, e.g. Leibold et al. (2004); Logue et al. (2011)). In a sense, distinctness can be effectively realized in such cases below the state of complete distinctness. Beyond this, it could simply be of interest to reveal the existence of collections of communities that are separated to a certain degree from other such collections, while coherence can be meaningfully stated within each collection. The coherence components corresponding to these collections would then be required to realize MST-edges of weight less than the threshold, while edges connecting nodes from different components would show weights equal to or greater than the threshold.

Indeed, the proof is exactly analogous to that for complete distinctness given in Section 3.1 (including the footnote), where essentially the original MST is decomposed into separate subtrees by removing (cutting) MST-edges of weight equal to or greater than the threshold value. With each such cut, the weights of edges connecting nodes between the resulting subtrees are equal to or greater than the weight of the cut edge by the cycle property of MSTs. Ultimately, this procedure leads to the above decomposition into coherence components. Similar thoughts apparently led Xu et al. (2002) to the design of clustering algorithms akin to single-linkage clustering. In fact, the present decomposition into coherence components at specified threshold levels could be conceived of as a method of clustering communities.

As an illustration, Box 3 shows the coherence components (square boxes) of the four MSTs in Box 2 at two thresholds of community distinctness \(\rho\). The figures confirm three relations:

  1. (1)

    The number of coherence components for given \(\lambda\) is a non-increasing function of \(\rho\). For \(\lambda {=}0.05\), the deletion of the two edges of weight 0.400 for \(\rho {=}0.3\) but not for \(\rho {=}0.5\) causes the four components for \(\rho {=}0.3\) to collapse to two components for \(\rho {=}0.5\). Analogously, for \(\lambda {=}0.15\), the edge of weight 0.300 determines that the three components collapse to two.

  2. (2)

    The number of coherence components for given \(\rho\) is a non-increasing function of \(\lambda\). By increasing \(\lambda\), the resolution between object types is lowered, leading to a reduction in the distinctness of the communities. For \(\rho {=}0.3\), the number of components falls from four to two with increasing \(\lambda\), while for \(\rho {=}0.5\), the number of components equals two for all four \(\lambda\). Other pairs of components maintain their distinctness over all \(\rho\) and \(\lambda\) (e.g. 0.667 between the component containing B and that containing C).

  3. (3)

    In cases where the same set of communities forms a coherence component for given \(\rho\) but different \(\lambda\), the coherence of the component is a non-increasing function of \(\lambda\). The reduction in the community distinctness as \(\lambda\) increases implies that the mean edge weight does not increase with \(\lambda\). For \(\rho {=}0.5\), the mean edge weight of component AB falls from 0.400 to 0.200 to 0 and the mean edge weight of the component CDEF falls from 0.289 to 0.189 to 0.078 to 0.056 as \(\lambda\) increases.

5.3 Extensions

Frequently, the conceptual consistency of a method becomes most apparent in its extreme realizations. The present concept considers a collection of objects that can be decomposed (or partitioned) into classes, within which the objects are conceived to be indistinguishable or equivalent at a specified threshold difference. The decomposition is performed by G-clustering at the respective threshold. It does not depend on the community affiliations of the objects. Communities can therefore be determined in various ways, including the extreme situation where each object is regarded as a community of its own. In this case, the present method arrives at community differences that are 0 or 1 according to whether the respective objects belong to the same or to different G-clusters. The coherence components thus coincide with the G-clusters. While this may appear trivial and relates to ordinary clustering of individuals into communities at a specified level of similarity, it yet demonstrates the consistency of the present method of coherence assessment.

In less extreme situations, decomposition of the total collection may not be explicitly characterized by community properties. Relevant structures may not even be directly observable but may rather be suspected to exist as a result of cryptic fragmentation or migration barriers, for example. Decomposition into potential communities could then be motivated by the observation of a discontinuity in the spatial distribution of environmental factors. Depending on the factor under consideration, various decompositions could be of interest, each with its specific degree of coherence. Comparisons between the factors as to their associated degrees of coherence would then allow the factors to be ranked according to their involvement in fragmentation.

The method of coherence assessment is also applicable, for example, to ”interactions” in the metanetwork analyses of Emer et al. (2018) and Li et al. (2020) in fragmented forest landscapes. An interaction between a frugivorous bird and a plant is present in a fragment (community), if the bird is observed to feed on a fruit of the plant. Each such interaction can be considered as a single object that consists of a pair characterized by the species affiliations of the bird and the plant or by their respective sizes. In this way, each object is assigned a type, such as the pair (bird species, plant species) or one of the four combinations (large/small bird, large/small seed). This corresponds to the general setting of discrete traits in the present approach and consequently allows application of the above methods for the assessment of coherence, including the decomposition of the collection of forest fragments into coherence components.

5.4 Coherence and variable differences

If differences between objects vary, ideas of nestedness among communities and associated coherence relationships could be problematic, since type identity between assigned objects may practically not exist. In this case, clusters or classes of objects take the position of types, with the restriction that clusters are less strictly separated from each other. While the individual objects are distinct entities by definition, their separation for trait characteristics may become the more ambiguous, the more continuously these characteristics vary. Particularly for G-clustering with its untransformed reference to the measured differences, this could imply that little clustering is possible above the level of individual objects. Thus, finding objects in different communities that belong to the same G-cluster is unlikely, so that communities tend to be largely distinct, and coherence among them is consequently low.

At first sight, this might contradict intuitive expectations in that small differences among objects should exclude sizable community differentiation, so that all communities would be assessed to form a single coherence component. Yet, identification of objects that represent each other depends on the chosen level \(\lambda\) of resolution. If this is low, identification of objects from different communities is indeed rarely possible. On the other hand, as \(\lambda\) increases in such a situation, no further G-clustering could occur until all objects finally end up in a single G-cluster, in which case all communities form a single coherence component. Well-argued levels of resolution must therefore be specified in studies involving variable object differences.

A note on implementation of the method: Numerical calculations of the method as applied in the example in Boxes 1-3 were programmed by EMG. The Fortran program, which implements the data clustering utilities “cluster” and “den” of P.C.J. Kleiweg (https://www.let.rug.nl/kleiweg/clustering), is available from this author.