Collective rationality and functional wisdom of the crowd in far-from-rational institutional investors

Abstract

The average portfolio structure of institutional investors is shown to reproduce the structure which optimally accounts for transaction costs when investment constraints are weak. Strikingly, this result emerges even though these investors are not aware of the existence of such law and despite the fact that their aims and tools are very heterogeneous. This extends the so-called wisdom of the crowd to much more complex situations in two important ways. First, wisdom of the crowd also holds for whole functions instead of a point-wise estimates. Second, this shows that in socio-economic systems, the optimal individual choice may only be found when the diversity of individual decisions is averaged out. Thus, rationality at a collective level does not need nearly rational individuals with well-aligned incentives. Finally we discuss the importance of accounting for constraints when assessing the presence of wisdom of the crowd.

Appendices

Appendix: Optimal number of assets in a portfolio

Let us briefly summarize how to derive the relationship between wealth and the optimal number of assets to hold in a portfolio in a special case: We refer the reader to de Lachapelle and Challet (2010) for a more general case and for a more comprehensive discussion of the various assumptions. Assume that a trader has no position and wishes to spread his capital W into N (fixed) assets. Transacting an amount w incurs a cost $F(w)=Cw^{\delta }$. Denoting by $R_{P}$ the return of the portfolio over a given period, a mean–variance cost is

$$\begin{aligned} L_{\lambda }(R_{P})=\lambda E(R_{P})-\text {Var}(R_{P}). \end{aligned}$$

(5)

Now, the portfolio return over that period is nothing else than the weighted sum of the returns of individual assets plus the return on cash held minus the total transaction costs. Setting $x_{i}=w_{i}/W$ and $x=\sum _{i}x_{i}$ and assuming zero interest rate, one has

$$\begin{aligned} R_{P}=\sum _{i=1}^{N}x_{i}R_{i}-C\sum _{i=1}^{N}\left( \frac{(x_{i}W)^{\delta }}{W}\right) . \end{aligned}$$

Since most funds (and retail traders) default for equally weighted portfolios, we set $x_{i}=x/N$, which yields

$$\begin{aligned} R_{P}=\frac{x}{N}\sum _{i}R_{i}-\frac{C}{W^{1-\delta }}\sum _{i}x^{\delta }. \end{aligned}$$

It is straightforward to derive the optimal fraction of capital to invest by maximizing L in Eq. (5), which yields, assuming negligible cross-correlation between the assets,

$$\begin{aligned} x^{*}=\frac{\lambda }{2}\left[ \overline{R_{i}}-\delta C(N/x^{*}W)^{1-\delta }\right] . \end{aligned}$$

Noting that L depends on x, $\lambda$, and N, one can alternatively optimize N at fixed x and $\lambda$, which gives

$$\begin{aligned} \lambda =\frac{W^{1-\delta }}{(1-\delta )C(N^{*}/x)^{2-\delta }}. \end{aligned}$$

These two last equations make it possible to eliminate ${{\lambda }}$ and thus to find that, for large N, $N^{2-\delta }\propto W^{1-\delta }$ and hence that $N^{*}$ scales like $W^{\frac{1-\delta }{2-\delta }}$.

Filtering

In order to remove inconsistencies in the dataset, we applied several filters.

1.1 Country of origin

Our dataset is sparse and heterogeneous. Indeed, the quality of the sources of data is directly related to each country’s disclosure regulations. For these reasons, we decided to keep only the entities which use an US-based mail address.

About 60% of the total market capitalization of the dataset is concentrated in US-based securities. Figure 5 shows two large clouds of dots, and each of them corresponds to a different region of origin: Green (resp. orange) cloud corresponds to non-US (resp. US)-based securities. The origin of this large difference between these two regions is not clear: It could, for example, come from differences in regulations in non-US countries. It turns out that the ratio of the investment values in US and non-US assets varies little as a function of time (see Fig. 5), which does not affect the exponent $\mu$ in Eq. (1). As a consequence, we focused on US securities.

Fig. 5

Top: Market capitalization as a function of the number of investors for US securities (orange points) and non-US securities (green points). Bottom: Temporal evolution of the aggregated market capitalization of US securities over the total market capitalization (color figure online)

1.2 Frequency

Large funds are requested to report their positions at a frequency which depends on the applicable regulation. As a result, reporting frequency ranges from monthly to yearly, most funds filing quarterly reports. We therefore focused on the latter.

1.3 Penny Stocks

“Penny stocks,” i.e., usually securities which trade below $5 per share in the USA, are not listed on a national exchange. Since they are considered highly speculative investments and are subject to different regulations, we filtered them out.

1.4 Size

We also filtered out small founds and securities and applied the following filters: $W_{i}>10^{5}$ USD, $C_{\alpha }>10^{5}$ USD, $n_{i}\ge 5$, $m_{\alpha }\ge 10$.

1.5 Output

We restricted our study to 36 quarterly snapshots starting from the first quarter of 2005 and ending with the last quarter of 2013. Figure 6 reports the evolution of the number of securities and funds in the database before and after filtering.

Fig. 6

Temporal evolution of the number of funds $N_{i}$ and securities $N_{\alpha }$ in the database. Unfiltered in dashed lines and US based only in solid lines

Determination of the crossover point $\mathbf {n^{*}}$

For each date t,  we define the cross-over point $n^{*}$ between the two regions which appear in the local polynomial regression. We determine this point value with a likelihood maximization of the model

$$\begin{aligned} \log W=\mu _{<}\log n+(\mu _{>}-\mu _{<})(\log n-\log n^{*})\theta (\log n-\log n^{*}), \end{aligned}$$

(6)

where $\theta (x)$ is the Heaviside function which encodes an if-clause so that $\mu =\mu _{<}$ if $n<n^{*}$ and $\mu =\mu _{>}$ otherwise. We use the method introduced by Muggeo (2003) to find parameters $\mu _{<}$, $\mu _{>}$, and $n^{*}$, which is implemented by its author in the R package segmented. In essence, this method consists in linearizing Eq. (6) and determining the relevant parameters recursively. Figure 3 shows that $n^{*}$ is stable as a function of time.

Asset selection: a model

The framework we introduce in this paper follows a series of a few elementary steps described below. The aim is for the model to be sensitive to the different constraints which dominates the portfolio selection of a fund.

1.1 Asset selection in the small diversification region $n_{i}<n^{*}$

In this region, we assume that portfolios are equally weighted. Each position has a size $\frac{W_{i}}{n_{i}^{\text {opt }}}$, where $n_{i}^{\text {opt}}$ is the optimal number of position computed with Eq. (1). The funds select their asset randomly with a probability proportional to $C_{\alpha }$. Also, in order to build an equally weighted portfolio, a position is valid only if it is of size $\frac{W_{i}}{n_{i}^{\text {opt}}}$.

1.2 Asset selection in the large diversification region $n_{i}\ge n^{*}$

In this region, the liquidity constraints make it harder for funds to keep an equally weighted portfolio and portfolio values are thus spread on a larger number of assets. We propose here a stochastic model of asset selection based on two main ingredients: The first is that the selection probability of asset $\alpha$ by fund i depends on the diversification of a fund $n_{i}$ and on the scaled rank of the capitalization of asset $\alpha$ and the second is that the investment is bounded by an hard constraint on the fraction of market capitalization of asset $\alpha$.

We chose a security selection mechanism which rests on the scaled rank of capitalization of security $\alpha$, defined as $\rho _{\alpha }=\frac{r_{\alpha }}{M}$, where $r_{\alpha }$ is the rank of capitalization $C_{\alpha }$ and M the number of securities at a given time. The selection probability $P(W_{i\alpha }>0|\rho _{\alpha })$ is then obtained by parametric fit to a beta distribution in each logarithmic bin. Note that we do not use the same rank-based selection mechanism in the low-diversification region because in this case it is harder to have a good fit with the beta distribution. This is, however, only a minor point since the capitalization is approximately power-law distributed and the two selection mechanisms are basically equivalent (the rank is proportional to a power of the capitalization), and indeed, results are very similar in both cases.

Fig. 7

Top: Empirical probability of investing in a security of scaled capitalization rank $\rho$ for each fund diversification bin $[n_{i}]$ . Bottom: Probability density function of investing in a security of scaled capitalization rank $\rho$ given the diversification $n_{i}$ of the fund, given by the model

Figure 7 shows that the distribution of the ranks in which a fund is invested is sensitive to its diversification $n_{i}$ ($t=$2013-03-31). The beta distribution is defined as

$$\begin{aligned} f(x;a,b)=\frac{1}{B(a,b)}x^{a-1}(1-x)^{b-1}, \end{aligned}$$

(7)

where a and b are the shape parameters of the distribution and B is a normalization constant, is limited to a $\left[ 0,1\right]$ interval, and is flexible enough to describe the asset selection mechanism of funds.

1.2.1 Maximum investment ratio

Funds limit their investment in a given asset by using a simple rule of thumb: Defining the investment ratio $f_{i,\alpha }=\frac{W_{i\alpha }}{C_{\alpha }}$, one easily sees in Fig. 8 that each fund has its own maximum investment ratio

$$\begin{aligned} f_{i}^{\text {max}}=\text {max}_{\alpha }\left( \frac{W_{i\alpha }}{C_{\alpha }}\right) . \end{aligned}$$

(8)

Since the average exchange volume dollar of an asset is proportional to its capitalization (Fig. 9), the existence of $f_{i}^{\text {max}}$ is a way to account for the available liquidity.

Fig. 8

Fraction of the market capitalization of a security held by a fund. Each color represents a different fund. Top: Funds with a large diversification ($n_{i}>800$). We can clearly see a delimitation for most of the funds, which correspond to the maximum fraction $f_{i}^{\text {max}}.$ The value of $f_{i}^{\text {max}}$ widely differs from one fund to another. Bottom: Funds with a low diversification ($n_{i}<60$); $f_{i}^{\text {max}}$ does not appear

Although that limit is clear for an individual fund, the range of empirical values $f_{i}^{\text {max}}$ is remarkably large (see Fig. 10).

Fig. 9

Market capitalization as a function of the daily exchange volume dollar averaged over the previous three months, for March 31, 2013. Fitting data to $C_{\alpha }=W_{\alpha ,\text {daily}}^{\eta }$. We find $\eta \simeq 1$ for all the dates in our database, confirming the hypothesis that the daily exchange volume dollar of an asset is approximately proportional to its market capitalization

Fig. 10

Empirical probability density function of $f_{i}^{\text {max}}$ for all the funds in the $n>n^{*}$ region

Fig. 11

Coefficients a and b of the beta distribution 7 as a function of $n_{i}$ . Linear fits are for eye guidance only

Fig. 12

We separate the contribution from the low and highly diversified region. The origin of the discrepancy observed in Fig. 4 appears to be mainly due to the highly diversified region

Simulation of asset selection

The simulation is done in a few simple steps:

• 1. For a given time t,  compute $n^{*}$ from the data using the segmented model Eq. (6).

• 2. Iterate over all the funds: for fund i, with a number of assets $n_{i}$,

  • (a) If $n_{i}<n^{*}$:

    • i Compute its optimal portfolio value using Eq. (1). The fund will invest $\frac{W_{i}^{\text {opt}}}{n_{i}}$ for every position.

    • ii Select assets randomly with a probability proportional to $C_{\alpha }$.

  • (b) Else if $n_{i}\ge n^{*}$:

    • i. Compute its $f_{i}^{\text {max}}$, so that the fund i will invest $f_{i}^{\text {max}}$ in $n_{i}$ assets.

    • ii. Select assets according to their capitalization rank following a beta probability distribution in Fig. 7 with the parameters found in Fig. 11.¹

By iterating those steps, we obtain Fig. 2.

Since the simulation outputs a portfolio for every fund, we can directly infer the number of investors $m_{\alpha }$ of every security (Fig. 12).