Does music really work? The two-stage audiovisual cross-modal correspondence effect on consumers’ shopping behavior

Abstract

In this research, we explore the impacts of cross-modal correspondence between sound frequency and color lightness on consumers’ shopping behavior. Compared to previous studies that relied on a stable single-stage information environment, our study is based on a two-stage (i.e., elimination and choice stages) cognitive model to account for the dynamic cross-modal correspondence effect on shopping behavior. After conducting two laboratory experiments and one field experiment, we find that although consumers tend to pay more attention to light (vs. dark) products in the high (vs. low)-frequency sound condition in the elimination stage, this effect is less salient at the choice stage. We further find that consumer involvement acts as a moderator. Specifically, the correspondence effect is attenuated for highly involved consumers.

Appendices

Appendix 1. Laboratory experiment design for study 1

Procedures:

• 1) Briefly explain that the objective of the laboratory experiment is to understand consumer online shopping behavior and inform participants that they will be compensated with gift certificates valued at 20 CNY for a convenience store in the university.

• 2) Ask participants whether they have an initial preferred shade of red. If yes, ask them to indicate their preferred shade from the following five different levels of red lightness.

• 3) Participants are then seated at the eye-tracker computer. The guide mentions the following to the participants: “Imagine you are going to buy a new hat and find some hats at an online store. Please keep your eyes on the screen.”²

• 4) Lead the participants to go to the online store website. Tell the participants that the store website will display two available hats to them quickly. Use the following screen that was captured from the website as an example.³

  

• 5) (Elimination stage) The website will show two red hats for 10 s where one is light (100% value) and the other is dark (60% value). The order of the two red hats is randomized. Each hat was priced at approximately 68 CNY (approximately $9.81) and introduced simply without any brand information. Participants will be exposed to either a high-frequency tone (approximately 1800 Hz) or a low-frequency (approximately 120 Hz) tone of “Turkey in the Straw” at a predetermined volume displayed by the research assistant on his mobile phone.

• 6) (Choice stage) Show the participants the following screen as an example. Tell the participants that they are going to consider whether to make a purchase and remind them to keep their eyes on the screen. The screen will be displayed for 30 s (there was no “end” button on the screen). Participants will still be exposed to either the high-frequency or the low-frequency tone.

  

• 7) Instruct them to fill in their perceived sound frequency and background information:

  • Perceived sound frequency, gender, age, monthly living expenses, online shopping frequency

• 8) End of the session. Thank the subjects for their participation.

Appendix 2. Laboratory experiment design for study 2

Procedures:

• 1) Briefly explain that the objective of the laboratory experiment is to understand consumer online shopping behavior.

• 2) Ask participants whether they have an initial preferred shade of blue. If yes, ask them to indicate their preferred shade of blue from the following five different shades of blue.

  

• 3) Participants in the condition of low involvement were instructed that “Imagine you are going to buy a new T-shirt and find some T-shirts at an online store.” Participants in the condition of high involvement were instructed to “Imagine that our Business and Management School is going to launch some T-shirts to the students at our school. The school has posted the T-shirts to its online store. You are going to buy a new T-shirt from the online store.”

• 4) Lead the participants to go to the online store website. Tell the participants that the store website will display two available T-shirts to them quickly. Use the following screen that was captured from the website as an example.

• 5) (Elimination stage) The website will show two blue T-shirts for 10 s where one is light (100% value) and the other is dark (60% value). The order of the two blue T-shirts is randomized. Participants will be exposed to either a high-frequency tone (approximately 1800 Hz) or a low-frequency (approximately 120 Hz) tone of “Turkey in the Straw” at a predetermined volume displayed by the research assistant on his mobile phone.

• 6) Participants were asked whether they would like to click on the light/dark blue T-shirt to obtain more information and instruct them to fill in their click intention on each T-shirt (1 = extremely unlikely, and 7 = extremely likely).

• 7) (Choice stage) Show the participants the following screen as an example. Tell the participants that they are to consider whether to make a purchase. The screen will be displayed for 30 s (there was no “end” button on the screen). Participants will still be exposed to either the high-frequency or the low-frequency tone.

• 8) Instruct the subjects to fill in their purchase intention on each T-shirt (1 = extremely unlikely, and 7 = extremely likely).

• 9) Instruct them to fill in their perceived sound frequency and background information, including perceived sound frequency, involvement, gender, age, monthly living expenses, and online shopping frequency.

• 10) End of the session. Thank the subjects for their participation.

Appendix 3. Results of models with correlated error terms in laboratory studies

To estimate the cross-modal effects on participants’ visual attention in study 1, we specify the model below:

$$ {Y}_{irt}={\alpha}_0+{\alpha}_1{\mathrm{Correspondence}}_{\mathrm{i}\mathrm{r}}+{\alpha}_2{\mathrm{Preference}}_{\mathrm{i}}+{\alpha}_3{\mathrm{Gender}}_{\mathrm{i}}+{\alpha}_4{\mathrm{Age}}_{\mathrm{i}}+{\alpha}_5{\mathrm{Expense}}_{\mathrm{i}}+{\alpha}_6{\mathrm{Frequency}}_{\mathrm{i}}+{\varepsilon}_{irt}, $$

(2)

where Y_itr represents participant i’s visual attention ratio (i.e., the ratio of the fixation time to total exposure time) on hat r (0 for the light red hat and 1 for the dark red hat) on stage t (0 for the elimination stage and 1 for the choice stage). The Correspondence_ir represents the correspondence of background music and product color that was displayed for consumer i (1 for correspondence (i.e., high-frequency music with the light red hat or the low-frequency music with the dark red hat) and 0 for non-correspondence (i.e., the other combinations of color and sound)). In the model, we also include controls for participant i’s initial color preference (Preference_i), gender (Gender_i), age (Age_i), living expenses (Expense_i) and online shopping frequency (Frequency_i).

Given that a participant’s attention levels at the two stages are not independent, we follow previous literature (Agarwal et al., 2011) and correlate the error terms of the models for participant i’s attention ratio at the two stages as follows:

$$ {\displaystyle \begin{array}{c}\mathrm{where}\ {\varSigma}_{\varepsilon }=\left[\begin{array}{cc}{\sigma}_{ARES}^2& {\sigma}_{ARES, ARCS}\\ {}{\sigma}_{ARCS, ARES}& {\sigma}_{ARCS}^2\end{array}\right],\\ {}{\varepsilon}_{irt}^{ARES},{\varepsilon}_{irt}^{ARCS}\sim N\left(0,{\varSigma}_{\varepsilon}\right)\end{array}} $$

(3)

where ${\varepsilon}_{irt}^{ARES}$ represents the error term of participant i’s attention ratio at the elimination stage, and ${\varepsilon}_{irt}^{ARCS}$ represents the error term of participant i’s attention ratio at the choice stage.

Table 3 shows the results of the models with correlated error terms. The results indicate that the audiovisual cross-modal correspondence effects for the attention ratio are significant at both the elimination stage (0.136, p < 0.001) and the choice stage (0.027, p < 0.001). Moreover, the coefficient of correspondence for the attention ratio was significantly greater at the elimination stage than that at the choice stage (χ²(1) = 94.92, p < 0.001). These findings imply that compared to the elimination stage, the cross-modal correspondence effect on consumers’ visual attention is attenuated at the choice stage.

Table 3 Results of models with correlated error terms in study 1

Similarly, we estimate the cross-modal effects on participants’ visual attention and the moderating effect of involvement in study 2 with correlated error terms as follows:

$$ {Y}_{irt}={\alpha}_0+{\alpha}_1{\mathrm{Correspondence}}_{\mathrm{i}\mathrm{r}}+{\alpha}_2{\mathrm{Involvement}}_{\mathrm{i}}+{\alpha}_3{\mathrm{Correspondence}}_{\mathrm{i}\mathrm{r}}\ast {\mathrm{Involvement}}_{\mathrm{i}}+\kern1.25em {\alpha}_4{\mathrm{Preference}}_{\mathrm{i}}+{\alpha}_5{\mathrm{Gender}}_{\mathrm{i}}+{\alpha}_6{\mathrm{Age}}_{\mathrm{i}}+{\alpha}_7{\mathrm{Expense}}_{\mathrm{i}}+{\alpha}_8{\mathrm{Frequency}}_{\mathrm{i}}+{\varepsilon}_{irt}, $$

(4)

where Y_irt represents participant i’s intention on T-shirt r (0 for the light blue T-shirt and 1 for the dark blue T-shirt) on stage t (click intention at the elimination stage and purchase intention at the choice stage). The Correspondence_ir represents the correspondence of background music and product color that was displayed for consumer i (1 for correspondence (i.e., high-frequency music with the light blue T-shirt or the low-frequency music with the dark blue T-shirt), and 0 for noncorrespondence (i.e., the other combinations of color and sound)). Involvement_i represents consumer i’s perception of personal relevance related to product categories or shopping tasks (1 for high involvement and 0 for low involvement). In the model, we also include controls for participant i’s initial color preference (Preference_i), gender (Gender_i), age (Age_i), living expenses (Expense_i) and online shopping frequency (Frequency_i).

Given that a participant’s intentions at the two stages are not independent, we correlate the error terms of the models for participant i’s intention at the two stages as follows:

$$ \kern0.5em {\displaystyle \begin{array}{c} where\ {\varSigma}_{\varepsilon }=\left[\begin{array}{cc}{\sigma}_{IES}^2& {\sigma}_{IES, ICS}\\ {}{\sigma}_{ICS, IES}& {\sigma}_{ICS}^2\end{array}\right],\\ {}{\varepsilon}_{it}^{IES},{\varepsilon}_{it}^{ICS}\sim N\left(0,{\varSigma}_{\varepsilon}\right)\end{array}} $$

(5)

where ${\varepsilon}_{it}^{IES}$ represents the error term of participant i’s click intention at the elimination stage, and ${\varepsilon}_{it}^{ICS}$ represents the error term of participant i’s purchase intention at the choice stage.

Table 4 shows the results of the models with correlated error terms. The results indicate that the audiovisual cross-modal correspondence effect for intention is significant at the elimination stage (2.917, p < 0.001) and at the choice stage (0.733, p < 0.01). In addition, the coefficient of correspondence for intention at the elimination stage was significantly greater than that at the choice stage (χ²(1) = 49.880, p < 0.001). Moreover, the results also indicate that the moderating effects of involvement are negative at both the elimination stage (−2.328, p < 0.001) and at the choice stage (−0.511, p < 0.1), which provides more evidence for the robustness of our results.

Table 4 Results of models with correlated error terms in study 2

Appendix 4. Measures of main variables

All measures were reported on a 1–7 scale (1 = extremely disagree, 7 = extremely agree).

Construct	Item	Source
Visual attention4	Visual attention was measured by the total fixation duration in s within the area of interest (AIO).	Hagtvedt and Brasel (2016)
Click intention (α = 0.89)	1. I feel like clicking the item now. 2. I would like to click on the item as soon as possible. 3. I would like to click on the item right away.	Taylor and Todd (1995); Yang et al. (2017)
Purchase intention (α = 0.88)	1. I feel like buying this item now. 2. I would like to buy the item as soon as possible. 3. I would like to buy the item right away.	Taylor and Todd (1995); Song et al. (2018)
Consumer involvement (α = 0.87)	1. I perceive this item as very important. 2. I perceive this item as very significant. 3. I perceive this item as very valuable. 4. This item matters a lot to me. 5. This item means a lot to me.	Zaichkowsky (1985)

1. Note: α, Cronbach’s alpha

Appendix 5. Experimental design of field study 3

Fig. 4

Platforms used to conduct the field experiment. a The control platform. b The treatment platform. Note: Music was played only for PC customers. We thus collected data only from PC customers. In addition, there is no music stop/pause button on the platforms during the whole experimental period

Fig. 5

Pages used to display products in elimination and choice stages. a The elimination stage. b The choice stage. Note: All products (with a product picture, the price, and a description) at the platform would be exposed to consumers when they visited the platform (the elimination stage). If a consumer was interested in a product, she/he would click on the product and be led to the landing page of the product’s page to learn about the details (e.g., size information, materials) of the product (the choice stage). Both platforms display the product pictures on the left and details of the product on the right and bottom of the page. If the consumers took an action to purchase, a conversion was recorded

Appendix 6. Differences in prices, pre-treatment clicks, and pre-treatment conversions of treated and control products

There were 16 and 13 products at the control and treatment platforms, respectively. There are no significant differences in product prices, pre-treatment clicks, and pre-treatment conversions between the products at the two platforms (ps > 0.1).

Analysis	Product type	Mean	St. Dev.	p-value (t-value)
Price	Treated	9.888	3.665	0.221 (1.278)
	Control	13.438	9.460
Clicks	Treated	55.500	74.437	0.785 (0.276)
	Control	64.538	101.805
Conversions	Treated	6.375	9.946	0.300 (1.058)
	Control	3.077	5.766

1. Note: Numbers in the above table present the average performance of each product on platforms during the pre-treatment period. For example, on average, products at the treatment platform received 55.500 clicks and 6.375 conversions in the pre-treatment period, while products at the control platform received 64.538 clicks and 3.077 conversions in this period

Appendix 7. Robustness checks in field study 3

We note that both the click-through rate (mean: 0.070; Std.: 0.115) and conversion rate (mean: 0.049; Std.: 0.135) have a relatively high SD-vs-value ratio. This could be because the utility of the individual choice in clicking on a product or converting it to a sale follows an independent and identically distributed (i.i.d.) extreme value distribution rather than a normal distribution (Agarwal et al., 2011). To address this issue, we checked the robustness of the cross-modal correspondence effects in some alternative specifications (see Table 3). Column (1) followed Agarwal et al.’s (2011) approach and used a logit model to check the robustness of our results. In Column (2), we modeled the logarithm of the dependent variables, and in Column (3), we used the standardized values of dependent variables in the model. The results of Columns (1)-(3) were similar to those in Table 2; thus, our results are robust.

Table 5 Robustness checks of study 3