Skip to main content
Log in

Performance on Video-Based Situational Judgment Test Items: Simulated Interracial Interactions

  • Original Paper
  • Published:
Journal of Business and Psychology Aims and scope Submit manuscript

Abstract

Individuals are known to categorize others into social groups based on cues like race and gender and to experience relative discomfort when interacting with “outgroup” members. Two experimental studies were used to examine whether actor demographic cues in situational judgment assessment items completed by test takers in a simulated employee selection context may lead to differences in their performance and reactions to the hiring organization. In both studies, test takers assumed the perspective of actors shown in video-based scenarios and indicated how they would respond to interaction partners (IPs) to whom they were racially similar or dissimilar. In Study 1, a given test taker responded to IPs of a constant gender; in Study 2, IPs’ gender varied across scenarios within each condition. In Study 1, Black test takers spent more time and scored better on two of the four scenarios when responding to racially similar IPs. These effects were not found in Study 2, but demographic cues showed new interactive effects on performance and reactions. We discuss the implications of different findings across the two studies.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

Notes

  1. Campion et al. (2014) reviewed SJT research and correlated SJT attributes with their coefficient alpha values. Alpha values were lower when video versus written SJT formats were used (r = − .28, p < .05).

  2. Brief interactions with or observations of strangers differ from the more sustained interactions over time examined in the context of cross-group friendships (e.g., West and Dovidio 2013). Friendship development involves more complex and dynamic processes where interaction partners influence each other and evidence change in personal attitudes and behavior over time (West and Dovidio 2013). For example, as individuals get to know outgroup members, they can find out individuating information that may be inconsistent with their held stereotypes (Rothbart and John 1985). Thus, we bound our literature review and theoretical rationale to contexts of interracial interactions between individuals who are new interaction partners or who have less close relationships and not to discussions of long-standing, well-developed close relationships.

  3. As hypothesis testing was based on White, Asian, and Black actors, additional participants who were not from those demographic groups (N = 21) were excluded from analyses.

  4. Research on avatar gender has shown that as many as 79% of those in multiplayer online games have used avatars of an opposite gender and 30% do so on a regular basis (Hussain and Griffiths 2008; Yee & Ducheneaut, 2011, as cited by Martey et al. 2014). Findings fairly consistently show that when assuming an avatar of a different gender people do not tend to behave differently (i.e., movement within a game), but when men take on a female role, they tend to adopt stereotypic language patterns in chat functions (i.e., use more emotional language, use more exclamation points; Yee and Bailenson 2007; Martey et al. 2014). As our response options were not gendered in language patterns and individuals were responding to a multiple choice item, this research would suggest that gender assumed should not have an effect. Men and women in the current research generally did not perform differently on the SJT scenarios.

  5. Two measures of reactions to the SJT scenarios—perceived opportunity to perform and job relatedness—were also included in the current studies, but there were no differences in these measures across conditions in either Study 1 or Study 2. In the interests of brevity, these measures are not discussed. Ratings of interaction partner attractiveness were also collected. As these ratings were generally uncorrelated with the outcomes of interest and controlling for these ratings in testing the hypotheses and research questions in studies 1 and 2 did not substantively change our results, these measures are likewise not discussed.

  6. The pattern of significant results did not change when rerunning analyses without excluding multivariate outliers.

  7. We crossed participant race and IP race to check if results differed depending on the race of the IP and found that for the conflict situation, Black test takers performed better when interacting with a Black IP than when interacting with the White IP. For the helping situation, Black test takers performed better when interacting with a Black IP than when interacting with either the White IP or the Asian IP. However, sample sizes are small for cells crossing Black participants with Asian and White IPs (Ns = 8–9).

  8. When crossing test taker race and IP race, Black test takers spent significantly more time responding to Black IPs than to Asian IPs for all four scenarios. Response time differences for Black IPs relative to White IPs were statistically significant for conflict situation 2 and helping situation 2. Again, sample sizes are small for cells crossing Black participants with Asian and White IPs (Ns = 8–9).

  9. Additional exploratory analyses examined whether test taker-IP gender similarity affected SJT item scores, item response times, or perceived fit with the organization and whether test taker gender interacted with test taker race, test taker-IP racial similarity, or level of diversity represented in the SJT videos. There were no statistically significant effects in Study 1.

  10. The pattern of significant results did not change when rerunning analyses without excluding multivariate outliers.

  11. There were also no statistically significant differences in performance when crossing test taker race and IP race.

  12. We also crossed test taker race and IP race and found that Black test takers spent more time responding to the two conflict situations when interacting with Asian IPs. For conflict situation 1, the difference was statistically significant relative to White IPs; for conflict situation 2, the difference was statistically significant relative to both White and Black IPs.

  13. Additional exploratory analyses examined whether test taker-IP gender similarity affected SJT item scores, item response times, or perceived fit with the organization and whether test taker gender interacted with test taker race, test taker-IP racial similarity, or level of diversity represented in the SJT videos. Like in Study 1, there were no statistically significant effects.

  14. For interactions involving six groups (e.g., race similarity × test taker race), a sample size of approximately 35 per group is recommended to detect medium effect sizes at a .05 significance level with a power of .80 (Cohen 1992). After data cleaning, our sample sizes of minority test takers ranged from 11 to 18 in Study 1 and 15 to 36 in Study 2.

References

Download references

Acknowledgments

The authors wish to thank Jessie Olien, Abraham Kim, and Julie Klopp for their assistance with data collection for the current studies. The authors also thank Neal Schmitt, Danielle King, Dia Chatterjee, and two anonymous reviewers for their comments on an earlier version of this paper.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Juliya Golubovich.

Ethics declarations

This research was approved by the Institutional Review Board at Michigan State University.

Conflict of Interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This research was conducted while the first author was a graduate student at the Michigan State University.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Golubovich, J., Ryan, A.M. Performance on Video-Based Situational Judgment Test Items: Simulated Interracial Interactions. J Bus Psychol 36, 693–711 (2021). https://doi.org/10.1007/s10869-020-09697-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10869-020-09697-1

Keywords

Navigation