In today’s knowledge-based economy, with its rapidly changing job requirements, cognitive competencies have been identified as a crucial prerequisite for social and economic engagement (OECD [Organisation for Economic Cooperation and Development] 2012). The number of available routine and manual labor tasks is in decline, while the demand for high-level cognitive and information-processing competencies is rising (OECD 2013). Individuals who lack the necessary competencies have difficulty finding employment and face depreciation of the competencies they do possess when they are not put to use. An inventory of the existing competencies of nations’ workforces is therefore necessary to identify deficits and their consequences.

In October 2013, the Programme for the International Assessment of Adult Competencies (PIAAC) provided up-to-date information on the competencies of the adult populations in 24 countries. Nine more countries followed in June 2016, with five more countries scheduled to follow in the third round of the survey. The data is of high quality, based on an extensive sample in each country and collected according to robust methodological standards (Rammstedt and Maehler 2014), enabling meaningful international comparisons. This special issue of Large-Scale Assessments in Education features four papers that have used the rich database that PIAAC offers to provide new insights into a number of topics. These include the relationships between competencies and education, labor market outcomes (such as incomes), and personality. Three articles also address specific methodological aspects of PIAAC, such as the coverage of the migrant population, test-taking engagement in large-scale assessments, and measurement invariance in the motivation-to-learn scale for group comparisons across countries. The issue also features a paper on PIAAC-Longitudinal (PIAAC-L), a follow-up study of German PIAAC respondents in three further waves (2014, 2015, 2016). As a group, the articles in this special issue recommend potential policy actions to tackle problems associated with skill inequality and propose useful further research avenues to deepen understanding of these problems. They also offer suggestions for methodological advancements in PIAAC.

Beginning this special issue are Kirsch and Lennon (2017), who provide an overview of the historic developments that have led to PIAAC being conducted in 38 countries so far, and review the innovations that have been implemented as part of PIAAC. A particularly important feature of PIAAC is the use of computer-based assessment to evaluate respondents’ technological literacy in an environment that reflects the demands of such technologies in the workplace and in everyday life. Introducing this technology has also widened the scope of what large-scale assessments can efficiently measure, which means that PIAAC marks a new era in large-scale assessment. Kirsch and Lennon also present analytical strategies that extend the use of PIAAC, for example offering new opportunities for researching subpopulations beyond the selected samples by applying small-area estimation.

Education was, and is often used as a proxy for competencies and human capital (see for example Mincer 1974). This is because data on competencies were not available before international large-scale assessments were developed, and have only recently become available for a large number of countries and population groups of interest. Assessing competencies can also be costly, and, for specific purposes, surveying educational attainment may be viewed as a much easier and less expensive proxy. However, formal qualifications and competencies are different constructs, and both can differ widely in their ability to predict, for example, a nation’s growth rate (Hanushek and Woessmann 2015, 2016). Massing and Schneider (2017) address this topic by exploring the net relationship between educational attainment and literacy skills. Despite finding a high degree of heterogeneity of skills across countries for equivalent education categories, they also found that (broad) International Standard Classification of Education (ISCED) categories might not be appropriate for cross-national comparison of human capital, as they do not always follow the assumed hierarchy of “higher education entails higher competencies” (Massing and Schneider 2017, p. 1).

Heisig (2018) focuses on the signaling value of educational degrees (Spence 1973; Weiss 1995) and emphasizes the distribution of skills by analyzing the internal homogeneity of educational groups in terms of formal education. By using PIAAC data from 21 countries, Heisig suggests a direct country-level measure of internal homogeneity (for example, operationalized by literacy). Country-level analyses show that educational groups tend to be more homogeneous in countries with stronger (ability-related) tracking in secondary education. However, internal homogeneity can be also negatively associated with standardization of input (such as curricula, and textbooks). The size of the skills gap and the extent of internal homogeneity can indeed account for cross-national variation in labor market inequalities between adults with low and medium levels of education. Heisig also presents findings on whether the skills gap and index of internal homogeneity can actually account for labor market inequalities by educational attainment. The aim is to provide some evidence of the credential effect of the educational degree on the probability that an applicant is chosen in the labor market.

In this vein, one aspect that is still difficult to capture with a cross-sectional study such as PIAAC is the causal link between competencies and labor market outcomes. By reviewing recent studies and presenting new evidence, Hampf et al. (2017) shed light on this causal link and provide evidence that the ordinary least squares estimates do not overestimate the true returns to competencies in the labor market. To this end, Hampf et al. analyze the compensation of competencies by using instrumental-variable estimation. They back up their results by reviewing findings from two natural experiments, one of which exploited changes in compulsory-schooling laws in the USA (Hanushek et al. 2015) and one which investigated differences in ICT skills as a result of broadband Internet availability in German municipalities (Falck et al. 2016).

Despite the importance of cognitive skills in predicting key indicators of life success, a growing body of evidence has suggested that non-cognitive skills, such as personality, can also be a significant predictor for life outcomes (Roberts et al. 2007; Heckman and Kautz 2012). Using PIAAC-L data, Rammstedt et al. (2017a) thus investigate the relationship of cognitive (e.g., numeracy) and non-cognitive competencies (e.g., the Big Five; see John et al. 2008). They further examine the extent to which personality is related to important life outcomes above and beyond competencies and sociodemographic characteristics, and are able to show that personality was incrementally predictive of life satisfaction and health in particular and, to a lesser extent, of educational attainment and income. These results thus stress the inclusion of measures such as personality traits in upcoming cycles of PIAAC.

Another important aim of large-scale assessments is to derive policy advice to better meet social and economic challenges. One such challenge is the heavy influx of refugees that the European Union faced in 2015 (OECD 2016). A solid database on immigrants, their social-demographic background, and the competencies that they bring with them is crucial for deriving strategies for their successful social and labor market integration. However, compiling such a database requires representatively surveying this population group in a large-scale assessment. Maehler et al. (2017) give an overview of the representation of the migrant population in the German PIAAC sample as a case study of a country that currently has a high migration rate. They evaluate reasons for the under-coverage of this population in terms of sampling frames, patterns in response behavior, and contact times.

Today’s knowledge society and the changing requirements of the labor market make it necessary to continuously sustain and improve competencies. An important way to do so is through further education (Desjardins 2003), which bears a strong relationship to motivation to learn (see e.g., Gorges et al. 2016). Gorges et al. (2016) used four PIAAC background questionnaire items to compile a motivation-to-learn scale that showed measurement invariance across 21 countries. In this PIAAC special issue, Gorges et al. (2017) focus on the validity of the motivation-to-learn scale for group comparisons within each country. In most countries under investigation, they found at least partial strong measurement invariance across gender, age groups, level of education, and migration background.

Goldhammer et al. (2017) focus on test-taking (dis)engagement in large-scale assessments. They find that a lack of motivation among study participants to engage with assessment items will likely influence the results of such assessments, and this could threaten the validity of any interpretations of their ability based on their test-score results (Asseburg and Frey 2013). Using Canadian PIAAC data to test measurement models for test-taking engagement and their relationship to person and item characteristics, Goldhammer, Martens, and Lüdtke calculate a binary disengagement index for the domains of literacy, numeracy, and problem-solving using a response-time threshold. The results show that disengagement is influenced by individual differences in test-taker characteristics, item difficulty, and position in the assessment. They conclude that high cognitive ability correlates with low disengagement and high item difficulty correlates with high disengagement. Test-takers also strategically skipped items that they considered too difficult.

PIAAC data permits researchers to investigate key predictors and outcomes of skills. As a consequence, the data now make it possible to explain processes of skill maintenance and development versus skill loss over time, or to disentangle whether self-selection into certain professions might be a crucial determining factor in the correlation between earnings and competences. To investigate questions related to causality, a longitudinal design is needed. In Germany, the Programme for the International Assessment of Adult Competencies was therefore converted into a panel study, the PIAAC-Longitudinal (PIAAC-L; Zabal et al. 2016). Rammstedt et al. (2017b) present the conceptual and methodological framework of PIAAC-L, describe the design of the longitudinal study, and address its analytic potential. PIAAC-L aims to investigate the effects of skill outcomes over an individual’s lifespan, and the development of the competencies assessed in PIAAC (for example, in terms of the effects of competencies on labor market participation over an individual’s lifetime). PIAAC-L consists of three follow-up waves to the initial PIAAC 2012 assessment, conducted in 2014, 2015, and 2016. In particular, the additional and alternative background information that was collected (Rammstedt and Maehler 2017) and extension of the design to survey members of the entire household both offer vast analytic potential. These changes provide avenues for overcoming methodological challenges and promise advancements to improve data quality in future PIAAC cycles.

The scientific community has already demonstrated the possibilities offered by comprehensive interdisciplinary analysis of the PIAAC data (Rammstedt and Maehler 2017). To summarize:

• Currently, there are 31 public use files (PUF) available.^{Footnote 1}

• There have been at least 150^{Footnote 2} publications based on PIAAC data since the first data release in 2013.

• Nine conferences with international scope and a focus on PIAAC have already taken place.^{Footnote 3}

• An international research data center (RDC PIAAC) has been established (https://www.gesis.org/en/piaac/rdc/).

• An international PIAAC information platform (piaacgateway.com) has been created.

• Countless data analysis workshops have taken place (such as those organized by the International Association for the Evaluation of Educational Achievement (IEA), Educational Testing Service (ETS), or the RDC PIAAC).

• Importantly, the number of follow-up studies and projects based on PIAAC has grown. For instance, in addition to the German PIAAC-L study, Canada (Heisz and Oikawa 2017), Poland (Palczynska 2017), and Italy (Roma et al. 2017) have also implemented a longitudinal design, and, in the USA, a complementary PIAAC study of incarcerated adults was undertaken (Rampey et al. 2016).

• Finally, the PIAAC log files are available for research purposes (OECD 2017a, b) and should improve understanding of participants’ test-taking behaviors.

The interdisciplinary contributions in this special issue on Results, Methodological Aspects and Advancements of the Programme for the International Assessment of Adult Competencies (PIAAC) in Large-Scale Assessments in Education present innovative findings and contribute to the theoretical development of competence assessment in adulthood, support the better dissemination of the PIAAC data, and aim to further methodological developments in large-scale assessment in education.

1. 1.

   Please see the OECD webpage: http://www.oecd.org/skills/piaac/publicdataandanalysis/#d.en.408927.

2. 2.

   Please consult the publication list on the Research Data Center PIAAC webpage: https://www.gesis.org/en/piaac/rdc/.

3. 3.

   For an overview, please see the Research Data Center PIAAC webpage.

1. Asseburg, R., & Frey, A. (2013). Too hard, too easy, or just right? The relationship between effort or boredom and ability-difficulty fit. Psychological Test and Assessment Modeling, 55(1), 92–104.

   Google Scholar

2. Desjardins, R. (2003). Determinants of literacy proficiency: a lifelong-lifewide learning perspective. International Journal of Educational Research, 39(3), 205–245.

   Article Google Scholar

3. Falck, O., Heimisch, A., & Wiederhold, S. (2016). Returns to ICT skills. CESifo Working Paper No. 5720.

4. Goldhammer, F., Martens, T., & Lüdtke, O. (2017). Conditioning factors of test-taking engagement in PIAAC: an exploratory IRT modelling approach considering person and item characteristics. Large-scale Assessments in Education, 5, 18.

   Article Google Scholar

5. Gorges, J., Koch, T., Maehler, D. B., & Offerhaus, J. (2017). Same but different? Measurement invariance of the PIAAC motivation-to-learn scale across key socio-demographic groups. Large-scale Assessments in Education, 5, 13.

   Article Google Scholar

6. Gorges, J., Maehler, D. B., Koch, T., & Offerhaus, J. (2016). Who likes to learn new things: Measuring adult motivation to learn with PIAAC data from 21 countries. Large-scale Assessments in Education, 4, 1.

   Article Google Scholar

7. Hampf, F., Wiederhold, S., & Woessmann, L. (2017). Skills, earnings, and employment: exploring causality in the estimation of returns to skills. Large-scale Assessments in Education, 5, 12.

   Article Google Scholar

8. Hanushek, E. A., Schwerdt, G., Wiederhold, S., & Woessmann, L. (2015). Returns to skills around the world: Evidence from PIAAC. European Economic Review, 73(C), 103–130.

   Article Google Scholar

9. Hanushek, E. A., & Woessmann, L. (2015). The knowledge capital of nations. Cambridge, MA: MIT Press.

   Google Scholar

10. Hanushek, E. A., & Woessmann, L. (2016). Knowledge capital, growth, and the East Asian miracle. Science, 351(6271), 344–445.

    Article Google Scholar

11. Heckman, J. J., & Kautz, T. (2012). Hard evidence on soft skills. Labour Economics, 19(4), 451–464.

    Article Google Scholar

12. Heisig, J.-P. (2018). Measuring the signaling value of educational degrees: secondary education systems and the internal homogeneity of educational groups. Large-scale Assessments in Education, 6, 9.

    Article Google Scholar

13. Heisz, A. & Oikawa, C. (2017). The longitudinal and international study of Adults: A Canadian PIAAC-Longitudinal. In Presentation at the International Conference on PIAAC and PIAAC-Longitudinal. Mannheim, Germany. Retrieved from https://www.gesis.org/en/piaac/conference/.

14. John, O. P., Naumann, L. P., & Soto, C. J. (2008). Paradigm shift to the integrative Big five trait taxonomy. In O. P. John, R. W. Robins, & L. A. Pervin (Eds.), Handbook of Personality Theory and Research (3rd ed., pp. 114–158). New York, NY: Guilford Publications.

    Google Scholar

15. Kirsch, I., & Lennon, M. L. (2017). PIAAC: a new design for a new era. Large-scale Assessments in Education, 5, 11.

    Article Google Scholar

16. Maehler, D. B., Martin, S., & Rammstedt, B. (2017). Coverage of the migrant population in large-scale assessment surveys. Experiences from PIAAC in Germany. Large-scale Assessments in Education, 5, 9.

    Article Google Scholar

17. Massing, N., & Schneider, S. L. (2017). Degrees of competency: the relationship between educational qualifications and adult skills across countries. Large-scale Assessments in Education, 5, 6.

    Article Google Scholar

18. Mincer, J. (1974). Schooling, experience and earnings. New York, NY: National Bureau of Economic Research.

    Google Scholar

19. OECD. (2012). Better skills, better jobs, better lives: A strategic approach to skills policies. Paris: OECD Publishing. https://doi.org/10.1787/9789264177338-en.

    Google Scholar

20. OECD. (2013). OECD skills outlook: First results from the Survey of Adult Skills. Paris: OECD Publishing.

    Google Scholar

21. OECD. (2016). International migration outlook 2016. Paris: OECD Publishing. https://doi.org/10.1787/migr_outlook-2016-en.

    Google Scholar

22. OECD. (2017a). Programme for the International Assessment of Adult Competencies (PIAAC), log files [version: public use file]. Data available at http://www.oecd.org/skills/piaac/.

23. OECD. (2017b). Programme for the International Assessment of Adult Competencies (PIAAC), log files [version: scientific use file]. Data access via http://www.oecd.org/skills/piaac/.

24. Palczyńska, M. (2017). Getting to know PIAAC respondents better: Polish follow-up study. Presentation at the International Conference on PIAAC and PIAAC-Longitudinal. Mannheim. Retrieved from https://www.gesis.org/en/piaac/conference/.

25. Rammstedt, B., & Maehler, D. B. (2014). PIAAC and its methodological challenges. Methods, Data, Analysis, 8(2), 125–136. https://doi.org/10.12758/mda.2014.005.

    Article Google Scholar

26. Rammstedt, B. & Maehler, D. B. (2017). Welcome to the International Conference on PIAAC and PIAAC-Longitudinal. In Presentation at the International Conference on PIAAC and PIAAC-Longitudinal. Mannheim. Retrieved from https://www.gesis.org/en/piaac/conference/.

27. Rammstedt, B., Danner, D., & Lechner, C. (2017a). Personality, competencies, and life outcomes: results from the German PIAAC longitudinal study. Large-scale Assessments in Education, 5, 2.

    Article Google Scholar

28. Rammstedt, B., Martin, S., Zabal, A., Carstensen, C., & Schupp, J. (2017b). The PIAAC longitudinal study in Germany: rationale and design. Large-scale Assessments in Education, 5, 4.

    Article Google Scholar

29. Rampey, B. D., Keiper, S., Mohadjer, L., Krenzke, T., Li, J., Thornton, N., & Hogan, J. (2016). Highlights from the U.S. PIAAC Survey of Incarcerated Adults: Their skills, work experience, education, and training: Program for the International Assessment of Adult Competencies: 2014 (NCES 2016-040). U.S. Department of Education. Washington, DC: National Center for Education Statistics. Retrieved from http://nces.ed.gov/pubsearch.

30. Roberts, B. W., Kuncel, N. R., Shiner, R., Caspi, A., & Goldberg, L. R. (2007). The power of personality: The comparative validity of personality traits, socioeconomic status, and cognitive ability for predicting important life outcomes. Perspectives on Psychological Science: A Journal of the Association for Psychological Science, 2(4), 313–345. https://doi.org/10.1111/j.1745-6916.2007.00047.x.

    Article Google Scholar

31. Roma, F., Bastianelli, M., Mineo, S., & Giancola, O. (2017). Skills, non-cognitive dimensions and job complexity: A new framework for analysis from the “PIAAC Italy Survey”. In Presentation at the International Conference on PIAAC and PIAAC-Longitudinal. Mannheim. Retrieved from https://www.gesis.org/en/piaac/conference/.

32. Spence, M. (1973). Job market signaling. Quarterly Journal of Economics, 87(3), 355–374.

    Article Google Scholar

33. Weiss, A. (1995). Human capital vs. signalling explanations of wages. The Journal of Economic Perspectives, 9(4), 133–154.

    Article Google Scholar

34. Zabal, A., Martin, S., & Rammstedt, B. (2016). PIAAC-L Data Collection 2014: technical report: Follow-up to PIAAC Germany 2012. GESIS Papers 17. Koln: GESIS.

Download references

AP and DM wrote the manuscript. BR revised the manuscript. All authors read and approved the final manuscript.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Affiliations

1. GESIS-Leibniz-Institute for the Social Sciences, Cologne, Germany

   Anja Perry

2. GESIS-Leibniz-Institute for the Social Sciences, Mannheim, Germany

   Débora B. Maehler & Beatrice Rammstedt

1. Anja PerryView author publications

   You can also search for this author in PubMed Google Scholar

2. Débora B. MaehlerView author publications

   You can also search for this author in PubMed Google Scholar

3. Beatrice RammstedtView author publications

   You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anja Perry.

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and Permissions

Cite this article

Perry, A., Maehler, D.B. & Rammstedt, B. Introduction to the special issue on Results, Methodological Aspects, and Advancements of the Programme for the International Assessment of Adult Competencies (PIAAC). Large-scale Assess Educ 6, 14 (2018). https://doi.org/10.1186/s40536-018-0066-x

Download citation

• Received: 13 October 2018

• Accepted: 21 October 2018

• Published: 03 November 2018

• DOI: https://doi.org/10.1186/s40536-018-0066-x