No CrossRef data available.
Article contents
Beyond integrative experiment design: Systematic experimentation guided by causal discovery AI
Published online by Cambridge University Press: 05 February 2024
Abstract
Integrative experiment design is a needed improvement over ad hoc experiments, but the specific proposed method has limitations. We urge a further break with tradition through the use of an enormous untapped resource: Decades of causal discovery artificial intelligence (AI) literature on optimizing the design of systematic experimentation.
- Type
- Open Peer Commentary
- Information
- Copyright
- Copyright © The Author(s), 2024. Published by Cambridge University Press
References
Bareinboim, E., & Pearl, J. (2016). Causal inference and the data-fusion problem. Proceedings of the National Academy of Sciences of the United States of America, 113(27), 7345–7352.CrossRefGoogle ScholarPubMed
Bronstein, M. V., Everaert, J., Kummerfeld, E., Haynos, A. F., & Vinogradov, S. (2022a). Biased and inflexible interpretations of ambiguous social situations: Associations with eating disorder symptoms and socioemotional functioning. International Journal of Eating Disorders, 55(4), 518–529. https://doi.org/10.1002/eat.23688CrossRefGoogle ScholarPubMed
Bronstein, M. V., Kummerfeld, E., MacDonald, A III, & Vinogradov, S. (2022b). Willingness to vaccinate against SARS-CoV-2: The role of reasoning biases and conspiracist ideation. Vaccine, 40(2), 213–222.CrossRefGoogle ScholarPubMed
Chen, W., Zhang, K., Cai, R., Huang, B., Ramsey, J., Hao, Z., & Glymour, C. (2021). FRITL: A hybrid method for causal discovery in the presence of latent confounders. arXiv [cs.LG]. arXiv. http://arxiv.org/abs/2103.14238Google Scholar
Ghassami, A., Salehkaleybar, S., Kiyavash, N., & Bareinboim, E. (2018). Budgeted experiment design for causal structure learning. In Dy, J. & Krause, A. (Eds.), Proceedings of the 35th international conference on machine learning (Vol. 80, pp. 1724–1733). PMLR.Google Scholar
Hoyer, P., Janzing, D., Mooij, J. M., Peters, J., & Schölkopf, B. (2008). Nonlinear causal discovery with additive noise models. Advances in Neural Information Processing Systems, 21, 689–696. https://proceedings.neurips.cc/paper/2008/hash/f7664060cc52bc6f3d620bcedc94a4b6-Abstract.htmlGoogle Scholar
Huang, B., Low, C. J. H., Xie, F., Glymour, C., & Zhang, K. (2022). Latent hierarchical causal structure discovery with rank constraints. Advances in Neural Information Processing Systems, 35, 5549–5561.Google Scholar
Huang, B., Zhang, K., Zhang, J., Ramsey, J., Sanchez-Romero, R., Glymour, C., & Schölkopf, B. (2020). Causal discovery from heterogeneous/nonstationary data. Journal of Machine Learning Research: JMLR, 21(1), 3482–3534.Google Scholar
Hyttinen, A., Eberhardt, F., & Hoyer, P. O. (2013a). Experiment selection for causal discovery. Journal of Machine Learning Research: JMLR, 14, 3041–3071.Google Scholar
Hyttinen, A., Hoyer, P. O., Eberhardt, F., & Jarvisalo, M. (2013b). Discovering cyclic causal models with latent variables: A general SAT-based procedure. In Proceedings of the Twenty-Ninth Conference on Uncertainty in Artificial Intelligence (pp. 301–310).Google Scholar
Kummerfeld, E., & Ramsey, J. (2016). Causal Clustering for 1-Factor Measurement Models. In KDD '16: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 1655–1664). https://doi.org/10.1145/2939672.2939838CrossRefGoogle ScholarPubMed
Lindgren, E., Kocaoglu, M., Dimakis, A. G., & Vishwanath, S. (2018). Experimental design for cost-aware learning of causal graphs. Advances in Neural Information Processing Systems, 31, 5284–5294. https://proceedings.neurips.cc/paper/2018/hash/ba3e9b6a519cfddc560b5d53210df1bd-Abstract.htmlGoogle Scholar
Mayo-Wilson, C. (2011). The problem of piecemeal induction. Philosophy of Science, 78(5), 864–874.CrossRefGoogle Scholar
Mayo-Wilson, C. (2014). The limits of piecemeal causal inference. The British Journal for the Philosophy of Science, 65(2), 213–249.CrossRefGoogle Scholar
Mooij, J. M., Magliacane, S., & Claassen, T. (2020). Joint causal inference from multiple contexts. Journal of Machine Learning Research: JMLR, 21(1), 3919–4026.Google Scholar
Ogarrio, J. M., Spirtes, P., & Ramsey, J. (2016). A hybrid causal search algorithm for latent variable models. JMLR Workshop and Conference Proceedings, 52, 368–379.Google ScholarPubMed
Peters, J., Bühlmann, P., & Meinshausen, N. (2016). Causal inference by using invariant prediction: Identification and confidence intervals. Journal of the Royal Statistical Society. Series B, Statistical Methodology, 78(5), 947–1012.CrossRefGoogle Scholar
Peters, J., Janzing, D., & Schölkopf, B. (2011). Causal inference on discrete data using additive noise models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(12), 2436–2450.CrossRefGoogle ScholarPubMed
Peters, J., Mooij, J., Janzing, D., & Schölkopf, B. (2014). Causal discovery with continuous additive noise models. Journal of Machine Learning Research, 15, 2009–2053. https://www.jmlr.org/papers/volume15/peters14a/peters14a.pdfGoogle Scholar
Ramsey, J., Glymour, M., Sanchez-Romero, R., & Glymour, C. (2017). A million variables and more: The fast greedy equivalence search algorithm for learning high-dimensional graphical causal models, with an application to functional magnetic resonance images. International Journal of Data Science and Analytics, 3(2), 121–129.CrossRefGoogle ScholarPubMed
Shen, X., Ma, S., Vemuri, P., & Simon, G., & Alzheimer's Disease Neuroimaging Initiative. (2020). Challenges and opportunities with causal discovery algorithms: Application to Alzheimer's pathophysiology. Scientific Reports, 10(1), 2975.CrossRefGoogle ScholarPubMed
Shimizu, S., Hoyer, P. O., Hyvärinen, A., & Kerminen, A. (2006). A linear non-Gaussian acyclic model for causal discovery. Journal of Machine Learning Research: JMLR, 7(Oct), 2003–2030.Google Scholar
Shimizu, S., Inazumi, T., Sogawa, Y., Hyvarinen, A., Kawahara, Y., Washio, T., … Bollen, K. (2011). DirectLiNGAM: A direct method for learning a linear non-Gaussian structural equation model. The Journal of Machine Learning Research, 12, 1225–1248. https://www.jmlr.org/papers/volume12/shimizu11a/shimizu11a.pdfGoogle Scholar
Spirtes, P., Glymour, C. N., Scheines, R., Heckerman, D., Meek, C., Cooper, G., & Richardson, T. (2000). Causation, prediction, and search. MIT Press.Google Scholar
Stevenson, B. L., Kummerfeld, E., Merrill, J. E., Blevins, C., Abrantes, A. M., Kushner, M. G., & Lim, K. O. (2022). Quantifying heterogeneity in mood–alcohol relationships with idiographic causal models. Alcoholism, Clinical and Experimental Research, 46(10), 1913–1924.CrossRefGoogle ScholarPubMed
Xie, F., Huang, B., Chen, Z., He, Y., Geng, Z., & Zhang, K. (2022). Identification of linear non-Gaussian latent hierarchical structure. In Chaudhuri, K., Jegelka, S., Song, L., Szepesvari, C., Niu, G., & Sabato, S. (Eds.), Proceedings of the 39th international conference on machine learning (Vol. 162, pp. 24370–24387). PMLR.Google Scholar
Zhang, J. (2008). On the completeness of orientation rules for causal discovery in the presence of latent confounders and selection bias. Artificial Intelligence, 172(16), 1873–1896.CrossRefGoogle Scholar
Target article
Beyond playing 20 questions with nature: Integrative experiment design in the social and behavioral sciences
Related commentaries (31)
Against naïve induction from experimental data
Are language–cognition interactions bigger than a breadbox? Integrative modeling and design space thinking temper simplistic questions about causally dense phenomena
Assume a can opener
Beyond integrative experiment design: Systematic experimentation guided by causal discovery AI
Commensurability engineering is first and foremost a theoretical exercise
Confidence in research findings depends on theory
Consensus meetings will outperform integrative experiments
Dimensional versus conceptual incommensurability in the social and behavioral sciences
Discovering the unknown unknowns of research cartography with high-throughput natural description
Diversity of contributions is not efficient but is essential for science
Don't let perfect be the enemy of better: In defense of unparameterized megastudies
Eliminativist induction cannot be a solution to psychology's crisis
Experiment commensurability does not necessitate research consolidation
Explore your experimental designs and theories before you exploit them!
Getting lost in an infinite design space is no solution
Individual differences do matter
Integrative design for thought-experiments
Integrative experiments require a shared theoretical and methodological basis
Is generalization decay a fundamental law of psychology?
Measurement validity and the integrative approach
Neuroadaptive Bayesian optimisation can allow integrative design spaces at the individual level in the social and behavioural sciences… and beyond
Phenomena complexity, disciplinary consensus, and experimental versus correlational research in psychological science
Representative design: A realistic alternative to (systematic) integrative design
Sampling complex social and behavioral phenomena
Some problems with zooming out as scientific reform
Test many theories in many ways
The elephant's other legs: What some sciences actually do
The future of experimental design: Integrative, but is the sample diverse enough?
The miss of the framework
The social sciences needs more than integrative experimental designs: We need better theories
There are no shortcuts to theory
Author response
Replies to commentaries on beyond playing 20 questions with nature