Direct-to-consumer medical artificial intelligence/machine learning applications are increasingly used for a variety of diagnostic assessments, and the emphasis on telemedicine and home healthcare during the COVID-19 pandemic may further stimulate their adoption. In this Perspective, we argue that the artificial intelligence/machine learning regulatory landscape should operate differently when a system is designed for clinicians/doctors as opposed to when it is designed for personal use. Direct-to-consumer applications raise unique concerns due to the nature of consumer users, who tend to be limited in their statistical and medical literacy and risk averse about their health outcomes. This creates an environment where false alarms can proliferate and burden public healthcare systems and medical insurers. While similar situations exist elsewhere in medicine, the ease and frequency with which artificial intelligence/machine learning apps can be used, and their increasing prevalence in the consumer market, calls for careful reflection on how to effectively regulate them. We suggest regulators should strive to better understand how consumers interact with direct-to-consumer medical artificial intelligence/machine learning apps, particularly diagnostic ones, and this requires more than a focus on the system’s technical specifications. We further argue that the best regulatory review would also consider such technologies’ social costs under widespread use.

直接面向消费者的医疗人工智能/机器学习应用程序越来越多地用于各种诊断评估，并且在COVID-19大流行期间对远程医疗和家庭医疗保健的重视可能会进一步刺激其采用。在此观点中，我们认为，针对系统设计用于临床医生/医生的人工智能/机器学习监管环境应与针对个人使用的系统有所不同。由于消费者用户的性质，直接面向消费者的应用程序引起了独特的关注，他们的统计和医学素养往往受到限制，并且对他们的健康结果不愿承担风险。这创造了一种环境，在这种环境中，虚假警报会扩散并加重公共医疗保健系统和医疗保险公司的负担。尽管医学上其他地方也存在类似的情况，但人工智能/机器学习应用程序的易用性和使用频率以及它们在消费市场中的日益普及，都需要仔细思考如何有效地对其进行监管。我们建议监管机构应努力更好地了解消费者如何与直接面向消费者的医疗人工智能/机器学习应用程序（尤其是诊断性应用程序）进行交互，这不仅需要关注系统的技术规范。我们进一步认为，最佳监管审查还将考虑在广泛使用情况下此类技术的社会成本。要求仔细思考如何有效地调节它们。我们建议监管机构应努力更好地了解消费者如何与直接面向消费者的医疗人工智能/机器学习应用程序（尤其是诊断性应用程序）进行交互，这不仅需要关注系统的技术规范。我们进一步认为，最佳监管审查还将考虑在广泛使用情况下此类技术的社会成本。要求仔细思考如何有效地调节它们。我们建议监管机构应努力更好地了解消费者如何与直接面向消费者的医疗人工智能/机器学习应用程序（尤其是诊断性应用程序）进行交互，这不仅需要关注系统的技术规范。我们进一步认为，最佳监管审查还将考虑在广泛使用情况下此类技术的社会成本。