Significant effort is being put into developing industrial applications for artificial intelligence (AI), especially those using machine learning (ML) techniques. Despite the intensive support for building ML applications, there are still challenges when it comes to evaluating, assuring, and improving the quality or dependability. The difficulty stems from the unique nature of ML, namely, system behavior is derived from training data not from logical design by human engineers. This leads to black-box and intrinsically imperfect implementations that invalidate many principles and techniques in traditional software engineering. In light of this situation, the Japanese industry has jointly worked on a set of guidelines for the quality assurance of AI systems (in the Consortium of Quality Assurance for AI-based Products and Services) from the viewpoint of traditional quality-assurance engineers and test engineers. We report on the second version of these guidelines, which cover a list of quality evaluation aspects, catalogue of current state-of-the-art techniques, and domain-specific discussions in five representative domains. The guidelines provide significant insights for engineers in terms of methodologies and designs for tests driven by application-specific requirements.

中文翻译：

---

基于机器学习的人工智能质量保证指南

正在大力开发人工智能 (AI) 的工业应用，尤其是那些使用机器学习 (ML) 技术的工业应用。尽管对构建 ML 应用程序提供了大力支持，但在评估、保证和提高质量或可靠性方面仍然存在挑战。困难源于机器学习的独特性，即系统行为来源于训练数据，而不是人类工程师的逻辑设计。这导致了黑盒和本质上不完美的实现，使传统软件工程中的许多原则和技术无效。鉴于这种情况，日本业界从传统的质量保证工程师和测试工程师的角度，共同制定了一套人工智能系统质量保证指南（在基于人工智能的产品和服务质量保证联盟中）。我们报告了这些指南的第二版，其中涵盖了质量评估方面的列表、当前最先进技术的目录以及五个代表性领域的特定领域讨论。该指南为工程师在应用特定要求驱动的测试方法和设计方面提供了重要的见解。和五个代表性领域的特定领域讨论。该指南为工程师在应用特定要求驱动的测试方法和设计方面提供了重要的见解。和五个代表性领域的特定领域讨论。该指南为工程师在应用特定要求驱动的测试方法和设计方面提供了重要的见解。