TECHNICAL ISO/IEC TR REPORT 29119-11 First edition 2020-11 Software and systems engineering : Software testing - Part 11: Guidelines on the testing of AI-based systems Reference number IEC IS0/IEC TR29119-11:2020(E) os1 @ IS0/IEC 2020 IS0/IEC TR 29119-11:2020(E) COPYRIGHT PROTECTED DOCUMENT @IS0/IEC2020 All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting below or Iso's member body in the country of the requester. ISO copyright office CP 40i : Ch. de Blandonnet 8 CH-1214 Vernier, Geneva Phone: +4122 749 01 11 Email: [email protected] Website: www.iso.org Published in Switzerland ii @ IS0/IEC 2020 - All rights reserved IS0/IEC TR 29119-11:2020(E) Contents Page Foreword ..V Introduction.. ..vi 1 Scope. 2 Normative references 3 Terms, definitions and abbreviated terms. 3.1 Terms and definitions 3.2 Abbreviated terms... ..10 4 Introduction to AI and testing ..11 4.1 Overview of AI and testing. ..11 4.2 Artificial intelligence (AI) ..11 4.2.1 Definition of 'artificial intelligence .11 4.2.2 AI use cases. ..12 4.2.3 AI usage and market. ..12 4.2.4 AI technologies. ..13 4.2.5 AI hardware ..15 4.2.6 AI development frameworks. .16 4.2.7 Narrow vs general Al .16 4.3 Testing of Al-based systems ..16 4.3.1 The importance of testing for Al-based systems. ..16 4.3.2 Safety-related Al-based systems ..17 4.3.3 Standardization and Al. 17 5 AI system characteristics ..19 5.1 Al-specific characteristics .19 5.1.1 General. 19 5.1.2 Flexibility and adaptability .20 5.1.3 Autonomy. 20 5.1.4 Evolution. 21 5.1.5 Bias 21 5.1.6 Complexity 21 5.1.7 Transparency, interpretability and explainability 22 5.1.8 Non-determinism. 22 5.2 Aligning Al-based systems with human values 23 5.3 Side-effects 23 5.4 Reward hacking 24 5.5 Specifying ethical requirements for Al-based systems 24 6 Introduction to the testing of Al-based systems .25 6.1 Challenges in testing Al-based systems. 25 6.1.1 Introduction to challenges testing Al-based systems .25 6.1.2 System specifications. 25 6.1.3 Test input data 25 6.1.4 Self-learning systems. 26 6.1.5 Flexibility and adaptability 26 6.1.6 Autonomy 26 6.1.7 Evolution. 26 6.1.8 Bias 26 6.1.9 Transparency, interpretability and explainability 27 6.1.10 Complexity 27 6.1.11 Probabilistic and non-deterministic systems .27 6.1.12 The test oracle problem for Al-based systems. 27 6.2 Testing Al-based systems across the life cycle 27 6.2.1 General. .27 6.2.2 Unit/component testing. .28 @ IS0/IEC 2020 - All rights reserved ii IS0/IEC TR 29119-11:2020(E) 6.2.3 Integration testing 28 6.2.4 System testing 28 6.2.5 System integration testing 29 6.2.6 Acceptance testing. 29 6.2.7 Maintenance testing. 29 7 Testing and QA of ML systems 29 7.1 Introduction to the testing and QA of ML systems.. 29 7.2 ReviewofMLworkflow 29 7.3 Acceptance criteria. .29 7.4 Framework, algorithm/model and hyperparameter selection 30 7.5 Training data quality 30 7.6 Test data quality 30 7.7 Model updates 30 7.8 Adversarial examples and testing 30 7.9 Benchmarks for machine learning .31 8 Black-box testing of Al-based systems 31 8.1 Combinatorial testing. 31 8.2 Back-to-back testing.. 32 8.3 A/Btesting.. .32 8.4 Metamorphic testing. 33 8.5 Exploratory testing. 34 9 White-box testing of neural networks .34 9.1 Structure of a neural network. 34 9.2 Test coverage measures for neural networks 36 9.2.1 Introduction to test coverage levels 36 9.2.2 Neuron coverage 36 9.2.3 Threshold coverage 36 9.2.4 Sign change coverage .36 9.2.5 Value change coverage 36 9.2.6 Sign-sign coverage 36 9.2.7 Layercoverage 37 9.3 Test effectiveness of the white-box measures 37 9.4 White-box testing tools for neural networks. 37 10 Test environments for Al-based systems .38 10.1 Test environments for AI-based systems. 38 10.2 Test scenario derivation. 39 10.3 Regulatory test scenarios and