Building AI Agents to Automate Software Test Case Creation

Table of Contents

1. Introduction
2. Value of Test Automation
3. Context-aware Test Generation
4. Requirements Extraction
5. Test Implementation Generation
6. Future Enhancements

1. Introduction

The DriveOS team at NVIDIA developed HEPH, an internal generative AI framework for automatic test generation to streamline the software testing process. HEPH automates the design and implementation of various tests, including integration and unit tests, saving engineering teams time and effort.

2. Value of Test Automation

HEPH utilizes an LLM agent to automate the entire testing workflow, from document traceability to code generation. This automation significantly reduces the time spent on creating test specifications and implementations, providing value to development, security, and QA teams.

3. Context-aware Test Generation

HEPH uses project documentation and interface specifications to generate test specifications and implementations. It extracts requirement details from the requirement storage system, traces them to relevant documentation fragments, and generates tests in C/C++ based on the aligned requirements and documentation.

4. Requirements Extraction

HEPH extracts requirement information from systems like Jama using requirement identifiers, ensuring that the test generation process is based on accurate and up-to-date requirements.

5. Test Implementation Generation

Based on the generated test specifications and interface control document (ICD) details, HEPH creates tests in C/C++ for execution. It compiles and executes these tests, collects coverage data, and repeats the generation process for any missing test cases.

6. Future Enhancements

Future enhancements of HEPH focus on supporting different test workflows, integrating real-time human feedback, and improving modularity. These enhancements will enable customization of test generation processes and refinement for higher accuracy and relevance.