Dynamic Control Flow in CUDA Graphs with Conditional Nodes

Table of Contents

1. Conditional Nodes
   • IF nodes
   • WHILE nodes

---

Conditional Nodes

Conditional nodes in CUDA Graphs provide the ability to conditionally or repeatedly execute portions of a graph without involving the CPU in decision-making, leading to improved performance. There are two types of conditional nodes: IF nodes and WHILE nodes.

IF nodes

IF nodes execute their body once if the condition value is true. They are container nodes that contain a graph whose execution depends on the value of a condition variable. The condition value is accessed through a handle created before the node, and it can be set in a CUDA kernel using a specific API function. An IF node with a body graph is an example of this type of node.

WHILE nodes

WHILE nodes execute the body graph repeatedly as long as the condition is non-zero. To create a WHILE loop where the body executes conditionally, calculations must be performed and the conditional handle set appropriately in a preceding node. The body graph object of a WHILE node is created along with the node itself.

By utilizing conditional nodes in CUDA Graphs, it is possible to execute specific parts of a graph multiple times without the need to return control to the CPU, reducing CPU overhead and latency in the workflow.