How NVIDIA Isaac ROS 5.0 Uses AI Agents to Accelerate ROS 2 Nodes

Technical Blog

sessionStorage.removeItem('facets.search.text'); sessionStorage.removeItem('facets.search.text-copy'); sessionStorage.removeItem('facets.search.filters');

Accelerating a ROS 2 Node with an AI Agent and NVIDIA Isaac ROS

Sep 22, 2026

rosidl::Buffer and the NVIDIA-contributed CUDA buffer backend enable ROS 2 nodes to exchange GPU-resident payloads through zero-copy transport when runtime conditions allow, while preserving standard ROS 2 messages and node boundaries. All nodes in NVIDIA Isaac ROS 5.0 have been updated to use the CUDA buffer backend for more efficient data movement. An AI coding agent uses the migrate-node-to-rosidl-buffer skill to audit a CUDA-accelerated node, trace data movement, plan a minimal interface-preserving refactor, and verify that the CUDA transport path is enabled. The Depth Anything 3 TensorRT ROS 2 node demonstrates the migration: subscription options accept CUDA-backed buffers, TensorRT inference writes results directly into CUDA buffer-backed message storage, and optional CPU work remains separate from the optimized publication path. Verification uses NVIDIA Nsight Systems to confirm the absence of payload-sized host-device transfers at the ROS boundary and checks that msg->data.get_backend_type() reports "cuda" when both endpoints meet requirements. Next Steps Download NVIDIA Isaac ROS 5.0 to access the accelerated robotics software stack. Review the ROS 2 Lyrical rosidl::Buffer and CUDA buffer backend documentation to understand the upstream capability. Install the migrate-node-to-rosidl-buffer agent skill used in this post.

Powered by NVIDIA Nemotron. AI-generated content may summarize information incompletely. Verify important information. Learn more

GPU acceleration can speed up compute-intensive robotics workloads, but a fast CUDA kernel alone does not guarantee a fast ROS 2 graph. As messages move between nodes, they may continue to be serialized or copied through CPU memory, eroding the benefits of keeping perception and AI workloads on the GPU (Figure 1).

With the upstream rosidl::Buffer abstraction and the CUDA buffer backend that NVIDIA recently contributed to ROS Lyrical , ROS 2 nodes can exchange GPU-resident payloads through zero-copy transport when runtime conditions allow, while preserving standard ROS 2 messages and node boundaries. All nodes in NVIDIA Isaac ROS 5.0 have been updated to use the CUDA buffer backend and benefit from the more efficient data movement enabled by rosidl::Buffer .

Existing ROS 2 nodes can adopt rosidl::Buffer with minimal changes. The more challenging task is identifying the correct boundaries to update. This requires a careful audit of allocations, serialization, stream ownership, and fallback behavior.

This tutorial walks you through how to turn that audit into an agent-driven workflow. An AI coding agent uses the purpose-built migrate-node-to-rosidl-buffer skill to inspect an existing CUDA-accelerated node, trace data movement, plan a minimal interface-preserving refactor, and verify that the CUDA transport path is actually enabled. You’ll learn how to use the agent skill to update the node to adopt the CUDA buffer backend. The resulting accelerated workload can then be deployed on NVIDIA Jetson AGX Thor .

In ROS 2 Lyrical, variable-length primitive array fields such as uint8[] are represented in generated C++ code by rosidl::Buffer . The default CPU-backed rosidl::Buffer behaves like the std::vector interface existing ROS 2 code expects, preserving source compatibility. The pluggable abstraction also allows platform vendors to support externally managed storage without defining a separate ROS message type.

Figure 1. A typical path for a message through ROS 2 graphs accelerated by non-CUDA-buffer backends

Figure 2. The new CUDA buffer backend streamlines CUDA acceleration throughout the pipeline for ROS developers

Together, rosidl::Buffer and the CUDA buffer backend move memory sharing and data-lifetime management behind a standard ROS 2 field. This means the upstream capability is easier to adopt in GPU-accelerated robotics applications, so you can focus on node logic while retaining CPU fallback for incompatible peers.

This tutorial uses the Depth Anything 3 (DA3) TensorRT ROS 2 node as the example. The DA3 model predicts spatially consistent geometry from an arbitrary number of visual inputs, with or without known camera poses.

We aim to update this node to adopt the introduced CUDA buffer backend to take advantage of the performance improvement offered by the rosidl::Buffer feature. The node is particularly useful as a migration example because its algorithm is already GPU-accelerated.

This node’s callback converts the incoming ROS image to an OpenCV view, runs monocular metric-depth inference with NVIDIA TensorRT , converts the resulting cv::Mat back to a ROS image, and publishes it as a floating-point depth image.

The code is straightforward, but the CPU-backed ROS boundary surrounds a GPU-native algorithm. That CPU boundary is appropriate for a CPU producer or consumer, but it is unnecessary when the nodes on both sides can already produce and consume CUDA memory. In that case, the two payload-sized host transfers, host allocation, and serialization work become an optimization opportunity at the interface.

The goal is therefore not to redesign the model or replace its standard messages; rather, it is to preserve the existing ROS contract while allowing the output Image.data field to carry storage from an appropriate backend.

An AI coding agent is well suited to investigative work: following payloads through callbacks and helper libraries, finding host-device boundaries, preserving the node contract, and coordinating source, dependency, launch, and test changes.

The migrate-node-to-rosidl-buffer skill turns this analysis into a repeatable workflow. Rather than replacing the node with a template or rewriting code automatically, it directs the agent to:

Using the rosidl::Buffer migration skill, the agent updates the node’s dependencies and interfaces to adopt the CUDA buffer backend. Most changes adapt the TensorRT wrapper to accept CUDA buffer handles for input and output data while preserving its existing API. The ROS transport change remains small: one subscription option, one CUDA allocation, two stream-aware handle extractions, and one publish. No custom message, duplicate CUDA topic, or CPU/CUDA publisher branch is required.

The following sections explain the key changes you can expect from the skill for the node migration.

First, the skill helps add CUDA buffer backend packages ( cuda_buffer and cuda_buffer_backend ) as additional dependencies. The message definition does not change—the node continues using sensor_msgs/msg/Image .

rclcpp::SubscriptionOptions options; options.acceptable_buffer_backends = "cuda";

The existing image_transport and message_filters topology remains in place. The subscription options are simply forwarded through it.

The following excerpt contains the essential changes that leverage CUDA buffer APIs:

auto depth_msg = std::make_unique (); depth_msg->header = bgr_image_msg->header; depth_msg->height = bgr_image_msg->height; depth_msg->width = bgr_image_msg->width; depth_msg->encoding = sensor_msgs::image_encodings::TYPE_32FC1; depth_msg->is_bigendian = false; depth_msg->step = depth_msg->width * sizeof(float);

depth_msg->data = cuda_buffer_backend::allocate_buffer( static_cast (depth_msg->step) * depth_msg->height);

const cudaStream_t stream = tensorrt_depth_anything_->getCudaStream(); { auto input = cuda_buffer_backend::from_input_buffer( bgr_image_msg->data, stream); auto output = cuda_buffer_backend::from_output_buffer( depth_msg->data, stream);

tensorrt_depth_anything_->doInferenceCuda( input.get_ptr(), bgr_image_msg->width, bgr_image_msg->height, bgr_image_msg->step, *camera_info_msg, reinterpret_cast (output.get_ptr()), node_param_.point_cloud_downsample_factor, node_param_.colorize_point_cloud, node_param_.publish_point_cloud, node_param_.enable_debug); } // Release the CUDA event-tracked handles before publishing.

pub_depth_image_->publish(std::move(depth_msg));

Each line has a narrow purpose:

The skill keeps the non-CUDA route intact. Point-cloud construction and debug visualization are local CPU consumers in the original node. When enabled, they may still require a device-to-host copy and synchronization. They do not determine the representation delivered on the depth topic, so the migration leaves them as explicit optional boundaries rather than complicating the optimized publication path.

The rosidl::Buffer feature was introduced in ROS 2 Lyrical, so the migrated node is expected to work with Lyrical and above with supported RMW implementations ( rmw_fastrtps_cpp and rmw_zenoh_cpp ).

During the migration, the core functions and boundary message types are kept the same and add cuda_buffer and cuda_buffer_backend as additional dependencies to the package for enabling CUDA buffer backend. As a result, the overall build process and setup remain similar to the original node.

To enable CUDA buffer backend, build the packages from source. Start by cloning the source from the rosidl_buffer_backends repository where all the currently supported backends and companion packages are hosted:

Note that the core functions of rosidl::Buffer are already built in ROS 2 Lyrical, so there is no need to rebuild the ROS 2 core packages.

The rosidl::Buffer backends are designed to be ROS 2 plugins. Building and sourcing the CUDA buffer backend packages in the same workspace is sufficient to make the backend available to the nodes at runtime.

colcon build --symlink-install --packages-up-to cuda_buffer_backend source install/setup.bash

colcon build --symlink-install --packages-up-to depth_anything_v3 source install/setup.bash export RMW_IMPLEMENTATION=rmw_fastrtps_cpp

You can then follow the same model preparation process and run the same launch file with the updated TensorRT node as instructed in the original repository.

The migration leaves the TensorRT computation unchanged and targets the transport around it. To inspect GPU activity and memory transfers, use NVIDIA Nsight Systems . On an eligible CUDA path, the migrated node should not show payload-sized host-to-device or device-to-host transfers at its ROS boundary. Record comparable latency measurements before and after the change.

Figure 3. The migrated DA3 node preserves the RGB-to-depth result while using CUDA-backed ROS 2 message storage

rclcpp::SubscriptionOptions options; options.acceptable_buffer_backends = "cuda";

subscription_ = create_subscription ( "/depth_anything_v3/output/depth_image", rclcpp::QoS(1), [this](sensor_msgs::msg::Image::ConstSharedPtr msg) { const std::string backend = msg->data.get_backend_type(); RCLCPP_INFO(get_logger(), "received backend=%s", backend.c_str());

if (backend != "cuda") { throw std::runtime_error("CUDA transport was not negotiated"); } auto input = cuda_buffer_backend::from_input_buffer(msg->data, stream_); consume_on_cuda(input.get_ptr(), stream_); }, options);

Note that the production code will often try to accept CPU fallback without throwing the error.

With the provided CUDA buffer APIs, from_input_buffer() automatically handles the CPU fallback internally. Users don’t have to distinguish the CPU path and GPU path in the callback for incoming messages. All the CUDA memory sharing and CPU-to-GPU conversion, if needed, are taken care of by the CUDA buffer backend.

The skill also contains a verification step that helps produce custom source and sink nodes for testing and validation. This is done by creating two pipelines based on the generated source and sink nodes to test the same migrated node working under both CPU and GPU setup without code changes.

The same workflow can be applied to other CUDA-accelerated ROS 2 nodes with variable-length primitive message fields. The key is to treat optimization as an end-to-end systems task. The AI agent traces data movement, identifies which fields benefit from GPU-backed storage, preserves standard ROS 2 interfaces, and verifies both the optimized path and CPU fallback. That makes the migration repeatable instead of a one-off refactor.

NVIDIA Isaac ROS 5.0 brings this workflow into an accelerated robotics software stack, while NVIDIA Jetson AGX Thor provides the edge compute platform for running demanding ROS 2 perception, inference, and autonomy workloads on the robot.

Accelerating a ROS 2 node requires optimizing GPU computation as well as data movement. With rosidl::Buffer , the NVIDIA CUDA buffer backend, and an Isaac ROS 5.0 AI-guided migration skill, existing CUDA-enabled nodes can exchange GPU-resident data with minimal code changes. This avoids unnecessary serialization and CPU copies while preserving standard ROS 2 message interface.