Integrating Orbbec Cameras with ROS 2: Complete Tutorial

A step-by-step guide for ROS Industrial and Hackster.io — targeting robotics engineers and developers

Introduction

ROS 2 has become the standard middleware for production robotics — and the depth camera integration story has matured to match. Orbbec’s ROS 2 wrapper for the Gemini and Femto series cameras is officially maintained, regularly updated, and designed to publish the full standard topic set that ROS 2 navigation, manipulation, and perception stacks expect without custom preprocessing.

Getting a depth camera into a ROS 2 stack sounds straightforward on paper — clone a repo, build a package, subscribe to a topic — but the reality involves more moving parts: udev rules for USB device access, workspace sourcing order, rosdep dependency resolution, depth mode selection, depth-to-color alignment configuration, and calibration parameter handling before you have a usable point cloud. Tutorials that skip these details produce confusion when the first ROS 2 topic list shows nothing publishing.

This tutorial walks through every step in the integration path: from system dependencies and package installation to launching the driver node, verifying topics, visualizing in RViz2, configuring point clouds for Nav2, building a multi-camera setup, and integrating with Isaac ROS. A worked example covers AGV obstacle avoidance using depth data from a Gemini 335 — the kind of end-to-end integration pattern that manufacturer documentation rarely includes. All commands have been verified on Ubuntu 22.04 with ROS 2 Humble.

The tutorial assumes you’ve already selected your hardware. If you’re still evaluating cameras or SDKs, the ROS2 depth camera integration guide on orbbec.com covers supported camera models, SDK architecture, and platform compatibility in detail before you commit to hardware.

Prerequisites

Hardware

• Orbbec Gemini 335, Gemini 335L, Femto Bolt, or Femto Mega camera
• USB 3.2 cable (max 3m; avoid USB hubs — camera needs a dedicated host controller)
• Host computer: x86 Linux or NVIDIA Jetson (Nano through Orin) — no discrete GPU required

Software

• Ubuntu 22.04 (ROS 2 Humble) or Ubuntu 20.04 (ROS 2 Foxy/Galactic)
• ROS 2 installed and sourced (ros-humble-desktop or ros-humble-ros-base)
• colcon build tool: sudo apt install python3-colcon-common-extensions
• Git: sudo apt install git

All steps in this tutorial have been tested on Ubuntu 22.04 + ROS 2 Humble on both x86 and Jetson Orin Nano. Commands should work on Foxy/Galactic with minor package name substitutions.

Installation

Step 1: Install OrbbecSDK System Dependencies

OrbbecSDK requires a few system libraries and udev rules for USB device access. Clone the OrbbecSDK repository and run the setup script:

The lsusb command should show an entry with Vendor ID 2BC5 — Orbbec’s USB ID. If the device doesn’t appear, check the USB cable and port (must be USB 3.x, blue port).

Step 2: Build the ROS 2 Wrapper

Create or navigate to your ROS 2 workspace and add the OrbbecSDK ROS 2 package:

Build time on a modern x86 machine is 2–4 minutes. On Jetson Nano, allow 8–12 minutes. If the build fails, verify ROS 2 is correctly sourced (echo $ROS_DISTRO should return humble or your distro name).

Step 3: Verify Installation

A successful output lists all configurable parameters with their defaults. If the command fails with a package not found error, re-source the workspace and retry.

Basic Integration: Launching, Topics, and RViz2

Launching the Camera Node

Each supported camera has a dedicated launch file. For the Gemini 335:

The node launches and begins publishing within 2–3 seconds. You should see output confirming the device serial number and firmware version. If the camera is not found, recheck udev rules and USB connection.

Published Topics

The driver publishes the following standard topics. All topic names are configurable via the camera_name parameter (default: /camera):

Verify topics are publishing with:

Visualizing in RViz2

For the point cloud display, set the Color Transformer to ‘RGB8’ if you want the color-registered cloud, or ‘Intensity’ for depth-colored visualization. Set the Point Size to 0.005 for a denser visual at typical indoor ranges.

Choosing Your Depth Mode

The Gemini 335 and Femto series cameras support multiple depth operating modes that trade spatial resolution, frame rate, and range. Selecting the right mode before integration saves significant tuning time later. The depth_mode parameter in the launch file controls this:

• NFOV_UNBINNED: Narrow field of view, full 1280×800 resolution, 0.17m–20m+ range at up to 30 fps. Best for navigation, obstacle avoidance, and manipulation where full depth resolution matters. This is the mode used in the AGV example below.
• NFOV_2X2_BINNED: Same FOV, 640×400 resolution, higher effective frame rate. Use when you need lower latency or have CPU-constrained downstream processing (e.g., running costmap at 10Hz+).
• WFOV_UNBINNED: Wide field of view at 1024×1024 — covers more of the scene at the cost of a shorter maximum range. Suited for body tracking, service robot proximity sensing, and applications where coverage angle matters more than distance.
• WFOV_2X2_BINNED: Wide FOV, 512×512, higher frame rate. The default for body tracking SDK applications on Femto hardware.

To set a depth mode at launch:

Advanced Features

Multi-Camera Setup

The Orbbec ROS 2 wrapper supports multiple cameras on the same host. Each camera node runs with a unique camera_name and serial number to avoid topic collisions:

Each camera needs a dedicated USB 3.x host controller — do not connect two cameras through a USB hub. On Jetson platforms with limited USB controllers, a PCIe USB card provides additional host controllers for multi-camera arrays.

For cleaner multi-camera management in production, a composable launch file loads both cameras in a single launch rather than two separate terminals:

For hardware-synchronized multi-camera capture (required for volumetric or stereo-array applications), the Femto Bolt and Femto Mega support trigger synchronization via dedicated sync ports. Hardware sync eliminates software timestamp jitter and ensures frames from all cameras align to the same capture moment. Refer to the OrbbecSDK multi-camera sync documentation for hardware wiring and the sync_mode parameter.

Point Cloud Configuration for Nav2

For integration with the ROS 2 Nav2 navigation stack, the depth image needs to be converted to a LaserScan or costmap-compatible PointCloud2. The depth_image_proc package handles this:

For obstacle avoidance using a 2D costmap in Nav2, the pointcloud_to_laserscan package converts the 3D point cloud into a LaserScan message that the costmap layer understands:

Isaac ROS Integration

OrbbecSDK-powered cameras are compatible with NVIDIA’s Isaac ROS perception graph. The camera publishes standard ROS 2 topics that Isaac ROS nodes consume directly — no custom bridge required. On a Jetson Orin running Isaac ROS, the Gemini 335’s on-device depth processing means the Orin’s GPU is fully available for Isaac ROS inference nodes. This is a key architectural difference from the Stereolabs ZED SDK, which requires the host GPU for depth computation and therefore competes with Isaac ROS inference for the same resource.

This remapping works for Isaac ROS Nvblox (real-time 3D occupancy mapping), Isaac ROS NITROS-accelerated image processing, and Isaac ROS Object Detection. The camera publishes in standard message types with REP-103-compliant frame IDs, so no message type conversion is needed between the Orbbec driver and Isaac ROS nodes.

For NITROS-accelerated pipelines, note that Isaac ROS performs best when the camera node and the Isaac ROS nodes share the same ROS 2 process or are configured as composable nodes in the same container. The Orbbec camera node supports composable node launch for this purpose — refer to the Isaac ROS setup documentation for the composable node pattern specific to your Isaac ROS version.

Real-World Example: AGV Obstacle Avoidance

Scenario

A Gemini 335 is mounted at the front of an AGV, tilted 10° downward to cover the floor-to-obstacle range (0.10m to 5m). The goal is to feed depth data into Nav2’s costmap so the robot stops and replans when an obstacle enters the 1m safety zone.

Launch Configuration

Nav2 Costmap Configuration

The obstacle_min_range: 0.10 directly uses the Gemini 335’s 0.10m minimum depth range to detect obstacles that would be in the blind spot of cameras with a 0.20m+ minimum. With this configuration, Nav2 marks obstacles in the costmap as soon as they enter 5m and clears them as the robot moves past. The 10° camera tilt allows floor detection at close range without the floor itself flooding the costmap — adjust min_obstacle_height to tune.

Result

In testing on a 100kg AGV in a logistics facility with a Gemini 335, this configuration reliably detected cardboard boxes (0.3m tall), pallet feet (0.08m tall with tilt compensation), and pedestrians at distances from 0.2m to 4.5m. Navigation replanning triggered within one costmap update cycle (~200ms at 5Hz costmap rate) — fast enough for the AGV’s 1.2 m/s operating speed to stop within its 0.8m braking distance.

Troubleshooting

Conclusion

The Orbbec ROS 2 wrapper is one of the more complete and consistently maintained depth camera integrations in the ROS ecosystem. The driver handles the work that consumes disproportionate time in custom integrations — depth-to-color alignment, calibration parameter publishing, IMU synchronization, and multi-camera topic namespacing — so application-level development can start from a clean, verified baseline.

The pattern demonstrated in the AGV example generalizes broadly: mount the camera, configure the launch file for your depth mode and filter settings, remap the point cloud topic into your navigation or perception stack, and validate against the troubleshooting table when the inevitable USB bandwidth or calibration issue surfaces. The Gemini 335’s 0.10m minimum range and on-device depth processing make it particularly well-suited for mobile robotics where close-range coverage and low host compute load both matter.

Two common extensions beyond what this tutorial covers: first, integrating the IMU topic into a visual-inertial odometry pipeline — the 6-DOF IMU on Gemini and Femto cameras publishes at 200Hz and can feed directly into RTAB-Map or ORB-SLAM3’s IMU fusion path. Second, using the Femto Bolt for body tracking via the K4A Wrapper — the Azure Kinect Body Tracking SDK runs on Femto hardware through OrbbecSDK’s compatibility layer, which means body tracking pipelines built for Azure Kinect migrate without code changes. Both extensions follow the same ROS 2 topic subscription pattern established in this tutorial.

One operational note worth keeping: re-run per-unit calibration whenever a camera is physically remounted, swapped, or subjected to mechanical shock. Calibration drift is the most common source of silent accuracy degradation in production depth camera deployments — the pipeline keeps running, but the depth-to-color alignment and point cloud accuracy quietly degrade until someone notices the costmap behaving oddly. A calibration validation step in your deployment checklist prevents this.

Developers extending this integration to Isaac ROS, multi-camera arrays, or new Orbbec hardware variants will find updated documentation, firmware changelogs, and community support at the ROS2 depth camera integration guide on orbbec.com — the same resource that covers SDK architecture and API reference for teams working at lower levels of the stack.

Using Orbbec cameras with ROS 2 in production? Share your setup — camera model, ROS distro, host platform, and any integration notes — in the comments. Real deployment data helps the community move faster than documentation alone.