Hailo export guide (#24596)

Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
Co-authored-by: UltralyticsAssistant <web@ultralytics.com>
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
Co-authored-by: Ultralytics Assistant <135830346+UltralyticsAssistant@users.noreply.github.com>
This commit is contained in:
Francesco Mattioli 2026-05-26 10:12:02 +02:00 committed by GitHub
parent ede916d59c
commit 1c8bca06db
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
5 changed files with 617 additions and 2 deletions

View File

@ -20,7 +20,7 @@ For a deeper look at the motivation behind this architectural shift, see the [Ul
- **Using the Ultralytics API or CLI?** No changes needed — just swap your model name to `yolo26n.pt`.
- **Using custom inference code (ONNX Runtime, TensorRT, etc.)?** Update your post-processing — detection output is now `(N, 300, 6)` in `xyxy` format, no NMS required. Other tasks append extra data (mask coefficients, keypoints, or angle).
- **Exporting?** Most formats support end-to-end output natively. However, a few formats (NCNN, RKNN, PaddlePaddle, ExecuTorch, IMX, and Edge TPU) automatically fall back to traditional output due to unsupported operator constraints (e.g., `torch.topk`).
- **Exporting?** Most formats support end-to-end output natively. However, a few formats (NCNN, RKNN, PaddlePaddle, ExecuTorch, IMX, Edge TPU, and QNN) automatically fall back to traditional output due to unsupported operator constraints (e.g., `torch.topk`). Hailo HEF workflows compile from ONNX with Hailo-specific scripts, so verify the detection head and NMS configuration for your model.
## How End-to-End Detection Works
@ -141,6 +141,8 @@ The following formats **do not** support end-to-end and automatically fall back
When you export to one of these formats, Ultralytics automatically switches to the one-to-many head and logs a warning — no manual intervention needed. This means **you'll need NMS in your inference pipeline** for these formats, just like with [YOLOv8](../models/yolov8.md) or [YOLO11](../models/yolo11.md).
For [Hailo HEF](../integrations/hailo.md), the compile step happens outside `model.export(format=...)` after an ONNX export. Use the Hailo DFC logs, `.alls` model script, and NMS JSON that match your exact detection model; if an end-to-end YOLO26 graph is not supported by your Hailo toolchain, export the ONNX model with `end2end=False` and compile the traditional detection head.
!!! note "TensorRT + INT8"
[TensorRT](../integrations/tensorrt.md) supports end-to-end, but it is **auto-disabled** when exporting with `int8=True` on TensorRT ≤10.3.0.

View File

@ -159,6 +159,20 @@ TF Edge TPU is designed for high-speed, efficient computing on Google's Edge TPU
- **Security Considerations**: Integrates with Google's robust security for IoT and edge devices.
- **Hardware Acceleration**: Custom-designed to take full advantage of Google Coral devices.
### Hailo HEF
Hailo HEF is a compiled executable format for Hailo AI accelerators, including Hailo-8, Hailo-8L, and Hailo-15 devices. Ultralytics YOLO detection models are exported to ONNX first, then compiled to HEF with the external Hailo Dataflow Compiler. HEF is not a direct Ultralytics export target; for supported edge acceleration workflows, compare [Axelera AI](../integrations/axelera.md) and [DeepX](../integrations/deepx.md) first.
- **Performance Benchmarks**: Depends on Hailo hardware, Hailo SDK version, model script, NMS configuration, and calibration data.
- **Compatibility and Integration**: Only for Hailo-powered embedded systems, industrial gateways, and Raspberry Pi AI Kit deployments.
- **Community Support and Ecosystem**: Supported through Hailo Developer Zone, HailoRT, TAPPAS, and the Hailo Model Zoo.
- **Case Studies**: Useful for real-time object detection on cameras, robotics, access control, smart city, and industrial inspection devices.
- **Maintenance and Updates**: Depends on Hailo SDK, firmware, and model-zoo updates for new accelerator targets.
- **Security Considerations**: Supports local, on-device inference where data stays at the edge.
- **Hardware Acceleration**: Uses Hailo NPU execution through compiled HEF artifacts.
For a step-by-step workflow, see the [Hailo integration guide](../integrations/hailo.md).
### TF.js
TensorFlow.js (TF.js) is a library that brings machine learning capabilities directly to the browser, offering a new realm of possibilities for web developers and users alike. It allows for the integration of machine learning models in web applications without the need for back-end infrastructure.
@ -223,6 +237,7 @@ The following table provides a snapshot of the various deployment options availa
| TF GraphDef | Stable for static computation graphs | Integrates well with TensorFlow infrastructure | Resources for optimizing static graphs | Scenarios requiring static graphs | Updates alongside TensorFlow core | Established TensorFlow security practices | TensorFlow acceleration options |
| TF Lite | Speed and efficiency on mobile/embedded | Wide range of device support | Robust community, Google backed | Mobile applications with minimal footprint | Latest features for mobile | Secure environment on end-user devices | GPU and DSP among others |
| TF Edge TPU | Optimized for Google's Edge TPU hardware | Exclusive to Edge TPU devices | Growing with Google and third-party resources | IoT devices requiring real-time processing | Improvements for new Edge TPU hardware | Google's robust IoT security | Custom-designed for Google Coral |
| Hailo HEF | Hardware-specific and externally compiled | Hailo devices and Raspberry Pi AI Kit | Hailo Developer Zone and Model Zoo | Existing Hailo deployments | Hailo SDK and firmware updates | On-device inference keeps data local | Hailo NPU via HEF artifacts |
| TF.js | Reasonable in-browser performance | High with web technologies | Web and Node.js developers support | Interactive web applications | TensorFlow team and community contributions | Web platform security model | Enhanced with WebGL and other APIs |
| PaddlePaddle | Competitive, easy to use and scalable | Baidu ecosystem, wide application support | Rapidly growing, especially in China | Chinese market and language processing | Focus on Chinese AI applications | Emphasizes data privacy and security | Including Baidu's Kunlun chips |
| MNN | High-performance for mobile devices. | Mobile and embedded ARM systems and X86-64 CPU | Mobile/embedded ML community | Mobile systems efficiency | High performance maintenance on Mobile Devices | On-device security advantages | ARM CPUs and GPUs optimizations |

View File

@ -0,0 +1,595 @@
---
comments: true
description: Convert Ultralytics YOLO detection models from ONNX to Hailo HEF for Hailo-8, Hailo-8L, Raspberry Pi AI Kit, AI HAT+, and Hailo-15 devices.
keywords: Hailo export, Hailo HEF, export YOLO to Hailo, YOLO Hailo, Hailo-8, Hailo-8L, Hailo-15, Raspberry Pi AI Kit, Raspberry Pi AI HAT+, Hailo Dataflow Compiler, Hailo DFC, HailoRT, Hailo Model Zoo, Hailo AI accelerator, edge AI deployment, embedded AI, model quantization, INT8 quantization, ONNX to HEF, Ultralytics YOLO, YOLO11, YOLOv8, object detection
---
# Hailo Export for Ultralytics YOLO Models
!!! warning "Not a direct Ultralytics export format"
Hailo HEF is **not officially supported** as a direct Ultralytics `model.export(format="hailo")` target. The workflow below exports to ONNX first, then uses Hailo's external Dataflow Compiler toolchain to produce a `.hef` file. For better performance per watt than older Hailo HEF deployments, use newer direct Ultralytics export formats such as [Axelera AI](axelera.md) or [DeepX](deepx.md) instead.
The [Hailo](https://hailo.ai/) toolchain uses HEF files for embedded platforms including the [Raspberry Pi AI Kit](https://www.raspberrypi.com/products/ai-kit/) and [AI HAT+](https://www.raspberrypi.com/documentation/accessories/ai-hat-plus.html), industrial cameras, edge gateways, and AI PCs.
This guide walks through exporting [Ultralytics YOLO](https://github.com/ultralytics/ultralytics) detection models to Hailo's **HEF (Hailo Executable Format)** using the **Hailo Dataflow Compiler (DFC)** SDK. The workflow starts from a YOLO `.pt` model, exports to [ONNX](onnx.md), compiles with Hailo tools, and produces a `.hef` file ready for Hailo-8, Hailo-8L, and Hailo-15 accelerators.
## When to Use Hailo HEF
HEF is the compiled artifact consumed by HailoRT on Hailo target devices. Use this guide only when your deployment hardware specifically requires Hailo HEF. If you are still choosing edge hardware or export targets, start with newer direct Ultralytics export formats such as [Axelera AI](axelera.md) or [DeepX](deepx.md), which provide a supported `model.export(...)` workflow and better performance-per-watt options than older Hailo deployments.
HEF is similar in deployment role to hardware-specific formats such as [RKNN](rockchip-rknn.md) for Rockchip NPUs, [IMX500](sony-imx500.md) for Raspberry Pi AI Cameras, and [Qualcomm QNN](qnn.md) for Snapdragon NPUs, but it is not currently generated directly by Ultralytics.
This workflow is relevant when you need:
- **Raspberry Pi AI Kit compatibility**: Hailo-8L is used in the official Raspberry Pi AI Kit and AI HAT+.
- **HailoRT post-processing**: HailoRT can include YOLO [non-maximum suppression](https://www.ultralytics.com/glossary/non-maximum-suppression-nms) in the compiled inference pipeline.
- **INT8 compilation**: The Hailo DFC quantizes the model with representative calibration images to produce an INT8 graph for Hailo hardware. Learn more about [model quantization](https://www.ultralytics.com/glossary/model-quantization).
## Hailo HEF Export Format
HEF is a hardware-specific executable generated by the Hailo Dataflow Compiler. It contains the quantized model graph, memory allocation, scheduling, and optional post-processing configured for a target Hailo architecture. Unlike standard YOLO [Export mode](../modes/export.md) formats that are produced directly by `model.export(format=...)`, HEF compilation currently uses a two-stage flow:
1. Export YOLO to [ONNX](onnx.md) with Ultralytics.
2. Use Hailo DFC tools to parse, optimize, quantize, and compile the ONNX model into HEF.
The full workflow expands into the following pipeline:
```text
YOLO (.pt) -> ONNX -> HAR (parse) -> HAR (optimize/quantize) -> HEF (compile)
```
1. **Export to ONNX** using Ultralytics [Export mode](../modes/export.md)
2. **Parse** the ONNX model into Hailo's intermediate HAR format
3. **Load a model script** (`.alls`) with normalization and post-processing directives
4. **Calibrate and quantize** using representative images
5. **Compile** to a deployable HEF file
## Supported Tasks
This guide focuses on Ultralytics YOLO **object detection** models, because the Hailo model script and NMS configuration are detection-head specific.
| Task | Supported |
| :-------------------------------------------- | :-------- |
| [Object Detection](../tasks/detect.md) | ✅ Yes |
| [Instance Segmentation](../tasks/segment.md) | ❌ No |
| [Semantic Segmentation](../tasks/semantic.md) | ❌ No |
| [Pose Estimation](../tasks/pose.md) | ❌ No |
| [OBB Detection](../tasks/obb.md) | ❌ No |
| [Classification](../tasks/classify.md) | ❌ No |
For instance segmentation, semantic segmentation, pose, OBB, and classification deployments, compare other edge formats in the [Export mode](../modes/export.md) table or use a generic ONNX pipeline where your target runtime supports the task.
### Compatibility Notes
Hailo export compatibility depends on the model head, input image size, class count, Hailo architecture, model script (`.alls`), and NMS configuration. Static files from the [Hailo Model Zoo](https://github.com/hailo-ai/hailo_model_zoo) are useful references, but they are not universal templates. For example, an NMS JSON created for a COCO 80-class YOLO11n model is not correct for a custom 3-class model or for a different fixed `imgsz`.
| Scope | Expected Support | Notes |
| :-------------------------------------- | :--------------- | :----------------------------------------------------------------------------------------------------------------------------------------- |
| YOLOv8 / YOLO11 detection, stock models | ✅ Good | Shared decoupled detection head; `.alls`, end nodes, and NMS config still need to match the exported graph and fixed `imgsz`. |
| Custom YOLOv8 / YOLO11 detection | ✅ Possible | Requires per-model NMS configuration generated from class count, strides, and detection-head layout; static Model Zoo JSON will not match. |
| YOLOv9 detection | ⚠️ Validate | Similar detection-head pattern, but compile and output parsing should be tested before treating it as supported. |
| YOLOv10 / YOLO26 end-to-end detection | ❌ Not supported | End-to-end/NMS-free exports do not match the Hailo NMS post-processing path; use a traditional detection head if testing manually. |
| Dynamic or arbitrary image sizes | ❌ Not supported | Hailo compilation uses a fixed input shape; `.alls` and NMS settings must match the exported `imgsz`. |
## Installation
### Step 1: Install Ultralytics
```bash
pip install ultralytics
```
### Step 2: Install Hailo DFC SDK
The Hailo DFC is required for parsing, optimization, and compilation. Download the Python wheel from the [Hailo Developer Zone](https://hailo.ai/developer-zone/) (free registration required) and install it:
```bash
pip install /path/to/hailo_sdk_client-*.whl
```
!!! note
The Hailo DFC SDK requires a **Linux x86_64** machine. Export and compilation cannot be performed on ARM devices such as Raspberry Pi. Copy the resulting `.hef` file to your Hailo-powered device for deployment with HailoRT.
## YOLO11n HEF Export Example
The script below compiles a YOLO11n detection model from `.pt` to `.hef` at a fixed 640-pixel input size. It exports to ONNX using Ultralytics, then compiles with Hailo DFC using COCO128 as a small calibration dataset.
Before running the script, download the matching YOLO11n NMS config file from the [Hailo Model Zoo](https://github.com/hailo-ai/hailo_model_zoo) or create your own Hailo NMS JSON for the model. Reuse this script as a known YOLO11n starting point; custom models need matching end nodes, `.alls` directives, and NMS settings.
!!! example "Full Pipeline"
```python
import random
import numpy as np
from hailo_sdk_client import ClientRunner
from PIL import Image
from ultralytics import YOLO
from ultralytics.data.utils import check_det_dataset
from ultralytics.utils import DATASETS_DIR
# Configuration
MODEL = "yolo11n"
HW_ARCH = "hailo8" # hailo8 | hailo8l | hailo15h
IMGSZ = 640
CALIB_IMAGES = 128
NMS_CONFIG = "yolo11n_nms_config.json" # Download or generate for your exact model.
# YOLO11 detection head end nodes. See "Supported Models and End Nodes" for YOLOv8 and other families.
END_NODES = [
"/model.23/cv2.0/cv2.0.2/Conv",
"/model.23/cv3.0/cv3.0.2/Conv",
"/model.23/cv2.1/cv2.1.2/Conv",
"/model.23/cv3.1/cv3.1.2/Conv",
"/model.23/cv2.2/cv2.2.2/Conv",
"/model.23/cv3.2/cv3.2.2/Conv",
]
# Step 1: Export to ONNX
model = YOLO(f"{MODEL}.pt")
model.export(format="onnx", imgsz=IMGSZ, opset=11) # creates an ONNX file named after MODEL
# Step 2: Parse ONNX with Hailo DFC
# The DFC prints the detected end nodes after parsing; use them if unsure.
runner = ClientRunner(hw_arch=HW_ARCH)
runner.translate_onnx_model(f"{MODEL}.onnx", end_node_names=END_NODES)
# Step 3: Load model script (normalization + HailoRT NMS)
# The conv layer names are generated by DFC and can change for other model sizes/families.
alls = (
"normalization1 = normalization([0.0, 0.0, 0.0], [255.0, 255.0, 255.0])\n"
"change_output_activation(conv54, sigmoid)\n"
"change_output_activation(conv65, sigmoid)\n"
"change_output_activation(conv80, sigmoid)\n"
f'nms_postprocess("{NMS_CONFIG}", meta_arch=yolov8, engine=cpu)\n'
"allocator_param(width_splitter_defuse=disabled)"
)
runner.load_model_script(alls)
# Step 4: Build calibration dataset (auto-downloads COCO128)
check_det_dataset("coco128.yaml")
calib_dir = DATASETS_DIR / "coco128" / "images" / "train2017"
image_files = list(calib_dir.glob("*.jpg")) + list(calib_dir.glob("*.png"))
if not image_files:
raise FileNotFoundError(f"No calibration images found in {calib_dir}")
calibset = np.zeros((CALIB_IMAGES, IMGSZ, IMGSZ, 3), dtype=np.float32)
for i in range(CALIB_IMAGES):
img = Image.open(random.choice(image_files)).convert("RGB").resize((IMGSZ, IMGSZ))
calibset[i] = np.array(img, dtype=np.float32)
# Step 5: Optimize and quantize
runner.optimize(calibset)
runner.save_har(f"{MODEL}.o.har") # optional intermediate HAR
# Step 6: Compile to HEF
hef = runner.compile()
with open(f"{MODEL}.hef", "wb") as f:
f.write(hef)
print(f"Compiled HEF saved to: {MODEL}.hef")
```
The resulting HEF file, such as `yolo11n.hef`, is ready to deploy on a compatible Hailo device. If you are compiling for Raspberry Pi AI Kit, set `HW_ARCH = "hailo8l"` before running the compile step.
## Step-by-Step Breakdown
### Step 1: Export to ONNX
Ultralytics exports your trained model to ONNX format, which the Hailo DFC ingests as input. Set `opset=11` for broad DFC compatibility.
```python
from ultralytics import YOLO
MODEL = "yolo11n"
model = YOLO(f"{MODEL}.pt")
model.export(format="onnx", imgsz=640, opset=11)
```
### Step 2: Parse the ONNX Model
The `translate_onnx_model` call converts the ONNX graph into Hailo's intermediate HAR representation. The `end_node_names` list tells the DFC where to cut the graph before NMS so Hailo can attach its own hardware post-processing.
```python
from hailo_sdk_client import ClientRunner
runner = ClientRunner(hw_arch="hailo8")
runner.translate_onnx_model(f"{MODEL}.onnx", end_node_names=END_NODES)
```
!!! tip "Finding end nodes"
The DFC prints a suggestion after parsing:
```
[info] In order to use HailoRT post-processing capabilities, these end node names should be used: ...
```
Copy those node names if you are unsure which ones to use, or if you are working with a custom or less common architecture.
### Step 3: Load the Model Script
The model script (`.alls`) configures input normalization, output activation, and NMS post-processing. The `meta_arch=yolov8` setting applies to both YOLOv8 and YOLO11 since they share the same detection head layout.
```python
MODEL = "yolo11n"
NMS_CONFIG = "yolo11n_nms_config.json"
alls = (
"normalization1 = normalization([0.0, 0.0, 0.0], [255.0, 255.0, 255.0])\n"
"change_output_activation(conv54, sigmoid)\n"
"change_output_activation(conv65, sigmoid)\n"
"change_output_activation(conv80, sigmoid)\n"
f'nms_postprocess("{NMS_CONFIG}", meta_arch=yolov8, engine=cpu)\n'
"allocator_param(width_splitter_defuse=disabled)"
)
runner.load_model_script(alls)
```
!!! note
The `change_output_activation` layer names (`conv54`, `conv65`, `conv80`) are assigned by the DFC during parsing and are **model-specific**. If you are compiling a different model size or architecture, check the DFC output for the correct names, or use a predefined `.alls` file from the [Hailo Model Zoo](https://github.com/hailo-ai/hailo_model_zoo).
The `NMS_CONFIG` file is also model-specific. Use the config that matches your exported model, or start from the [Hailo Model Zoo](https://github.com/hailo-ai/hailo_model_zoo) configuration for the closest YOLO variant.
`engine=cpu` runs NMS through HailoRT on the host CPU. Use `engine=nn_core` only for model/script combinations that Hailo documents as supported by the target hardware and SDK version.
Remove the `nms_postprocess` line if you prefer to run NMS fully in your application code. If you do this, update the inference parser because the HEF will output raw detection-head tensors instead of grouped NMS detections.
### Step 4: Build the Calibration Dataset
INT8 quantization requires a representative set of images. The script below uses COCO128, which Ultralytics downloads automatically via `check_det_dataset`:
```python
from ultralytics.data.utils import check_det_dataset
from ultralytics.utils import DATASETS_DIR
check_det_dataset("coco128.yaml") # downloads to DATASETS_DIR if not present
calib_dir = DATASETS_DIR / "coco128" / "images" / "train2017"
```
!!! tip
Use at least 64 images for calibration. More images generally improve quantization quality. For best results, use images from your deployment domain rather than COCO128.
### Step 5: Optimize and Quantize
```python
runner.optimize(calibset)
runner.save_har(f"{MODEL}.o.har") # optional intermediate checkpoint
```
This step applies quantization-aware fine-tuning and layer noise analysis. A GPU is strongly recommended; without one, this step can take several hours.
### Step 6: Compile to HEF
```python
hef = runner.compile()
with open(f"{MODEL}.hef", "wb") as f:
f.write(hef)
```
## Supported Models and End Nodes
For detection models, `end_node_names` identifies the ONNX detection-head outputs that Hailo should compile before attaching its NMS post-processing. These names vary by architecture and can change when the exported graph changes.
### YOLO11 and YOLOv8
YOLO11 and YOLOv8 share the same decoupled detection head. The layer index differs by one between the two families:
| Model Family | Detection Head Layer | End Node Pattern |
| ------------ | -------------------- | ---------------------------------------- |
| YOLO11 (all) | `model.23` | `/model.23/cv2.0/cv2.0.2/Conv` (6 nodes) |
| YOLOv8 (all) | `model.22` | `/model.22/cv2.0/cv2.0.2/Conv` (6 nodes) |
**YOLO11 end nodes** (all sizes: n, s, m, l, x):
```python
END_NODES = [
"/model.23/cv2.0/cv2.0.2/Conv",
"/model.23/cv3.0/cv3.0.2/Conv",
"/model.23/cv2.1/cv2.1.2/Conv",
"/model.23/cv3.1/cv3.1.2/Conv",
"/model.23/cv2.2/cv2.2.2/Conv",
"/model.23/cv3.2/cv3.2.2/Conv",
]
```
**YOLOv8 end nodes** (all sizes: n, s, m, l, x):
```python
END_NODES = [
"/model.22/cv2.0/cv2.0.2/Conv",
"/model.22/cv3.0/cv3.0.2/Conv",
"/model.22/cv2.1/cv2.1.2/Conv",
"/model.22/cv3.1/cv3.1.2/Conv",
"/model.22/cv2.2/cv2.2.2/Conv",
"/model.22/cv3.2/cv3.2.2/Conv",
]
```
### Other Architectures
For other detection architectures, run the parse step without `end_node_names` first, read the suggested nodes from the DFC log output, then re-run with those nodes:
```python
# First pass: let the DFC suggest end nodes
runner = ClientRunner(hw_arch=HW_ARCH)
runner.translate_onnx_model(f"{MODEL}.onnx")
# Check the printed log for: "[info] In order to use HailoRT post-processing..."
```
Predefined `.alls` scripts and NMS config files for many YOLO variants are available in the [Hailo Model Zoo](https://github.com/hailo-ai/hailo_model_zoo).
## Supported Hardware Architectures
| Architecture | Device | Peak Compute (Vendor Spec) | Common Use Case |
| ------------ | --------- | -------------------------- | ----------------------- |
| `hailo8` | Hailo-8 | 26 TOPS | Hailo accelerator card |
| `hailo8l` | Hailo-8L | 13 TOPS | Raspberry Pi AI Kit |
| `hailo15h` | Hailo-15H | 20 TOPS | Hailo-15 target devices |
Set `HW_ARCH` in the script to match your target device before compiling.
## Running Inference on Hailo Hardware
Once you have the `.hef` file, copy it to your Hailo-powered device and run inference using the **HailoRT** Python API (`hailo_platform` package). Unlike the DFC export steps, inference runs directly on the edge device.
!!! note
The inference code below runs **on the Hailo-powered device** (e.g. Raspberry Pi + AI Kit), not on the x86 machine used for compilation.
### Step 1: Install HailoRT on the Device
On the target device, install HailoRT and the Python bindings. For Raspberry Pi AI Kit and AI HAT+ users, the [official Raspberry Pi AI software guide](https://www.raspberrypi.com/documentation/computers/ai.html) installs HailoRT, the device driver, and Python bindings with:
```bash
sudo apt install dkms
sudo apt install hailo-all
```
For non-Raspberry Pi Hailo devices, install the HailoRT package that matches your device, driver, and SDK version from the [Hailo Developer Zone](https://hailo.ai/developer-zone/).
AI HAT+ 2 devices use a different Raspberry Pi package (`hailo-h10-all`) and Hailo-10H workflow. Follow the Raspberry Pi AI software guide for that hardware generation.
### Step 2: Quick Sanity Check
Before running Python inference, confirm the Hailo device is recognized:
```bash
hailortcli fw-control identify
```
You should see the device type, firmware version, and serial number printed.
### Step 3: Run Inference
The script below runs object detection on a single image using the compiled HEF file and the `hailo_platform` Python API. It handles preprocessing, inference, and drawing bounding boxes from the HailoRT NMS output.
!!! example "Inference Script"
```python
import numpy as np
from hailo_platform import (
HEF,
ConfigureParams,
FormatType,
HailoStreamInterface,
InferVStreams,
InputVStreamParams,
OutputVStreamParams,
VDevice,
)
from PIL import Image, ImageDraw
# Configuration
MODEL = "yolo11n"
HEF_PATH = f"{MODEL}.hef" # path to your compiled HEF file
SOURCE = "bus.jpg" # image path
IMGSZ = 640
CONF = 0.25
COCO_NAMES = [
"person",
"bicycle",
"car",
"motorcycle",
"airplane",
"bus",
"train",
"truck",
"boat",
"traffic light",
"fire hydrant",
"stop sign",
"parking meter",
"bench",
"bird",
"cat",
"dog",
"horse",
"sheep",
"cow",
"elephant",
"bear",
"zebra",
"giraffe",
"backpack",
"umbrella",
"handbag",
"tie",
"suitcase",
"frisbee",
"skis",
"snowboard",
"sports ball",
"kite",
"baseball bat",
"baseball glove",
"skateboard",
"surfboard",
"tennis racket",
"bottle",
"wine glass",
"cup",
"fork",
"knife",
"spoon",
"bowl",
"banana",
"apple",
"sandwich",
"orange",
"broccoli",
"carrot",
"hot dog",
"pizza",
"donut",
"cake",
"chair",
"couch",
"potted plant",
"bed",
"dining table",
"toilet",
"tv",
"laptop",
"mouse",
"remote",
"keyboard",
"cell phone",
"microwave",
"oven",
"toaster",
"sink",
"refrigerator",
"book",
"clock",
"vase",
"scissors",
"teddy bear",
"hair drier",
"toothbrush",
]
# Load HEF and connect to device
hef = HEF(HEF_PATH)
params = VDevice.create_params()
target = VDevice(params)
configure_params = ConfigureParams.create_from_hef(hef, interface=HailoStreamInterface.PCIe)
network_groups = target.configure(hef, configure_params)
network_group = network_groups[0]
network_group_params = network_group.create_params()
# Setup I/O virtual streams
input_vstreams_params = InputVStreamParams.make(network_group, quantized=False, format_type=FormatType.FLOAT32)
output_vstreams_params = OutputVStreamParams.make(network_group, quantized=False, format_type=FormatType.FLOAT32)
# Preprocess
orig = Image.open(SOURCE).convert("RGB")
ow, oh = orig.size
resized = orig.resize((IMGSZ, IMGSZ))
input_data = np.expand_dims(np.array(resized, dtype=np.float32), axis=0) # (1,640,640,3)
input_name = hef.get_input_vstream_infos()[0].name
# Inference
with InferVStreams(network_group, input_vstreams_params, output_vstreams_params) as pipeline:
with network_group.activate(network_group_params):
pipeline.send({input_name: input_data})
raw = pipeline.recv()
# Parse HailoRT NMS output and draw results
# When compiled with nms_postprocess the HEF outputs detections grouped by
# class: shape (batch, num_classes, max_dets, 5) where 5 = [y1,x1,y2,x2,score]
draw = ImageDraw.Draw(orig)
output_key = next(iter(raw.keys()))
batch_dets = raw[output_key][0] # shape: (num_classes, max_dets, 5)
for cls_idx, cls_dets in enumerate(batch_dets):
for det in cls_dets:
score = float(det[4])
if score < CONF:
continue
y1, x1, y2, x2 = det[:4]
# Scale from model coords (0-640) back to original image size
x1 = int(x1 * ow / IMGSZ)
y1 = int(y1 * oh / IMGSZ)
x2 = int(x2 * ow / IMGSZ)
y2 = int(y2 * oh / IMGSZ)
label = f"{COCO_NAMES[cls_idx]} {score:.2f}"
draw.rectangle([x1, y1, x2, y2], outline="red", width=2)
draw.text((x1 + 2, y1 + 2), label, fill="red")
orig.save("output.jpg")
print("Saved output.jpg")
```
!!! tip
The detection output format assumes the HEF was compiled with `nms_postprocess` in the `.alls` script. If you compiled **without** NMS, the raw outputs are the 6 detection head tensors and you must run NMS in your application separately.
### Raspberry Pi AI Kit and AI HAT+
The Raspberry Pi AI Kit and 13 TOPS AI HAT+ use Hailo-8L. To use either device:
1. Set `HW_ARCH = "hailo8l"` before compiling your HEF on the x86 machine.
2. Copy the `.hef` to your Raspberry Pi.
3. Install HailoRT by following the [official Raspberry Pi AI software guide](https://www.raspberrypi.com/documentation/computers/ai.html).
4. Run the inference script above.
For camera-based inference on Raspberry Pi, the [picamera2 Hailo examples](https://github.com/raspberrypi/picamera2/tree/main/examples/hailo) provide ready-to-use scripts for live detection with the Camera Module. You can also compare Raspberry Pi deployment paths in the [Coral Edge TPU on Raspberry Pi guide](../guides/coral-edge-tpu-on-raspberry-pi.md) and [Sony IMX500 integration guide](sony-imx500.md).
### Video Inference with TAPPAS
For high-throughput video pipelines, [TAPPAS](https://github.com/hailo-ai/tappas) provides GStreamer elements that stream video through the Hailo chip in real time:
```bash
MODEL=yolo11n
gst-launch-1.0 filesrc location=video.mp4 ! decodebin ! \
hailonet hef-path=${MODEL}.hef ! \
hailofilter function-name=yolov8 ! \
hailooverlay ! autovideosink
```
See the [TAPPAS documentation](https://github.com/hailo-ai/tappas) for full pipeline configuration options.
## Summary
This guide covered the complete workflow to export Ultralytics YOLO detection models to Hailo HEF format:
1. Export to ONNX with Ultralytics (`model.export(format="onnx")`).
2. Parse the ONNX model with the Hailo DFC and specify detection head end nodes.
3. Configure normalization and NMS via a model script.
4. Quantize with a calibration dataset (COCO128 via Ultralytics).
5. Compile to a `.hef` file ready for Hailo-8, Hailo-8L, or Hailo-15.
For further details, see the [Hailo Developer Zone](https://hailo.ai/developer-zone/), [Hailo documentation](https://hailo.ai/developer-zone/documentation/), and the [Hailo Model Zoo](https://github.com/hailo-ai/hailo_model_zoo). For other Ultralytics export targets, see the related [ONNX](onnx.md), [OpenVINO](openvino.md), [TensorRT](tensorrt.md), [NCNN](ncnn.md), [TFLite Edge TPU](edge-tpu.md), [RKNN](rockchip-rknn.md), [Sony IMX500](sony-imx500.md), and [Qualcomm QNN](qnn.md) guides. To compare exported model speed and accuracy across formats, use [Benchmark mode](../modes/benchmark.md). For the full list of formats and options, visit the [Export mode](../modes/export.md) documentation and the [integrations guide page](index.md).
## FAQ
### What Hailo devices are supported?
The Hailo DFC supports Hailo-8 (`hailo8`), Hailo-8L (`hailo8l`), and Hailo-15H (`hailo15h`). See the [Supported Hardware Architectures](#supported-hardware-architectures) table for the matching `HW_ARCH` value.
### Which Ultralytics models can be exported?
This guide focuses on detection models. See [Supported Tasks](#supported-tasks) for task-level scope, [Compatibility Notes](#compatibility-notes) for model compatibility limits, and [Supported Models and End Nodes](#supported-models-and-end-nodes) for YOLO11 and YOLOv8 end-node examples.
### Why does the model script use `meta_arch=yolov8` for YOLO11?
YOLO11 uses the same decoupled detection head architecture as YOLOv8. The Hailo DFC uses `meta_arch=yolov8` for NMS configuration for both model families.
### Do I need a GPU for the optimization step?
A GPU is strongly recommended for the quantization-aware fine-tuning in `runner.optimize()`. Without one, the process still works but is significantly slower (several hours vs. about 10-20 minutes with a GPU).
### How do I find the correct end nodes for my model?
Run `runner.translate_onnx_model(...)` without specifying `end_node_names`, then use the suggested detection-head nodes printed by the DFC. See [Other Architectures](#other-architectures) for the example command.
### Where can I get the Hailo DFC SDK and NMS config files?
The Hailo DFC SDK Python wheel is available from the [Hailo Developer Zone](https://hailo.ai/developer-zone/), while predefined `.alls` scripts and NMS config files are available from the [Hailo Model Zoo](https://github.com/hailo-ai/hailo_model_zoo).

View File

@ -1,7 +1,7 @@
---
comments: true
description: Discover Ultralytics integrations for streamlined ML workflows, dataset management, optimized model training, and robust deployment solutions.
keywords: Ultralytics, machine learning, ML workflows, dataset management, model training, model deployment, Roboflow, ClearML, Comet ML, DVC, MLFlow, Ultralytics Platform, Neptune, Ray Tune, TensorBoard, Weights & Biases, Amazon SageMaker, Paperspace Gradient, Google Colab, Neural Magic, Gradio, TorchScript, ONNX, OpenVINO, TensorRT, CoreML, TF SavedModel, TF GraphDef, TFLite, TFLite Edge TPU, TF.js, PaddlePaddle, NCNN
keywords: Ultralytics, machine learning, ML workflows, dataset management, model training, model deployment, Roboflow, ClearML, Comet ML, DVC, MLFlow, Ultralytics Platform, Neptune, Ray Tune, TensorBoard, Weights & Biases, Amazon SageMaker, Paperspace Gradient, Google Colab, Neural Magic, Gradio, TorchScript, ONNX, OpenVINO, TensorRT, CoreML, TF SavedModel, TF GraphDef, TFLite, TFLite Edge TPU, TF.js, PaddlePaddle, NCNN, Hailo, Hailo HEF, Qualcomm QNN, RKNN, edge AI
---
# Ultralytics Integrations
@ -71,6 +71,8 @@ Welcome to the Ultralytics Integrations page! This page provides an overview of
- [Gradio](gradio.md): Deploy Ultralytics models with Gradio for real-time, interactive object detection demos.
- [Hailo](hailo.md): Convert Ultralytics YOLO detection models from ONNX to Hailo HEF with Hailo's external Dataflow Compiler for Hailo-8, Hailo-8L, Raspberry Pi AI Kit, and Hailo-15 devices.
- [MNN](mnn.md): Developed by [Alibaba](https://www.alibabagroup.com/), MNN is a highly efficient and lightweight deep learning framework. It supports inference and training of deep learning models and has industry-leading performance for inference and training on-device.
- [NCNN](ncnn.md): Developed by [Tencent](http://www.tencent.com/), NCNN is an efficient [neural network](https://www.ultralytics.com/glossary/neural-network-nn) inference framework tailored for mobile devices. It enables direct deployment of AI models into apps, optimizing performance across various mobile platforms.

View File

@ -602,6 +602,7 @@ nav:
- ExecuTorch: integrations/executorch.md
- Google Colab: integrations/google-colab.md
- Gradio: integrations/gradio.md
- Hailo: integrations/hailo.md
- IBM Watsonx: integrations/ibm-watsonx.md
- JupyterLab: integrations/jupyterlab.md
- Kaggle: integrations/kaggle.md