Add guide for YOLOv7 ONNX inference (#18453)

Signed-off-by: Mohammed Yasin <32206511+Y-T-G@users.noreply.github.com>
Co-authored-by: UltralyticsAssistant <web@ultralytics.com>
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
Co-authored-by: Ultralytics Assistant <135830346+UltralyticsAssistant@users.noreply.github.com>
Co-authored-by: Laughing <61612323+Laughing-q@users.noreply.github.com>
Co-authored-by: Muhammad Rizwan Munawar <muhammadrizwanmunawar123@gmail.com>
This commit is contained in:
Mohammed Yasin 2025-05-28 15:38:22 +06:00 committed by GitHub
parent 48b72f9140
commit 712a7ad27d
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
2 changed files with 283 additions and 11 deletions

View File

@ -18,7 +18,7 @@ Here are some of the key models supported:
2. **[YOLOv4](yolov4.md)**: A darknet-native update to YOLOv3, released by Alexey Bochkovskiy in 2020.
3. **[YOLOv5](yolov5.md)**: An improved version of the YOLO architecture by Ultralytics, offering better performance and speed trade-offs compared to previous versions.
4. **[YOLOv6](yolov6.md)**: Released by [Meituan](https://www.meituan.com/) in 2022, and in use in many of the company's autonomous delivery robots.
5. **[YOLOv7](yolov7.md)**: Updated YOLO models released in 2022 by the authors of YOLOv4.
5. **[YOLOv7](yolov7.md)**: Updated YOLO models released in 2022 by the authors of YOLOv4. Only inference is supported.
6. **[YOLOv8](yolov8.md)**: A versatile model featuring enhanced capabilities such as [instance segmentation](https://www.ultralytics.com/glossary/instance-segmentation), pose/keypoints estimation, and classification.
7. **[YOLOv9](yolov9.md)**: An experimental model trained on the Ultralytics [YOLOv5](yolov5.md) codebase implementing Programmable Gradient Information (PGI).
8. **[YOLOv10](yolov10.md)**: By Tsinghua University, featuring NMS-free training and efficiency-accuracy driven architecture, delivering state-of-the-art performance and latency.

View File

@ -91,19 +91,289 @@ YOLOv7 introduces several key features:
## Usage Examples
As of the time of writing, Ultralytics does not currently support YOLOv7 models. Therefore, any users interested in using YOLOv7 will need to refer directly to the YOLOv7 GitHub repository for installation and usage instructions.
As of the time of writing, Ultralytics only supports ONNX and TensorRT inference for YOLOv7.
Here is a brief overview of the typical steps you might take to use YOLOv7:
### ONNX Export
1. Visit the YOLOv7 GitHub repository: [https://github.com/WongKinYiu/yolov7](https://github.com/WongKinYiu/yolov7).
To use YOLOv7 ONNX model with Ultralytics:
2. Follow the instructions provided in the README file for installation. This typically involves cloning the repository, installing necessary dependencies, and setting up any necessary environment variables.
0. (Optional) Install Ultralytics and export an ONNX model to have the required dependencies automatically installed:
3. Once installation is complete, you can train and use the model as per the usage instructions provided in the repository. This usually involves preparing your dataset, configuring the model parameters, training the model, and then using the trained model to perform object detection.
```bash
pip install ultralytics
yolo export model=yolo11n.pt format=onnx
```
Please note that the specific steps may vary depending on your specific use case and the current state of the YOLOv7 repository. Therefore, it is strongly recommended to refer directly to the instructions provided in the YOLOv7 GitHub repository.
1. Export the desired YOLOv7 model by using the exporter in the [YOLOv7 repo](https://github.com/WongKinYiu/yolov7):
We regret any inconvenience this may cause and will strive to update this document with usage examples for Ultralytics once support for YOLOv7 is implemented.
```bash
git clone https://github.com/WongKinYiu/yolov7
cd yolov7
python export.py --weights yolov7-tiny.pt --grid --end2end --simplify --topk-all 100 --iou-thres 0.65 --conf-thres 0.35 --img-size 640 640 --max-wh 640
```
2. Modify the ONNX model graph to be compatible with Ultralytics using the following script:
```python
import numpy as np
import onnx
from onnx import helper, numpy_helper
# Load the ONNX model
model_path = "yolov7/yolov7-tiny.onnx" # Replace with your model path
model = onnx.load(model_path)
graph = model.graph
# Fix input shape to batch size 1
input_shape = graph.input[0].type.tensor_type.shape
input_shape.dim[0].dim_value = 1
# Define the output of the original model
original_output_name = graph.output[0].name
# Create slicing nodes
sliced_output_name = f"{original_output_name}_sliced"
# Define initializers for slicing (remove the first value)
start = numpy_helper.from_array(np.array([1], dtype=np.int64), name="slice_start")
end = numpy_helper.from_array(np.array([7], dtype=np.int64), name="slice_end")
axes = numpy_helper.from_array(np.array([1], dtype=np.int64), name="slice_axes")
steps = numpy_helper.from_array(np.array([1], dtype=np.int64), name="slice_steps")
graph.initializer.extend([start, end, axes, steps])
slice_node = helper.make_node(
"Slice",
inputs=[original_output_name, "slice_start", "slice_end", "slice_axes", "slice_steps"],
outputs=[sliced_output_name],
name="SliceNode",
)
graph.node.append(slice_node)
# Define segment slicing
seg1_start = numpy_helper.from_array(np.array([0], dtype=np.int64), name="seg1_start")
seg1_end = numpy_helper.from_array(np.array([4], dtype=np.int64), name="seg1_end")
seg2_start = numpy_helper.from_array(np.array([4], dtype=np.int64), name="seg2_start")
seg2_end = numpy_helper.from_array(np.array([5], dtype=np.int64), name="seg2_end")
seg3_start = numpy_helper.from_array(np.array([5], dtype=np.int64), name="seg3_start")
seg3_end = numpy_helper.from_array(np.array([6], dtype=np.int64), name="seg3_end")
graph.initializer.extend([seg1_start, seg1_end, seg2_start, seg2_end, seg3_start, seg3_end])
# Create intermediate tensors for segments
segment_1_name = f"{sliced_output_name}_segment1"
segment_2_name = f"{sliced_output_name}_segment2"
segment_3_name = f"{sliced_output_name}_segment3"
# Add segment slicing nodes
graph.node.extend(
[
helper.make_node(
"Slice",
inputs=[sliced_output_name, "seg1_start", "seg1_end", "slice_axes", "slice_steps"],
outputs=[segment_1_name],
name="SliceSegment1",
),
helper.make_node(
"Slice",
inputs=[sliced_output_name, "seg2_start", "seg2_end", "slice_axes", "slice_steps"],
outputs=[segment_2_name],
name="SliceSegment2",
),
helper.make_node(
"Slice",
inputs=[sliced_output_name, "seg3_start", "seg3_end", "slice_axes", "slice_steps"],
outputs=[segment_3_name],
name="SliceSegment3",
),
]
)
# Concatenate the segments
concat_output_name = f"{sliced_output_name}_concat"
concat_node = helper.make_node(
"Concat",
inputs=[segment_1_name, segment_3_name, segment_2_name],
outputs=[concat_output_name],
axis=1,
name="ConcatSwapped",
)
graph.node.append(concat_node)
# Reshape to [1, -1, 6]
reshape_shape = numpy_helper.from_array(np.array([1, -1, 6], dtype=np.int64), name="reshape_shape")
graph.initializer.append(reshape_shape)
final_output_name = f"{concat_output_name}_batched"
reshape_node = helper.make_node(
"Reshape",
inputs=[concat_output_name, "reshape_shape"],
outputs=[final_output_name],
name="AddBatchDimension",
)
graph.node.append(reshape_node)
# Get the shape of the reshaped tensor
shape_node_name = f"{final_output_name}_shape"
shape_node = helper.make_node(
"Shape",
inputs=[final_output_name],
outputs=[shape_node_name],
name="GetShapeDim",
)
graph.node.append(shape_node)
# Extract the second dimension
dim_1_index = numpy_helper.from_array(np.array([1], dtype=np.int64), name="dim_1_index")
graph.initializer.append(dim_1_index)
second_dim_name = f"{final_output_name}_dim1"
gather_node = helper.make_node(
"Gather",
inputs=[shape_node_name, "dim_1_index"],
outputs=[second_dim_name],
name="GatherSecondDim",
)
graph.node.append(gather_node)
# Subtract from 100 to determine how many values to pad
target_size = numpy_helper.from_array(np.array([100], dtype=np.int64), name="target_size")
graph.initializer.append(target_size)
pad_size_name = f"{second_dim_name}_padsize"
sub_node = helper.make_node(
"Sub",
inputs=["target_size", second_dim_name],
outputs=[pad_size_name],
name="CalculatePadSize",
)
graph.node.append(sub_node)
# Build the [2, 3] pad array:
# 1st row -> [0, 0, 0] (no padding at the start of any dim)
# 2nd row -> [0, pad_size, 0] (pad only at the end of the second dim)
pad_starts = numpy_helper.from_array(np.array([0, 0, 0], dtype=np.int64), name="pad_starts")
graph.initializer.append(pad_starts)
zero_scalar = numpy_helper.from_array(np.array([0], dtype=np.int64), name="zero_scalar")
graph.initializer.append(zero_scalar)
pad_ends_name = "pad_ends"
concat_pad_ends_node = helper.make_node(
"Concat",
inputs=["zero_scalar", pad_size_name, "zero_scalar"],
outputs=[pad_ends_name],
axis=0,
name="ConcatPadEnds",
)
graph.node.append(concat_pad_ends_node)
pad_values_name = "pad_values"
concat_pad_node = helper.make_node(
"Concat",
inputs=["pad_starts", pad_ends_name],
outputs=[pad_values_name],
axis=0,
name="ConcatPadStartsEnds",
)
graph.node.append(concat_pad_node)
# Create Pad operator to pad with zeros
pad_output_name = f"{final_output_name}_padded"
pad_constant_value = numpy_helper.from_array(
np.array([0.0], dtype=np.float32),
name="pad_constant_value",
)
graph.initializer.append(pad_constant_value)
pad_node = helper.make_node(
"Pad",
inputs=[final_output_name, pad_values_name, "pad_constant_value"],
outputs=[pad_output_name],
mode="constant",
name="PadToFixedSize",
)
graph.node.append(pad_node)
# Update the graph's final output to [1, 100, 6]
new_output_type = onnx.helper.make_tensor_type_proto(
elem_type=graph.output[0].type.tensor_type.elem_type, shape=[1, 100, 6]
)
new_output = onnx.helper.make_value_info(name=pad_output_name, type_proto=new_output_type)
# Replace the old output with the new one
graph.output.pop()
graph.output.extend([new_output])
# Save the modified model
onnx.save(model, "yolov7-ultralytics.onnx")
```
3. You can then load the modified ONNX model and run inference with it in Ultralytics normally:
```python
from ultralytics import ASSETS, YOLO
model = YOLO("yolov7-ultralytics.onnx", task="detect")
results = model(ASSETS / "bus.jpg")
```
### TensorRT Export
1. Follow steps 1-2 in the [ONNX Export](#onnx-export) section.
2. Install the `TensorRT` Python package:
```python
pip install tensorrt
```
3. Run the following script to convert the modified ONNX model to TensorRT engine:
```python
# Based off of https://github.com/NVIDIA/TensorRT/blob/release/10.7/samples/python/introductory_parser_samples/onnx_resnet50.py
import tensorrt as trt
TRT_LOGGER = trt.Logger(trt.Logger.WARNING)
def GiB(val):
return val * 1 << 30
def build_engine_onnx(model_file):
builder = trt.Builder(TRT_LOGGER)
network = builder.create_network(0)
config = builder.create_builder_config()
parser = trt.OnnxParser(network, TRT_LOGGER)
config.set_memory_pool_limit(trt.MemoryPoolType.WORKSPACE, GiB(4))
with open(model_file, "rb") as model:
if not parser.parse(model.read()):
print("ERROR: Failed to parse the ONNX file.")
for error in range(parser.num_errors):
print(parser.get_error(error))
return None
config.set_flag(trt.BuilderFlag.FP16)
engine_bytes = builder.build_serialized_network(network, config)
with open(model_file.replace("onnx", "engine"), "wb") as f:
f.write(engine_bytes)
build_engine_onnx("yolov7-ultralytics.onnx") # path to the modified ONNX file
```
4. Load and run the model in Ultralytics:
```python
from ultralytics import ASSETS, YOLO
model = YOLO("yolov7-ultralytics.engine", task="detect")
results = model(ASSETS / "bus.jpg")
```
## Citations and Acknowledgements
@ -136,11 +406,11 @@ YOLOv7 introduces several innovations, including model re-parameterization and d
### Can I use YOLOv7 with Ultralytics tools and platforms?
As of now, Ultralytics does not directly support YOLOv7 in its tools and platforms. Users interested in using YOLOv7 need to follow the installation and usage instructions provided in the [YOLOv7 GitHub repository](https://github.com/WongKinYiu/yolov7). For other state-of-the-art models, you can explore and train using Ultralytics tools like [Ultralytics HUB](../hub/quickstart.md).
As of now, Ultralytics only supports YOLOv7 ONNX and TensorRT inference. To run the ONNX and TensorRT exported version of YOLOv7 with Ultralytics, check the [Usage Examples](#usage-examples) section.
### How do I install and run YOLOv7 for a custom object detection project?
### How do I train a custom YOLOv7 model using my dataset?
To install and run YOLOv7, follow these steps:
To install and train a custom YOLOv7 model, follow these steps:
1. Clone the YOLOv7 repository:
```bash
@ -154,6 +424,8 @@ To install and run YOLOv7, follow these steps:
3. Prepare your dataset and configure the model parameters according to the [usage instructions](https://github.com/WongKinYiu/yolov7) provided in the repository.
For further guidance, visit the YOLOv7 GitHub repository for the latest information and updates.
4. After training, you can export the model to ONNX or TensorRT for use in Ultralytics as shown in [Usage Examples](#usage-examples).
### What are the key features and optimizations introduced in YOLOv7?
YOLOv7 offers several key features that revolutionize real-time object detection: