From 712a7ad27d6f0b2d047cc577ba43fb09e699c76d Mon Sep 17 00:00:00 2001 From: Mohammed Yasin <32206511+Y-T-G@users.noreply.github.com> Date: Wed, 28 May 2025 15:38:22 +0600 Subject: [PATCH] Add guide for YOLOv7 ONNX inference (#18453) Signed-off-by: Mohammed Yasin <32206511+Y-T-G@users.noreply.github.com> Co-authored-by: UltralyticsAssistant Co-authored-by: Glenn Jocher Co-authored-by: Ultralytics Assistant <135830346+UltralyticsAssistant@users.noreply.github.com> Co-authored-by: Laughing <61612323+Laughing-q@users.noreply.github.com> Co-authored-by: Muhammad Rizwan Munawar --- docs/en/models/index.md | 2 +- docs/en/models/yolov7.md | 292 +++++++++++++++++++++++++++++++++++++-- 2 files changed, 283 insertions(+), 11 deletions(-) diff --git a/docs/en/models/index.md b/docs/en/models/index.md index dc977e9a..34c44453 100644 --- a/docs/en/models/index.md +++ b/docs/en/models/index.md @@ -18,7 +18,7 @@ Here are some of the key models supported: 2. **[YOLOv4](yolov4.md)**: A darknet-native update to YOLOv3, released by Alexey Bochkovskiy in 2020. 3. **[YOLOv5](yolov5.md)**: An improved version of the YOLO architecture by Ultralytics, offering better performance and speed trade-offs compared to previous versions. 4. **[YOLOv6](yolov6.md)**: Released by [Meituan](https://www.meituan.com/) in 2022, and in use in many of the company's autonomous delivery robots. -5. **[YOLOv7](yolov7.md)**: Updated YOLO models released in 2022 by the authors of YOLOv4. +5. **[YOLOv7](yolov7.md)**: Updated YOLO models released in 2022 by the authors of YOLOv4. Only inference is supported. 6. **[YOLOv8](yolov8.md)**: A versatile model featuring enhanced capabilities such as [instance segmentation](https://www.ultralytics.com/glossary/instance-segmentation), pose/keypoints estimation, and classification. 7. **[YOLOv9](yolov9.md)**: An experimental model trained on the Ultralytics [YOLOv5](yolov5.md) codebase implementing Programmable Gradient Information (PGI). 8. **[YOLOv10](yolov10.md)**: By Tsinghua University, featuring NMS-free training and efficiency-accuracy driven architecture, delivering state-of-the-art performance and latency. diff --git a/docs/en/models/yolov7.md b/docs/en/models/yolov7.md index 51eee8a2..ec4392b0 100644 --- a/docs/en/models/yolov7.md +++ b/docs/en/models/yolov7.md @@ -91,19 +91,289 @@ YOLOv7 introduces several key features: ## Usage Examples -As of the time of writing, Ultralytics does not currently support YOLOv7 models. Therefore, any users interested in using YOLOv7 will need to refer directly to the YOLOv7 GitHub repository for installation and usage instructions. +As of the time of writing, Ultralytics only supports ONNX and TensorRT inference for YOLOv7. -Here is a brief overview of the typical steps you might take to use YOLOv7: +### ONNX Export -1. Visit the YOLOv7 GitHub repository: [https://github.com/WongKinYiu/yolov7](https://github.com/WongKinYiu/yolov7). +To use YOLOv7 ONNX model with Ultralytics: -2. Follow the instructions provided in the README file for installation. This typically involves cloning the repository, installing necessary dependencies, and setting up any necessary environment variables. +0. (Optional) Install Ultralytics and export an ONNX model to have the required dependencies automatically installed: -3. Once installation is complete, you can train and use the model as per the usage instructions provided in the repository. This usually involves preparing your dataset, configuring the model parameters, training the model, and then using the trained model to perform object detection. +```bash +pip install ultralytics +yolo export model=yolo11n.pt format=onnx +``` -Please note that the specific steps may vary depending on your specific use case and the current state of the YOLOv7 repository. Therefore, it is strongly recommended to refer directly to the instructions provided in the YOLOv7 GitHub repository. +1. Export the desired YOLOv7 model by using the exporter in the [YOLOv7 repo](https://github.com/WongKinYiu/yolov7): -We regret any inconvenience this may cause and will strive to update this document with usage examples for Ultralytics once support for YOLOv7 is implemented. +```bash +git clone https://github.com/WongKinYiu/yolov7 +cd yolov7 +python export.py --weights yolov7-tiny.pt --grid --end2end --simplify --topk-all 100 --iou-thres 0.65 --conf-thres 0.35 --img-size 640 640 --max-wh 640 +``` + +2. Modify the ONNX model graph to be compatible with Ultralytics using the following script: + +```python +import numpy as np +import onnx +from onnx import helper, numpy_helper + +# Load the ONNX model +model_path = "yolov7/yolov7-tiny.onnx" # Replace with your model path +model = onnx.load(model_path) +graph = model.graph + +# Fix input shape to batch size 1 +input_shape = graph.input[0].type.tensor_type.shape +input_shape.dim[0].dim_value = 1 + +# Define the output of the original model +original_output_name = graph.output[0].name + +# Create slicing nodes +sliced_output_name = f"{original_output_name}_sliced" + +# Define initializers for slicing (remove the first value) +start = numpy_helper.from_array(np.array([1], dtype=np.int64), name="slice_start") +end = numpy_helper.from_array(np.array([7], dtype=np.int64), name="slice_end") +axes = numpy_helper.from_array(np.array([1], dtype=np.int64), name="slice_axes") +steps = numpy_helper.from_array(np.array([1], dtype=np.int64), name="slice_steps") + +graph.initializer.extend([start, end, axes, steps]) + +slice_node = helper.make_node( + "Slice", + inputs=[original_output_name, "slice_start", "slice_end", "slice_axes", "slice_steps"], + outputs=[sliced_output_name], + name="SliceNode", +) +graph.node.append(slice_node) + +# Define segment slicing +seg1_start = numpy_helper.from_array(np.array([0], dtype=np.int64), name="seg1_start") +seg1_end = numpy_helper.from_array(np.array([4], dtype=np.int64), name="seg1_end") +seg2_start = numpy_helper.from_array(np.array([4], dtype=np.int64), name="seg2_start") +seg2_end = numpy_helper.from_array(np.array([5], dtype=np.int64), name="seg2_end") +seg3_start = numpy_helper.from_array(np.array([5], dtype=np.int64), name="seg3_start") +seg3_end = numpy_helper.from_array(np.array([6], dtype=np.int64), name="seg3_end") + +graph.initializer.extend([seg1_start, seg1_end, seg2_start, seg2_end, seg3_start, seg3_end]) + +# Create intermediate tensors for segments +segment_1_name = f"{sliced_output_name}_segment1" +segment_2_name = f"{sliced_output_name}_segment2" +segment_3_name = f"{sliced_output_name}_segment3" + +# Add segment slicing nodes +graph.node.extend( + [ + helper.make_node( + "Slice", + inputs=[sliced_output_name, "seg1_start", "seg1_end", "slice_axes", "slice_steps"], + outputs=[segment_1_name], + name="SliceSegment1", + ), + helper.make_node( + "Slice", + inputs=[sliced_output_name, "seg2_start", "seg2_end", "slice_axes", "slice_steps"], + outputs=[segment_2_name], + name="SliceSegment2", + ), + helper.make_node( + "Slice", + inputs=[sliced_output_name, "seg3_start", "seg3_end", "slice_axes", "slice_steps"], + outputs=[segment_3_name], + name="SliceSegment3", + ), + ] +) + +# Concatenate the segments +concat_output_name = f"{sliced_output_name}_concat" +concat_node = helper.make_node( + "Concat", + inputs=[segment_1_name, segment_3_name, segment_2_name], + outputs=[concat_output_name], + axis=1, + name="ConcatSwapped", +) +graph.node.append(concat_node) + +# Reshape to [1, -1, 6] +reshape_shape = numpy_helper.from_array(np.array([1, -1, 6], dtype=np.int64), name="reshape_shape") +graph.initializer.append(reshape_shape) + +final_output_name = f"{concat_output_name}_batched" +reshape_node = helper.make_node( + "Reshape", + inputs=[concat_output_name, "reshape_shape"], + outputs=[final_output_name], + name="AddBatchDimension", +) +graph.node.append(reshape_node) + +# Get the shape of the reshaped tensor +shape_node_name = f"{final_output_name}_shape" +shape_node = helper.make_node( + "Shape", + inputs=[final_output_name], + outputs=[shape_node_name], + name="GetShapeDim", +) +graph.node.append(shape_node) + +# Extract the second dimension +dim_1_index = numpy_helper.from_array(np.array([1], dtype=np.int64), name="dim_1_index") +graph.initializer.append(dim_1_index) + +second_dim_name = f"{final_output_name}_dim1" +gather_node = helper.make_node( + "Gather", + inputs=[shape_node_name, "dim_1_index"], + outputs=[second_dim_name], + name="GatherSecondDim", +) +graph.node.append(gather_node) + +# Subtract from 100 to determine how many values to pad +target_size = numpy_helper.from_array(np.array([100], dtype=np.int64), name="target_size") +graph.initializer.append(target_size) + +pad_size_name = f"{second_dim_name}_padsize" +sub_node = helper.make_node( + "Sub", + inputs=["target_size", second_dim_name], + outputs=[pad_size_name], + name="CalculatePadSize", +) +graph.node.append(sub_node) + +# Build the [2, 3] pad array: +# 1st row -> [0, 0, 0] (no padding at the start of any dim) +# 2nd row -> [0, pad_size, 0] (pad only at the end of the second dim) +pad_starts = numpy_helper.from_array(np.array([0, 0, 0], dtype=np.int64), name="pad_starts") +graph.initializer.append(pad_starts) + +zero_scalar = numpy_helper.from_array(np.array([0], dtype=np.int64), name="zero_scalar") +graph.initializer.append(zero_scalar) + +pad_ends_name = "pad_ends" +concat_pad_ends_node = helper.make_node( + "Concat", + inputs=["zero_scalar", pad_size_name, "zero_scalar"], + outputs=[pad_ends_name], + axis=0, + name="ConcatPadEnds", +) +graph.node.append(concat_pad_ends_node) + +pad_values_name = "pad_values" +concat_pad_node = helper.make_node( + "Concat", + inputs=["pad_starts", pad_ends_name], + outputs=[pad_values_name], + axis=0, + name="ConcatPadStartsEnds", +) +graph.node.append(concat_pad_node) + +# Create Pad operator to pad with zeros +pad_output_name = f"{final_output_name}_padded" +pad_constant_value = numpy_helper.from_array( + np.array([0.0], dtype=np.float32), + name="pad_constant_value", +) +graph.initializer.append(pad_constant_value) + +pad_node = helper.make_node( + "Pad", + inputs=[final_output_name, pad_values_name, "pad_constant_value"], + outputs=[pad_output_name], + mode="constant", + name="PadToFixedSize", +) +graph.node.append(pad_node) + +# Update the graph's final output to [1, 100, 6] +new_output_type = onnx.helper.make_tensor_type_proto( + elem_type=graph.output[0].type.tensor_type.elem_type, shape=[1, 100, 6] +) +new_output = onnx.helper.make_value_info(name=pad_output_name, type_proto=new_output_type) + +# Replace the old output with the new one +graph.output.pop() +graph.output.extend([new_output]) + +# Save the modified model +onnx.save(model, "yolov7-ultralytics.onnx") +``` + +3. You can then load the modified ONNX model and run inference with it in Ultralytics normally: + +```python +from ultralytics import ASSETS, YOLO + +model = YOLO("yolov7-ultralytics.onnx", task="detect") + +results = model(ASSETS / "bus.jpg") +``` + +### TensorRT Export + +1. Follow steps 1-2 in the [ONNX Export](#onnx-export) section. + +2. Install the `TensorRT` Python package: + +```python +pip install tensorrt +``` + +3. Run the following script to convert the modified ONNX model to TensorRT engine: + +```python +# Based off of https://github.com/NVIDIA/TensorRT/blob/release/10.7/samples/python/introductory_parser_samples/onnx_resnet50.py + +import tensorrt as trt + +TRT_LOGGER = trt.Logger(trt.Logger.WARNING) + + +def GiB(val): + return val * 1 << 30 + + +def build_engine_onnx(model_file): + builder = trt.Builder(TRT_LOGGER) + network = builder.create_network(0) + config = builder.create_builder_config() + parser = trt.OnnxParser(network, TRT_LOGGER) + + config.set_memory_pool_limit(trt.MemoryPoolType.WORKSPACE, GiB(4)) + with open(model_file, "rb") as model: + if not parser.parse(model.read()): + print("ERROR: Failed to parse the ONNX file.") + for error in range(parser.num_errors): + print(parser.get_error(error)) + return None + config.set_flag(trt.BuilderFlag.FP16) + engine_bytes = builder.build_serialized_network(network, config) + with open(model_file.replace("onnx", "engine"), "wb") as f: + f.write(engine_bytes) + + +build_engine_onnx("yolov7-ultralytics.onnx") # path to the modified ONNX file +``` + +4. Load and run the model in Ultralytics: + +```python +from ultralytics import ASSETS, YOLO + +model = YOLO("yolov7-ultralytics.engine", task="detect") + +results = model(ASSETS / "bus.jpg") +``` ## Citations and Acknowledgements @@ -136,11 +406,11 @@ YOLOv7 introduces several innovations, including model re-parameterization and d ### Can I use YOLOv7 with Ultralytics tools and platforms? -As of now, Ultralytics does not directly support YOLOv7 in its tools and platforms. Users interested in using YOLOv7 need to follow the installation and usage instructions provided in the [YOLOv7 GitHub repository](https://github.com/WongKinYiu/yolov7). For other state-of-the-art models, you can explore and train using Ultralytics tools like [Ultralytics HUB](../hub/quickstart.md). +As of now, Ultralytics only supports YOLOv7 ONNX and TensorRT inference. To run the ONNX and TensorRT exported version of YOLOv7 with Ultralytics, check the [Usage Examples](#usage-examples) section. -### How do I install and run YOLOv7 for a custom object detection project? +### How do I train a custom YOLOv7 model using my dataset? -To install and run YOLOv7, follow these steps: +To install and train a custom YOLOv7 model, follow these steps: 1. Clone the YOLOv7 repository: ```bash @@ -154,6 +424,8 @@ To install and run YOLOv7, follow these steps: 3. Prepare your dataset and configure the model parameters according to the [usage instructions](https://github.com/WongKinYiu/yolov7) provided in the repository. For further guidance, visit the YOLOv7 GitHub repository for the latest information and updates. +4. After training, you can export the model to ONNX or TensorRT for use in Ultralytics as shown in [Usage Examples](#usage-examples). + ### What are the key features and optimizations introduced in YOLOv7? YOLOv7 offers several key features that revolutionize real-time object detection: