Segment Anything V3 TensorRT
Description
This operator is the TensorRT-accelerated version of Segment Anything V3, providing open-vocabulary object detection and instance segmentation: without task-specific pretraining, it detects and segments target objects in an image from English text descriptions. The node has two operating modes — Export mode converts a fixed Vision/Decoder ONNX package into a TensorRT engine package, while Inference mode only needs to load an already-exported engine package to run segmentation; multiple text prompts on the same image reuse the Vision image features for speed.
Overview
Converts Segment Anything V3’s ONNX model to a TensorRT engine and uses it for accelerated open-vocabulary detection and segmentation.
Scenarios
-
Production deployment where inference needs to be faster than plain torch
-
First-time deployment on a target GPU: use Export mode to convert the official ONNX package into a TensorRT engine specific to that machine
-
Day-to-day operation: use Inference mode to load an already-exported engine package for open-vocabulary detection and segmentation
-
Running multiple text prompts on the same image, reusing shared Vision image features to avoid redundant computation
Inputs and Outputs
Inputs |
Image: An input BGR color image. |
Outputs |
Detection Results: Returns the merged bounding boxes, scores, and polygon masks across all prompts. |
Parameters
Initialization Parameters
Operation Type
Description |
Selects the node’s operating mode: Export mode builds a TensorRT engine package from the ONNX model package; Inference mode loads an already-built engine package to run segmentation. |
Tuning Instructions |
Use Inference mode for day-to-day operation. Use Export mode on first deployment, after changing the runtime environment, or whenever the export batch parameters change and the engine must be rebuilt. |
Parameter Range |
Options: Export, Inference, Default: Inference |
Parameter Range |
Values/Range:
Default: TRT Inference |
ONNX Model File
Description |
Used only in Export mode. Upload a ZIP archive that contains at least all_segmentv3_vision.onnx, all_segmentv3_decoder_dynamic.onnx, and their related files. |
Tuning Instructions |
This parameter is not used during inference tuning; it only matters when exporting the engine. |
Download Link |
|
Parameter Range |
Options: .zip |
Parameter Range |
Values/Range:
|
Engine Save Directory
Description |
Used only in Export mode. Specifies the directory where the generated TensorRT engine ZIP is saved; the filename automatically includes the Decoder’s optimal and maximum batch sizes. |
Tuning Instructions |
Use a writable local directory. After changing the export batch parameters, you must export again — the old engine is not updated automatically. |
Build Workspace
Description |
Used only in Export mode, in GB. It is the temporary GPU memory ceiling during the TensorRT build, not the fixed memory usage during inference. |
Tuning Instructions |
If you run out of memory while exporting, lower this value or the batch parameters. If memory is plentiful and you want TensorRT to search more optimization tactics, raise it, but too high a value may crowd out other GPU memory. |
Parameter Range |
[1, 16], Default: 4 |
Parameter Range |
Values/Range: [1, 16] Default: 4 Unit: GB |
Optimal Prompt Batch
Description |
Used only in Export mode. The prompt batch size the Decoder is primarily optimized for during the build. Inference can still use anywhere from 1 up to the maximum batch size, but speed is usually best when it equals this value. |
Tuning Instructions |
Set according to the typical number of prompts per image: use 4 for the common 1-4 prompt case, and only use 8 if you regularly use 8 prompts. A larger value generally increases build time and GPU memory pressure. The engine must be re-exported after changing this. |
Parameter Range |
[1, 16], Default: 4, must not exceed the maximum batch |
Parameter Range |
Values/Range: [1, 16] Default: 4 |
Max Prompt Batch
Description |
Used only in Export mode. The maximum prompt batch size the engine allows. It determines the maximum number of prompts the engine can handle and also affects build time and the engine’s resident GPU memory; some memory may be reserved even if actual runs don’t use the full batch. |
Tuning Instructions |
Set it to the actual maximum number of prompts you expect, without blindly maxing it out: typically 8 for 16GB GPUs and 4 for 6GB GPUs. The engine must be re-exported after changing this. |
Parameter Range |
[1, 16], Default: 4, must not be less than the optimal batch |
Parameter Range |
Values/Range: [1, 16] Default: 4 |
TRT Engine File
Description |
Used only in Inference mode. Upload an all_segmentv3_engine_fp16_opt*_max*.zip engine package exported by this node. |
Tuning Instructions |
The engine is tied to the TensorRT/CUDA version and GPU architecture, so it’s best exported on the target machine. If you change the maximum or optimal batch size, you must switch to a freshly re-exported engine package. |
Parameter Range |
Options: .zip |
Parameter Range |
Values/Range:
|
Run Parameters
Prompt Source
Description |
Selects a built-in prompt scenario or custom prompts. Built-in scenarios are good for a quick check; choose Custom and fill in the parameter below when you need to detect a specific target. |
Tuning Instructions |
First use a built-in scenario to confirm the model and engine work correctly, then refine detection with specific, clear English phrases. |
Parameter Range |
Options: Bag, Box, Merchandise, Custom, Default: Custom |
Parameter Range |
Values/Range:
Default: Custom |
Custom Prompts
Description |
Only takes effect when Custom is selected. Enter the English words or phrases to detect, separated by commas, e.g. chair, table, person. |
Tuning Instructions |
Use specific, clear phrases tied to the target’s appearance. Too many prompts increases inference time — split the task across runs or increase the inference batch size instead. |
Confidence Threshold
Description |
Only instances with a model score above this value are kept. |
Tuning Instructions |
Raise the threshold if there are too many false positives; lower it if there are too many missed detections. A lower threshold generally means more results, more post-processing time, and a higher chance of low-quality results. |
Parameter Range |
[0.0, 1.0], Default: 0.5 |
Parameter Range |
Values/Range: [0.0, 1.0] Default: 0.5 |
NMS IoU Threshold
Description |
Runs global box deduplication across the results of all prompts: when two boxes' IoU exceeds this value, only the higher-scoring box is kept. |
Tuning Instructions |
Lower it if there are many duplicate boxes; raise it or disable deduplication (set above 1, e.g. 1.1) if neighboring targets are being incorrectly removed. A lower value means more aggressive deduplication. |
Parameter Range |
[0.0, 1.1], Default: 0.7 |
Parameter Range |
Values/Range: [0.0, 1.1] Default: 0.7 |
Inference Batch
Description |
The number of prompts fed to the Decoder per round; if there are more prompts than this, they are automatically split into batches and the results merged afterward. |
Tuning Instructions |
Increase it when GPU memory is plentiful and there are many prompts, to reduce batching and improve throughput; decrease it if memory is limited or runs are unstable. This parameter only affects inference and does not require re-exporting the engine. |
Parameter Range |
[1, 16], Default: 4, must not exceed the maximum batch size the engine was exported with |
Parameter Range |
Values/Range: [1, 16] Default: 4 |