Image-based Recognition Workflow

This workflow uses image recognition as the primary source of target perception. A 2D detection result is sufficient when only a class, position, or contour is required. Robot picking additionally requires an organized point cloud, camera parameters, and hand-eye calibration data to generate a 3D pose.

Suitable scenarios

  • Targets have distinguishable texture, color, or contour features.

  • Targets may deform and cannot be represented by a stable 3D point-cloud template.

  • Point clouds are incomplete, but color images are suitable for detection or segmentation.

Workflow

  1. Input images, point clouds, camera parameters, and hand-eye calibration data.

  2. Configure a 2D or 3D ROI when required.

  3. Obtain detection results through object detection, instance segmentation, or shape-based matching.

  4. Filter the results and combine them with point clouds to generate object poses.

  5. Correct poses, then generate and sort pick points.

Design guidance

Input data

Use EpicPro Input (Single Data Group) for a single-camera workflow, or EpicPro Input for multiple data groups. Both operators can receive live data or read .epicraw files for offline debugging.

For 3D pose generation, ensure that the original image, organized camera-frame point cloud, camera parameters, and camera-to-robot hand-eye calibration data belong to the same acquisition.

ROI and image alignment

An ROI can be obtained in the following ways:

With Image ROI, enable Keep Original Shape to preserve the original image size and fill the area outside the ROI with zeros. If the option is disabled, the output is cropped; use Align Post-ROI Detection Results before extracting points from the original point cloud.

When applying Point Cloud ROI before pixel-based mapping, enable Keep Organized and Keep Original Size. These options can be disabled after the workflow no longer depends on image-to-point correspondence.

Select a recognition method

Text-prompt Detection and Fast Prompt Segmentation are optional approaches. Validate model availability, runtime environment, and stability with production data before deployment.

Generate a 3D pose

Choose one of the following paths:

If Split Detection Result has already produced masks with the same dimensions as the organized point cloud, Extract Point Clouds from Mask List can also be used. Prefer the projection-based operator when the color and depth cameras require calibration-based mapping.

Correct poses and generate pick points

Parameter tuning

ROI and alignment

Start with an ROI that fully contains the target, then remove unnecessary background gradually. Keep image shape and point-cloud organization until the workflow no longer requires pixel correspondence. After a target cloud has been extracted, remove invalid padding and switch to ordinary unorganized point-cloud processing to avoid unnecessary computation.

Detection and segmentation

Select a model that matches the task and inference platform. Increase the confidence threshold to reduce weak detections, or decrease it to retain difficult targets, then use downstream filtering to remove false positives. When using detection orientation, verify the annotation convention, long-edge direction, and pick-point axes across many target rotations.

Shape-based matching

For Shape-based Matching, prepare a clean template, use a unique template name, and force training only while the template is being developed. Limit scale and rotation search ranges to realistic values. Smaller steps and more features improve search resolution but increase runtime. Tune score and overlap thresholds only after the template contour is stable.

Point-cloud extraction and pose generation

In Extract Point Cloud from 2D Detection Result, adjust the bounding-box scale to balance missing target edges against included background. Verify the detection type and image source. With Extract Point Clouds from Mask List, use contours for irregular targets, rectangles for faster broad extraction, or a center circle when only a stable center region is required.

For Generate Object Pose from Detection Result, tune the central-cloud ratio and Z-value method according to the target surface. For aligned target clouds, choose an appropriate circle radius, Z method, and alignment strategy. Enable detection orientation only when its angle is stable.

Pick-point generation and sorting

Generate Pick Points (Parameters) creates candidates from X/Y/Z offsets and rotational symmetries. Offsets follow the pick-point axes, so verify axis directions before setting values. Generate only physically valid symmetric orientations.

Pick-point Sorting V2 supports coordinate, target-axis angle, view-ray angle, reference-point distance, score, priority, and planar distance or angle strategies. Add the rule that best represents the business objective first, then use secondary rules to break ties. Sorting does not replace reachability, collision, or boundary filtering.

Debugging tips

  • If no target is detected, check the input image, model, confidence threshold, and ROI.

  • If detection is correct but the extracted cloud is shifted, check post-ROI alignment, image source, camera parameters, and whether the point cloud is organized.

  • If position is correct but orientation is wrong, check the detection-angle option and the coordinate system of normals or bounding boxes.

  • Use the current operator reference for parameter meaning and ranges instead of reusing fixed thresholds or timing figures from old projects.