Typical Algorithm Workflows
This section describes three common workflows by their primary recognition method. Choose a workflow based on the available input data, workpiece deformation, image features, point-cloud quality, and required pose accuracy.
| Workflow | Primary method | Suitable scenarios | Advantages | Considerations |
|---|---|---|---|---|
Detect or segment targets in images, or use shape-based matching, and then combine the result with an organized point cloud to generate poses. |
Objects with distinctive textures or contours; deformable objects or objects without a stable 3D shape. |
Handles deformation and makes full use of image features. |
Deep-learning methods normally require representative training data. Image, point cloud, and camera parameters must remain aligned when generating 3D poses. |
|
Perform 3D coarse and precise matching against a model point cloud. |
Rigid workpieces with stable geometry and reliable point-cloud templates. |
Estimates full 3D poses without relying on texture. |
Depends on the quality, scale, and normals of both scene and model point clouds. |
|
Use 2D recognition to isolate targets, then process the corresponding point clouds and optionally perform precise 3D matching. |
Closely packed targets that are difficult to separate using point clouds alone and require accurate 3D poses. |
Combines image-based separation with point-cloud geometry. |
The image, detection result, organized point cloud, camera parameters, and coordinate systems must remain consistent. |
Checks before selecting a workflow
-
Confirm the required output: If only a 2D result is required, do not continue to pose generation. Robot picking additionally requires point clouds, camera parameters, and hand-eye calibration data.
-
Confirm data correspondence: Workflows that extract point clouds or generate poses from image results must use data from the same acquisition. Pixel-level mapping requires an organized point cloud.
-
Confirm coordinate systems: Camera-frame point clouds are used for projection and image alignment. Data sent to the robot or used to correct pick poses must be transformed as required by each operator.
-
Validate intermediate results: Check the ROI, recognition result, target point cloud, object pose, and pick points stage by stage.
For operator inputs, outputs, and parameters, always use the current Operator Reference.
Point-cloud template matching
A complete template-matching graph contains workspace cropping, invalid-point removal, downsampling or filtering, normal estimation, 3D Coarse Matching, 3D Precise Matching, coordinate transformation, and pick-point processing. Coarse matching should retain candidates that cover the real targets; precise matching provides the final accuracy.
Multi-camera point-cloud stitching
Transform every camera cloud into a common coordinate system before merging. Verify each transformed cloud separately, especially in overlapping areas. Double surfaces or a constant offset normally indicate an incorrect hand-eye matrix or transformation direction.
For grouped stitching, the point-cloud, camera-parameter, and calibration lists must use the same camera order. If an intermediate filter changes one list, update the related lists together.
Image-recognition patterns
Pattern 1: extract and process each target cloud
Detect or segment the image, extract a cloud for every detection, remove invalid or unrelated points, and generate a pose with Generate Object Pose from Aligned Point Cloud and Detection Result. This pattern is recommended when a detection region contains background or neighboring workpieces.
Pattern 2: generate poses directly
Provide the detection, original image, organized camera-frame cloud, camera parameters, and hand-eye data to Generate Object Pose from Detection Result. If the image was cropped, keep its original shape or restore detection coordinates before point mapping.
Pattern 3: use point clouds to narrow the image region
Use a 3D ROI, clustering, or point-cloud sorting to obtain a reliable spatial region, then project it into the image for detection or segmentation. Track coordinate systems explicitly when the cloud is transformed to the robot frame for spatial processing and back to the camera frame for projection.
Combine image recognition with 3D matching
When 2D recognition separates targets but does not provide sufficient pose accuracy, extract each target cloud and initial pose, then run 3D Precise Matching. Use the same scale for target and model clouds and provide valid normals. A full coarse-plus-precise search is more tolerant of poor initial poses; direct precise matching is faster but depends on a reliable initial pose.
Treat legacy graph screenshots or imported graphs as structural references. Replace old operator names with their 1.4 equivalents and reconfigure the model, ROI, thresholds, coordinate systems, and pick strategy for the current project.