Technical Summary: UNCLE-Grasp

Problem Statement

Robotic strawberry harvesting in unstructured agricultural environments faces significant challenges due to partial occlusion, where leaves obscure fruit geometry. This occlusion leads to incomplete point cloud observations, causing geometric distortions that shift estimated centroids and result in unstable or unreachable grasps. Existing uncertainty-aware grasping methods typically estimate uncertainty for pose, shape, or individual grasp candidates but fail to explicitly aggregate grasp feasibility across multiple completed object hypotheses. Consequently, they lack a mechanism to support an object-level "attempt-or-abstain" decision. Furthermore, deterministic grasp prediction models often produce poorly calibrated confidence scores under distribution shifts, leading to brittle decision-making in heavily occluded scenes.

Methodology

The paper proposes UNCLE-Grasp (UNCertainty-aware grasping of LEaf-occluded strawberries), a task-adapted framework that integrates learned shape completion, Monte Carlo (MC) dropout-based variability estimation, physically grounded grasp evaluation, and risk-aware decision-making.

1. Pipeline Overview

The framework processes RGB-D observations of partially occluded strawberries through four main stages:

1. Perception & Preprocessing: RGB-D images are denoised and segmented to extract individual partial point clouds for detected strawberries.
2. MC-Dropout Completion & Uncertainty Estimation: A transformer-based point cloud completion network (PointAttN) reconstructs missing geometry. By retaining dropout layers during inference, the system generates $K$ stochastic completion samples. The variability across these samples serves as a proxy for epistemic uncertainty.
   • Global Filtering: Strawberries with a mean point-level variability exceeding a threshold ($\delta_{global}$) are rejected immediately.
3. Grasp Generation & Filtering: For each remaining completion sample, a grasp generation network (CGNet) produces candidate grasps. These candidates undergo a three-stage filtering process:
   • Local Uncertainty Filter: Removes candidates where the gripper closing region contains points with high disagreement across stochastic completions.
   • Geometric Filters: Enforces task-specific constraints, including approach direction (frontal access), vertical grasp orientation, and jaw-object clearance to prevent collisions.
4. Risk-Aware Decision (LCB): For each completion sample $k$, the retained grasp candidates are combined into a single grasp wrench space to compute a force-closure score $\epsilon_k$.
   • The system calculates the mean ($\bar{\epsilon}$) and standard deviation ($\sigma_\epsilon$) of these scores across all $K$ completions.
   • A Lower Confidence Bound (LCB) is computed: $LCB = \bar{\epsilon} - z_\alpha \sigma_\epsilon$.
   • The parameter $z_\alpha$ scales linearly with the estimated occlusion level $\alpha$, becoming stricter as occlusion increases.
   • Decision Rule: The system attempts to grasp the strawberry only if $LCB > 0$; otherwise, it abstains.

2. Key Technical Components

• Uncertainty Modeling: The framework approximates epistemic uncertainty using MC dropout variability rather than modeling aleatoric uncertainty explicitly. It does not disentangle these sources.
• Force-Closure Metric: Grasp feasibility is evaluated using the $\epsilon$ metric, defined as the radius of the largest ball centered at the origin contained within the convex hull of the grasp wrench space.
• Selective Abstention: The core decision logic prioritizes reliability over yield. By abstaining from targets with inconsistent grasp feasibility across plausible completions, the system avoids risky attempts that could damage fruit or fail.

Key Contributions

1. Object-Level Decision Framework: A novel pipeline that propagates uncertainty from stochastic shape completions through grasp filtering and force-closure evaluation to determine whether to attempt or defer a grasp.
2. Task-Adapted Integration: A method that combines global/local uncertainty filtering, strawberry-specific geometric constraints, and an LCB-style feasibility score using existing point-cloud completion and grasp-generation models without retraining them.
3. Controlled Evaluation: A comprehensive evaluation in both simulation and on a physical robot under increasing levels of synthetic and real leaf occlusion, including ablation studies on shape completion, geometric filtering, uncertainty-aware filtering, and object-level abstention.

Experimental Results

The framework was evaluated on a Unitree Z1 robotic arm with an Intel RealSense D435i camera, using high-fidelity plastic strawberries to ensure safety and repeatability. Experiments covered occlusion levels from 0% to approximately 87.45%.

• Simulation Performance: At the highest simulated occlusion (87.45%), UNCLE-Grasp improved the success rate over attempted grasps from 0.780 (strongest baseline, CGNet+Geom Completed) to 0.870. While the attempt rate was slightly lower (0.860 vs. 0.880), the overall success rate (success over total strawberries) increased from 0.680 to 0.740.
• Physical Robot Performance: Under approximately 87% synthetic occlusion, UNCLE-Grasp achieved a success rate over attempted grasps of 0.800, significantly outperforming the strongest baseline (0.483). However, this came with a lower attempt rate, reflecting the system's conservative risk-aversion.
• Real Leaf Occlusion: Under real leaf occlusion (~70%), UNCLE-Grasp improved grasp success from 0.367 (baseline) to 0.517.
• Ablation Studies: Results confirmed that shape completion provides the largest gains under severe occlusion, learned grasp generation (CGNet) outperforms centroid-based placement, and geometric filtering removes physically infeasible candidates. However, only the full UNCLE-Grasp pipeline, incorporating uncertainty-aware selection, consistently achieved the highest conditional success rates.

Significance and Claims

The paper claims that the primary benefit of UNCLE-Grasp is more reliable, risk-aware execution through selective abstention. The results demonstrate a trade-off between grasp reliability and harvesting yield: by deferring uncertain targets, the system significantly reduces the rate of failed or damaging grasps, even if it reduces the total number of attempts.

The authors emphasize that this work highlights a broader principle for risk-aware automation: when perception is ambiguous, selectively abstaining can be more desirable than always acting. The study asserts that uncertainty is most useful when connected to an explicit downstream decision (attempt vs. abstain) rather than merely reported as a perception score.

The paper acknowledges limitations, noting that the current pipeline is a research prototype with high computational costs (approx. 57.53 seconds per strawberry in the optimized version) and does not address missed detections or collision-aware approach planning. It explicitly states that the system does not guarantee the success of any particular grasp but indicates with high confidence that at least one feasible grasp exists despite geometric uncertainty. The authors do not claim readiness for real-time commercial deployment but rather demonstrate the feasibility of uncertainty-aware selective execution in agricultural robotics.