Task-Aware 6-DoF Grasp Pose Estimation through Cross-Modal Gated Fusion of Foundation-Model Semantic Embeddings

Abstract

Robotic grasping in cluttered scenes is dominated by methods that optimise either stability or a predefined task category, leaving a gap between “can this grasp hold the object” and “can this grasp complete the requested downstream task.” We address this gap with Task-Aware 6-DoF, a three-module detector that routes a natural- language task description through a frozen text encoder, fuses the resulting semantic vector with PointNet ++ point features through a learnable gated residual, and predicts task completion through a binary classifier trained on disjoint seeds. On a four-split procedurally generated benchmark with three random seeds and 25 trials per cell, our method reaches a task completion rate of 0.60 against seven reproduced baselines spanning two naive controllers and five recent published methods, beating six of seven decisively on the primary metric and tying the remaining LLM-only baseline within the simulator noise ban d. An honest ablation isolates the task-completion head as the dominant contribution while showing the gated attention module to be redundant. Calibration against three anchor points from prior work yields a mean absolute percentage error of 7.2 percent, comfortably below the 15 percent bound for analytic surrogates.