Researchers Propose TacEx to Enhance Robot Learning Efficiency Through Tactile Curiosity

Background

Reinforcement learning is a method where robots learn skills through trial and error, but it requires many attempts and thus has low efficiency. Existing methods find new strategies through random actions, during which robots spend most of their time in empty space without touching objects. Conventional curiosity methods, which reward parts the model does not know well, have also been criticized for potentially rewarding irregular movements unrelated to objects.

Method

The researchers developed TacEx, designed to separate the model's uncertainty by sensory modality and direct curiosity specifically toward the tactile channel. Through this, the robots explore physical changes occurring from contact without task goals. Using the data gathered through exploration, they learn pick-and-place policies offline and further train pre-trained vision-language-action models using tactile-based exploration.

Main results

The abstract explains that using TacEx allows robots to learn skills to grasp and manipulate objects even without task rewards. It states that pick-and-place policies can be learned using this collected interaction data without additional environment interaction. It reports that pre-trained vision-language-action models without touch, when post-trained with TacEx, show significantly improved performance and remain sample-efficient. Specific figures are not provided in the abstract.

Limitations & open points

The abstract does not provide specific figures or comparison targets such as the type of robot used in experiments, the scope of tasks, or success rates, making it impossible to judge the magnitude of performance improvement. Stability in real-world environments and generalization to other tasks cannot be confirmed from the abstract alone.

Glossary

• 강화학습 — A learning method where behavioral methods are learned through trial and error to maximize rewards received as a result of actions.
• 샘플 비효율성 — A property where efficiency is low because the amount of attempts or data required for learning is large.
• 내재적 동기(호기심) — A reward mechanism that drives the robot to find new or unfamiliar things on its own, without external rewards.
• 인식적 불확실성 — Refers to the degree to which an AI model does not yet know well. Exploring in the direction of reducing this can yield new experiences.
• 촉각 피드백 — A signal in which the robot receives pressure or contact information felt when touching an object.
• 오프라인 학습 — A learning method using only previously collected data without additional experiments.
• 시각-언어-행동(VLA) 모델 — A large AI model that takes images and language commands together to output robot actions.
• 사후 학습(post-training) — The process of further retraining a pre-trained model to suit a specific purpose.
• 픽 앤 플레이스(pick-and-place) — The action of picking up an object and placing it down at a different location.