Thanana
Home
Biography
Publication
Research
World Models and Control
Holonomic Systems
Human-robot Interaction
Cyber-physical Systems
Teaching
Reinforcement Learning
Introduction to Robotics
Software
Robotics
Image Processing

Tactile Intelligence

“To touch is to understand — robots that learn as they feel.”

Robots now see and plan well, yet they still fail at the moment of contact. Friction changes, objects slip, and the contact patch is hidden from cameras, so the quantities that matter most for manipulation are the ones a robot can measure least directly. Tactile intelligence treats touch as a closed loop: tactile signals update a belief about the hidden contact state, an uncertainty-aware controller acts on that belief, and the action itself creates the next tactile measurement.

Belief over contact state p(zt | history) 1 Touch: tactile signals yt 2 Infer latent stochastic model 3 Decide: uncertainty-aware MPC / RL, ut = π(bt) 4 Act grip force, surface haptics

Figure 1. The tactile intelligence loop. Contact produces tactile signals (1); a latent stochastic model turns them into a belief over the hidden contact state—slip, friction, object pose (2); a controller plans with that belief and its uncertainty (3); and the resulting grip or surface-haptic action changes the contact, which produces the next measurement (4).

Motivation

I want robots that can feel, adapt, and decide through touch. My work in robust, adaptive and Bayesian model predictive control, combined with reinforcement learning, suggests a shift in viewpoint: compliance need not be only a passive mechanical property; it can be an active principle of intelligence. A compliant robot should not resist uncertainty at contact, but yield to it, interpret it, and learn from it—with safety and stability treated as explicit requirements to be established under appropriate modeling and control assumptions.

Problem formulation

Let \(z_t\) be the latent contact state (e.g., slip velocity, friction coefficient, object pose), \(u_t\) the grip or haptic command, and \(y_t\) the tactile measurement, which is noisy and only partially informative:

\[ z_{t+1}=f_\vartheta(z_t,u_t)+w_t,\qquad y_t=h_\psi(z_t)+\nu_t,\qquad b_t=p\bigl(z_t\mid y_{0:t},u_{0:t-1}\bigr). \]

The controller chooses actions from the belief \(b_t\), not from the true state:

\[ u_t=\pi(b_t)=\arg\min_{u_{t:t+N-1}}\ \mathbb{E}_{b_t}\!\Bigl[\sum_{k=0}^{N-1}\ell(z_{t+k},u_{t+k})\Bigr] \quad\text{s.t.}\quad \Pr\bigl(\text{slip}_{t+k}\bigr)\le\epsilon . \]

The central difficulty is that \(f_\vartheta\) and \(h_\psi\) are learned and adapt online, so the belief that the controller trusts can itself change in ways that are not safe. Understanding and constraining that interface is a central theme of my current work. The formulation above describes a research objective; its tactile implementation and guarantees remain to be established.

(a) Correction path — every tick (b) Adaptation path — frozen context Reference belief predicted from the previous contact Tactile encoder proposes a correction from new contact Correction budget projects the belief within δE of the reference MPC grip / haptic command from the belief mean Input tolerance set by actuator and user comfort, ūc tolerance selects the budget Under the MPC assumptions: correction-induced command change is at most ūc Replay cache recent contacts and prediction errors Candidate update of encoder and contact model Update gate accept only if the score drops and the belief moves ≤ εB Accept or hold new parameters used at the next contact Update tolerance largest command change from learning, ūa accepted parameters reach the encoder

Figure 2. A proposed controller-aware interface for tactile learning, inspired by Neural Belief MPC. (a) A correction budget limits the distance from a reference belief. (b) A separate same-context budget limits belief changes induced by accepted parameter updates. In the MPC study, prescribed input tolerances determine these budgets under the stated covariance, domain, and affine-cell assumptions. The tactile and haptic blocks illustrate a prospective application; transferring these results requires validating the contact model and controller assumptions. Input-plan consistency alone does not establish physical safety, user comfort, or closed-loop stability.

Core ideas

Latent tactile state. Deep stochastic state-space models capture what touch cannot measure directly—slip, friction and contact geometry—together with their uncertainty and temporal structure.
Belief-aware control. The proposed direction investigates MPC for force and slip constraints alongside reinforcement learning for longer-term adaptation. The aim is to use belief information and uncertainty explicitly in decision making.
Bidirectional touch. Surface haptics actively modulate fingertip friction, offering a route to both sense and shape contact—blurring the line between perception and action.

Research questions

  • How can a learned tactile model keep a calibrated belief when contact is intermittent and sensors saturate or drop out?
  • How far may an online-adapted belief move before the controller's grip command changes by more than a prescribed input tolerance?
  • When should the controller trust touch, and when should it fall back to a conservative, model-based action?

Foundations in my current work

These questions draw on my work in learned state estimation and control under partial observation. Their transfer to tactile interaction is a research direction:

  • Seeing Beyond Vision — stochastic latent modeling and uncertainty propagation under occlusion, providing a modeling connection to intermittent tactile observations.
  • Robust CF Filtering and SafeCF-SSM — cognitive-flexible filtering and conditional physical-latent certification under online adaptation.
  • Neural Belief MPC — belief correction and adaptation budgets tied to input tolerances within a known affine MPC cell, under stated assumptions.

Objective

To develop robots that yield to uncertainty yet remain stable, safe and dexterous—advancing tactile robotics toward truly human-compatible interaction in manipulation, assistance and rehabilitation.

“To touch is to understand — robots that learn as they feel.”

Challenge. Change. Impact. — through tactile intelligence.

← Back to Home