Definition
Liquid time-constant (LTC) network is a continuous-time recurrent network in which each neuron's state follows an ODE whose decay rate is modulated by a learned, input-dependent gate. Its effective time constant therefore changes from moment to moment ("liquid") with the input, unlike an RNN or a leaky integrator with a fixed time constant.
LTCs were introduced by Ramin Hasani, Mathias Lechner, Alexander Amini, Daniela Rus and Radu Grosu (arXiv 2020, AAAI 2021). The design borrows from models of non-spiking neurons in the nematode C. elegans, where a synapse acts like a conductance that pulls the membrane potential toward a reversal potential. "Liquid neural network" is the umbrella name for this family: LTCs, their wiring-constrained Neural Circuit Policies, and the closed-form CfC variant.
From a leaky integrator to a liquid one
A continuous-time RNN neuron is a leaky integrator: it decays toward zero at a fixed rate and is driven by its input, . Its speed of response is set once by , and nothing stops from growing with the input.
An LTC neuron instead lets the drive enter multiplicatively, as a conductance that both opens a channel toward a target value and adds to the leak:
- hidden state of the neuron (its "membrane potential")
- input at time t
- base time constant, learned and positive
- a small network with a sigmoid output (the synaptic conductance), parameters
- learned bias vector (the reversal potential): where the gate pulls the state
Collect the terms that multiply and the equation reads with an input-dependent time constant:
- effective time constant: equal to when the gate is closed (), shorter as grows
- the value the state relaxes toward for a fixed input; always between 0 and
Two properties follow, and Hasani et al. prove both. If , the time constant stays in , so the neuron never becomes arbitrarily fast or slow. And starting inside the range, the state stays between and however large the input: a strong input just opens the gate wider and pulls harder toward , it cannot push past it.
Solving it: the fused semi-implicit Euler step
The ODE is stiff when is large (a fast pull toward ), and explicit Euler then overshoots or blows up. The LTC paper treats the decaying implicitly and the gate explicitly, which gives a closed-form update:
- solver step; the paper unfolds 6 steps per input sample
The result is a convex combination of and (both coefficients are positive), so the update itself respects the bound at any step size. Training is ordinary backpropagation through time through the unrolled solver steps.
| t | exact | fused, Δt = 0.5 | explicit Euler, Δt = 0.5 | explicit Euler, Δt = 1 |
|---|---|---|---|---|
| 0.5 | 0.4544 | 0.3528 | 0.8176 | — |
| 1.0 | 0.5760 | 0.5050 | 0.5579 | 1.6351 |
| 1.5 | 0.6086 | 0.5707 | 0.6404 | — |
| 2.0 | 0.6173 | 0.5990 | 0.6142 | −1.0386 |
Explicit Euler multiplies the distance to by each step. At that is , so it oscillates around but converges; at it is and the iterates grow (the next two are and ). The fused step's factor is , which lies in for every : it lags the exact curve slightly but never leaves .
1import torch, torch.nn as nn23class LTCCell(nn.Module):4 def __init__(self, n_in, n_hidden, unfolds=6):5 super().__init__()6 self.inp = nn.Linear(n_in, n_hidden)7 self.rec = nn.Linear(n_hidden, n_hidden, bias=False)8 self.log_tau = nn.Parameter(torch.zeros(n_hidden)) # tau = exp(.) > 09 self.A = nn.Parameter(torch.randn(n_hidden)) # reversal potentials10 self.unfolds = unfolds1112 def forward(self, I, x, dt): # dt: (B, 1) elapsed time13 tau, h = self.log_tau.exp(), dt / self.unfolds14 for _ in range(self.unfolds):15 f = torch.sigmoid(self.inp(I) + self.rec(x)) # conductance, in (0, 1)16 x = (x + h * f * self.A) / (1 + h * (1 / tau + f)) # fused semi-implicit step17 return x1819cell, head = LTCCell(3, 19), nn.Linear(19, 1)20x = torch.zeros(8, 19)21for I_t, dt_t in zip(inputs.unbind(1), gaps.unbind(1)): # (B, T, 3) inputs, (B, T, 1) time gaps22 x = cell(I_t, x, dt_t)23y_hat = head(x) # trained with ordinary BPTT2425# The authors' library: from ncps.torch import LTC, CfC; from ncps.wirings import AutoNCPNeural Circuit Policies and CfC
Neural Circuit Policies (Lechner, Hasani et al., Nature Machine Intelligence 2020) wire LTC neurons sparsely in four layers modelled on the C. elegans tap-withdrawal circuit: sensory, inter, command and motor neurons. Behind a convolutional feature extractor, a 19-neuron NCP steered a car for lane keeping from camera images, comparable to much larger LSTM controllers, with attention maps that stayed on the road and a circuit small enough to inspect neuron by neuron. Later work (Chahine et al., Science Robotics 2023) used liquid networks to fly drones toward targets in environments unlike those seen in training.
The cost is the solver: every input sample needs several sequential update steps. Closed-form continuous-time networks (CfC, Hasani et al., Nature Machine Intelligence 2022) approximate the LTC solution in closed form and drop the solver entirely. The elapsed time enters as an input to a time-dependent gate:
- three heads on a shared backbone network
- a learned, input-dependent interpolation from (short elapsed time) to (long elapsed time), playing the role of
The CfC paper reports training and inference one to five orders of magnitude faster than ODE-based counterparts, at similar accuracy on its time-series benchmarks.
Use when
- +Input-dependent time constant: fast reaction to strong inputs, slow integration otherwise.
- +Bounded state and bounded by construction; the fused solver is stable at any step size.
- +Elapsed time is explicit, so irregularly sampled series need no resampling.
- +Very small, sparse, inspectable networks (NCP) for robot control.
Avoid when
- −ODE unfolding per sample makes training and inference slow (CfC mitigates this).
- −Sequential in time: no parallel training across the sequence like a transformer.
- −Evidence is mostly small-scale; tooling and pretrained models are scarce.
Check yourself
An LTC neuron has τ = 2 and, for the current input, gate value f = 1.5. What is its effective time constant?
Check yourself
Why does a large input not push an LTC neuron's state beyond its reversal potential A?
Remember
- ✓LTC: with a learned gate.
- ✓Input-dependent time constant , bounded in ; state bounded between 0 and .
- ✓Fused semi-implicit Euler: , stable at any ; explicit Euler is not.
- ✓NCPs: sparse C. elegans-style wiring, 19 neurons steering a car; CfC (2022) replaces the solver with a closed-form approximation.
- ✓Good for small, irregularly sampled, control-style problems; slow and unproven at large scale.