FAST: Efficient Robot Action Tokenization 

Background/problem

1. Autoregressive VLAs predict discrete tokens, but robot actions are continuous (e.g., arm joint angles).
2. Current VLAs typically use a simple binning discretization scheme, where the range of values are split into, say, 256 buckets.
3. The issue here is at least two-fold: a. Too many tokens per action chunk (expensive), especially when control rate is high. b. High redundancy between tokens (the model can have low loss by copying the last token, learning stalls).

Their solution

• FAST tokenizer steps:

  1. Normalize action chunk.

     This initial normalization step is useful to bring the data into a specified range and also makes tokenization of cross-embodied datasets with different action scales easier.

  2. Apply Discrete Cosine Transform (DCT) (and compress via rounding)

     • The DCT rewrites a signal as a mix of smooth cosine waves at different frequencies. Instead of storing every sample, you just store the strengths of those waves, which capture both the overall trend (low frequency) and fine details (high frequency).

  3. Quantize to get sparse frequency matrix.

     To compress the DCT-converted signal we can simply omit insignificant coefficients, which we implement through a scale-and-round operation, where the scaling coefficient is a hyperparameter that trades off between lossiness and compression rate of the tokenization operation.

  4. Low-frequency components first.

     Flattening is done column-first, so low-frequency coefficients across all action dimensions appear first, followed by higher-frequency ones. (So autoregressive decoding predicts the global coarse motion early and only later refines fine-grained details.)

     Note that the order of flattening the |A|×H DCT coefficient matrix prior to BPE encoding can have significant impact on policy training. There are two options: column-first flattening, i.e., concatenate the lowest-frequency components for each dimension first, or row-first flattening, i.e., concatenating all frequency components for a single action dimension first. We choose the former, since we find that predicting the low-frequency components, that characterize the overall shape of the output sequence, first during autoregressive prediction leads to more stable policy rollouts.

  5. Byte Pair Encoding (BPE) to get compressed action tokens.

     The BPE step “squashes” the zero-valued components and merges frequently-occurring coefficient combinations across action dimensions. We choose BPE to compress the DCT matrix, since many efficient implementations exist and it can produce a fixed-size output vocabulary that can be easily integrated into the existing vocabulary of vision-language models for VLA training.

Empirics, briefly

• π0-FAST, trains 5x faster than the original π0 model, and achieves similar performance.