How LeRobotDataset turns one frame index into a temporal window. delta_timestamps lists offsets in seconds; each one becomes a frame index, gets clamped to its episode, and comes back stacked in a tensor. Drag t, edit the offsets, and watch the sample change.
dataset[i] fetchesdelta_timestampsfps = 30 · tolerance_s = 1e-4Each offset is converted with round(offset × fps) and must be a multiple of 1/30 s ≈ 0.0333 s, within tolerance_s = 1e-4, or the dataset refuses to build. Order is kept: the stacked tensor follows the list. Keys you leave out of delta_timestamps come back as a single frame with no extra dimension.
dataset[i] picks one row of the whole dataset; on this page t is that row's position inside episode 17 (0 … 239).delta_timestamps, in seconds relative to t. Negative is the past, positive the future, 0.0 is t itself. It has to be a whole number of frames.t + round(offset × fps) (the "raw" index), then clamped to the episode's first and last frame (the "used" index). It never crosses into a neighbouring episode.{feature}_is_pad. True when the raw index fell outside the episode, so the frame you got is a copy of the episode's first or last frame. Loss functions should mask those rows.action values over the episodesynthetic pick-and-placelerobot/svla_so101_pickplace| episodes · frames · fps | 50 episodes · 11,939 frames · 30 fps, so ≈ 239 frames ≈ 8 s per episode. This page uses one 240-frame stand-in episode. |
| action | float32 [6]: shoulder_pan.pos, shoulder_lift.pos, elbow_flex.pos, wrist_flex.pos, wrist_roll.pos, gripper.pos. The commanded target sent to the follower arm. |
| observation.state | float32 [6], same six names. The measured position, i.e. where the arm actually is. |
| observation.images.up observation.images.side | Two cameras, video, 480 × 640 × 3 each. Decoded on the fly to [3, 480, 640] tensors. |
| timestamp · frame_index episode_index · index · task_index | Bookkeeping per row: seconds within the episode, frame number within the episode, which episode, the global row id, which task string. |
| install | pip install "lerobot[dataset]" (Python ≥ 3.12), then from lerobot.datasets.lerobot_dataset import LeRobotDataset. |