#### NOTE
This is a **Hugging Face dataset**. For large datasets, ensure `huggingface_hub>=1.1.3` to avoid rate limits. Learn more in the Hugging Face integration docs.
# TaF-Dataset → FiftyOne (Native Multimodal MCAP)
[jiamig/taf-dataset](https://huggingface.co/datasets/jiamig/taf-dataset)
converted to native multimodal MCAP episodes.
Every frame pairs what a tactile sensor sees with what a force sensor
measures. A vision-based tactile sensor records the gel deforming, a 12x12
piezoelectric array records where the contact lands, and an ATI six-axis
sensor records the wrench that caused it, all on one 30 fps clock. Contacts
reach 90.8 N and 1.58 Nm.
3,594 episodes across 408 sequences, 10,057,742 frames, 93.1 hours.
## Installation
```bash
pip install fiftyone
```
## Usage
```python
import fiftyone as fo
import fiftyone.utils.huggingface as fouh
dataset = fouh.load_from_hub(
"Voxel51/TaF-Tactile-Force",
name="TaF-Tactile-Force",
persistent=True,
)
fo.launch_app(dataset)
```
The firmest contacts:
```python
view = dataset.match({"peak_force": {"$gt": 50}})
```
## What you get
Each episode carries:
- `/tactile-camera`, the tactile image, as `foxglove.CompressedVideo`
- `/pressure-map`, the 12x12 piezoelectric array as a heatmap, as
`foxglove.CompressedImage`
- `/force-torque`, the six-axis wrench, with `/force-torque.plot` carrying
`fx`, `fy`, `fz`, `tx`, `ty` and `tz` as a timeline
- `/pressure.plot` carrying the summed pressure, the peak cell, and how many
cells are in contact
- `/instruction`, naming the sensor configuration
Episodes carry the fields `sequence`, `sensor_family`, `episode_index`,
`num_frames`, `duration`, `fps`, `peak_force`, `peak_fz`, `peak_torque` and
`peak_pressure`, so a view can select on how hard the contact got.
Six sensor configurations are represented:
| Configuration | Episodes |
|-------------------------------|------------|
| GelSight Mini with markers | 1,103 |
| GelSight Mini without markers | 810 |
| Custom sensor without markers | 657 |
| Custom sensor, 6\*6 markers | 650 |
| Custom sensor, 8\*8 markers | 285 |
| Custom sensor, 7\*7 markers | 89 |
Peak force runs to a median of 25.0 N and a maximum of 90.8 N; peak pressure
to a median of 884 and a maximum of 4052.
## Notes on the conversion
The tactile video is re-encoded to Annex-B H.264 without B-frames. The source
is AV1, which the viewer does not decode.
The pressure array is 12x12, written as an image and upscaled with
nearest-neighbour so each cell stays a square. Colour is scaled per episode
against that episode’s 99.5th percentile, so a light touch and a hard press
are both readable. The `/pressure.plot` channel carries the unscaled values.
Three of the six sensor configurations spell their marker grid with an
asterisk, as in `custom_designed_6*6_mark_obj1`. That character cannot appear
in a Windows path, so directories on disk carry `6x6`, `7x7` and `8x8` while
the `sequence` field keeps the name as the release writes it.
Tactile image resolution varies by sequence.
This conversion holds 3,594 episodes and 10,057,742 frames against the 3,592
and 10,053,265 the source card states, across the same 408 sequences.
## License & attribution
TaF-Dataset is released under the
[MIT license](https://opensource.org/licenses/MIT), and this conversion is
distributed under the same license.
Changes from the source: conversion to the FiftyOne MCAP flavor, re-encoding
of the tactile video to H.264, and rendering of the pressure array as an
image stream alongside its scalar channels.
The dataset accompanies **TaF-VLA: Tactile-Force Alignment in
Vision-Language-Action Models for Force-aware Manipulation**.