#### NOTE
This is a **Hugging Face dataset**. For large datasets, ensure `huggingface_hub>=1.1.3` to avoid rate limits. Learn more in the Hugging Face integration docs.
# Dataset Card for PIVOT-mcap
This is a [FiftyOne](https://github.com/voxel51/fiftyone) dataset with 103 samples.
## Installation
If you haven’t already, install FiftyOne:
```bash
pip install -U fiftyone
```
## Usage
```python
import fiftyone as fo
from fiftyone.utils.huggingface import load_from_hub
# Load the dataset
# Note: other available arguments include 'max_samples', etc
dataset = load_from_hub("Voxel51/PIVOT-mcap")
# Launch the App
session = fo.launch_app(dataset)
```
## Dataset Details
### Dataset Description
PIVOT-mcap re-encodes the [PIVOT dataset](https://huggingface.co/datasets/MaryRaymond/PIVOT) (Raymond, 2026) — a five-scene, multi-trajectory drone photogrammetry dataset for evaluating NeRF and 3D Gaussian Splatting under realistic capture conditions — as 103 MCAP episodes, one per drone trajectory. Each episode pairs the trajectory’s RGB frame stream with PIVOT’s dual pose representation (sensor-derived measured pose and COLMAP-optimized pose per frame), dual camera calibration (physical/offline calibration and COLMAP-optimized intrinsics), per-frame pose error, and the scene’s shared sparse point cloud, so that a trajectory can be scrubbed through as a synchronized 3D episode in FiftyOne/Foxglove instead of browsed as a folder of independently posed frames.
- **Curated by:** Mary Raymond (original PIVOT dataset, capture, and processing pipeline); Harpreet Sahota (this MCAP re-encoding and FiftyOne dataset)
- **Funded by:** [More Information Needed]
- **Shared by:** Harpreet Sahota (this derived MCAP dataset); Mary Raymond (original PIVOT dataset, shared as `MaryRaymond/PIVOT` on the Hugging Face Hub)
- **Language(s):** en (dataset card only — the underlying data is imagery, camera poses, and a point cloud, with no natural-language content)
- **License:** CC BY-NC 4.0. The source PIVOT dataset is released under CC BY-NC 4.0 (non-commercial); the PIVOT toolchain source code is released separately under the MIT License.
### Dataset Sources
- **Repository:** [MaryRaymond/PIVOT](https://huggingface.co/datasets/MaryRaymond/PIVOT) (source dataset); [maryraymond/PIVOT](https://github.com/maryraymond/PIVOT) (processing/visualization/benchmark toolchain)
- **Paper:** Mary Raymond, “PIVOT: A Multi-Trajectory Dataset and Testbed for Pose, Intrinsics, and Novel Viewpoint Evaluation in Real-World 3D Reconstruction,” [arXiv:2608.25401](https://arxiv.org/abs/2608.25401)
- **Demo:** [More Information Needed]
## Uses
### Direct Use
- Exploring and visualizing PIVOT’s dual-pose, dual-intrinsic drone trajectories in 3D: scrubbing through a camera moving along its measured GPS/IMU/gimbal path and, where COLMAP registered the frame, its optimized pose, against the scene’s shared sparse point cloud.
- Prototyping or debugging FiftyOne/Foxglove multimodal (MCAP) pipelines against a real, non-synthetic example of “one moving camera against one static map with more than one competing pose source.”
- Non-commercial research uses consistent with the source dataset’s benchmark design: studying novel-view trajectory generalization (seen vs. unseen trajectories), pose-source sensitivity (measured vs. COLMAP-optimized poses), and intrinsic-source sensitivity (calibrated vs. COLMAP-optimized intrinsics), as defined in the PIVOT paper’s three benchmark families.
### Out-of-Scope Use
- Commercial use: the dataset inherits PIVOT’s CC BY-NC 4.0 license and is non-commercial only.
- Anything relying on real elapsed flight time or frame-to-frame timing: MCAP `log_time` values in every episode are a synthetic 10 fps sequence clock (`timestamps_are_synthetic=True` on every sample), not the drone’s real capture timestamps, since PIVOT ships no real per-frame timestamps in the source data.
- Broad claims about camera platforms, environments, or motion diversity beyond this release: PIVOT v1 (and this re-encoding) covers five scenes captured with a single DJI Mini 4 Pro drone, as noted in the paper’s own limitations discussion.
## Dataset Structure
This is a FiftyOne dataset with `media_type == "multimodal"` and 103 samples — one sample per `(scene, trajectory)` pair across PIVOT’s five scenes (`church`: 24, `backyard`: 22, `frontyard`: 22, `victorian_garden`: 18, `village_street`: 17). No splits or tags are defined; PIVOT leaves train/eval trajectory selection as a downstream, benchmark-specific choice rather than a fixed property of the data. Each sample’s `filepath` points to one `.mcap` file (an “episode”) containing that trajectory’s full frame sequence, both pose streams, camera calibration, and the scene’s point cloud.
### Sample fields
| Field | FiftyOne type | Description |
|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------|------------------------------------------------------------------------------------------------------------------------------------|
| `filepath` | `StringField` | Path to the episode’s `.mcap` file |
| `scene` | `StringField` | Scene name (5 values, verbatim from PIVOT) |
| `trajectory` | `StringField` | Trajectory name within the scene (verbatim from PIVOT) |
| `resolution` | `ListField` | `[width, height]` pixel resolution of the trajectory’s frames |
| `duration_s` | `FloatField` | Episode duration under the synthetic 10 fps clock (derived) |
| `message_count` | `IntField` | Total MCAP messages in the episode (derived, from the `.mcap` file) |
| `channel_count` | `IntField` | Total MCAP channels/topics in the episode (derived) |
| `topics` | `ListField` | MCAP topic names present in the episode (derived) |
| `schemas` | `ListField` | Foxglove/JSON schema names present in the episode (derived) |
| `has_image` | `BooleanField` | Whether a `foxglove.CompressedImage` channel is present (derived; always `True`) |
| `has_pointcloud` | `BooleanField` | Whether a `foxglove.PointCloud` channel is present (derived; always `True`) |
| `has_colmap_pose` | `BooleanField` | Whether any frame in the episode has a registered COLMAP pose (derived; `False` only for `church/rocket_upward`) |
| `n_points_kept` | `IntField` | Point-cloud points retained after outlier trimming (derived) |
| `n_points_dropped` | `IntField` | Point-cloud points dropped as outliers, farther than 15 m from the point cloud’s centroid (derived) |
| `timestamps_are_synthetic` | `BooleanField` | Always `True`; flags that MCAP `log_time` is a synthetic clock, not real capture time |
| `mandatory_level` | `StringField` | `core` or `optional`, per PIVOT’s trajectory taxonomy (verbatim) |
| `motion_type` | `StringField` | e.g. `orbit`, `traversal`, `bev_orbit`, `bev_traverse`, `traverse_loop`, `vertical_ascent`, `scattered`, `panorama_360` (verbatim) |
| `altitude_band` | `StringField` | `low` / `mid` / `high` (verbatim) |
| `camera_direction` | `StringField` | e.g. `scene_inward`, `along_track`, `nadir` (verbatim) |
| `lens_type` | `StringField` | `standard_fov` or `wide_fov` (verbatim) |
| `capture_mode` | `StringField` | `video_frames` or `photos` (verbatim) |
| `capture_device` | `StringField` | `dji_drone_mini_4_pro` (verbatim) |
| `aspect_ratio` | `StringField` | `16:9` or `4:3` (verbatim) |
| `fov_h`, `fov_v`, `fov_diag` | `FloatField` | Trajectory field of view in degrees (verbatim; `0` on some zero-COLMAP-registration trajectories, per PIVOT’s own export) |
| `average_rot_error` | `FloatField` | Trajectory-mean rotation error between measured and COLMAP poses, degrees (verbatim) |
| `average_cam_center_error_distance` | `FloatField` | Trajectory-mean camera-center distance error, meters (verbatim) |
| `missing_colmap_frames` | `IntField` | Count of frames without a registered COLMAP pose (verbatim) |
| `number_frames_in_traj` | `IntField` | Total frame count in the trajectory (verbatim) |
| `scene_total_frames_number`, `scene_colmap_reg_frames_number`, `scene_pointcloud_number`, `scene_observations`, `scene_scene_diameter`, `scene_aabb_diagonal`, `scene_max_rotation_angle`, `scene_mean_track_length`, `scene_colmap_per_image_observation`, `scene_mean_observations_per_image`, `scene_mean_reprojection_error_px` | `IntField` / `FloatField` | Scene-level reconstruction statistics, copied verbatim from PIVOT’s `scene_data.json` onto every sample belonging to that scene |
### Label types and why
This dataset carries no per-sample FiftyOne label field (no `Classification`, `Detections`, etc.). PIVOT’s actual content — a moving camera with two competing pose sources against one static map — is a synchronized multi-topic time series, not a set of independent per-image annotations, so it is encoded as MCAP topics inside each sample’s file instead:
- `/camera_measured/image_raw` (`foxglove.CompressedImage`) — one JPEG per frame
- `/camera_measured/calibration` (`foxglove.CameraCalibration`) — physical camera intrinsics, paired to the image channel (via `mcap.calibration_topic` metadata) so camera frustums render in the viewer
- `/camera_colmap/calibration` (`foxglove.CameraCalibration`) — COLMAP-optimized intrinsics; frustum geometry only, not paired to an image channel, since only one calibration topic can pair with a given image topic
- `/tf` (`foxglove.FrameTransform`) — `world -> camera_measured` on every frame; `world -> camera_colmap` only on frames with a registered COLMAP pose
- `/map/sparse_pointcloud` (`foxglove.PointCloud`) — the scene’s sparse reconstruction, logged once at the start of the episode, world frame
- `/pose_error` (custom JSON schema `pivot_pose_error`) — `rot_error_deg`, `camera_center_error_distance_m`, only on frames with a registered COLMAP pose
- `/pose_error/line` (`foxglove.SceneUpdate`, `LinePrimitive`) — a visual line segment connecting the measured and COLMAP camera centers, only on frames with a registered COLMAP pose
### `dataset.info`
Empty. All dataset-level facts (scene/trajectory taxonomy, pose-error statistics, reconstruction statistics) are attached as per-sample scalar fields (see the field table above) rather than stored on `dataset.info`.
### Parsing decisions
- **Coordinate systems converted.** PIVOT stores poses as 4x4 camera-to-world matrices in a NED world frame (X-north, Y-east, Z-down) with an OpenGL-style camera convention (verified against PIVOT’s own `dji_drone_mini_4.py` and `viser_visualization.py`). These are rotated into a Z-up world frame and an OpenCV/ROS optical camera convention for Foxglove/FiftyOne rendering — applied identically to camera poses and to point-cloud vertices, since both live in the same source frame.
- **Timestamps are synthetic.** PIVOT ships no real per-frame capture timestamps (JPEG EXIF `DateTimeOriginal`/`CreateDate` are stripped; only GPS and gimbal angles survive). MCAP `log_time` uses a synthetic 100 ms/frame (10 fps) sequence clock to preserve frame order without claiming a real elapsed flight duration.
- **Missing COLMAP poses are never fabricated.** Frames without a registered COLMAP pose (`colmap_pose_c2w` absent from the source JSON, not `null`) get no `world -> camera_colmap` transform and no `/pose_error` message for that frame. `church/rocket_upward` has zero registered frames across its entire trajectory — a documented issue in PIVOT’s own dataset card, confirmed independently, not introduced by this re-encoding.
- **Distortion model picked from available keys, not from the `camera_type` label.** Standard-FOV trajectories use OpenCV `plumb_bob` (`k1, k2, p1, p2, k3`). Wide-FOV trajectories are tagged `OPENCV_FISHEYE` in PIVOT, but only COLMAP’s own per-scene intrinsics carry a genuine 4-coefficient (`k1`-`k4`) fisheye fit; the measured/calibration-file version of the same physical lens has `p1 = p2 = 0` and is missing `k4` in PIVOT’s `scene_data.json` export (an upstream data-loss bug). The correct `k4` was recovered from PIVOT’s own raw per-camera calibration files and cross-checked against the other six shared intrinsic values before substitution.
- **Fisheye monotonicity guard.** PIVOT’s `k1`-`k4` fisheye fits are high-order (theta^8) polynomials that can become non-invertible within the lens’ own field of view. When that happens within a trajectory’s own computed half-diagonal FOV, `k4` is dropped to `0` (leaving a `k1`-`k3` fit) rather than shipping a calibration that a fisheye rectifier cannot invert; this is logged whenever it occurs.
- **Point-cloud outlier trimming.** Sparse-reconstruction points farther than 15 m from the point cloud’s centroid are dropped before logging (`n_points_kept` / `n_points_dropped` record the counts per sample).
- **In-pipeline geometric check.** For every episode, the median distance from measured camera centers to the point cloud’s centroid is asserted to be smaller than the scene’s own `aabb_diagonal`, as a sanity check on the coordinate conversion (median, not mean, to stay robust to isolated GPS glitches without silently dropping frames).
- **Scene- and trajectory-level statistics are copied verbatim** from PIVOT’s own `scene_data.json`, not recomputed.
- **Raw COLMAP intermediates were never read.** `PYCOLMAP_soft_prior/` (roughly 26 GB of feature-matching database and candidate sparse models per scene) was excluded from the source download and never touched during MCAP authoring; only PIVOT’s curated `scene_data.json` and `sparse_model.ply` per scene were used, since PIVOT itself already summarizes the winning reconstruction into those two files.
## Dataset Creation
### Curation Rationale
PIVOT was designed to separate several favorable assumptions that are typically bundled together in NeRF/3DGS benchmarks (offline-optimized poses, per-scene-optimized intrinsics, reconstruction-friendly trajectories, held-out views drawn from the same trajectory family used in training) by making trajectory identity, pose source, and intrinsic source independently selectable. This MCAP/FiftyOne re-encoding preserves that dual-pose, dual-intrinsic design while making each trajectory explorable as a single synchronized 3D episode, rather than a folder of independently posed frames with metadata attached.
### Source Data
#### Data Collection and Processing
PIVOT v1 covers five real-world scenes (Church, Village Street, Victorian Garden, Frontyard, Backyard), each captured with a single DJI Mini 4 Pro drone flown along 17-24 named trajectories covering reconstruction-friendly orbits, robot-like traversals, and extrapolation-oriented motion (bird’s-eye-view sweeps, vertical ascent, scattered viewpoints, 360-degree panoramas). PIVOT’s own raw-data pipeline (as described in the paper): read trajectory metadata and raw captures, sample video frames using translation/rotation thresholds (or use captured stills directly), extract EXIF/XMP metadata, compute measured poses from device position/orientation metadata, transform poses into the NED world frame and OpenGL camera convention, run COLMAP feature extraction and matching with measured positions injected as soft position priors, select the best reconstruction, retain optimized poses and intrinsics alongside measured poses and calibrated intrinsics, compute per-frame pose errors and trajectory/scene statistics, and compute a directed pose-Chamfer trajectory-distance matrix.
For this MCAP/FiftyOne re-encoding specifically: only PIVOT’s already-curated `scene_data.json` (poses, calibration, taxonomy, statistics) and `sparse_model.ply` (the winning sparse point cloud) per scene were read. One `.mcap` episode was authored per `(scene, trajectory)` pair (103 total) using `foxglove-sdk`, applying the coordinate conversions and edge-case handling listed under Parsing decisions above, then loaded into FiftyOne as one `fo.Sample` per episode.
#### Who are the source data producers?
Mary Raymond (independent researcher), who designed and executed the PIVOT capture, processing, and benchmark pipeline, including piloting the DJI Mini 4 Pro drone. Per the paper’s acknowledgements, drone captures took place at a site made accessible with support from Clare County Council and Bunratty Castle & Folk Park.
### Annotations
#### Annotation process
There is no separate human annotation process. Per-frame pose, pose-error, and calibration values are produced entirely by PIVOT’s capture and processing pipeline: measured poses come from the drone’s onboard GPS, flight attitude, and gimbal attitude sensors; COLMAP-optimized poses and per-scene intrinsics come from PYCOLMAP structure-from-motion using the measured positions as soft priors; physical/offline camera calibration comes from a checkerboard calibration procedure documented in PIVOT’s own toolchain.
#### Who are the annotators?
Not applicable — no human annotators. All pose and calibration values are either sensor-derived or algorithmically computed (COLMAP structure-from-motion), as described above.
#### Personal and Sensitive Information
[More Information Needed]. The scenes are outdoor drone captures at a real, named site (Bunratty Castle & Folk Park, per the paper’s acknowledgements) and could incidentally contain identifiable people, vehicles, or property; neither the paper nor the toolchain describes an anonymization process.
## Citation
**BibTeX:**
```bibtex
@article{raymond2026pivot,
title = {PIVOT: A Multi-Trajectory Dataset and Testbed for Pose, Intrinsics, and Novel Viewpoint Evaluation in Real-World 3D Reconstruction},
author = {Raymond, Mary},
journal = {arXiv preprint arXiv:2608.25401},
year = {2026}
}
```
**APA:**
Raymond, M. (2026). PIVOT: A multi-trajectory dataset and testbed for pose, intrinsics, and novel viewpoint evaluation in real-world 3D reconstruction. *arXiv preprint arXiv:2608.25401*.
## More Information
- Source dataset: [MaryRaymond/PIVOT](https://huggingface.co/datasets/MaryRaymond/PIVOT)
- PIVOT toolchain (processing, visualization, export, benchmarking): [github.com/maryraymond/PIVOT](https://github.com/maryraymond/PIVOT)
- Known issue carried over from the source dataset: `church/rocket_upward` has no COLMAP-registered frames, documented in the source dataset’s own card. This re-encoding preserves that by omitting the `world -> camera_colmap` transform and `/pose_error` topic entirely for that episode, rather than fabricating data.
- This is a derived, third-party re-encoding of PIVOT into MCAP/FiftyOne format, not an official PIVOT release. The source dataset and paper are the authoritative reference for PIVOT itself.
## Dataset Card Authors
[Harpreet Sahota](https://huggingface.co/harpreetsahota) (MCAP re-encoding and FiftyOne dataset card)