#### NOTE This is a **Hugging Face dataset**. For large datasets, ensure `huggingface_hub>=1.1.3` to avoid rate limits. Learn more in the Hugging Face integration docs. ![Hugging Face](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Dataset-yellow) # Dataset Card for Dur360BEV-Extended ![preview](https://huggingface.co/datasets/Voxel51/Dur360BEV-Extended/resolve/main/dur360.gif) Dur360BEV-Extended is a real-world autonomous driving dataset extending the original Dur360BEV benchmark with 15,400 additional synchronized frames. Each frame pairs a 360° dual-fisheye RGB image (Ricoh Theta S, 1280×720), a dense 128-channel LiDAR point cloud (Ouster OS1-128), and RTK-corrected GPS/IMU (OxTS RT3000v3). Frames are annotated with 3D bounding boxes for vehicles, pedestrians, and bicycles in a 100 m × 100 m area centered on the ego vehicle. In FiftyOne the dataset is represented as 15 multimodal MCAP episodes (~102 s each). Each episode carries image, LiDAR, GPS, 3D box wireframes (both image overlay and 3D scene), and ego-pose streams that play back in the FiftyOne multimodal viewer. ## Installation ```bash pip install -U fiftyone ``` ## Usage ```python import fiftyone as fo from fiftyone.utils.huggingface import load_from_hub # Load the dataset dataset = load_from_hub("Voxel51/Dur360BEV-Extended") # Launch the App session = fo.launch_app(dataset) ``` ## Dataset Details ### Dataset Description - **Curated by:** Wenke E, Chao Yuan, Li Li, Yixin Sun, Yona Falinie A. Gaus, Amir Atapour-Abarghouei, Toby P. Breckon (Durham University, UK) - **Funded by:** [More Information Needed] - **Shared by:** Durham University Computer Science and Engineering - **Language(s):** English - **License:** CC-BY-4.0 ### Dataset Sources - **Repository:** https://github.com/Tom-E-Durham/Dur360BEV - **Paper:** https://arxiv.org/abs/2503.00675 (ICRA 2025) - **Demo:** [More Information Needed] ## Uses ### Direct Use Training and evaluating Bird’s-Eye View (BEV) map generation models that consume single spherical camera imagery. The 3D bounding box annotations support vehicle, pedestrian, and bicycle detection research. The GPS/IMU data supports ego-motion estimation. The dataset is intended to be used alongside the original Dur360BEV dataset for combined training. ### Out-of-Scope Use The dataset was collected in Durham, UK in August 2024 and reflects traffic conditions and road layouts specific to that geography and time. Applications requiring geographic or temporal generalization should account for this scope. ## Dataset Structure ### Topology The FiftyOne dataset contains **15 samples**, one per MCAP episode. `media_type = multimodal`. Each episode covers approximately 102 seconds at 10 Hz (~1,027 frames). The 15,400 raw frames are divided 90/10 into train and test by the official split indices (`dataset_ext_indices.pkl`): 13,866 train frames and 1,534 test frames. Each sample’s `split` field holds the dominant split for that episode. ### Sample Fields | Field | FiftyOne type | Description | |-------------------|--------------------------|-----------------------------------------------------------------------------------------------------------------------------------| | `filepath` | `StringField` | Absolute path to the episode’s `.mcap` file | | `episode_num` | `IntField` | Episode number 1–15 | | `frame_count` | `IntField` | Number of frames in this episode (~1,027) | | `first_frame` | `StringField` | 10-digit stem of the first raw frame (e.g. `1000000042`) | | `last_frame` | `StringField` | 10-digit stem of the last raw frame | | `split` | `StringField` | Dominant official split for the episode (`"train"` or `"test"`) | | `train_frames` | `IntField` | Count of train-split frames within this episode | | `test_frames` | `IntField` | Count of test-split frames within this episode | | `duration_s` | `FloatField` | Episode wall-clock duration in seconds | | `message_count` | `IntField` | Total MCAP messages across all topics | | `channel_count` | `IntField` | Number of MCAP channels | | `topics` | `ListField(StringField)` | MCAP topic names present in this episode | | `schemas` | `ListField(StringField)` | Foxglove schema names present in this episode | | `has_image` | `BooleanField` | True — episode contains `foxglove.CompressedImage` | | `has_pointcloud` | `BooleanField` | True — episode contains `foxglove.PointCloud` | | `has_gps` | `BooleanField` | True — episode contains `foxglove.LocationFix` | | `has_annotations` | `BooleanField` | True when `/annotations/objects` topic is present (frames without objects in the annotation area still log an empty scene update) | | `dataset_version` | `StringField` | `"extended"` — distinguishes from the original Dur360BEV split | | `sensor_camera` | `StringField` | Camera model string: `"Ricoh Theta S (dual-fisheye, 1280×720)"` | | `sensor_lidar` | `StringField` | LiDAR model string: `"Ouster OS1-128 (128-channel, 10 Hz)"` | | `sensor_gnss` | `StringField` | GNSS model string: `"OxTS RT3000v3 (RTK-corrected)"` | ### MCAP Topics per Episode Each `.mcap` file contains the following synchronized streams: | Topic | Foxglove schema | Content | |------------------------|------------------------------|-------------------------------------------------------------------------------------------------------------------------| | `/camera/image_raw` | `foxglove.CompressedImage` | JPEG-compressed dual-fisheye image, 1280×720, frame `camera` | | `/camera/annotations` | `foxglove.ImageAnnotations` | 3D box wireframes projected onto the dual-fisheye image (LineList, per-class colors) | | `/camera/camera_info` | `foxglove.CameraCalibration` | Approximate Kannala-Brandt calibration for the front fisheye half (f ≈ 181 px, D = [0,0,0,0]) | | `/lidar/points` | `foxglove.PointCloud` | Float32 point cloud with fields x, y, z, reflectivity (4 × 4 bytes/point, stride 16); frame `lidar` | | `/gps` | `foxglove.LocationFix` | RTK-corrected latitude, longitude, altitude | | `/tf_static` | `foxglove.FrameTransform` | Static transforms: `base_link → lidar` and `lidar → camera` (identity, no measured extrinsic) | | `/annotations/objects` | JSON | Raw 3D bounding boxes: list of `[cx, cy, cz, length_m, width_m, height_m, yaw_rad]` plus `class_names` | | `/annotations/scene` | `foxglove.SceneUpdate` | 3D wireframe entities in the `lidar` frame for the 3D viewer; one `SceneEntity` per box with `LinePrimitive (LineList)` | | `/ego_pose` | JSON | Ego attitude: `roll_rad`, `pitch_rad`, `yaw_rad` from OxTS | ### Annotation Details **3D bounding boxes** are stored in the LiDAR coordinate frame (x = forward, y = left, z = up). The raw label format is `class dz dy dx cx cy cz yaw`, where `dz` = height, `dy` = width, `dx` = length in meters. Yaw is in radians. Boxes are annotated within a 100 m × 100 m area centered on the ego vehicle using a combination of automated LiDAR detection (∼60%) and manual review on the Xtreme1 platform. | Class | Box count (Extended split) | |------------|------------------------------| | Car | 62,927 | | Pedestrian | 5,813 | | Bicycle | 228 | 651 of 15,400 frames (4.2%) have no annotated objects; these are included as valid frames with empty annotation streams. **Parsing decisions:** - LiDAR `.bin` files use Ouster float32 format with 9 fields (x, y, z, intensity, time, reflectivity, ring, ambient, range). Only x, y, z, and reflectivity are written to the MCAP `PointCloud` to keep file sizes manageable. - Images are stored as raw 1280×720 PNG in source. The official loader crops the bottom 80 rows (`img[:640]`); this FiftyOne import preserves the full 720-row image. The effective dual-fisheye region occupies the top 640 rows. - The `CameraCalibration` at `/camera/camera_info` is an approximation: a Kannala-Brandt model fit to the front fisheye half only (cx ≈ 960, cy = 320, f ≈ 181 px, D = [0,0,0,0]). No measured LiDAR–camera extrinsic is provided in this dataset; the `lidar → camera` transform is an identity. - The `Bicycle` class is present in the raw label files but ignored by the official dataset loader. This FiftyOne import includes all 228 bicycle boxes. - The 15 episodes are contiguous temporal splits of the full 15,400-frame recording, divided evenly by `numpy.array_split`. ## Dataset Creation ### Curation Rationale Dur360BEV-Extended was created to expand the scale and diversity of the Dur360BEV benchmark (ICRA 2025). The original dataset introduced the first autonomous driving benchmark targeting BEV map generation from a single spherical camera. The extended split adds 15,400 further frames from the same sensor rig and collection sites to enrich variety of traffic conditions and driving scenarios for model training and evaluation. ### Source Data #### Data Collection and Processing Data was collected in Durham, UK across four distinct driving environments: campus, highway, city center, and residential neighborhoods. Collection used a Renault Twizy equipped with a Ricoh Theta S spherical camera, Ouster OS1-128 LiDAR, and OxTS RT3000v3 GNSS/INS. The camera captures at 15 Hz; the LiDAR at 10 Hz. Temporal synchronization was achieved via ROS Noetic with a 30 ms slop window, yielding synchronized 10 Hz frames. Files are named with a `1000xxxxxx` prefix to distinguish the extended split from the original `0000xxxxxx` frames. Collection took place in August 2024. #### Who are the source data producers? Durham University researchers in the Department of Computer Science and the Department of Engineering. ### Annotations #### Annotation process 3D bounding boxes were labeled on the Xtreme1 open-source annotation platform. An integrated LiDAR object detection model provided initial annotations for approximately 60% of boxes. An experienced annotator manually reviewed and completed the remaining boxes within the 100 m × 100 m annotation area. #### Who are the annotators? Durham University research team. #### Personal and Sensitive Information The dataset contains GPS coordinates corresponding to roads in Durham, UK. No personally identifiable pedestrian imagery is retained beyond what is incidentally captured in a public driving dataset. ## Citation **BibTeX:** ```bibtex @article{wenke2025dur360bev, title={Dur360BEV: A Real-world 360-degree Single Camera Dataset and Benchmark for Bird-Eye View Mapping in Autonomous Driving}, author={Wenke E and Chao Yuan and Li Li and Yixin Sun and Yona Falinie A. Gaus and Amir Atapour-Abarghouei and Toby P. Breckon}, journal={arXiv preprint arXiv:2503.00675}, year={2025} } ``` **APA:** E, W., Yuan, C., Li, L., Sun, Y., Gaus, Y. F. A., Atapour-Abarghouei, A., & Breckon, T. P. (2025). Dur360BEV: A Real-world 360-degree Single Camera Dataset and Benchmark for Bird-Eye View Mapping in Autonomous Driving. *arXiv preprint arXiv:2503.00675*. ## More Information The original Dur360BEV dataset (initial split, `0000xxxxxx` frames) is available at https://huggingface.co/datasets/TomEeee/Dur360BEV. The two splits can be combined for fuller training coverage. The official benchmark code is at https://github.com/Tom-E-Durham/Dur360BEV. ## Dataset Card Authors [Harpreet Sahota](https://huggingface.co/harpreetsahota)