Note

This is a Hugging Face dataset. For large datasets, ensure huggingface_hub>=1.1.3 to avoid rate limits. Learn more in the Hugging Face integration docs.

Hugging Face

Dataset Card for HighwayScene#

image/png

Installation#

pip install -U fiftyone

Usage#

import fiftyone as fo
from fiftyone.utils.huggingface import load_from_hub

dataset = load_from_hub("Voxel51/HighwayScene")
session = fo.launch_app(dataset)

Dataset Details#

Dataset Description#

HighwayScene is a synchronized multi-LiDAR roadside dataset introduced for the cross-sensor background subtraction benchmark in “Beam-Wise Statistical Background Subtraction for Static Roadside LiDAR: A Cross-Sensor Benchmark Study” (ITSC 2026). Three statically mounted sensors with different sensing principles observe the same highway construction-site scene simultaneously:

Sensor

Principle

Rate

Ouster OS0

Rotating time-of-flight

10 Hz

Aeva Aeries II

FMCW

10 Hz

Blickfeld QB2

Solid-state time-of-flight

5 Hz

The scene features consistently high traffic volume under a legally enforced 40 km/h speed limit. The dataset supports research on background modeling, dynamic-point segmentation, and cross-sensor benchmarking for static roadside LiDAR.

This FiftyOne version converts the original protobuf .pb records to MCAP format so each episode can be explored interactively in the FiftyOne multimodal viewer, with all three sensor streams on a shared 20-second timeline.

  • Curated by: Alexander Baumann, Marcel Voßhans, Thao Dang — Institute for Intelligent Systems, Esslingen University of Applied Sciences, Germany

  • Funded by: [More Information Needed]

  • Shared by: Institute for Intelligent Systems (IIS), Esslingen University of Applied Sciences

  • Language(s): N/A

  • License: CC BY-NC-SA 4.0 (non-commercial use only)

Dataset Sources#

  • Repository: https://huggingface.co/datasets/iis-esslingen/HighwayScene (original protobuf records)

  • Paper: https://arxiv.org/abs/2608.14868

  • Demo: https://highwayscene.github.io/

Uses#

Direct Use#

  • Benchmarking beam-wise statistical background subtraction methods across heterogeneous LiDAR technologies

  • Cross-sensor evaluation of static/dynamic point-cloud segmentation

  • Research on roadside LiDAR perception: traffic monitoring, infrastructure-based sensing, inter-vehicle distance estimation

  • Multi-modal LiDAR data exploration via the FiftyOne App’s 3D tile and shared timeline

Out-of-Scope Use#

  • Moving-sensor or ego-vehicle applications — the benchmark assumes a statically mounted sensor with a fixed scan pattern

  • Commercial use — the CC BY-NC-SA 4.0 license prohibits commercial applications

  • Scenes with substantial long-term structural changes, changed sensor poses, or dynamically varying scan patterns

Dataset Structure#

FiftyOne Dataset Topology#

Media type: multimodal (MCAP files) Sample count: 30 episodes Splits: train (20), val (5), test (5), encoded as the split string field and queryable via dataset.match(fo.ViewField("split") == "test")

Each FiftyOne sample points to one .mcap file covering approximately 20 seconds of synchronized multi-sensor recording. The MCAP file contains three point-cloud topics (/ouster/points, /aeva/points, /blickfeld/points) and three static identity frame transforms on /tf_static. All topics use the foxglove.PointCloud schema and are rendered in the FiftyOne App’s 3D tile on a shared timeline.

Sample Fields#

Field

FiftyOne type

Description

filepath

StringField

Absolute path to the .mcap episode file

split

StringField

Benchmark split: "train", "val", or "test"

record_name

StringField

Original .pb filename stem (encodes recording date and global frame-ID range)

first_frame_id

IntField

First global frame ID in this episode (1-based, dataset-wide)

last_frame_id

IntField

Last global frame ID in this episode

duration_s

FloatField

Episode duration in seconds (~19.9 s for 200-frame episodes)

message_count

IntField

Total MCAP message count across all topics in this episode

ouster_frames

IntField

Number of Ouster OS0 point-cloud messages (10 Hz; 200 per full episode)

aeva_frames

IntField

Number of Aeva Aeries II point-cloud messages (10 Hz; 194–200 per episode)

blickfeld_frames

IntField

Number of Blickfeld QB2 point-cloud messages (5 Hz; 95–101 per episode)

topics

ListField(StringField)

MCAP topic names present in the episode

schemas

ListField(StringField)

MCAP schema names used (foxglove.PointCloud, foxglove.FrameTransform)

has_pointcloud

BooleanField

True for all 30 episodes

has_image

BooleanField

False — no camera data in this dataset

has_gps

BooleanField

False — no GPS data

has_imu

BooleanField

False — no IMU data

has_logs

BooleanField

False — no log messages

MCAP Topic Structure (per episode)#

Topic

Schema

Rate

Per-point fields

/ouster/points

foxglove.PointCloud

10 Hz

x, y, z (float32); intensity, ambient, reflectivity (float32); channel_id (float32)

/aeva/points

foxglove.PointCloud

10 Hz

x, y, z (float32); intensity, ambient, reflectivity, velocity (float32, radial m/s)

/blickfeld/points

foxglove.PointCloud

5 Hz

x, y, z (float32); reflectivity, channel_id, horizontal_id (float32)

/tf_static

foxglove.FrameTransform

static

Identity transforms for ouster, aeva, blickfeld frames relative to world

Typical point counts per frame: Ouster ~45K, Aeva ~92K, Blickfeld ~12K. Blickfeld has no intensity or ambient channel. Aeva’s velocity field (radial velocity in m/s) is the unique FMCW capability enabling velocity-based dynamic-point labeling.

Ground Truth#

No per-frame label files are stored in the FiftyOne dataset. The benchmark computes static/dynamic labels at evaluation time from parameters in metadata/ground_truth.yaml (in the original repository):

  • Ouster and Blickfeld: lane-aligned 3D bounding-box ROIs define dynamic regions; any point inside is labeled dynamic.

  • Aeva: a point is labeled dynamic if its measured radial velocity exceeds 1 m/s.

Parsing Decisions#

  • Original records are protobuf .pb files decoded with highwayscene-proto==1.0.0. Each .pb → one .mcap using foxglove-sdk==0.26.0.

  • All uint16 auxiliary channels (intensity, ambient, reflectivity, channel_id, horizontal_id) are cast to float32 in the MCAP point-cloud buffers.

  • Points with range ≤ 0.5 m are filtered (zero-return guard).

  • Static transforms use foxglove.FrameTransform with no embedded timestamp, placing them in the viewer’s static store for the full episode duration.

  • No extrinsic sensor-to-sensor calibration is provided by the dataset authors; all world → sensor transforms are identity.

  • Blickfeld operates at 5 Hz: frames where the sensor is absent are simply omitted from the /blickfeld/points topic.

Dataset Creation#

Curation Rationale#

Existing roadside LiDAR datasets focus on object detection or semantic segmentation with class-based labels, and are typically limited to a single sensor or scene. HighwayScene was created to fill the gap in systematic cross-sensor evaluation for static background subtraction — providing simultaneous recordings from three heterogeneous LiDAR technologies under real highway traffic conditions, with reproducible annotation strategies that do not require manual point-wise labeling.

Source Data#

Data Collection and Processing#

Data was recorded at a German highway construction site with a legally enforced 40 km/h speed limit. Sensors were mounted statically on roadside infrastructure. Recording rate is 10 Hz for the Ouster and Aeva sensors and 5 Hz for the Blickfeld. All streams are temporally synchronized. 5,998 global frames are captured across the full recording session (Feb 9, 2026), partitioned into train (frames 1–4000), val (4001–5000), and test (5001–5998) splits using a strict non-overlapping protocol.

Who are the source data producers?#

Institute for Intelligent Systems (IIS), Esslingen University of Applied Sciences, Germany. Contact: alexander.baumann@hs-esslingen.de.

Annotations#

Annotation process#

Static/dynamic ground-truth labels are not stored per frame in the dataset records. Instead, two reproducible annotation strategies are defined in metadata/ground_truth.yaml:

  1. Geometry-based (Ouster, Blickfeld): For each traffic lane, a static 3D bounding-box ROI aligned with the roadway is manually defined. Under empty-scene conditions no LiDAR returns fall inside these volumes. Since vehicles are constrained by concrete barriers, any return inside a lane volume originates from dynamic traffic. Points inside are labeled dynamic; all others static.

  2. Velocity-based (Aeva): A point is labeled dynamic if its measured FMCW radial velocity exceeds 1 m/s. This strategy requires no scene geometry and illustrates the FMCW sensor’s self-annotating capability.

Who are the annotators?#

Annotation parameters (lane ROIs, ground planes) were defined by the dataset authors. No manual point-wise labeling was performed — labels are reproducibly derived from geometry and physics.

Personal and Sensitive Information#

LiDAR point clouds do not record visually identifiable information (faces, license plates). No personal data is present in this dataset.

Citation#

BibTeX:

@inproceedings{baumann2026beamwise,
  author    = {Alexander Baumann and Marcel Vo{\ss}hans and Thao Dang},
  title     = {Beam-Wise Statistical Background Subtraction for Static
               Roadside {LiDAR}: A Cross-Sensor Benchmark Study},
  booktitle = {IEEE International Conference on Intelligent Transportation
               Systems (ITSC)},
  year      = {2026}
}

APA:

Baumann, A., Voßhans, M., & Dang, T. (2026). Beam-Wise Statistical Background Subtraction for Static Roadside LiDAR: A Cross-Sensor Benchmark Study. IEEE International Conference on Intelligent Transportation Systems (ITSC).

More Information#

  • Original dataset (protobuf format): https://huggingface.co/datasets/iis-esslingen/HighwayScene

  • Background subtraction code: https://github.com/HighwayScene/roadside-lidar-background-subtraction

  • Protobuf reader: https://github.com/HighwayScene/highwayscene-proto

  • Project page: https://highwayscene.github.io/

Dataset Card Authors#

Harpreet Sahota

Dataset Card Contact#

harpreetsahota07@gmail.com