L3Pilot project open-sourced their log file format

L3Pilot project open-sourced their log file format

24 January 2020

The European research project L3Pilot has a focus on large-scale piloting of SAE Level 3 functions. Its 34 partners are currently running tests and collecting data in 10 countries. All log data coming from test vehicles is converted into a Common Data Format (CDF), a file format that the L3Pilot project created and now promotes for open collaboration. The format has recently been made available at https://github.com/l3pilot/l3pilot-cdf.

The CDF defines a detailed structure for storing sensor data one trip at a time. Its signal list and definitions are based on lessons learned from many past field test projects. The signal list caters for various impact assessment areas, mainly: driver behaviour, user experience, mobility, safety, efficiency, environment, and socio-economics. Still, it’s a prioritized selection and offers a manageable file size.

The CDF builds on HDF5, a hierarchical data format developed by the HDF Group. Using HDF5 as a basis ensures that a variety of data tools and programming languages can readily interface the files. HDF5 also offers efficient compression and metadata features.

The CDF has already enabled joint software development of analysis scripts in L3Pilot, including indicator calculation, event and driving situation detection, and support for video annotation. The format and the harmonized indicators will be used to evaluate 13 OEM datasets. The L3Pilot CDF will be added soon to the CAD Knowledge Base.