FieldSAFE – Dataset for Obstacle Detection in Agriculture

The FieldSAFE dataset is a novel multi-modal dataset for obstacle detection in agriculture. It comprises 2 hours of raw sensor data from a tractor-mounted sensor system in a grass mowing scenario in Denmark, October 2016.


 

Sensing modalities include stereo camera, thermal camera, web camera, 360-degree camera, lidar, and radar, while precise localization is available from fused IMU and GNSS. Both static and moving obstacles are present including humans, mannequin dolls, rocks, barrels, buildings, vehicles, and vegetation. All obstacles have ground truth object labels and geographic coordinates.

Sensor Model Resolution FOV / ° Range / m Datarate / fps
Stereo camera Multisense S21
CMV2000
1024 x 544 85 x 50 1.5-50 10
Web camera Logitech HD Pro C920 1920 x 1080 70 x 43 n/a 20
360 camera Giroptic 360cam 2048 x 833 360 x 292 n/a 30
Thermal camera Flir A65, 13 mm lens 640 x 512 45 x 37 n/a 30
Lidar Velodyne HDL-32E 2172 x 32 360 x 40 1-100 10
Radar Delphi ESR 16 targets/frame
16 targets/frame
90 x 4.2
20 x 4.2
0-60
0-174
20
20

Usage

The entire dataset was recorded using the Robot Operating System (ROS) and is therefore contained in rosbags.
For detailed installation instructions and examples for how to use the dataset, see https://github.com/mikkelkh/FieldSAFE

In the ROS Details section below, ROS topics and data types for all sensors are listed along with a ROS transform tree.

Download

Filename Size / GB Duration / mm:ss Map Description
static_map_rgb.png 0.5 n/a Static RGB orthophoto of the field.
static_ground_truth.png 0.002 n/a Static ground truth map the field.
2016-10-25-11-41-21_example.bag 3.6 01:00 1 minute example bag. Extract from sequence with dynamic obstacles.
2016-10-25-11-09-42.bag 73.9 17:25 Static obstacle session #1. 3 laps around the grass field with several static obstacle along the way.
2016-10-25-11-34-25.bag 26.6 06:00 Repositioning of the tractor. At the end, the setup is ready for recording dynamic obstacles.
2016-10-25-11-41-21.bag 91.3 21:54 Dynamic obstacle session #1. Humans move in random patterns, crossing the path of the tractor.
2016-10-25-12-07-22.bag 95.2 22:17 Dynamic obstacle session #2. Humans and a mini digger move in random patterns, crossing the path of the tractor.
2016-10-25-12-37-57.bag 112.2 26:05 Static obstacle session #2. The rest of the field is traversed and mowed, while passing a number of static obstacles.

Ground Truth

Ground truth information on object location and class labels for both static and moving obstacles is available as timestamped global (geographic) coordinates.
By transforming local sensor data from the tractor into global coordinates, a simple look-up of class label into the annotated ground truth map is possible.

Static Obstacles


Orthophoto with tractor tracks overlaid. Black tracks include only static obstacles, whereas red and white tracks also have moving obstacles. Currently, red tracks have no ground truth for moving obstacles annotated.

Labeled orthophoto.

Dynamic Obstacles


Path of tractor and moving obstacles (humans) overlaid on the static ground truth map.

Citation

If you use this dataset in your research or elsewhere, please cite/reference the following paper:
FieldSAFE: Dataset for Obstacle Detection in Agriculture

@article{kragh2017fieldsafe,
AUTHOR = {Kragh, Mikkel Fly and Christiansen, Peter and Laursen, Morten Stigaard and Larsen, Morten and Steen, Kim Arild and Green, Ole and Karstoft, Henrik and Jørgensen, Rasmus Nyholm},
TITLE = {FieldSAFE: Dataset for Obstacle Detection in Agriculture},
JOURNAL = {Sensors},
VOLUME = {17},
YEAR = {2017},
NUMBER = {11},
ARTICLE NUMBER = {2579},
URL = {http://www.mdpi.com/1424-8220/17/11/2579},
ISSN = {1424-8220},
ABSTRACT = {In this paper, we present a multi-modal dataset for obstacle detection in agriculture. The dataset comprises approximately 2 h of raw sensor data from a tractor-mounted sensor system in a grass mowing scenario in Denmark, October 2016. Sensing modalities include stereo camera, thermal camera, web camera, 360     ∘     camera, LiDAR and radar, while precise localization is available from fused IMU and GNSS. Both static and moving obstacles are present, including humans, mannequin dolls, rocks, barrels, buildings, vehicles and vegetation. All obstacles have ground truth object labels and geographic coordinates.},
DOI = {10.3390/s17112579}
}

ROS Details

Below, ROS-specific details are given for which transforms and topics are available.
Extraction of the estimated extrinsic parameters for all sensors is described in the Transforms section.
Intrinsic camera parameters are described in the Camera Calibration section.
All available topics for each sensor are described in the Topics section.
More information and example code is available in the git repository.

Transforms

Below, a screenshot from rviz illustrates the sensor platform and sensor coordinate frames in ROS.
A full TF (transform) tree is available here.

To extract specific extrinsic parameters, the tf_echo package in ROS can be used.
Below, an example is given for finding translation and rotation between the lidar (velodyne) and the stereo camera (cam_stereo_left_frame).
The example requires the provided demo to be run in another terminal at the same time as described in the git repository.

rosrun tf tf_echo velodyne cam_stereo_left_frame
At time 0.000
- Translation: [0.012, 0.427, -0.295]
- Rotation: in Quaternion [0.564, -0.539, 0.428, -0.457]
            in RPY (radian) [-1.788, 0.010, -1.517]
            in RPY (degree) [-102.468, 0.590, -86.895]

Camera Calibration

The stereo and thermal cameras have been registered and calibrated using the camera calibration method available in the Computer Vision System Toolbox in MATLAB.
The intrinsic camera parameters are available as MATLAB mat files for the stereo camera and the thermal camera.
Details for how to show reprojection errors and visualize extrinsic parameters are available from the MathWorks Documentation.

Topics

The recorded rosbags include a number of different topics for each sensor.
The table below lists all the topics and their message types in ROS.

Sensor Topics Message type Description
Stereo camera /Multisense/depth sensor_msgs/Image Depth image
/Multisense/imu/accelerometer_vector geometry_msgs/Vector3Stamped Accelerometer data
/Multisense/imu/gyroscope_vector geometry_msgs/Vector3Stamped Gyroscope data
/Multisense/imu/imu_data sensor_msgs/Imu Packed accelerometer and gyroscope data
/Multisense/imu/magnetometer_vector geometry_msgs/Vector3Stamped Magnetometer data
/Multisense/left/camera_info sensor_msgs/CameraInfo Intrinsic calibration data (factory settings)
/Multisense/left/image_rect_color sensor_msgs/Image Rectified left color image
/Multisense/left/image_rect_color/camera_info sensor_msgs/CameraInfo Intrinsic calibration data (copy of above)
/Multisense/pps std_msgs/Time Pulse per second signal
/Multisense/right/camera_info sensor_msgs/CameraInfo Intrinsic calibration data (factory settings)
/Multisense/right/image_rect sensor_msgs/Image Right grayscale image
/Multisense/right/image_rect/camera_info sensor_msgs/CameraInfo Intrinsic calibration data (copy of above)
/Multisense/stamped_pps multisense_ros/StampedPps Timestamped pulse per second signal
/Multisense/status multisense_ros/DeviceStatus Device status message
Web camera /Logitech_webcam/camera_info sensor_msgs/CameraInfo Intrinsic calibration data
/Logitech_webcam/image_raw/compressed sensor_msgs/CompressedImage Compressed color image
360 camera /giroptic_360/image/compressed sensor_msgs/CompressedImage Compressed color image
Thermal camera /FlirA65/camera_info sensor_msgs/CameraInfo Intrinsic calibration data (do not use)
/FlirA65/image_raw sensor_msgs/Image Grayscale image
Lidar /velodyne_packets velodyne_msgs/VelodyneScan Raw laser data packets. Use provided software to generate 3D point clouds.
Radar /fmData/can_rx_radar msgs/can Raw CAN messages
/Delphi_ESR/RadarData htf_delphi_esr/Delphi_radar Radar detections parsed from CAN messages
GPS /fmData/gps_rx msgs/serial Raw serial messages from GPS. Use provided software to generate sensor_msgs/NavSatFix messages.
IMU /fmData/imu_rx msgs/serial Raw serial messages from IMU. Use provided software to generate sensor_msgs/Imu messages.
/fmData/imu_tx msgs/serial Raw serial message requests sent to the IMU.
System /diagnostics diagnostic_msgs/DiagnosticArray Diagnostics
/tf tf/tfMessage Outdated transform messages. Must be ignored when replaying bag. Use provided software to generate updated transforms.

Note that intrinsic calibration data for cameras are included directly in ROS messages.
These values, however, are factory defaults and not the ones estimated during camera calibration.
For the calibrated values, see the Calibration section, above.
Below, an example is given for extracting the intrinsic parameters for the left image of the stereo camera.
The example requires the provided demo to be run in another terminal at the same time as described in the git repository.

rostopic echo /Multisense/left/camera_info 
header: 
  seq: 21419
  stamp: 
    secs: 1477388586
    nsecs: 798077000
  frame_id: /Multisense/left_camera_optical_frame
height: 544
width: 1024
distortion_model: plumb_bob
D: [0.0030753163155168295, 0.002497022273018956, 0.0003005412872880697, 0.001575434347614646, -0.003454494522884488, 0.0, 0.0, 0.0]
K: [555.9204711914062, 0.0, 498.1905517578125, 0.0, 556.6275634765625, 252.35089111328125, 0.0, 0.0, 1.0]
R: [0.9999634027481079, -0.000500216381624341, 0.00853759702295065, 0.0005011018947698176, 0.9999998807907104, -0.00010158627264900133, -0.00853754486888647, 0.00010586075950413942, 0.9999635219573975]
P: [580.6427001953125, 0.0, 512.0, 0.0, 0.0, 580.6427001953125, 254.5, 0.0, 0.0, 0.0, 1.0, 0.0]
binning_x: 0
binning_y: 0
roi: 
  x_offset: 0
  y_offset: 0
  height: 0
  width: 0
  do_rectify: False
---

Contact

For questions, contact Mikkel Fly Kragh at mkha@eng.au.dk