cyber_record
cyber_record is a cyber record file offline parse tool. You can use cyber_record to read messages from record file, or write messages to the record file.
Quick start
First install “cyber_record” by the following command.
1pip3 install cyber_record
2// or update version
3pip3 install cyber_record -U
Command line mode
You can easily get the information in the record file by the following command.
Info
cyber_record info will output the statistics of the record file.
1$ cyber_record info -f example.record.00000
2
3record_file: example.record.00000
4version: 1.0
5begin_time: 2021-07-23 17:12:15.114944
6end_time: 2021-07-23 17:12:15.253911
7duration: 0.14 s
8size: 477.55 KByte
9message_number: 34
10channel_number: 8
11
12/apollo/planning , apollo.planning.ADCTrajectory , 1
13/apollo/routing_request , apollo.routing.RoutingRequest , 0
14/apollo/monitor , apollo.common.monitor.MonitorMessage , 0
15/apollo/routing_response , apollo.routing.RoutingResponse , 0
16/apollo/routing_response_history , apollo.routing.RoutingResponse , 1
17/apollo/localization/pose , apollo.localization.LocalizationEstimate, 15
18/apollo/canbus/chassis , apollo.canbus.Chassis , 15
19/apollo/prediction , apollo.prediction.PredictionObstacles , 2
Echo
cyber_record echo will print the message of the specified topic to the terminal.
1$ cyber_record echo -f example.record.00000 -t /apollo/canbus/chassis
2
3engine_started: true
4speed_mps: 0.0
5throttle_percentage: 0.0
6brake_percentage: 0.0
7driving_mode: COMPLETE_AUTO_DRIVE
8gear_location: GEAR_DRIVE
9header {
10 timestamp_sec: 1627031535.112813
11 module_name: "SimControl"
12 sequence_num: 76636
13}
Or you can reference cyber_record by
1from cyber_record.record import Record
Examples
Below are some examples to help you read and write messages from record files.
Read messages
You can read messages directly from the record file in the following ways.
1from cyber_record.record import Record
2
3file_name = "20210521122747.record.00000"
4record = Record(file_name)
5for topic, message, t in record.read_messages():
6 print("{}, {}, {}".format(topic, type(message), t))
The following is the output log of the program
/apollo/localization/pose, <class 'LocalizationEstimate'>, 1627031535246897752
/apollo/canbus/chassis, <class 'Chassis'>, 1627031535246913234
/apollo/canbus/chassis, <class 'Chassis'>, 1627031535253680838
Filter Read
You can also read messages filtered by topics and time. This will improve the speed of parsing messages.
1def read_filter_by_both():
2 record = Record(file_name)
3 for topic, message, t in record.read_messages('/apollo/canbus/chassis', \
4 start_time=1627031535164278940, end_time=1627031535215164773):
5 print("{}, {}, {}".format(topic, type(message), t))
Parse messages
To avoid introducing too many dependencies, you can save messages by record_msg.
1pip3 install record_msg -U
record_msg provides 3 types of interfaces
csv format
you can use to_csv to format objects so that they can be easily saved in csv format.
1f = open("message.csv", 'w')
2writer = csv.writer(f)
3
4def parse_pose(pose):
5 '''
6 save pose to csv file
7 '''
8 line = to_csv([pose.header.timestamp_sec, pose.pose])
9 writer.writerow(line)
10
11f.close()
image
you can use ImageParser to parse and save images.
1image_parser = ImageParser(output_path='../test')
2for topic, message, t in record.read_messages():
3 if topic == "/apollo/sensor/camera/front_6mm/image":
4 image_parser.parse(message)
5 # or use timestamp as image file name
6 # image_parser.parse(image, t)
lidar
you can use PointCloudParser to parse and save pointclouds.
1pointcloud_parser = PointCloudParser('../test')
2for topic, message, t in record.read_messages():
3 if topic == "/apollo/sensor/lidar32/compensator/PointCloud2":
4 pointcloud_parser.parse(message)
5 # other modes, default is 'ascii'
6 # pointcloud_parser.parse(message, mode='binary')
7 # pointcloud_parser.parse(message, mode='binary_compressed')
Write messages
You can now also build record by messages. You can write pb_message by record.write.
1def write_message():
2 pb_map = map_pb2.Map()
3 pb_map.header.version = 'hello'.encode()
4
5 with Record(write_file_name, mode='w') as record:
6 record.write('/apollo/map', pb_map, int(time.time() * 1e9))
Its application scenario is to convert dataset into record files.
Note
Please note that it must be written in chronological order.
If you want to write raw message, you should first use Builder to help convert raw data to pb_message.
image
You can write image to record file like below. ImageBuilder will help you convert image to pb_image. encoding should be rgb8,`bgr8` or gray, y.
1def write_image():
2 image_builder = ImageBuilder()
3 write_file_name = "example_w.record.00002"
4 with Record(write_file_name, mode='w') as record:
5 img_path = 'test.jpg'
6 pb_image = image_builder.build(img_path, encoding='rgb8')
7 record.write('/apollo/sensor/camera/front_6mm/image',
8 pb_image,
9 int(time.time() * 1e9))
lidar
You can write image to record file like below. PointCloudBuilder will help you convert pcd file to pb_point_cloud.
1def write_point_cloud():
2 point_cloud_builder = PointCloudBuilder()
3 write_file_name = "example_w.record.00003"
4 with Record(write_file_name, mode='w') as record:
5 pcd_path = 'test.pcd'
6 pb_point_cloud = point_cloud_builder.build(pcd_path)
7 record.write('/apollo/sensor/lidar32/compensator/PointCloud2',
8 pb_point_cloud,
9 int(time.time() * 1e9))