The data collection efficiency is slow

[huangyan28]

For a particular task, if we use the default configuration, it takes about 50 hours for our device to collect one piece of data. May I ask how much data training is good for a specific task, or how can we modify it to improve the efficiency of data collection.

[zxzm-zak]

hi！The slow speed of data collection is mainly because the resolution of the images collected in our basic settings is too large. If you want to save time, please reduce the image resolution in the launch file of the usb camera. And Delete
compression='gzip',
compression_opts=4
from
image_grp.create_dataset(
cam_name,
data=np.array(data_dict[f'/observations/images/{cam_name}'], dtype=np.uint8),
compression='gzip',
compression_opts=4
)

[huangyan28]

What is the resolution of the data and the average amount of data required for each task when you produce the data

[huangyan28]

I have modified the resolution in the launch file, but the recorded video appears obvious double shadow, and the video cannot be generated at the same time. May I ask if I need to adjust the Settings of Gopro, or do I need to make other changes

[AlexLiu-bit]

I have modified the resolution in the launch file, but the recorded video appears obvious double shadow, and the video cannot be generated at the same time. May I ask if I need to adjust the Settings of Gopro, or do I need to make other changes

In fact, the most time-consuming process is writing data into the hdf5 file. To improve the speed of data collection, we suggest saving the data in other formats first, such as mp4 and csv, and then converting it to hdf5 later. We have tested this on an old computer, and the speed is quite good.

Additionally, the resolution of our data images is 1920x1080. If a lower resolution is required, we recommend saving the images at a resolution of 640x480 or higher.

[huangyan28]

I have modified the resolution in the launch file, but the recorded video appears obvious double shadow, and the video cannot be generated at the same time. May I ask if I need to adjust the Settings of Gopro, or do I need to make other changes

In fact, the most time-consuming process is writing data into the hdf5 file. To improve the speed of data collection, we suggest saving the data in other formats first, such as mp4 and csv, and then converting it to hdf5 later. We have tested this on an old computer, and the speed is quite good.

Additionally, the resolution of our data images is 1920x1080. If a lower resolution is required, we recommend saving the images at a resolution of 640x480 or higher.

Thank you！My issue got solved

[buduz]

I have modified the resolution in the launch file, but the recorded video appears obvious double shadow, and the video cannot be generated at the same time. May I ask if I need to adjust the Settings of Gopro, or do I need to make other changes

In fact, the most time-consuming process is writing data into the hdf5 file. To improve the speed of data collection, we suggest saving the data in other formats first, such as mp4 and csv, and then converting it to hdf5 later. We have tested this on an old computer, and the speed is quite good.
Additionally, the resolution of our data images is 1920x1080. If a lower resolution is required, we recommend saving the images at a resolution of 640x480 or higher.

Thank you！My issue got solved

Hello, I also encountered this double shadow issue when capturing images with the GoPro, but I didn’t make any modifications. May I ask how you resolved this problem?