Image not rendered correctly #51

yoann-solana · 2018-09-03T13:04:55Z

Hi,
I would like to test a USB camera on ROS. I have followed the procedure to setup libuvc but the image on the raw topic is not rendered well.

This is a capture of the image on the topic /camera/image_raw, as you will see the image seems to be duplicated.

This is the output of the v4l2-ctl --list-formats-ext command:

ioctl: VIDIOC_ENUM_FMT
	Index       : 0
	Type        : Video Capture
	Pixel Format: 'UYVY'
	Name        : UYVY 4:2:2
		Size: Discrete 640x480
			Interval: Discrete 0.017s (60.000 fps)
			Interval: Discrete 0.022s (45.000 fps)
		Size: Discrete 960x540
			Interval: Discrete 0.017s (58.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 1280x960
			Interval: Discrete 0.029s (34.000 fps)
			Interval: Discrete 0.044s (22.500 fps)
		Size: Discrete 1280x720
			Interval: Discrete 0.022s (45.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 1920x1080
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.067s (15.000 fps)

And this is my launch file:

<launch>
  <group ns="camera">
    <node pkg="libuvc_camera" type="camera_node" name="mycam">
      <!-- Parameters used to find the camera -->
      <param name="vendor" value="0x2560"/>
      <param name="product" value="0xc120"/>
      <param name="serial" value=""/>
      <!-- If the above parameters aren't unique, choose the first match: -->
      <param name="index" value="3"/>

      <!-- Image size and type -->
      <param name="width" value="960"/>
      <param name="height" value="540"/>
      <!-- choose whichever uncompressed format the camera supports: -->
      <param name="video_mode" value="uyvy"/> <!-- or uncompressed/ yuyv/nv12/mjpeg -->
      <param name="frame_rate" value="30"/>

      <param name="timestamp_method" value="start"/> <!-- start of frame -->
      <param name="camera_info_url" value="package://robot/config/cam0_calib.yaml"/>

      <param name="auto_exposure" value="3"/> <!-- use aperture_priority auto exposure -->
      <param name="auto_white_balance" value="false"/>
    </node>
  </group>
</launch>

Do you see something wrong?

My configuration is a computer running Ubuntu 16.04 with ROS Kinetic.

Thanks in advance,
Yoann.

The text was updated successfully, but these errors were encountered:

7675t · 2018-09-05T03:04:19Z

Have you check your camera with other camera softwares, say, guvcview?

yoann-solana · 2018-09-05T06:56:46Z

Hi,
Thank you for the help.
Yes I tried Guvcview for the 960x540 resolution at 30fps. And it works for all the next formats:

UYVY
RGB3
BGR3
YU12
YV12

If it can help, I tried the usb_cam ROS package and it works.
This is the configuration I used:

 <node name="cam_$(arg cam_id)" pkg="usb_cam" type="usb_cam_node" output="screen" >
    <param name="video_device" value="$(arg dev_path)" />
    <param name="image_width" value="960" />
    <param name="image_height" value="540" />
    <param name="pixel_format" value="uyvy" />
    <param name="camera_frame_id" value="$(arg cam_frame_id)" />
    <param name="io_method" value="mmap"/>
    <param name="framerate" value="30"/>
    <param name="camera_info_url" value="$(arg camera_info_url)" />
  </node>

7675t · 2018-09-05T07:32:25Z

In that case, it may be from libuvc_camera driver itself... I personally cannot debug this, but someone could if you share the information of your camera product.

yoann-solana · 2018-09-05T07:46:08Z

Hi,
Thank you, the camera is this model
Is it enough information?

7675t · 2018-09-05T08:10:47Z

I'm not quite sure, it seems from the pixel format of UYVY that not supported by ROS Image.

http://docs.ros.org/jade/api/sensor_msgs/html/image__encodings_8h_source.html

Can you show a image message by rostopic echo? encoding member should be yuv422.

yoann-solana · 2018-09-05T09:02:55Z

I have made the test with usb_cam and libuvc_ros.
Indeed the encoding of the image is different.

This is the short version of a message with usb_cam:

header: 
  seq: 16
  stamp: 
    secs: 1536135825
    nsecs: 368116116
  frame_id: "cam_2_lens_link"
height: 540
width: 960
encoding: "rgb8"
is_bigendian: 0
step: 2880
data: [32, 49, 50,

This is the short version of a message with libuvc_ros:

header: 
  seq: 199
  stamp: 
    secs: 1536137775
    nsecs: 629784377
  frame_id: camera
height: 540
width: 960
encoding: yuv422
is_bigendian: 0
step: 2880

I have also attached the complete message files.
cam_2_image_raw.txt
libuvc-cam_2_image_raw.txt

7675t · 2018-09-05T13:06:46Z

What happens if you set video_mode to "rgb"?

yoann-solana · 2018-09-05T14:23:39Z

The rgb mode is not supported.

process[camera/mycam-2]: started with pid [2465]
unsupported descriptor subtype VS_STILL_IMAGE_FRAME
uvc_get_stream_ctrl_format_size: Invalid mode (-51)
[ERROR] [1536156261.372000402]: check video_mode/width/height/frame_rate are available

If you have an idea, I can look at the source to help.

7675t · 2018-09-05T23:57:42Z

This camera seems to produce raw bayer images to image_raw. In case of that, you may need to use image_proc/debayer node or nodelet. Please try it.

image_proc - ROS Wiki

yoann-solana · 2018-09-06T08:31:44Z

Thank you. I just test it and the image is still the same.
Below is a copy of may launch file and the output of the terminal if it can help.
Launch file:

<launch>
  <!-- Push down all topics/nodelets into "camera" namespace -->
  <group ns="camera">

    <node pkg="nodelet" type="nodelet" name="camera_nodelet_manager" args="manager" />


    <node pkg="libuvc_camera" type="camera_node" name="mycam">
      <!-- Parameters used to find the camera -->
      <param name="vendor" value="0x2560"/>
      <param name="product" value="0xc120"/>
      <param name="serial" value=""/>
      <!-- If the above parameters aren't unique, choose the first match: -->
      <param name="index" value="2"/>

      <!-- Image size and type -->
      <param name="width" value="960"/>
      <param name="height" value="540"/>
      <!-- choose whichever uncompressed format the camera supports: -->
      <param name="video_mode" value="uncompressed"/> <!-- or uncompressed/rgb/yuyv/nv12/mjpeg, not work uyvy-->
      <param name="frame_rate" value="30"/>

      <param name="timestamp_method" value="start"/> <!-- start of frame -->
      <param name="camera_info_url" value="package://platform_misumi_bringup/config/cam0_calib.yaml"/>

      <param name="auto_exposure" value="3"/> <!-- use aperture_priority auto exposure -->
      <param name="auto_white_balance" value="false"/>
    </node>
    <node pkg="nodelet" type="nodelet" name="image_proc_debayer"
          args="load image_proc/debayer camera_nodelet_manager">
      <param name="debayer" value="2" />
    </node>

  </group> <!-- camera ns -->


</launch>

Terminal output:

started roslaunch server http://10.42.10.1:39059/

SUMMARY
========

PARAMETERS
 * /camera/image_proc_debayer/debayer: 2
 * /camera/mycam/auto_exposure: 3
 * /camera/mycam/auto_white_balance: False
 * /camera/mycam/camera_info_url: package://platfor...
 * /camera/mycam/frame_rate: 30
 * /camera/mycam/height: 540
 * /camera/mycam/index: 2
 * /camera/mycam/product: 0xc120
 * /camera/mycam/serial: 
 * /camera/mycam/timestamp_method: start
 * /camera/mycam/vendor: 0x2560
 * /camera/mycam/video_mode: uncompressed
 * /camera/mycam/width: 960
 * /rosdistro: kinetic
 * /rosversion: 1.12.13

NODES
  /camera/
    camera_nodelet_manager (nodelet/nodelet)
    image_proc_debayer (nodelet/nodelet)
    mycam (libuvc_camera/camera_node)

auto-starting new master
process[master]: started with pid [29554]
ROS_MASTER_URI=http://10.42.10.1:11311

setting /run_id to 9ff23a20-b1ae-11e8-8f3c-0007324983dc
process[rosout-1]: started with pid [29567]
started core service [/rosout]
process[camera/camera_nodelet_manager-2]: started with pid [29583]
process[camera/mycam-3]: started with pid [29585]
process[camera/image_proc_debayer-4]: started with pid [29586]
unsupported descriptor subtype VS_STILL_IMAGE_FRAME
attempt to claim already-claimed interface 1

7675t · 2018-09-06T09:05:39Z

Hmm, running out of my idea, sorry.
Could you share an image bag file for several seconds?
Anyway it may difficult to debug without the device if it is from the driver code.

yoann-solana · 2018-09-13T10:36:19Z

Hi,
Sorry for the delay.
I have recorded the bag file with the configuration from the launch file I have posted before.

The bag file was recorded for 0.3s and its size is 14.8MB.
2018-09-13-12-34-03.bag.zip

Regards,
Yoann

7675t · 2018-09-21T01:34:20Z

I have no idea, sorry. The image shows the data is inerlace, but I can't find any de-interlace code in libuvc.

https://github.com/ktossell/libuvc/search?l=C&q=interlace

So, I guess you may better to stick to other packages (usb_cam, gscam, etc) if you can.

skohlbr · 2018-10-29T09:59:50Z

Stumbled across the same issue. So in https://github.com/ros-drivers/libuvc_ros/blob/master/libuvc_camera/src/camera_driver.cpp#L187, the contents of the YUV buffer get directly memcpy'd into the RGB frame. That appears wrong, as a proper conversion makes 6 RGB bytes out of 4 YUV ones. As this is not done, it would be expected that only 2/3 of the image contain actual image data, and that's exactly what can be observed in the image in original post. It looks like maybe the conversion code for the other YUV convention (https://github.com/ros-drivers/libuvc_ros/blob/master/libuvc_camera/src/camera_driver.cpp#L192) should be also employed.

skohlbr · 2018-10-29T10:49:04Z

Fixed (and tested with the CU20 camera) via #52.

7675t · 2018-10-29T23:33:24Z

Wow, I should've noticed! Great job! Good for all and thank you @skohlbr , @yoann-solana

skohlbr added a commit to tu-darmstadt-ros-pkg/libuvc_ros that referenced this issue Oct 29, 2018

Fix ROS image output for uyvy uvc frame format input (ros-drivers#51)

b3376e2

skohlbr mentioned this issue Oct 29, 2018

Fix ROS image output for uyvy uvc frame format input (#51) #52

Merged

ktossell pushed a commit that referenced this issue Jul 6, 2019

Fix ROS image output for uyvy uvc frame format input (#51)

b9e6443

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Image not rendered correctly #51

Image not rendered correctly #51

yoann-solana commented Sep 3, 2018

7675t commented Sep 5, 2018

yoann-solana commented Sep 5, 2018

7675t commented Sep 5, 2018

yoann-solana commented Sep 5, 2018

7675t commented Sep 5, 2018

yoann-solana commented Sep 5, 2018

7675t commented Sep 5, 2018

yoann-solana commented Sep 5, 2018

7675t commented Sep 5, 2018

yoann-solana commented Sep 6, 2018

7675t commented Sep 6, 2018

yoann-solana commented Sep 13, 2018

7675t commented Sep 21, 2018

skohlbr commented Oct 29, 2018

skohlbr commented Oct 29, 2018

7675t commented Oct 29, 2018

Image not rendered correctly #51

Image not rendered correctly #51

Comments

yoann-solana commented Sep 3, 2018

7675t commented Sep 5, 2018

yoann-solana commented Sep 5, 2018

7675t commented Sep 5, 2018

yoann-solana commented Sep 5, 2018

7675t commented Sep 5, 2018

yoann-solana commented Sep 5, 2018

7675t commented Sep 5, 2018

yoann-solana commented Sep 5, 2018

7675t commented Sep 5, 2018

yoann-solana commented Sep 6, 2018

7675t commented Sep 6, 2018

yoann-solana commented Sep 13, 2018

7675t commented Sep 21, 2018

skohlbr commented Oct 29, 2018

skohlbr commented Oct 29, 2018

7675t commented Oct 29, 2018