Graphics, sound and multimedia • Re: Question about v4l2 bcm2835-codec encoding and color range

Thanks for the answer and explanation. But I tried this input format, without luck, nothing changed.

Code:

    struct v4l2_format inputFormat = {0};    inputFormat.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;    inputFormat.fmt.pix_mp.width = size.width();    inputFormat.fmt.pix_mp.height = size.height();    inputFormat.fmt.pix_mp.pixelformat = V4L2_PIX_FMT_RGBA32;    inputFormat.fmt.pix_mp.field = V4L2_FIELD_ANY;    inputFormat.fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;    inputFormat.fmt.pix_mp.quantization = V4L2_QUANTIZATION_FULL_RANGE;    inputFormat.fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_601; // Also tried V4L2_YCBCR_ENC_709, V4L2_YCBCR_DEFAULT    inputFormat.fmt.pix_mp.num_planes = 1;

That makes no sense as RGB is always full range, and isn't ycbcr encoded.

Code:

    struct v4l2_format outputFormat = {0};    outputFormat.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;    outputFormat.fmt.pix_mp.width = size.width();    outputFormat.fmt.pix_mp.height = size.height();    outputFormat.fmt.pix_mp.pixelformat = V4L2_PIX_FMT_H264;    outputFormat.fmt.pix_mp.field = V4L2_FIELD_ANY;    outputFormat.fmt.pix_mp.colorspace = V4L2_COLORSPACE_REC709;    outputFormat.fmt.pix_mp.quantization = V4L2_QUANTIZATION_FULL_RANGE;    outputFormat.fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT;    outputFormat.fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_709;

Memory says that codecs are meant to ignore all colourspace properties on the encoded side unless they can do conversion between colourspaces.

Looks like the driver just ignore all options what I pass. I tried to find examples of how to set the encoder to the full range mode, but without luck.

Rereading the driver, yes it appears to ignore all properties. For RGB this shouldn't matter at all, but could be fixed up for encoding YUV.

On the decoder side I tried play my h264 raw frame using ffmpeg, and openh264 library, the same result.

Decode to YUV and compare. Conversion to/from RGB is always the place that gets messed up.
Using "ffmpeg -i output.mp4 output.yuv" I get all the pixels being Y=218, U=127, V=127.

As a test I've used:
gst-launch-1.0 videotestsrc ! video/x-raw,format=I420,width=640,height=480 ! filesink location=test.i420
and
gst-launch-1.0 videotestsrc ! video/x-raw,format=I420,width=640,height=480 ! v4l2h264enc ! "video/x-h264,level=(string)4.1" ! filesink location=test.h264
to get a raw and encoded YUV stream. Decoded with ffmpeg -i test.h264 test_decode.yuv. Compared values in Vooya, and they are within 1.

Repeat encoding via
gst-launch-1.0 videotestsrc ! video/x-raw,format=BGRx,width=640,height=480 ! v4l2h264enc ! "video/x-h264,level=(string)4.1" ! filesink location=test.h264
and the colours look muted when viewing the decoded output as I420. That's the opposite to what you posted.

I can have a look to see if there is an obvious mismatch, but as above getting all the colourspace conversions right is a real pain. The H264 and JPEG encoders on the Pi only work in the YUV domain, so keeping to that is generally easiest.

Statistics: Posted by 6by9 — Tue Jan 16, 2024 6:47 pm

---