From patchwork Thu Aug 5 14:21:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 13213 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id AE4EFC3238 for ; Thu, 5 Aug 2021 14:22:02 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 60A3368864; Thu, 5 Aug 2021 16:22:01 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="NUnl2ko0"; dkim-atps=neutral Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CB10A6026D for ; Thu, 5 Aug 2021 16:21:59 +0200 (CEST) Received: by mail-wr1-x431.google.com with SMTP id d8so6805303wrm.4 for ; Thu, 05 Aug 2021 07:21:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XWhF3ysRr+r3b1PFO1nBDUmwHoN8VFvpRQhLtXCbKFM=; b=NUnl2ko0SGYkQb4siyCtCFX9e3Nuxw73hjhe1u1cz72Uld+GMCb3znu4AjDraY8QKj Aqtu8EhWNm7DLJaCfAjHc7WXApZ1iDFq8/Ff/3o9+X+hENOp87Zc1wjdpSw5xBYR21gV Ff8EAko6zS9WHTSasRrqb+tEhz5TVB3ZJLPzV3Krh6xatKH3N7UTnGdn0A9m52+6qkVR 3C/Cer0K0gTJXk2WcWFfZP5gU9+kLZBWASjBjDNy41y9oCcyfCzpoQt4vrzF8EQD5Hrk h6mWtGYjsgWi7NzKtRGX+cMTugoLvdaTU6346k2ltmM3Tp6SiFRx+hgLlZw/SPFcUeiz S3aA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XWhF3ysRr+r3b1PFO1nBDUmwHoN8VFvpRQhLtXCbKFM=; b=RXirKCBTyTSmdEkSwXhAPUuzY3zAN4Ph/vrFuVn7H01IXHNvidCCwhKXr5vO9RDgPN /MwrcjV9g07hP2njNYxZ8KiBhR/DQyX++I+pIKaW9Ab92iNOCQCxWXgZu8HCk7oAOQ14 /16GFOOuDZJL7XhJEubCleGiFsbFzYGYeBvzosbDNrIx9ULxWoyuTBlCsPyelda5M7lq rLL3BeuKTbn2HUs60C8n3jKy9ggdJ+o++smJvsKebTyhLca0JyRysDjk0+lcCR63cmz5 T2drKX8S9D/FcFKqFPiG79QdNDY+eU0qSzd0HSzhEHAgUCVlTeYEeZhui8lWk9caxWbB WdNA== X-Gm-Message-State: AOAM531fS9V3F8HTtaousWdVxWA3kwtjRrZeDSYLqsKPMhd+I0LnX+y2 IMSqggrg0UT6XxpZ8xsR14++GhXccFk0yQ== X-Google-Smtp-Source: ABdhPJxGv+AduxiUcceoo0qBaK/OHF4Oe0/f7xTlODkMkzvgWhyyJObZsjxZwKPbjsMdF5b8/vtQPw== X-Received: by 2002:a5d:68cb:: with SMTP id p11mr5652424wrw.364.1628173319268; Thu, 05 Aug 2021 07:21:59 -0700 (PDT) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id k17sm6410004wrw.53.2021.08.05.07.21.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 05 Aug 2021 07:21:59 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Thu, 5 Aug 2021 15:21:53 +0100 Message-Id: <20210805142154.20324-3-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210805142154.20324-1-david.plowman@raspberrypi.com> References: <20210805142154.20324-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 2/3] libcamera: Support passing ColorSpaces to V4L2 drivers X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The ColorSpace class is added to the StreamConfiguration, and is now passed to V4L2 devices where it is handled appropriately. Note how this means that the colour space is configured per-stream (though platforms may restrict this). Signed-off-by: David Plowman --- include/libcamera/internal/v4l2_videodevice.h | 2 + include/libcamera/stream.h | 3 + src/libcamera/v4l2_videodevice.cpp | 117 ++++++++++++++++++ 3 files changed, 122 insertions(+) diff --git a/include/libcamera/internal/v4l2_videodevice.h b/include/libcamera/internal/v4l2_videodevice.h index e767ec84..c05899cf 100644 --- a/include/libcamera/internal/v4l2_videodevice.h +++ b/include/libcamera/internal/v4l2_videodevice.h @@ -20,6 +20,7 @@ #include #include +#include #include #include #include @@ -163,6 +164,7 @@ public: V4L2PixelFormat fourcc; Size size; + ColorSpace colorSpace; std::array planes; unsigned int planesCount = 0; diff --git a/include/libcamera/stream.h b/include/libcamera/stream.h index 0c55e716..131f7733 100644 --- a/include/libcamera/stream.h +++ b/include/libcamera/stream.h @@ -12,6 +12,7 @@ #include #include +#include #include #include #include @@ -47,6 +48,8 @@ struct StreamConfiguration { unsigned int bufferCount; + ColorSpace colorSpace; + Stream *stream() const { return stream_; } void setStream(Stream *stream) { stream_ = stream; } const StreamFormats &formats() const { return formats_; } diff --git a/src/libcamera/v4l2_videodevice.cpp b/src/libcamera/v4l2_videodevice.cpp index da2af6a1..7f30412c 100644 --- a/src/libcamera/v4l2_videodevice.cpp +++ b/src/libcamera/v4l2_videodevice.cpp @@ -366,6 +366,11 @@ bool V4L2BufferCache::Entry::operator==(const FrameBuffer &buffer) const * \brief The image size in pixels */ +/** + * \var V4L2DeviceFormat::colorSpace + * \brief The color space of the pixels + */ + /** * \var V4L2DeviceFormat::fourcc * \brief The fourcc code describing the pixel encoding scheme @@ -731,6 +736,114 @@ int V4L2VideoDevice::getFormat(V4L2DeviceFormat *format) return getFormatSingleplane(format); } +static const std::vector> colorSpaceToV4l2 = { + { ColorSpace::RAW, V4L2_COLORSPACE_RAW }, + { ColorSpace::JFIF, V4L2_COLORSPACE_JPEG }, + { ColorSpace::SMPTE170M, V4L2_COLORSPACE_SMPTE170M }, + { ColorSpace::REC709, V4L2_COLORSPACE_REC709 }, + { ColorSpace::REC2020, V4L2_COLORSPACE_BT2020 }, +}; + +static const std::map encodingToV4l2 = { + { ColorSpace::Encoding::REC601, V4L2_YCBCR_ENC_601 }, + { ColorSpace::Encoding::REC709, V4L2_YCBCR_ENC_709 }, + { ColorSpace::Encoding::REC2020, V4L2_YCBCR_ENC_BT2020 }, +}; + +static const std::map transferFunctionToV4l2 = { + { ColorSpace::TransferFunction::IDENTITY, V4L2_XFER_FUNC_NONE }, + { ColorSpace::TransferFunction::SRGB, V4L2_XFER_FUNC_SRGB }, + { ColorSpace::TransferFunction::REC709, V4L2_XFER_FUNC_709 }, +}; + +static const std::map rangeToV4l2 = { + { ColorSpace::Range::FULL, V4L2_QUANTIZATION_FULL_RANGE }, + { ColorSpace::Range::LIMITED, V4L2_QUANTIZATION_LIM_RANGE }, +}; + +template +static void setColorSpace(const ColorSpace &colorSpace, T &v4l2PixFormat) +{ + if (!colorSpace.isFullyDefined()) + LOG(V4L2, Warning) << "Setting non-fully defined colour space" + << colorSpace.toString(); + + v4l2PixFormat.colorspace = V4L2_COLORSPACE_DEFAULT; + v4l2PixFormat.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; + v4l2PixFormat.xfer_func = V4L2_XFER_FUNC_DEFAULT; + v4l2PixFormat.quantization = V4L2_QUANTIZATION_DEFAULT; + + auto itColor = std::find_if(colorSpaceToV4l2.begin(), colorSpaceToV4l2.end(), + [&colorSpace](const std::pair &item) { return colorSpace == item.first; }); + if (itColor != colorSpaceToV4l2.end()) + v4l2PixFormat.colorspace = itColor->second; + + auto itEncoding = encodingToV4l2.find(colorSpace.encoding); + if (itEncoding != encodingToV4l2.end()) + v4l2PixFormat.ycbcr_enc = itEncoding->second; + + auto itTransfer = transferFunctionToV4l2.find(colorSpace.transferFunction); + if (itTransfer != transferFunctionToV4l2.end()) + v4l2PixFormat.xfer_func = itTransfer->second; + + auto itRange = rangeToV4l2.find(colorSpace.range); + if (itRange != rangeToV4l2.end()) + v4l2PixFormat.quantization = itRange->second; +} + +static const std::map v4l2ToColorSpace = { + { V4L2_COLORSPACE_RAW, ColorSpace::RAW }, + { V4L2_COLORSPACE_JPEG, ColorSpace::JFIF }, + { V4L2_COLORSPACE_SRGB, ColorSpace::JFIF }, + { V4L2_COLORSPACE_SMPTE170M, ColorSpace::SMPTE170M }, + { V4L2_COLORSPACE_REC709, ColorSpace::REC709 }, + { V4L2_COLORSPACE_BT2020, ColorSpace::REC2020 }, +}; + +static const std::map v4l2ToEncoding = { + { V4L2_YCBCR_ENC_601, ColorSpace::Encoding::REC601 }, + { V4L2_YCBCR_ENC_709, ColorSpace::Encoding::REC709 }, + { V4L2_YCBCR_ENC_BT2020, ColorSpace::Encoding::REC2020 }, +}; + +static const std::map v4l2ToTransferFunction = { + { V4L2_XFER_FUNC_NONE, ColorSpace::TransferFunction::IDENTITY }, + { V4L2_XFER_FUNC_SRGB, ColorSpace::TransferFunction::SRGB }, + { V4L2_XFER_FUNC_709, ColorSpace::TransferFunction::REC709 }, +}; + +static const std::map v4l2ToRange = { + { V4L2_QUANTIZATION_FULL_RANGE, ColorSpace::Range::FULL }, + { V4L2_QUANTIZATION_LIM_RANGE, ColorSpace::Range::LIMITED }, +}; + +template +static ColorSpace getColorSpace(const T &v4l2PixFormat) +{ + ColorSpace colorSpace; + + auto itColor = v4l2ToColorSpace.find(v4l2PixFormat.colorspace); + if (itColor != v4l2ToColorSpace.end()) + colorSpace = itColor->second; + + auto itEncoding = v4l2ToEncoding.find(v4l2PixFormat.ycbcr_enc); + if (itEncoding != v4l2ToEncoding.end()) + colorSpace.encoding = itEncoding->second; + + auto itTransfer = v4l2ToTransferFunction.find(v4l2PixFormat.xfer_func); + if (itTransfer != v4l2ToTransferFunction.end()) + colorSpace.transferFunction = itTransfer->second; + + auto itRange = v4l2ToRange.find(v4l2PixFormat.quantization); + if (itRange != v4l2ToRange.end()) + colorSpace.range = itRange->second; + + if (!colorSpace.isFullyDefined()) + LOG(V4L2, Warning) << "Returning non-fully defined colour space" + << colorSpace.toString(); + return colorSpace; +} + /** * \brief Try an image format on the V4L2 video device * \param[inout] format The image format to test applicability to the video device @@ -840,6 +953,7 @@ int V4L2VideoDevice::getFormatMultiplane(V4L2DeviceFormat *format) format->size.width = pix->width; format->size.height = pix->height; format->fourcc = V4L2PixelFormat(pix->pixelformat); + format->colorSpace = getColorSpace(*pix); format->planesCount = pix->num_planes; for (unsigned int i = 0; i < format->planesCount; ++i) { @@ -862,6 +976,7 @@ int V4L2VideoDevice::trySetFormatMultiplane(V4L2DeviceFormat *format, bool set) pix->pixelformat = format->fourcc; pix->num_planes = format->planesCount; pix->field = V4L2_FIELD_NONE; + setColorSpace(format->colorSpace, *pix); ASSERT(pix->num_planes <= std::size(pix->plane_fmt)); @@ -910,6 +1025,7 @@ int V4L2VideoDevice::getFormatSingleplane(V4L2DeviceFormat *format) format->size.width = pix->width; format->size.height = pix->height; format->fourcc = V4L2PixelFormat(pix->pixelformat); + format->colorSpace = getColorSpace(*pix); format->planesCount = 1; format->planes[0].bpl = pix->bytesperline; format->planes[0].size = pix->sizeimage; @@ -929,6 +1045,7 @@ int V4L2VideoDevice::trySetFormatSingleplane(V4L2DeviceFormat *format, bool set) pix->pixelformat = format->fourcc; pix->bytesperline = format->planes[0].bpl; pix->field = V4L2_FIELD_NONE; + setColorSpace(format->colorSpace, *pix); ret = ioctl(set ? VIDIOC_S_FMT : VIDIOC_TRY_FMT, &v4l2Format); if (ret) { LOG(V4L2, Error)