From patchwork Mon Dec 6 10:50:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 15042 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4F220BDB13 for ; Mon, 6 Dec 2021 10:51:25 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E006060894; Mon, 6 Dec 2021 11:51:24 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="gR9vVPwQ"; dkim-atps=neutral Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0207B6011A for ; Mon, 6 Dec 2021 11:51:22 +0100 (CET) Received: by mail-wm1-x32e.google.com with SMTP id az34-20020a05600c602200b0033bf8662572so7328543wmb.0 for ; Mon, 06 Dec 2021 02:51:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=80QCsLGyD1uStMsIR5B43lhR2OyW4eywkK7xvq30nzo=; b=gR9vVPwQDaMIgZo4mT1/I3kSms79qYqwPStFTV1eRGE34UHWcner0xW7K9dnUHhqdD bdarhEcABbl9dZOY/R1SxSWrOnCD2aBR8kgCJ3b/OEbL5HjGe8ltds+hkIZ0vMthx6sL +YDrawF47wjPJtGD16pZUYC++0BA7NUnLg7vsmJXKS4Q7bLSf8DVJkfh9K1POHuLZva2 piBfUk1D6t5scWE9wsQZ8S2JR/ytdaNJ70xzgvELINhqkl03O1vwga+VfchRdKI4XbBi hOXz2Pj49S96bbldWwBhrmL52ggPJpmJOdtQbAr0NxPXYMyXDrCvfb633FD5te5BA2Ik jE4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=80QCsLGyD1uStMsIR5B43lhR2OyW4eywkK7xvq30nzo=; b=Y3gb898fvXpLFLIEH9Vo//zfITjZu47Xx+bnomdkmoKQvk1s2lzsaK8GOl+DWzGn3P dMHE4eI79ozDplotjKyS5R37V0K0eboevEoM1owa4HdRuEAUTvFWNNbvm/Tq84HVh28+ JGV6icJPnzxQnf9rWPtbPUqGJ5GYSHz4DqddNSoTmj/wEDX95PP4rBJjhDzoED9fMMzj VTH29bjL3R/dJrVim2kRDeQuCb/QfDLDHTK6USf29IFOEzVlXUWiRzEM9Q7crQh/Qiq0 BlN7A+DqdL9fhDStZ5nkzRv4Bc4gjZCuooCz5fWsGgOnCtJkMK/c9k+JQ5zrsc5edoTx BXfA== X-Gm-Message-State: AOAM531N+6UZy/une7WoCQfjtGgyz/OtjP9P12E8hOYRuQdls65wp6pC UczQF90NMejCoSe3GuCtcUqQuBjVxdqIRcCL X-Google-Smtp-Source: ABdhPJzOwf1SH/jo6H3uKjjuhCCvbjk4r0CPjLY0KY3fVLqL4u3y0R1BXmhL+jLYSIvCKY5jyIYTkg== X-Received: by 2002:a7b:c194:: with SMTP id y20mr39135853wmi.61.1638787881269; Mon, 06 Dec 2021 02:51:21 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id r7sm10878186wrq.29.2021.12.06.02.51.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Dec 2021 02:51:20 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org, naush@raspberrypi.com, jacopo@jmondi.org, tfiga@google.com, hverkuil-cisco@xs4all.nl, kieran.bingham@ideasonboard.com, laurent.pinchart@ideasonboard.com Date: Mon, 6 Dec 2021 10:50:24 +0000 Message-Id: <20211206105032.13876-2-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211206105032.13876-1-david.plowman@raspberrypi.com> References: <20211206105032.13876-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v9 1/8] libcamera: Add ColorSpace class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This class represents a color space by defining its color primaries, YCbCr encoding, the transfer (gamma) function it uses, and whether the output is full or limited range. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck Reviewed-by: Umang Jain Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- include/libcamera/color_space.h | 69 ++++++++ include/libcamera/meson.build | 1 + src/libcamera/color_space.cpp | 305 ++++++++++++++++++++++++++++++++ src/libcamera/meson.build | 1 + 4 files changed, 376 insertions(+) create mode 100644 include/libcamera/color_space.h create mode 100644 src/libcamera/color_space.cpp diff --git a/include/libcamera/color_space.h b/include/libcamera/color_space.h new file mode 100644 index 00000000..0fe6b580 --- /dev/null +++ b/include/libcamera/color_space.h @@ -0,0 +1,69 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Raspberry Pi (Trading) Limited + * + * color_space.h - color space definitions + */ + +#pragma once + +#include +#include + +namespace libcamera { + +class ColorSpace +{ +public: + enum class Primaries { + Raw, + Smpte170m, + Rec709, + Rec2020, + }; + + enum class YcbcrEncoding { + Rec601, + Rec709, + Rec2020, + }; + + enum class TransferFunction { + Linear, + Srgb, + Rec709, + }; + + enum class Range { + Full, + Limited, + }; + + constexpr ColorSpace(Primaries p, YcbcrEncoding e, TransferFunction t, Range r) + : primaries(p), ycbcrEncoding(e), transferFunction(t), range(r) + { + } + + static const ColorSpace Raw; + static const ColorSpace Jpeg; + static const ColorSpace Srgb; + static const ColorSpace Smpte170m; + static const ColorSpace Rec709; + static const ColorSpace Rec2020; + + Primaries primaries; + YcbcrEncoding ycbcrEncoding; + TransferFunction transferFunction; + Range range; + + const std::string toString() const; + static const std::string toString(const std::optional &colorSpace); +}; + +bool operator==(const ColorSpace &lhs, const ColorSpace &rhs); +static inline bool operator!=(const ColorSpace &lhs, const ColorSpace &rhs) +{ + return !(lhs == rhs); +} + +} /* namespace libcamera */ diff --git a/include/libcamera/meson.build b/include/libcamera/meson.build index 5f42977c..fd767b11 100644 --- a/include/libcamera/meson.build +++ b/include/libcamera/meson.build @@ -5,6 +5,7 @@ libcamera_include_dir = 'libcamera' / 'libcamera' libcamera_public_headers = files([ 'camera.h', 'camera_manager.h', + 'color_space.h', 'controls.h', 'framebuffer.h', 'framebuffer_allocator.h', diff --git a/src/libcamera/color_space.cpp b/src/libcamera/color_space.cpp new file mode 100644 index 00000000..5fe4fcc2 --- /dev/null +++ b/src/libcamera/color_space.cpp @@ -0,0 +1,305 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Raspberry Pi (Trading) Limited + * + * color_space.cpp - color spaces. + */ + +#include + +#include +#include +#include +#include +#include + +/** + * \file color_space.h + * \brief Class and enums to represent color spaces + */ + +namespace libcamera { + +/** + * \class ColorSpace + * \brief Class to describe a color space + * + * The ColorSpace class defines the color primaries, the Y'CbCr encoding, + * the transfer function associated with the color space, and the range + * (sometimes also referred to as the quantisation) of the color space. + * + * Certain combinations of these fields form well-known standard color + * spaces such as "JPEG" or "REC709". + * + * In the strictest sense a "color space" formally only refers to the + * color primaries and white point. Here, however, the ColorSpace class + * adopts the common broader usage that includes the transfer function, + * Y'CbCr encoding method and quantisation. + * + * For more information on the specific color spaces described here, please + * see: + * + * sRGB + * JPEG + * SMPTE 170M + * Rec.709 + * Rec.2020 + */ + +/** + * \enum ColorSpace::Primaries + * \brief The color primaries for this color space + * + * \var ColorSpace::Primaries::Raw + * \brief These are raw colors directly from a sensor + * + * \var ColorSpace::Primaries::Smpte170m + * \brief SMPTE 170M color primaries + * + * \var ColorSpace::Primaries::Rec709 + * \brief Rec.709 color primaries + * + * \var ColorSpace::Primaries::Rec2020 + * \brief Rec.2020 color primaries + */ + +/** + * \enum ColorSpace::YcbcrEncoding + * \brief The Y'CbCr encoding + * + * \var ColorSpace::YcbcrEncoding::Rec601 + * \brief Rec.601 Y'CbCr encoding + * + * \var ColorSpace::YcbcrEncoding::Rec709 + * \brief Rec.709 Y'CbCr encoding + * + * \var ColorSpace::YcbcrEncoding::Rec2020 + * \brief Rec.2020 Y'CbCr encoding + */ + +/** + * \enum ColorSpace::TransferFunction + * \brief The transfer function used for this color space + * + * \var ColorSpace::TransferFunction::Linear + * \brief This color space uses a linear (identity) transfer function + * + * \var ColorSpace::TransferFunction::Srgb + * \brief sRGB transfer function + * + * \var ColorSpace::TransferFunction::Rec709 + * \brief Rec.709 transfer function + */ + +/** + * \enum ColorSpace::Range + * \brief The range (sometimes "quantisation") for this color space + * + * \var ColorSpace::Range::Full + * \brief This color space uses full range pixel values + * + * \var ColorSpace::Range::Limited + * \brief This color space uses limited range pixel values, being + * 16 to 235 for Y' and 16 to 240 for Cb and Cr (8 bits per sample) + * or 64 to 940 for Y' and 16 to 960 for Cb and Cr (10 bits) + */ + +/** + * \fn ColorSpace::ColorSpace(Primaries p, Encoding e, TransferFunction t, Range r) + * \brief Construct a ColorSpace from explicit values + * \param[in] p The color primaries + * \param[in] e The Y'CbCr encoding + * \param[in] t The transfer function for the color space + * \param[in] r The range of the pixel values in this color space + */ + +/** + * \brief Assemble and return a readable string representation of the + * ColorSpace + * + * If the color space matches a standard ColorSpace (such as ColorSpace::Jpeg) + * then the short name of the color space ("Jpeg") is returned. Otherwise + * the four constituent parts of the ColorSpace are assembled into a longer + * string. + * + * \return A string describing the ColorSpace + */ +const std::string ColorSpace::toString() const +{ + /* Print out a brief name only for standard color spaces. */ + + static const std::vector> colorSpaceNames = { + { ColorSpace::Raw, "Raw" }, + { ColorSpace::Jpeg, "Jpeg" }, + { ColorSpace::Srgb, "Srgb" }, + { ColorSpace::Smpte170m, "Smpte170m" }, + { ColorSpace::Rec709, "Rec709" }, + { ColorSpace::Rec2020, "Rec2020" }, + }; + auto it = std::find_if(colorSpaceNames.begin(), colorSpaceNames.end(), + [this](const auto &item) { + return *this == item.first; + }); + if (it != colorSpaceNames.end()) + return std::string(it->second); + + /* Assemble a name made of the constituent fields. */ + + static const std::map primariesNames = { + { Primaries::Raw, "Raw" }, + { Primaries::Smpte170m, "Smpte170m" }, + { Primaries::Rec709, "Rec709" }, + { Primaries::Rec2020, "Rec2020" }, + }; + static const std::map encodingNames = { + { YcbcrEncoding::Rec601, "Rec601" }, + { YcbcrEncoding::Rec709, "Rec709" }, + { YcbcrEncoding::Rec2020, "Rec2020" }, + }; + static const std::map transferNames = { + { TransferFunction::Linear, "Linear" }, + { TransferFunction::Srgb, "Srgb" }, + { TransferFunction::Rec709, "Rec709" }, + }; + static const std::map rangeNames = { + { Range::Full, "Full" }, + { Range::Limited, "Limited" }, + }; + + auto itPrimaries = primariesNames.find(primaries); + std::string primariesName = + itPrimaries == primariesNames.end() ? "Invalid" : itPrimaries->second; + + auto itEncoding = encodingNames.find(ycbcrEncoding); + std::string encodingName = + itEncoding == encodingNames.end() ? "Invalid" : itEncoding->second; + + auto itTransfer = transferNames.find(transferFunction); + std::string transferName = + itTransfer == transferNames.end() ? "Invalid" : itTransfer->second; + + auto itRange = rangeNames.find(range); + std::string rangeName = + itRange == rangeNames.end() ? "Invalid" : itRange->second; + + std::stringstream ss; + ss << primariesName << "/" << encodingName << "/" << transferName << "/" << rangeName; + + return ss.str(); +} + +/** + * \brief Assemble and return a readable string representation of an + * optional ColorSpace + * \return A string describing the optional ColorSpace + */ +const std::string ColorSpace::toString(const std::optional &colorSpace) +{ + if (!colorSpace) + return "Unknown"; + + return colorSpace->toString(); +} + +/** + * \var ColorSpace::primaries + * \brief The color primaries of this color space + */ + +/** + * \var ColorSpace::ycbcrEncoding + * \brief The Y'CbCr encoding used with this color space + */ + +/** + * \var ColorSpace::transferFunction + * \brief The transfer function used with this color space + */ + +/** + * \var ColorSpace::range + * \brief The pixel range used with this color space + */ + +/** + * \brief A constant representing a raw color space (from a sensor) + */ +const ColorSpace ColorSpace::Raw = { + Primaries::Raw, + YcbcrEncoding::Rec601, + TransferFunction::Linear, + Range::Full +}; + +/** + * \brief A constant representing the JPEG color space used for + * encoding JPEG images. + */ +const ColorSpace ColorSpace::Jpeg = { + Primaries::Rec709, + YcbcrEncoding::Rec601, + TransferFunction::Srgb, + Range::Full +}; + +/** + * \brief A constant representing the sRGB color space. This is + * identical to the JPEG color space except that the range Y'CbCr + * range is limited rather than full. + */ +const ColorSpace ColorSpace::Srgb = { + Primaries::Rec709, + YcbcrEncoding::Rec601, + TransferFunction::Srgb, + Range::Limited +}; + +/** + * \brief A constant representing the SMPTE170M color space + */ +const ColorSpace ColorSpace::Smpte170m = { + Primaries::Smpte170m, + YcbcrEncoding::Rec601, + TransferFunction::Rec709, + Range::Limited +}; + +/** + * \brief A constant representing the Rec.709 color space + */ +const ColorSpace ColorSpace::Rec709 = { + Primaries::Rec709, + YcbcrEncoding::Rec709, + TransferFunction::Rec709, + Range::Limited +}; + +/** + * \brief A constant representing the Rec.2020 color space + */ +const ColorSpace ColorSpace::Rec2020 = { + Primaries::Rec2020, + YcbcrEncoding::Rec2020, + TransferFunction::Rec709, + Range::Limited +}; + +/** + * \brief Compare color spaces for equality + * \return True if the two color spaces are identical, false otherwise + */ +bool operator==(const ColorSpace &lhs, const ColorSpace &rhs) +{ + return lhs.primaries == rhs.primaries && + lhs.ycbcrEncoding == rhs.ycbcrEncoding && + lhs.transferFunction == rhs.transferFunction && + lhs.range == rhs.range; +} + +/** + * \fn bool operator!=(const ColorSpace &lhs, const ColorSpace &rhs) + * \brief Compare color spaces for inequality + * \return True if the two color spaces are not identical, false otherwise + */ + +} /* namespace libcamera */ diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build index 2e54cc04..4045e24e 100644 --- a/src/libcamera/meson.build +++ b/src/libcamera/meson.build @@ -9,6 +9,7 @@ libcamera_sources = files([ 'camera_manager.cpp', 'camera_sensor.cpp', 'camera_sensor_properties.cpp', + 'color_space.cpp', 'controls.cpp', 'control_serializer.cpp', 'control_validator.cpp', From patchwork Mon Dec 6 10:50:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 15043 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 21A34BDB13 for ; Mon, 6 Dec 2021 10:51:27 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2F54C60889; Mon, 6 Dec 2021 11:51:26 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="pz/nsoPZ"; dkim-atps=neutral Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3984360872 for ; Mon, 6 Dec 2021 11:51:22 +0100 (CET) Received: by mail-wm1-x330.google.com with SMTP id 137so7787515wma.1 for ; Mon, 06 Dec 2021 02:51:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=S8VVl7/+9S4D8aM7tuAng1H4cosPqk1LN3e9Ply+FyA=; b=pz/nsoPZTNrA+iB4gG2vFGL8u54tylLEhKC6wEM9cTdD7W6d6BKpR3cbqcprZaOOP1 RinK3pRsL5SvlCxjHR9s2Dk+Qy1rG+/Mp2HTfqSSG/8JtUPeQMYVggfOldg5h+rxpAme JE623mEZoOgn83XWU+mW1JZpun2f517DlPStIOJBSBTTKKMdzbo4A2V1foxwiNfbAfpl TY2sX+sCQCFnKp1C+hYziNvxVNccmKMMUtWm+wHUR9K20vOcYGeQLEpwsO4Gp6+TMdaJ Gm5aVAHq+LsYVK96GFHj/A+FLw5+JfrfXTKqTfT0ziOagSbMEmHvXr8HAHs8AI+n3n+1 GiCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=S8VVl7/+9S4D8aM7tuAng1H4cosPqk1LN3e9Ply+FyA=; b=h1mCtCaM+9ETho5CKyL3LDUZTUhCJF7xQ/OpHIeUCKakl/YgSH6hCg93A8nUxYcybR dUvGn8ihJpVkjV+iiyLEp8MsgJFEHNlQ7PiOf997yqy/qcdRgwUKM1CELUEt63oic4hU rJGStblYhDnu/O2xqlQ38sMVjBu/4wuDcQ3jSEKJLqY0J7f8+gxMu3/Ghgo5lEPYtZCI uBYcfR3B+oe5lYav97g/GMq8g+n65ukOywa3tnefP9Jj6Iuu101AeQck4YQT0r2A39ip UZV1Qk0Xmn9F53x8BP7HPdJz4XnFT7bmAn8zqvnH2glYJWPST/aZW0BkVIkvSxp4foTB oaFA== X-Gm-Message-State: AOAM530K0yauo8V+83RZxrrJ66oDjs/2itmaUAvHX8cBs8uHmDxJDihv hatSlsA4rdk4YL311htLisjobQhVlohrkHif X-Google-Smtp-Source: ABdhPJzDqd18HhseDaf+4MiQwDJT8dEy7O3BZ5ud3wKGODEKd4zIckwJrgvlo3Guzd5ab6yj1I2+ag== X-Received: by 2002:a05:600c:3788:: with SMTP id o8mr36845761wmr.82.1638787881790; Mon, 06 Dec 2021 02:51:21 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id r7sm10878186wrq.29.2021.12.06.02.51.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Dec 2021 02:51:21 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org, naush@raspberrypi.com, jacopo@jmondi.org, tfiga@google.com, hverkuil-cisco@xs4all.nl, kieran.bingham@ideasonboard.com, laurent.pinchart@ideasonboard.com Date: Mon, 6 Dec 2021 10:50:25 +0000 Message-Id: <20211206105032.13876-3-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211206105032.13876-1-david.plowman@raspberrypi.com> References: <20211206105032.13876-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v9 2/8] libcamera: Add ColorSpace fields to StreamConfiguration X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This is so that applications can choose appropriate color spaces which will then be passed down to the V4L2 devices. The ColorSpace field is actually optional. If it is not set you will get the driver's default color space. Signed-off-by: David Plowman Reviewed-by: Umang Jain Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- include/libcamera/stream.h | 3 +++ src/libcamera/stream.cpp | 20 ++++++++++++++++++++ 2 files changed, 23 insertions(+) diff --git a/include/libcamera/stream.h b/include/libcamera/stream.h index 41ec02b1..f0ae7e62 100644 --- a/include/libcamera/stream.h +++ b/include/libcamera/stream.h @@ -12,6 +12,7 @@ #include #include +#include #include #include #include @@ -47,6 +48,8 @@ struct StreamConfiguration { unsigned int bufferCount; + std::optional colorSpace; + Stream *stream() const { return stream_; } void setStream(Stream *stream) { stream_ = stream; } const StreamFormats &formats() const { return formats_; } diff --git a/src/libcamera/stream.cpp b/src/libcamera/stream.cpp index b421e17e..686e693b 100644 --- a/src/libcamera/stream.cpp +++ b/src/libcamera/stream.cpp @@ -329,6 +329,26 @@ StreamConfiguration::StreamConfiguration(const StreamFormats &formats) * \brief Requested number of buffers to allocate for the stream */ +/** + * \var StreamConfiguration::colorSpace + * \brief The ColorSpace for this stream + * + * This field allows a ColorSpace to be selected for this Stream. + * + * The field is optional and an application can choose to leave it unset. + * Platforms that support the use of color spaces may provide default + * values through the generateConfiguration() method. An application can + * override these when necessary. + * + * If a specific ColorSpace is requested but the Camera cannot deliver it, + * then the StreamConfiguration will be adjusted to a value that can be + * delivered. In this case the validate() method will indicate via its + * return value that the CameraConfiguration has been adjusted. + * + * Note that platforms will typically have different constraints on what + * color spaces can be supported and in what combinations. + */ + /** * \fn StreamConfiguration::stream() * \brief Retrieve the stream associated with the configuration From patchwork Mon Dec 6 10:50:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 15044 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 32D40C324B for ; Mon, 6 Dec 2021 10:51:28 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 98B226088E; Mon, 6 Dec 2021 11:51:26 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="N0+0dVtT"; dkim-atps=neutral Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id F218F60822 for ; Mon, 6 Dec 2021 11:51:22 +0100 (CET) Received: by mail-wm1-x331.google.com with SMTP id i12so7760918wmq.4 for ; Mon, 06 Dec 2021 02:51:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hHpZgPXC/4/IxY3SDldvUMtViVO+F7GYSyD6EMZUEmE=; b=N0+0dVtTajOjFThOUReFqm++kymxFo+MFBMxLznoJTUK1PUfJWoak77QDNJIMO2llD T/87Vg+VpBMGU/zmzkjYlYg0OZggKSd56jIT+HZP+aVnth13WmnLRaq1o/70D+QalXi/ cP2Pm7yMMxl0XVmAK8g1JMLQJ1BZ20Six01p+ARZaD5H/Bfjk0XqVwoy9f3VL/bf5h2x s2NGdj6q13bIbHvOBgeL1uG3q7hJT3N1jAgWiKoJS0bUj2BwbqEJNMaUzIuwCRYta3wq Q0fGwgTyV5Ab78V2e8UepqQ92LBm3EOc4L1GchfcR/JLTJINAoSYIeqquPuuQ3SgCoM9 OKIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hHpZgPXC/4/IxY3SDldvUMtViVO+F7GYSyD6EMZUEmE=; b=BUAFxSWJoNznMFSQ0Ua6o4Ya+67pM0AO8IgH2zUWSQ9EUoGFpwsUWeyFafXvq2odG8 yrTokdD5r6XHmIPNFvUBeW+EUV19F0zikJnyoLKPutOTBPNcBoFjHqHwEPXPIMIwdNNh zTnGNV0S+sMm2l37msltgJFeP0YNa6mKZsxwt1XjQ4NWer04Qq55RZ4C7A2yIKFTaTmR Ju1KUL4F0YVKNuBOaTzW7Q85UR4g7iqJ81zrdQQI4Eisv67dWwKAHOOlgK7xq4TXWWr+ 5WBE14CgNeVhI1B5PX1VrY2MSksl/Fci0pSY0i5AJC+45IsR0f9oUDEULaMqF59Z/lBH uk0g== X-Gm-Message-State: AOAM532KhNQSb+fAAupp6MhnKn7qZipL5wu/b1B42JeYKdM6IpefLkT1 QBS4Gbl1WK73s9+6rBm1ZdhFEL2mNDuwGtP8 X-Google-Smtp-Source: ABdhPJyuMcYh8s55wwAZyPCrB7gTS71W4RfOltsqDetsijk+S8BVRoDVGi9LYwtD8b+0MsSLfJdLmw== X-Received: by 2002:a05:600c:35cb:: with SMTP id r11mr38405584wmq.190.1638787882457; Mon, 06 Dec 2021 02:51:22 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id r7sm10878186wrq.29.2021.12.06.02.51.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Dec 2021 02:51:22 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org, naush@raspberrypi.com, jacopo@jmondi.org, tfiga@google.com, hverkuil-cisco@xs4all.nl, kieran.bingham@ideasonboard.com, laurent.pinchart@ideasonboard.com Date: Mon, 6 Dec 2021 10:50:26 +0000 Message-Id: <20211206105032.13876-4-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211206105032.13876-1-david.plowman@raspberrypi.com> References: <20211206105032.13876-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v9 3/8] libcamera: Convert between ColorSpace class and V4L2 formats X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add functions to the V4L2Device class to convert to and from libcamera ColorSpace. These methods are added to the base V4L2Device class so that they can be shared both by the video device class and subdevices. With the ColorSpace class, the color space and related other fields are stored together, corresponding to a number of fields in the various different V4L2 format structures. Template methods are therefore a convenient implementation, and we must explicitly instantiate the templates that will be needed. Note that unset color spaces are converted to requests for the device's "default" color space. Signed-off-by: David Plowman Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- include/libcamera/internal/v4l2_device.h | 7 + src/libcamera/v4l2_device.cpp | 192 +++++++++++++++++++++++ 2 files changed, 199 insertions(+) diff --git a/include/libcamera/internal/v4l2_device.h b/include/libcamera/internal/v4l2_device.h index 8886b750..38cf93a6 100644 --- a/include/libcamera/internal/v4l2_device.h +++ b/include/libcamera/internal/v4l2_device.h @@ -18,6 +18,7 @@ #include #include +#include #include namespace libcamera { @@ -45,6 +46,12 @@ public: void updateControlInfo(); + template + static std::optional toColorSpace(const T &v4l2Format); + + template + static int fromColorSpace(const std::optional &colorSpace, T &v4l2Format); + protected: V4L2Device(const std::string &deviceNode); ~V4L2Device(); diff --git a/src/libcamera/v4l2_device.cpp b/src/libcamera/v4l2_device.cpp index 39f36009..cd8a5b1b 100644 --- a/src/libcamera/v4l2_device.cpp +++ b/src/libcamera/v4l2_device.cpp @@ -16,6 +16,8 @@ #include #include +#include + #include #include #include @@ -728,4 +730,194 @@ void V4L2Device::eventAvailable() frameStart.emit(event.u.frame_sync.frame_sequence); } +static const std::map v4l2ToColorSpace = { + { V4L2_COLORSPACE_RAW, ColorSpace::Raw }, + { V4L2_COLORSPACE_JPEG, ColorSpace::Jpeg }, + { V4L2_COLORSPACE_SRGB, ColorSpace::Srgb }, + { V4L2_COLORSPACE_SMPTE170M, ColorSpace::Smpte170m }, + { V4L2_COLORSPACE_REC709, ColorSpace::Rec709 }, + { V4L2_COLORSPACE_BT2020, ColorSpace::Rec2020 }, +}; + +static const std::map v4l2ToYcbcrEncoding = { + { V4L2_YCBCR_ENC_601, ColorSpace::YcbcrEncoding::Rec601 }, + { V4L2_YCBCR_ENC_709, ColorSpace::YcbcrEncoding::Rec709 }, + { V4L2_YCBCR_ENC_BT2020, ColorSpace::YcbcrEncoding::Rec2020 }, +}; + +static const std::map v4l2ToTransferFunction = { + { V4L2_XFER_FUNC_NONE, ColorSpace::TransferFunction::Linear }, + { V4L2_XFER_FUNC_SRGB, ColorSpace::TransferFunction::Srgb }, + { V4L2_XFER_FUNC_709, ColorSpace::TransferFunction::Rec709 }, +}; + +static const std::map v4l2ToRange = { + { V4L2_QUANTIZATION_FULL_RANGE, ColorSpace::Range::Full }, + { V4L2_QUANTIZATION_LIM_RANGE, ColorSpace::Range::Limited }, +}; + +static const std::vector> colorSpaceToV4l2 = { + { ColorSpace::Raw, V4L2_COLORSPACE_RAW }, + { ColorSpace::Jpeg, V4L2_COLORSPACE_JPEG }, + { ColorSpace::Srgb, V4L2_COLORSPACE_SRGB }, + { ColorSpace::Smpte170m, V4L2_COLORSPACE_SMPTE170M }, + { ColorSpace::Rec709, V4L2_COLORSPACE_REC709 }, + { ColorSpace::Rec2020, V4L2_COLORSPACE_BT2020 }, +}; + +static const std::map primariesToV4l2 = { + { ColorSpace::Primaries::Raw, V4L2_COLORSPACE_RAW }, + { ColorSpace::Primaries::Smpte170m, V4L2_COLORSPACE_SMPTE170M }, + { ColorSpace::Primaries::Rec709, V4L2_COLORSPACE_REC709 }, + { ColorSpace::Primaries::Rec2020, V4L2_COLORSPACE_BT2020 }, +}; + +static const std::map ycbcrEncodingToV4l2 = { + { ColorSpace::YcbcrEncoding::Rec601, V4L2_YCBCR_ENC_601 }, + { ColorSpace::YcbcrEncoding::Rec709, V4L2_YCBCR_ENC_709 }, + { ColorSpace::YcbcrEncoding::Rec2020, V4L2_YCBCR_ENC_BT2020 }, +}; + +static const std::map transferFunctionToV4l2 = { + { ColorSpace::TransferFunction::Linear, V4L2_XFER_FUNC_NONE }, + { ColorSpace::TransferFunction::Srgb, V4L2_XFER_FUNC_SRGB }, + { ColorSpace::TransferFunction::Rec709, V4L2_XFER_FUNC_709 }, +}; + +static const std::map rangeToV4l2 = { + { ColorSpace::Range::Full, V4L2_QUANTIZATION_FULL_RANGE }, + { ColorSpace::Range::Limited, V4L2_QUANTIZATION_LIM_RANGE }, +}; + +/** + * \brief Convert the color space fields in a V4L2 format to a ColorSpace + * \param[in] v4l2Format A V4L2 format containing color space information + * + * The colorspace, ycbcr_enc, xfer_func and quantization fields within a + * V4L2 format structure are converted to a corresponding ColorSpace. + * + * If any V4L2 fields are not recognised then we return an "unset" + * color space. + * + * \return The ColorSpace corresponding to the input V4L2 format + * \retval std::nullopt One or more V4L2 color space fields were not recognised + */ +template +std::optional V4L2Device::toColorSpace(const T &v4l2Format) +{ + auto itColor = v4l2ToColorSpace.find(v4l2Format.colorspace); + if (itColor == v4l2ToColorSpace.end()) + return std::nullopt; + + /* This sets all the color space fields to the correct "default" values. */ + ColorSpace colorSpace = itColor->second; + + if (v4l2Format.ycbcr_enc != V4L2_YCBCR_ENC_DEFAULT) { + auto itYcbcrEncoding = v4l2ToYcbcrEncoding.find(v4l2Format.ycbcr_enc); + if (itYcbcrEncoding == v4l2ToYcbcrEncoding.end()) + return std::nullopt; + + colorSpace.ycbcrEncoding = itYcbcrEncoding->second; + } + + if (v4l2Format.xfer_func != V4L2_XFER_FUNC_DEFAULT) { + auto itTransfer = v4l2ToTransferFunction.find(v4l2Format.xfer_func); + if (itTransfer == v4l2ToTransferFunction.end()) + return std::nullopt; + + colorSpace.transferFunction = itTransfer->second; + } + + if (v4l2Format.quantization != V4L2_QUANTIZATION_DEFAULT) { + auto itRange = v4l2ToRange.find(v4l2Format.quantization); + if (itRange == v4l2ToRange.end()) + return std::nullopt; + + colorSpace.range = itRange->second; + } + + return colorSpace; +} + +template std::optional V4L2Device::toColorSpace(const struct v4l2_pix_format &); +template std::optional V4L2Device::toColorSpace(const struct v4l2_pix_format_mplane &); +template std::optional V4L2Device::toColorSpace(const struct v4l2_mbus_framefmt &); + +/** + * \brief Fill in the color space fields of a V4L2 format from a ColorSpace + * \param[in] colorSpace The ColorSpace to be converted + * \param[out] v4l2Format A V4L2 format containing color space information + * + * The colorspace, ycbcr_enc, xfer_func and quantization fields within a + * V4L2 format structure are filled in from a corresponding ColorSpace. + * + * An error is returned if any of the V4L2 fields do not support the + * value given in the ColorSpace. Such fields are set to the V4L2 + * "default" values, but all other fields are still filled in where + * possible. + * + * If the color space is completely unset, "default" V4L2 values are used + * everywhere, so a driver would then choose its preferred color space. + * + * \return 0 on success or a negative error code otherwise + * \retval -EINVAL The ColorSpace does not have a representation using V4L2 enums + */ +template +int V4L2Device::fromColorSpace(const std::optional &colorSpace, T &v4l2Format) +{ + v4l2Format.colorspace = V4L2_COLORSPACE_DEFAULT; + v4l2Format.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; + v4l2Format.xfer_func = V4L2_XFER_FUNC_DEFAULT; + v4l2Format.quantization = V4L2_QUANTIZATION_DEFAULT; + + if (!colorSpace) + return 0; + + auto itColor = std::find_if(colorSpaceToV4l2.begin(), colorSpaceToV4l2.end(), + [&colorSpace](const auto &item) { + return colorSpace == item.first; + }); + if (itColor != colorSpaceToV4l2.end()) { + v4l2Format.colorspace = itColor->second; + /* Leaving all the other fields as "default" should be fine. */ + return 0; + } + + /* + * If the colorSpace doesn't precisely match a standard color space, + * then we must choose a V4L2 colorspace with matching primaries. + */ + int ret = 0; + + auto itPrimaries = primariesToV4l2.find(colorSpace->primaries); + if (itPrimaries != primariesToV4l2.end()) + v4l2Format.colorspace = itPrimaries->second; + else + ret = -EINVAL; + + auto itYcbcrEncoding = ycbcrEncodingToV4l2.find(colorSpace->ycbcrEncoding); + if (itYcbcrEncoding != ycbcrEncodingToV4l2.end()) + v4l2Format.ycbcr_enc = itYcbcrEncoding->second; + else + ret = -EINVAL; + + auto itTransfer = transferFunctionToV4l2.find(colorSpace->transferFunction); + if (itTransfer != transferFunctionToV4l2.end()) + v4l2Format.xfer_func = itTransfer->second; + else + ret = -EINVAL; + + auto itRange = rangeToV4l2.find(colorSpace->range); + if (itRange != rangeToV4l2.end()) + v4l2Format.quantization = itRange->second; + else + ret = -EINVAL; + + return ret; +} + +template int V4L2Device::fromColorSpace(const std::optional &, struct v4l2_pix_format &); +template int V4L2Device::fromColorSpace(const std::optional &, struct v4l2_pix_format_mplane &); +template int V4L2Device::fromColorSpace(const std::optional &, struct v4l2_mbus_framefmt &); + } /* namespace libcamera */ From patchwork Mon Dec 6 10:50:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 15045 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BD88EBDB13 for ; Mon, 6 Dec 2021 10:51:28 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9E95D60890; Mon, 6 Dec 2021 11:51:27 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="bKx91MYY"; dkim-atps=neutral Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9BB7C6086A for ; Mon, 6 Dec 2021 11:51:23 +0100 (CET) Received: by mail-wm1-x329.google.com with SMTP id o29so7775083wms.2 for ; Mon, 06 Dec 2021 02:51:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=7Z9S0RSj2zeDGxeJJB1gRj13Za+pW7JcT9rNai8iw68=; b=bKx91MYYzfyV+8sIYIX9zlHtdw9StqfQBy+Ls7++Pz2U+exMqlzLX5353YmcdHTq4Q zvWkqiA0CjYO6rr7SyRILcj7Qz5PWiOvDucBV1UaksuOX2qDb1t780lEVyUXh8hOVxpV xDGQboglgErJUEnS8pA4Ytw137Jn6mI5rMReskx7JAcgWWfz3lEI5639jICdklNBFn3O nwiZXv6huxKkOtZ+o9OPOmdU+jf1OHGOmvBsm9mTNB5dk1OGEtR+54zzLA7JFRt2tG7p KWeXwZpVUHfxbqk3Qi8ZXZn1EqHGPX52UD2aLYL+HVk/ziqcY+9JkhNB5NB5RqqNwD7Y +izA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=7Z9S0RSj2zeDGxeJJB1gRj13Za+pW7JcT9rNai8iw68=; b=Xq1r5FYmLv7rlOA/f3Ya8hCDkkVN/ZoQmCHnqrblXGG02qrOqCCYYO6xkBrSSPkvOz AK6DKYRUyj53HbKDR2PQHbkhBs6ogqjwwzh/k3J7wTWcisWkVbHoypX11us0cWx2FFJS FPnGigXqsPlcjMYrYYLlZ2fDFpGslTufg8OkiTUxAk5V/atIq7tCBwAFsqKrZqscK+P2 DD4yzSNF0g+3BQsrXKw3MVRsdMy6O+eP3MZyG3qmdAqaU/kwvoTsaW1/lNK4/hBfvFVR UCUxZX0jsHCJmIw1JG/kqXq73j+DRWkoTFKAheBLoopQqzcdvWxxhUWnrhke4SAXiKFn i3aA== X-Gm-Message-State: AOAM530PvoroyoY+q99Dcrh+3MYsMUdJDexBi6yZTtVgihCoUeZI6Tuu rb6hIGZtdCVX7rhGA/4k5Ah+KgbaLaLyAW/u X-Google-Smtp-Source: ABdhPJydW02aJHN1Ge8TK1wc17quwLO2bFKiUu8Rutb6brv70TlhUEFrPvZymHgu8wpCsgK8GkqnMw== X-Received: by 2002:a7b:c30e:: with SMTP id k14mr37191597wmj.156.1638787883145; Mon, 06 Dec 2021 02:51:23 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id r7sm10878186wrq.29.2021.12.06.02.51.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Dec 2021 02:51:22 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org, naush@raspberrypi.com, jacopo@jmondi.org, tfiga@google.com, hverkuil-cisco@xs4all.nl, kieran.bingham@ideasonboard.com, laurent.pinchart@ideasonboard.com Date: Mon, 6 Dec 2021 10:50:27 +0000 Message-Id: <20211206105032.13876-5-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211206105032.13876-1-david.plowman@raspberrypi.com> References: <20211206105032.13876-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v9 4/8] libcamera: Support passing ColorSpaces to V4L2 video devices X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The ColorSpace from the StreamConfiguration is now handled appropriately in the V4L2VideoDevice. Signed-off-by: David Plowman Reviewed-by: Laurent Pinchart --- include/libcamera/internal/v4l2_videodevice.h | 2 ++ src/libcamera/v4l2_videodevice.cpp | 32 +++++++++++++++++++ 2 files changed, 34 insertions(+) diff --git a/include/libcamera/internal/v4l2_videodevice.h b/include/libcamera/internal/v4l2_videodevice.h index 9b2ec3af..6a507a51 100644 --- a/include/libcamera/internal/v4l2_videodevice.h +++ b/include/libcamera/internal/v4l2_videodevice.h @@ -21,6 +21,7 @@ #include #include +#include #include #include #include @@ -167,6 +168,7 @@ public: V4L2PixelFormat fourcc; Size size; + std::optional colorSpace; std::array planes; unsigned int planesCount = 0; diff --git a/src/libcamera/v4l2_videodevice.cpp b/src/libcamera/v4l2_videodevice.cpp index b4b89e27..80a8c8e4 100644 --- a/src/libcamera/v4l2_videodevice.cpp +++ b/src/libcamera/v4l2_videodevice.cpp @@ -383,6 +383,21 @@ bool V4L2BufferCache::Entry::operator==(const FrameBuffer &buffer) const * that identifies the image format pixel encoding scheme. */ +/** + * \var V4L2DeviceFormat::colorSpace + * \brief The color space of the pixels + * + * The color space of the image. When setting the format this may be + * unset, in which case the driver gets to use its default color space. + * After being set, this value should contain the color space that + * was actually used. If this value is unset, then the color space chosen + * by the driver could not be represented by the ColorSpace class (and + * should probably be added). + * + * It is up to the pipeline handler or application to check if the + * resulting color space is acceptable. + */ + /** * \var V4L2DeviceFormat::planes * \brief The per-plane memory size information @@ -871,6 +886,7 @@ int V4L2VideoDevice::getFormatMultiplane(V4L2DeviceFormat *format) format->size.height = pix->height; format->fourcc = V4L2PixelFormat(pix->pixelformat); format->planesCount = pix->num_planes; + format->colorSpace = toColorSpace(*pix); for (unsigned int i = 0; i < format->planesCount; ++i) { format->planes[i].bpl = pix->plane_fmt[i].bytesperline; @@ -893,6 +909,12 @@ int V4L2VideoDevice::trySetFormatMultiplane(V4L2DeviceFormat *format, bool set) pix->num_planes = format->planesCount; pix->field = V4L2_FIELD_NONE; + ret = fromColorSpace(format->colorSpace, *pix); + if (ret < 0) + LOG(V4L2, Warning) + << "Setting color space unrecognised by V4L2: " + << ColorSpace::toString(format->colorSpace); + ASSERT(pix->num_planes <= std::size(pix->plane_fmt)); for (unsigned int i = 0; i < pix->num_planes; ++i) { @@ -920,6 +942,7 @@ int V4L2VideoDevice::trySetFormatMultiplane(V4L2DeviceFormat *format, bool set) format->planes[i].bpl = pix->plane_fmt[i].bytesperline; format->planes[i].size = pix->plane_fmt[i].sizeimage; } + format->colorSpace = toColorSpace(*pix); return 0; } @@ -943,6 +966,7 @@ int V4L2VideoDevice::getFormatSingleplane(V4L2DeviceFormat *format) format->planesCount = 1; format->planes[0].bpl = pix->bytesperline; format->planes[0].size = pix->sizeimage; + format->colorSpace = toColorSpace(*pix); return 0; } @@ -959,6 +983,13 @@ int V4L2VideoDevice::trySetFormatSingleplane(V4L2DeviceFormat *format, bool set) pix->pixelformat = format->fourcc; pix->bytesperline = format->planes[0].bpl; pix->field = V4L2_FIELD_NONE; + + ret = fromColorSpace(format->colorSpace, *pix); + if (ret < 0) + LOG(V4L2, Warning) + << "Set color space unrecognised by V4L2: " + << ColorSpace::toString(format->colorSpace); + ret = ioctl(set ? VIDIOC_S_FMT : VIDIOC_TRY_FMT, &v4l2Format); if (ret) { LOG(V4L2, Error) @@ -977,6 +1008,7 @@ int V4L2VideoDevice::trySetFormatSingleplane(V4L2DeviceFormat *format, bool set) format->planesCount = 1; format->planes[0].bpl = pix->bytesperline; format->planes[0].size = pix->sizeimage; + format->colorSpace = toColorSpace(*pix); return 0; } From patchwork Mon Dec 6 10:50:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 15046 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 55AFAC3258 for ; Mon, 6 Dec 2021 10:51:29 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 346A46011A; Mon, 6 Dec 2021 11:51:28 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="Pqa97l7n"; dkim-atps=neutral Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3E76660882 for ; Mon, 6 Dec 2021 11:51:24 +0100 (CET) Received: by mail-wm1-x334.google.com with SMTP id i8-20020a7bc948000000b0030db7b70b6bso10070615wml.1 for ; Mon, 06 Dec 2021 02:51:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=zrDnyKLJISq8ZIBK5MpItxEAiAskZ2+gyT4ETPipmZ0=; b=Pqa97l7nxT+bSUQCWzieCFEPUVJmHlfNdphq1yPMPn6f2GEjNSVsI9N4PWWHRhR7cy fzHtFK4xhnpcyRs6H5vuBrqsa0RumjQ4xxGTomVLOLq6IyqFIFmZNIRrKvZ7po6AMXOd w/T21neeru380gGHzWZBZn+CtgfOE4X3FEhTyxPBPof+l1iw/XXfSB7Qgwmi95/SR88T 5H2/IA4dOH1YWicX2rRxKPwnmlVsP5K6tB5a65UL3MQHZaxTB391k+dpoo+rrWvgpCjo IDRuFgCNsw4XmywKGBC7clOSWF3g0DrjKXbpcR6k9LwoJAhOn+KQj/p5c6xUwlFF2zAa ZQeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zrDnyKLJISq8ZIBK5MpItxEAiAskZ2+gyT4ETPipmZ0=; b=ifXhmwX7K9zP0Ys5FbbBcq2ZVMCeYbz2lTFEadqjHQYeTW1FXBxqAQlSXEABZj/oOa /KMiL/C71SeIAPcfi6Y3rhnEZhEAxTa1AVpRTsQw2uYqNaUChxS8lWX7xy6CbxL6e7UX PXRynd+T+nMN3WbXgmrLgjEJ6dvs1JDXeE3fu5GyiGHdxJ/nf0h+cs+NNq27Ts8ZqOgh xk6DfAdkE10yZdrhl16OS5aDhU/pieyO9A77bT+8LauQDtuTY5ijZgy2WhErcUogOy6h 8mOt3bA4R+cuQRfzGS7dzZdwTkcGY0fkxipuAszsYA90D66k8F2fDuoOTCxAyKom4DbP aGiQ== X-Gm-Message-State: AOAM531aIkhQcxoI8TatiFpzlsBXaolzUqWshCYJBVT6tqrohFdK6lbS JE8q/kvF5y6cZVmQORjH95IvGCvC8UqdActv X-Google-Smtp-Source: ABdhPJxk9nOkt6gEyZohX0YWRH1K9gaCBOSk539n1hu1SSUJ1LmKUr9ZbY0dW0X6Lk6BLYYhDI+H8w== X-Received: by 2002:a05:600c:4e8f:: with SMTP id f15mr23292684wmq.76.1638787883790; Mon, 06 Dec 2021 02:51:23 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id r7sm10878186wrq.29.2021.12.06.02.51.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Dec 2021 02:51:23 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org, naush@raspberrypi.com, jacopo@jmondi.org, tfiga@google.com, hverkuil-cisco@xs4all.nl, kieran.bingham@ideasonboard.com, laurent.pinchart@ideasonboard.com Date: Mon, 6 Dec 2021 10:50:28 +0000 Message-Id: <20211206105032.13876-6-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211206105032.13876-1-david.plowman@raspberrypi.com> References: <20211206105032.13876-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v9 5/8] libcamera: Add colorSpace field to V4L2SubdeviceFormat X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This adds a ColorSpace field to the V4L2SubdeviceFormat so that we can set and request particular color spaces from V4L2. This commit simply adds the field and fixes some occurrences of brace initializers that would otherwise be broken. A subsequent commit will pass and retrieve the value correctly to/from V4l2 itself. Signed-off-by: David Plowman Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- include/libcamera/internal/v4l2_subdevice.h | 2 ++ src/libcamera/camera_sensor.cpp | 2 ++ src/libcamera/pipeline/ipu3/cio2.cpp | 7 +++---- src/libcamera/pipeline/simple/simple.cpp | 8 ++++++-- src/libcamera/v4l2_subdevice.cpp | 11 +++++++++++ 5 files changed, 24 insertions(+), 6 deletions(-) diff --git a/include/libcamera/internal/v4l2_subdevice.h b/include/libcamera/internal/v4l2_subdevice.h index a6873b67..358bf2b6 100644 --- a/include/libcamera/internal/v4l2_subdevice.h +++ b/include/libcamera/internal/v4l2_subdevice.h @@ -14,6 +14,7 @@ #include #include +#include #include #include "libcamera/internal/formats.h" @@ -27,6 +28,7 @@ class MediaDevice; struct V4L2SubdeviceFormat { uint32_t mbus_code; Size size; + std::optional colorSpace; const std::string toString() const; uint8_t bitsPerPixel() const; diff --git a/src/libcamera/camera_sensor.cpp b/src/libcamera/camera_sensor.cpp index 4c142a58..14358333 100644 --- a/src/libcamera/camera_sensor.cpp +++ b/src/libcamera/camera_sensor.cpp @@ -15,6 +15,7 @@ #include #include +#include #include #include @@ -586,6 +587,7 @@ V4L2SubdeviceFormat CameraSensor::getFormat(const std::vector &mbu V4L2SubdeviceFormat format{ .mbus_code = bestCode, .size = *bestSize, + .colorSpace = ColorSpace::Raw, }; return format; diff --git a/src/libcamera/pipeline/ipu3/cio2.cpp b/src/libcamera/pipeline/ipu3/cio2.cpp index 59dda56b..f4e8c663 100644 --- a/src/libcamera/pipeline/ipu3/cio2.cpp +++ b/src/libcamera/pipeline/ipu3/cio2.cpp @@ -322,10 +322,9 @@ V4L2SubdeviceFormat CIO2Device::getSensorFormat(const std::vector return {}; } - V4L2SubdeviceFormat format{ - .mbus_code = bestCode, - .size = bestSize, - }; + V4L2SubdeviceFormat format{}; + format.mbus_code = bestCode; + format.size = bestSize; return format; } diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 701fb4be..a3108fc0 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -457,7 +457,9 @@ int SimpleCameraData::init() * formats on the video node. */ for (unsigned int code : sensor_->mbusCodes()) { - V4L2SubdeviceFormat format{ code, sensor_->resolution() }; + V4L2SubdeviceFormat format{}; + format.mbus_code = code; + format.size = sensor_->resolution(); ret = setupFormats(&format, V4L2Subdevice::TryFormat); if (ret < 0) { @@ -908,7 +910,9 @@ int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c) return ret; const SimpleCameraData::Configuration *pipeConfig = config->pipeConfig(); - V4L2SubdeviceFormat format{ pipeConfig->code, data->sensor_->resolution() }; + V4L2SubdeviceFormat format{}; + format.mbus_code = pipeConfig->code; + format.size = data->sensor_->resolution(); ret = data->setupFormats(&format, V4L2Subdevice::ActiveFormat); if (ret < 0) diff --git a/src/libcamera/v4l2_subdevice.cpp b/src/libcamera/v4l2_subdevice.cpp index 61e15b69..981645e0 100644 --- a/src/libcamera/v4l2_subdevice.cpp +++ b/src/libcamera/v4l2_subdevice.cpp @@ -169,6 +169,17 @@ const std::map formatInfoMap = { * \brief The image size in pixels */ +/** + * \var V4L2SubdeviceFormat::colorSpace + * \brief The color space of the pixels + * + * The color space of the image. When setting the format this may be + * unset, in which case the driver gets to use its default color space. + * If this value is unset after a call to validate(), then the color space + * chosen by the driver could not be represented by the ColorSpace class + * (and should probably be added). + */ + /** * \brief Assemble and return a string describing the format * \return A string describing the V4L2SubdeviceFormat From patchwork Mon Dec 6 10:50:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 15047 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A7341C3259 for ; Mon, 6 Dec 2021 10:51:29 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 973F2608A1; Mon, 6 Dec 2021 11:51:28 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="OmnOPz7K"; dkim-atps=neutral Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D7D1260890 for ; Mon, 6 Dec 2021 11:51:24 +0100 (CET) Received: by mail-wm1-x32c.google.com with SMTP id d72-20020a1c1d4b000000b00331140f3dc8so7291649wmd.1 for ; Mon, 06 Dec 2021 02:51:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=mbZZhCZgEn6YYtA3/51M6wXPUxmxfiWlFLNpGT7zY88=; b=OmnOPz7KK1PUrnlrkVkEGKE2DOb0pnrTc4hS4ZYKAk6g4gEPJvG2tmLRILvIfvaxcM dwyloi+e7jgTaUrmQvZMqFTxQVIAhIFlCT5drGSygg/r49OkDHlt8V0FBJZueBByAtnO oDkPbaFY78wuUc04tm1pD2WtnWSLPenQIhtNrnabTrHfT9L0Ft2/ZXlzDIGVUcofgr1o EfSEjrdEzGiJAIDaUQS9G0CveokcimmFNWP4qIvuI6QQdaNwLazgqSeTqCTlfFvLy6dm 2A6WCdbgw/aWaEVZwoEToYMgciIMTXZRQqWqDr0wt6KShWv33xenB8C50K2Z2c2OnxW5 LJGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mbZZhCZgEn6YYtA3/51M6wXPUxmxfiWlFLNpGT7zY88=; b=Jn/pUKxmi3dv93WIlU0SGZMkeBtC0PvmDOxnh1aksw9aONY4niiRkgL4alnvNQRbRx bV+jnSgGPxPyL+i4sGBp8IcMH82fABoM3qcpfZJy0YJGcPPZgc2kJBW8VWWvEvkudvLQ pGG/DttRIH6j74Mm2DyvrES7WqvrqX9xBGPxJ1iT8wKX5KWVGdvUnNtTpl5OYC2LZOcs 93H9d+Odj04ZxaTAKu+p3Z7Y2W4iY6NpT7GKVh+2yZFvsfTetz+r8HXQm6G6pYb3yVSj k+7j/9bu0aJeArCQ0ilrFo3h61SDGrzgQYiDSkAODaEa8rAjdmGOI1VcLEVa7glgOb8Z Dz3g== X-Gm-Message-State: AOAM530ugX3SSC3PjbA2ST42hzTjALsAH4ZZAmX4LOtQiT0E+PSB6RB4 dWmIfxZlmveoopIN3H0Gzer/H4BXHKbQAQM+ X-Google-Smtp-Source: ABdhPJzwgttQ9pRArkqaJc+VFVWVn+2f+jVthpKcKwbdNM6RyefMwZ5f26zwtP5SAan7ts9JQecMbg== X-Received: by 2002:a7b:c109:: with SMTP id w9mr38978470wmi.114.1638787884430; Mon, 06 Dec 2021 02:51:24 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id r7sm10878186wrq.29.2021.12.06.02.51.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Dec 2021 02:51:24 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org, naush@raspberrypi.com, jacopo@jmondi.org, tfiga@google.com, hverkuil-cisco@xs4all.nl, kieran.bingham@ideasonboard.com, laurent.pinchart@ideasonboard.com Date: Mon, 6 Dec 2021 10:50:29 +0000 Message-Id: <20211206105032.13876-7-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211206105032.13876-1-david.plowman@raspberrypi.com> References: <20211206105032.13876-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v9 6/8] libcamera: Support passing ColorSpaces to V4L2 subdevices X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The ColorSpace from the StreamConfiguration is now handled appropriately in the V4L2Subdevice. Signed-off-by: David Plowman Reviewed-by: Laurent Pinchart --- src/libcamera/v4l2_subdevice.cpp | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/src/libcamera/v4l2_subdevice.cpp b/src/libcamera/v4l2_subdevice.cpp index 981645e0..f5ec6901 100644 --- a/src/libcamera/v4l2_subdevice.cpp +++ b/src/libcamera/v4l2_subdevice.cpp @@ -411,6 +411,7 @@ int V4L2Subdevice::getFormat(unsigned int pad, V4L2SubdeviceFormat *format, format->size.width = subdevFmt.format.width; format->size.height = subdevFmt.format.height; format->mbus_code = subdevFmt.format.code; + format->colorSpace = toColorSpace(subdevFmt.format); return 0; } @@ -439,7 +440,13 @@ int V4L2Subdevice::setFormat(unsigned int pad, V4L2SubdeviceFormat *format, subdevFmt.format.code = format->mbus_code; subdevFmt.format.field = V4L2_FIELD_NONE; - int ret = ioctl(VIDIOC_SUBDEV_S_FMT, &subdevFmt); + int ret = fromColorSpace(format->colorSpace, subdevFmt.format); + if (ret < 0) + LOG(V4L2, Warning) + << "Setting color space unrecognised by V4L2: " + << ColorSpace::toString(format->colorSpace); + + ret = ioctl(VIDIOC_SUBDEV_S_FMT, &subdevFmt); if (ret) { LOG(V4L2, Error) << "Unable to set format on pad " << pad @@ -450,6 +457,7 @@ int V4L2Subdevice::setFormat(unsigned int pad, V4L2SubdeviceFormat *format, format->size.width = subdevFmt.format.width; format->size.height = subdevFmt.format.height; format->mbus_code = subdevFmt.format.code; + format->colorSpace = toColorSpace(subdevFmt.format); return 0; } From patchwork Mon Dec 6 10:50:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 15048 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BC887BDB13 for ; Mon, 6 Dec 2021 10:51:31 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 64AD560872; Mon, 6 Dec 2021 11:51:31 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="QvfVYvKD"; dkim-atps=neutral Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 96C2760882 for ; Mon, 6 Dec 2021 11:51:25 +0100 (CET) Received: by mail-wm1-x335.google.com with SMTP id az34-20020a05600c602200b0033bf8662572so7328657wmb.0 for ; Mon, 06 Dec 2021 02:51:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=tv3O2cAwn+Kn6H1MBehwoCfF/wdQzrC35hxIG5RHyWI=; b=QvfVYvKD7oPnr4rLWDite3Xu2xnCv67rZV0E3i8R1CIRnjtfSzB2FbXnRiYPQW803n TIVGlDv8h9TBFruetqQDxgx7LwUyMpEsfxh4dnsMDBNqpbFirCt1jWcTm678v7TpHoPw ex07iUGlFSybpP+0R9KkHwP/pUopLMXcfkcr0iiZ93qq74MRTwRevk8TW1CPJkgh5x6Z rKK4LwtKYgy6wWkn9THhyBX3Tohb16vvSD5tHYY+MG18fnqKA7MbjRLJnvsYqI/Lj4iz KaK1E5Od7lTqa0p/r6zR7fLP0vjsndqCirh/mW/fIDpTX/WmygmiNEGXjbLb6yGgQZOm N7uA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tv3O2cAwn+Kn6H1MBehwoCfF/wdQzrC35hxIG5RHyWI=; b=NvlDk21y/0KeONqikbaKF8kYJTvH6UyNJuo2SYLPw8Fvr2FNKHapOMGKVbqMJRSty2 72VwNWdtIWJFrN9DphLDogpyToNUxYL9kr40Q1Fzg/uGoM+wxFS8qtGx0UEmYtsJa1Kh 7F38RwD73VVkEacwJIhQalIT551vIIvxA4Lx9zmJI77UFc3aM3ZJXaC2OGvq1LbnbVt4 MyR84E1CV4ZPl+P9hWFfLzoPxzC2UpUp7NNndmkMS3KsBkggqtEMmrFsvCJClyy3ZUrd i9lrkrjojbvDmy8pTlBMloGtUKo+oOqlCkVH2c+VfbHEwi0gvX3zgdujxIUty3nuvfQs X/PA== X-Gm-Message-State: AOAM533IS/ynXsWg6Pm+EdHLbSk1kUxBpXnsn6MB3zCpt2SxkhB08Rw+ RV9s/j1h7YqYDkFKuDv20D1rKcR/3p1P4nh+ X-Google-Smtp-Source: ABdhPJyT89CBV3D6FitkSCS9gqNX/NlMSgadA9P9qpGfVWlqqOcExiMYkIHVBtKVEJpJtYfJz9j5Rw== X-Received: by 2002:a05:600c:350a:: with SMTP id h10mr22752368wmq.62.1638787885135; Mon, 06 Dec 2021 02:51:25 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id r7sm10878186wrq.29.2021.12.06.02.51.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Dec 2021 02:51:24 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org, naush@raspberrypi.com, jacopo@jmondi.org, tfiga@google.com, hverkuil-cisco@xs4all.nl, kieran.bingham@ideasonboard.com, laurent.pinchart@ideasonboard.com Date: Mon, 6 Dec 2021 10:50:30 +0000 Message-Id: <20211206105032.13876-8-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211206105032.13876-1-david.plowman@raspberrypi.com> References: <20211206105032.13876-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v9 7/8] libcamera: Add validateColorSpaces to CameraConfiguration class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This method forces raw streams to have the "raw" color space, and also optionally makes all output streams to share the same color space as some platforms may require this. Signed-off-by: David Plowman Reviewed-by: Jacopo Mondi --- include/libcamera/camera.h | 2 ++ src/libcamera/camera.cpp | 59 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 61 insertions(+) diff --git a/include/libcamera/camera.h b/include/libcamera/camera.h index a7759ccb..fdcb66ab 100644 --- a/include/libcamera/camera.h +++ b/include/libcamera/camera.h @@ -69,6 +69,8 @@ public: protected: CameraConfiguration(); + Status validateColorSpaces(bool shareOutputColorSpaces); + std::vector config_; }; diff --git a/src/libcamera/camera.cpp b/src/libcamera/camera.cpp index 400a7cf0..388af4f7 100644 --- a/src/libcamera/camera.cpp +++ b/src/libcamera/camera.cpp @@ -14,12 +14,14 @@ #include #include +#include #include #include #include #include "libcamera/internal/camera.h" #include "libcamera/internal/camera_controls.h" +#include "libcamera/internal/formats.h" #include "libcamera/internal/pipeline_handler.h" /** @@ -314,6 +316,63 @@ std::size_t CameraConfiguration::size() const return config_.size(); } +static bool isRaw(const PixelFormat &pixFmt) +{ + const PixelFormatInfo &info = PixelFormatInfo::info(pixFmt); + return info.isValid() && + info.colourEncoding == PixelFormatInfo::ColourEncodingRAW; +} + +/** + * \brief Check and update the color spaces requested for each stream + * \param[in] shareOutputColorSpaces Force all non-raw output streams to share + * the same color space + * + * This function performs certain consistency checks on the color spaces of + * the streams and may adjust them so that: + * + * - Any raw streams have the Raw color space + * - If shareOutputColorSpaces is set, all output streams are forced to share + * the same color space (this may be a constraint on some platforms). + * + * It is optional for a pipeline handler to use this method. + * + * \return A CameraConfiguration::Status value that describes the validation + * status (same as CameraConfiguration::validate). + */ +CameraConfiguration::Status CameraConfiguration::validateColorSpaces(bool shareOutputColorSpaces) +{ + Status status = Valid; + + /* + * Set all raw streams to the Raw color space, and make a note of the largest + * non-raw stream with a defined color space (if there is one). + */ + int index = -1; + for (auto [i, cfg] : utils::enumerate(config_)) { + if (isRaw(cfg.pixelFormat) && cfg.colorSpace != ColorSpace::Raw) { + cfg.colorSpace = ColorSpace::Raw; + status = Adjusted; + } + else if (cfg.colorSpace && (index == -1 || cfg.size > config_[i].size)) + index = i; + } + + if (index < 0 || !shareOutputColorSpaces) + return status; + + /* Make all output color spaces the same, if requested. */ + for (auto &cfg : config_) { + if (!isRaw(cfg.pixelFormat) && + cfg.colorSpace != config_[index].colorSpace) { + cfg.colorSpace = config_[index].colorSpace; + status = Adjusted; + } + } + + return status; +} + /** * \var CameraConfiguration::transform * \brief User-specified transform to be applied to the image From patchwork Mon Dec 6 10:50:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 15049 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3C7F3C324B for ; Mon, 6 Dec 2021 10:51:32 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C4BEA60725; Mon, 6 Dec 2021 11:51:31 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="tCE+VYLv"; dkim-atps=neutral Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com [IPv6:2a00:1450:4864:20::32d]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 50B51608A4 for ; Mon, 6 Dec 2021 11:51:26 +0100 (CET) Received: by mail-wm1-x32d.google.com with SMTP id p18so7746141wmq.5 for ; Mon, 06 Dec 2021 02:51:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=2mECcTwXK1O2rNwFlwteHR9qSm4YdLlUAiGG44luOJs=; b=tCE+VYLvWR/uo/Gfyxyb2oNH3dhg2PcTphd1Q5ol1s2qHeMhPxbF+IPutGMZqQ+Wfd s1v+UEV0+wLBw4egy2tNoXqAmtuDnBDDPN4Vxq2XpYyVH1OT6Vuw/aViL6aCZWOPe+v5 WUjHsiU2+IqXSSShxQrl2hj4AtV3XgbdzzMnHV9okuLoZqJyf2ssppOH1SUJtdktixzy LbX+T17POI4dbM+uAh5nVHIAzP3GD8pX2LliMFUHM9qKKkhXcBack+JYhjHSZlqRsOEZ g51PuXK8XPGng2QSwByxgVAKqBCphZCx4hqrnJ4CPOiJE9eH7qdErNRET49s2TlUcr3c ES1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2mECcTwXK1O2rNwFlwteHR9qSm4YdLlUAiGG44luOJs=; b=drV4gfHIAZSDP00V0vzMHpdbL2KaN2TunmdDgUC9z36P6Jk4s8JWVYwfZKEDKjpl9U nb4WSGe7XgC5hSlQmVgFvDyXvqscCNGUdo47+sRUZVdIjmlzMxc/AAiewIoJv/XKmJIp S5rviSfW+SIvVKPn441T0JUoBewGpXW8vPUmuuWLPzWrltyBRTuolATnrXxGKhG+MhyV RD64hiMfMqICbxvo9ah/zzL82r3DEnQ+0kF+Cx11QwNpJTtAP51x3kKRw1NGb2eFVsFX 8tYZ6aHTYkxzcC8DDSAHtrvQsFPoS0IxXqksl2mxpbuZ9S9BTv/Cj9MErrPAPpcqfbML 3MeA== X-Gm-Message-State: AOAM530+B6ZokvX00nqjhMMN2UDkpazyjU7lalE2OCfoBLP2JRbLs2Ik bWTZZdSjkwUdJcr4g9hRofgnSrRvvInLA6rJ X-Google-Smtp-Source: ABdhPJxkUoEKdbDhdiTZR8PVUECDf3a0GamNnVkBuXbw9TX8p+aGobzkKhQNUtftqQTF1wQv3wc6/A== X-Received: by 2002:a05:600c:6d2:: with SMTP id b18mr38826167wmn.98.1638787885827; Mon, 06 Dec 2021 02:51:25 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id r7sm10878186wrq.29.2021.12.06.02.51.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Dec 2021 02:51:25 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org, naush@raspberrypi.com, jacopo@jmondi.org, tfiga@google.com, hverkuil-cisco@xs4all.nl, kieran.bingham@ideasonboard.com, laurent.pinchart@ideasonboard.com Date: Mon, 6 Dec 2021 10:50:31 +0000 Message-Id: <20211206105032.13876-9-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211206105032.13876-1-david.plowman@raspberrypi.com> References: <20211206105032.13876-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v9 8/8] libcamera: pipeline: raspberrypi: Support color spaces X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The Raspberry Pi pipeline handler now sets color spaces correctly. In generateConfiguration() it sets them to reasonable default values based on the stream role. validate() now calls validateColorSpaces() to ensure that the requested color spaces are sensible, before proceeding to check what the hardware can deliver. Signed-off-by: David Plowman --- .../pipeline/raspberrypi/raspberrypi.cpp | 43 +++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 321b72ad..eb4fbf90 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -97,6 +97,7 @@ V4L2DeviceFormat toV4L2DeviceFormat(const V4L2SubdeviceFormat &format, deviceFormat.fourcc = V4L2PixelFormat::fromPixelFormat(pix); deviceFormat.size = format.size; + deviceFormat.colorSpace = format.colorSpace; return deviceFormat; } @@ -133,6 +134,7 @@ V4L2SubdeviceFormat findBestFormat(const SensorFormats &formatsMap, const Size & { double bestScore = std::numeric_limits::max(), score; V4L2SubdeviceFormat bestFormat; + bestFormat.colorSpace = ColorSpace::Raw; constexpr float penaltyAr = 1500.0; constexpr float penaltyBitDepth = 500.0; @@ -330,6 +332,8 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() if (config_.empty()) return Invalid; + status = validateColorSpaces(true); + /* * What if the platform has a non-90 degree rotation? We can't even * "adjust" the configuration and carry on. Alternatively, raising an @@ -497,11 +501,23 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() V4L2DeviceFormat format; format.fourcc = V4L2PixelFormat::fromPixelFormat(cfg.pixelFormat); format.size = cfg.size; + format.colorSpace = cfg.colorSpace; + LOG(RPI, Debug) + << "Try color space " << ColorSpace::toString(cfg.colorSpace); int ret = dev->tryFormat(&format); if (ret) return Invalid; + if (!format.colorSpace || cfg.colorSpace != format.colorSpace) { + status = Adjusted; + LOG(RPI, Warning) + << "Color space changed from " + << ColorSpace::toString(cfg.colorSpace) << " to " + << ColorSpace::toString(format.colorSpace); + } + cfg.colorSpace = format.colorSpace; + cfg.stride = format.planes[0].bpl; cfg.frameSize = format.planes[0].size; @@ -525,6 +541,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, PixelFormat pixelFormat; V4L2VideoDevice::Formats fmts; Size size; + std::optional colorSpace; if (roles.empty()) return config; @@ -539,6 +556,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, pixelFormat = mbusCodeToPixelFormat(sensorFormat.mbus_code, BayerFormat::Packing::CSI2); ASSERT(pixelFormat.isValid()); + colorSpace = ColorSpace::Raw; bufferCount = 2; rawCount++; break; @@ -546,6 +564,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, case StreamRole::StillCapture: fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::NV12; + colorSpace = ColorSpace::Jpeg; /* Return the largest sensor resolution. */ size = data->sensor_->resolution(); bufferCount = 1; @@ -563,6 +582,8 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, */ fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::YUV420; + /* This will be reasonable for many applications. */ + colorSpace = ColorSpace::Rec709; size = { 1920, 1080 }; bufferCount = 4; outCount++; @@ -571,6 +592,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, case StreamRole::Viewfinder: fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::ARGB8888; + colorSpace = ColorSpace::Jpeg; size = { 800, 600 }; bufferCount = 4; outCount++; @@ -602,6 +624,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, StreamConfiguration cfg(formats); cfg.size = size; cfg.pixelFormat = pixelFormat; + cfg.colorSpace = colorSpace; cfg.bufferCount = bufferCount; config->addConfiguration(cfg); } @@ -709,6 +732,7 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) V4L2PixelFormat fourcc = V4L2PixelFormat::fromPixelFormat(cfg.pixelFormat); format.size = cfg.size; format.fourcc = fourcc; + format.colorSpace = cfg.colorSpace; LOG(RPI, Debug) << "Setting " << stream->name() << " to " << format.toString(); @@ -724,6 +748,23 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) return -EINVAL; } + if (!format.colorSpace || cfg.colorSpace != format.colorSpace) { + /* + * We should have been through validate() before so this + * shouldn't be possible, but we mustn't sweep color space + * problems under the carpet. + */ + LOG(RPI, Warning) + << "Unexpected color space (" + << ColorSpace::toString(cfg.colorSpace) << " to " + << ColorSpace::toString(format.colorSpace) << ") in stream " + << stream->name(); + cfg.colorSpace = format.colorSpace; + } + LOG(RPI, Debug) + << "Stream " << stream->name() << " has color space " + << ColorSpace::toString(cfg.colorSpace); + cfg.setStream(stream); stream->setExternal(true); @@ -748,6 +789,8 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) format = {}; format.size = maxSize; format.fourcc = V4L2PixelFormat::fromPixelFormat(formats::YUV420); + /* No one asked for output, so the color space doesn't matter. */ + format.colorSpace = ColorSpace::Jpeg; ret = data->isp_[Isp::Output0].dev()->setFormat(&format); if (ret) { LOG(RPI, Error)