From patchwork Thu Nov 4 13:57:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 14458 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8FAC7BDB1C for ; Thu, 4 Nov 2021 13:58:27 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 332C66035A; Thu, 4 Nov 2021 14:58:26 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="tAi2ROXZ"; dkim-atps=neutral Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com [IPv6:2a00:1450:4864:20::332]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 19CAC60128 for ; Thu, 4 Nov 2021 14:58:24 +0100 (CET) Received: by mail-wm1-x332.google.com with SMTP id a20-20020a1c7f14000000b003231d13ee3cso7277706wmd.3 for ; Thu, 04 Nov 2021 06:58:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=tlUI2V+eGxAjDG+xyKZcONVDwUS7KK1blhVHHWSuJ+I=; b=tAi2ROXZpuI0tHu4N+vkZr3HtUo56wNoWKdMVAqE8+VMpnyaT/KegiU7sGQP+obvBd 7uZEMgajYBzijzYVBgb1FS9SVsgQgVOs2iif+1WqKcr8d2FgmChfWHxiKaoqPxuOwSDt NV5bsxPLgiBQZoP0obaJ44wmyvTJ3mYfpcekKacffkvK7cdPukGUwb1Q+bDrpqTWkEMN l35m2EsNdir93S0YAEgkNJ5YYJ7P0r7WqfGPLwornITOekAtYvFHbm7q5AeLaPdQICFE hitU8oooJOLplmqlamDpJOtcvOkoOTGerYvzXK+9D9aDYd1PVCniulsCD3vFelpnUuH9 chZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tlUI2V+eGxAjDG+xyKZcONVDwUS7KK1blhVHHWSuJ+I=; b=edfy5YdzAJ0uKTjaGW/svjGTJtC7cJYvlDYR3rM/GDKgpEn9IltMhVVN2n+Q93jU5e 8kUdLva1q8GPLF6/Zd5H1wtffZNFkR3YJzlS359/h/cjbUu6/xNDP79H0mfFLLeHYyZt oxBNFL5GMDKkkUHZQmkF54aWIKv1p//kl3uMofs/DozdlU7MQChtscidK1cZNNMqgNcl QDqYy7A605/tthnMi465hCm1Zp7AK5754tbPXxHgQFdrPglhK04PEstJkA7r+WAbpWao 5Fu2zpFuc+RkXy6eaIQjtyJ3RWbxB5Gk+96a6Va4+XJ/jyP4Kbojf/a77xfp3UOzdbZf Ve5A== X-Gm-Message-State: AOAM5315PVwC8jBH7ZeNY2+HFz7EOWy5ofHx2H/1AQ1rgF7Yu4qnoZa5 bCC+J6CfS+lbhVqBcRo4BCG0FdcjDzSy8A== X-Google-Smtp-Source: ABdhPJzoC58WeWXuo11G2Y8mVpNk/X7gQ3kaIfAU+r/zkmvfU+ztcm6JLPC2daMDW2/MU896IWRTIw== X-Received: by 2002:a1c:a905:: with SMTP id s5mr23572381wme.150.1636034303468; Thu, 04 Nov 2021 06:58:23 -0700 (PDT) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id i20sm2264736wmq.41.2021.11.04.06.58.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 04 Nov 2021 06:58:23 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Thu, 4 Nov 2021 13:57:59 +0000 Message-Id: <20211104135805.5269-2-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211104135805.5269-1-david.plowman@raspberrypi.com> References: <20211104135805.5269-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 1/7] libcamera: Add ColorSpace class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This class represents a color space by defining its color primaries, YCbCr encoding, the transfer (gamma) function it uses, and whether the output is full or limited range. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- include/libcamera/color_space.h | 88 +++++++++++ include/libcamera/meson.build | 1 + src/libcamera/color_space.cpp | 257 ++++++++++++++++++++++++++++++++ src/libcamera/meson.build | 1 + 4 files changed, 347 insertions(+) create mode 100644 include/libcamera/color_space.h create mode 100644 src/libcamera/color_space.cpp diff --git a/include/libcamera/color_space.h b/include/libcamera/color_space.h new file mode 100644 index 00000000..2af9da31 --- /dev/null +++ b/include/libcamera/color_space.h @@ -0,0 +1,88 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Raspberry Pi (Trading) Limited + * + * color_space.h - color space definitions + */ + +#ifndef __LIBCAMERA_COLOR_SPACE_H__ +#define __LIBCAMERA_COLOR_SPACE_H__ + +#include + +namespace libcamera { + +class ColorSpace +{ +public: + enum class Primaries : int { + Undefined, + Raw, + Smpte170m, + Rec709, + Rec2020, + }; + + enum class YcbcrEncoding : int { + Undefined, + Rec601, + Rec709, + Rec2020, + }; + + enum class TransferFunction : int { + Undefined, + Linear, + Srgb, + Rec709, + }; + + enum class Range : int { + Undefined, + Full, + Limited, + }; + + constexpr ColorSpace() + : ColorSpace(Primaries::Undefined, YcbcrEncoding::Undefined, TransferFunction::Undefined, Range::Undefined) + { + } + + constexpr ColorSpace(Primaries p, YcbcrEncoding e, TransferFunction t, Range r) + : primaries(p), ycbcrEncoding(e), transferFunction(t), range(r) + { + } + + static const ColorSpace Undefined; + static const ColorSpace Raw; + static const ColorSpace Jpeg; + static const ColorSpace Smpte170m; + static const ColorSpace Rec709; + static const ColorSpace Rec2020; + + Primaries primaries; + YcbcrEncoding ycbcrEncoding; + TransferFunction transferFunction; + Range range; + + bool isFullyDefined() const; + + const std::string toString() const; +}; + +constexpr ColorSpace ColorSpace::Undefined = { Primaries::Undefined, YcbcrEncoding::Undefined, TransferFunction::Undefined, Range::Undefined }; +constexpr ColorSpace ColorSpace::Raw = { Primaries::Raw, YcbcrEncoding::Rec601, TransferFunction::Linear, Range::Full }; +constexpr ColorSpace ColorSpace::Jpeg = { Primaries::Rec709, YcbcrEncoding::Rec601, TransferFunction::Srgb, Range::Full }; +constexpr ColorSpace ColorSpace::Smpte170m = { Primaries::Smpte170m, YcbcrEncoding::Rec601, TransferFunction::Rec709, Range::Limited }; +constexpr ColorSpace ColorSpace::Rec709 = { Primaries::Rec709, YcbcrEncoding::Rec709, TransferFunction::Rec709, Range::Limited }; +constexpr ColorSpace ColorSpace::Rec2020 = { Primaries::Rec2020, YcbcrEncoding::Rec2020, TransferFunction::Rec709, Range::Limited }; + +bool operator==(const ColorSpace &lhs, const ColorSpace &rhs); +static inline bool operator!=(const ColorSpace &lhs, const ColorSpace &rhs) +{ + return !(lhs == rhs); +} + +} /* namespace libcamera */ + +#endif /* __LIBCAMERA_COLOR_SPACE_H__ */ diff --git a/include/libcamera/meson.build b/include/libcamera/meson.build index 7155ff20..131e1740 100644 --- a/include/libcamera/meson.build +++ b/include/libcamera/meson.build @@ -5,6 +5,7 @@ libcamera_include_dir = 'libcamera' / 'libcamera' libcamera_public_headers = files([ 'camera.h', 'camera_manager.h', + 'color_space.h', 'compiler.h', 'controls.h', 'file_descriptor.h', diff --git a/src/libcamera/color_space.cpp b/src/libcamera/color_space.cpp new file mode 100644 index 00000000..dfd9fa1d --- /dev/null +++ b/src/libcamera/color_space.cpp @@ -0,0 +1,257 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Raspberry Pi (Trading) Limited + * + * color_space.cpp - color spaces. + */ + +#include + +#include +#include +#include + +/** + * \file color_space.h + * \brief Class and enums to represent color spaces + */ + +namespace libcamera { + +/** + * \class ColorSpace + * \brief Class to describe a color space + * + * The ColorSpace class defines the color primaries, the Y'CbCr encoding, + * the transfer function associated with the color space, and the range + * (sometimes also referred to as the quantisation) of the color space. + * + * Certain combinations of these fields form well-known standard color + * spaces such as "JPEG" or "REC709". Applications must not request color + * spaces with undefined fields, but the "Undefined" value may be + * returned if the camera drivers decide to use a color space that is + * not recognised by the ColorSpace class. + * + * For more information on the specific color spaces described here, please + * see: + * + * sRGB and JPEG +>* SMPTE 170M + * Rec.709 + * Rec.2020 + */ + +/** + * \enum ColorSpace::Primaries + * \brief The color primaries for this color space + * + * \var ColorSpace::Primaries::Undefined + * \brief The color primaries are undefined + * \var ColorSpace::Primaries::Raw + * \brief These are raw colors directly from a sensor + * \var ColorSpace::Primaries::Smpte170m + * \brief SMPTE 170M color primaries + * \var ColorSpace::Primaries::Rec709 + * \brief Rec.709 color primaries + * \var ColorSpace::Primaries::Rec2020 + * \brief Rec.2020 color primaries + */ + +/** + * \enum ColorSpace::YcbcrEncoding + * \brief The Y'CbCr encoding + * + * \var ColorSpace::YcbcrEncoding::Undefined + * \brief The Y'CbCr encoding is undefined + * \var ColorSpace::YcbcrEncoding::Rec601 + * \brief Rec.601 Y'CbCr encoding + * \var ColorSpace::YcbcrEncoding::Rec709 + * \brief Rec.709 Y'CbCr encoding + * \var ColorSpace::YcbcrEncoding::Rec2020 + * \brief Rec.2020 Y'CbCr encoding + */ + +/** + * \enum ColorSpace::TransferFunction + * \brief The transfer function used for this color space + * + * \var ColorSpace::TransferFunction::Undefined + * \brief The transfer function is not specified + * \var ColorSpace::TransferFunction::Linear + * \brief This color space uses a linear (identity) transfer function + * \var ColorSpace::TransferFunction::Srgb + * \brief sRGB transfer function + * \var ColorSpace::TransferFunction::Rec709 + * \brief Rec709 transfer function + */ + +/** + * \enum ColorSpace::Range + * \brief The range (sometimes "quantisation") for this color space + * + * \var ColorSpace::Range::Undefined + * \brief The range is not specified + * \var ColorSpace::Range::Full + * \brief This color space uses full range pixel values + * \var ColorSpace::Range::Limited + * \brief This color space uses limited range pixel values, being + * 16 to 235 for Y' and 16 to 240 for Cb and Cr (8 bits per sample) + * or 64 to 940 for Y' and 16 to 960 for Cb and Cr (10 bits) + */ + +/** + * \fn ColorSpace::ColorSpace(Encoding e, TransferFunction t, Range r) + * \brief Construct a ColorSpace from explicit values + * \param[in] e The Y'CbCr encoding + * \param[in] t The transfer function for the color space + * \param[in] r The range of the pixel values in this color space + */ + +/** + * \fn ColorSpace::ColorSpace() + * \brief Construct a color space with undefined encoding, transfer function + * and range + */ + +/** + * \brief Check if all the fields of the color space are defined + * \return Return true if all the fields of the color space are defined, + * otherwise false + */ +bool ColorSpace::isFullyDefined() const +{ + return primaries != Primaries::Undefined && + ycbcrEncoding != YcbcrEncoding::Undefined && + transferFunction != TransferFunction::Undefined && + range != Range::Undefined; +} + +/** + * \brief Assemble and return a readable string representation of the + * ColorSpace + * \return A string describing the ColorSpace + */ +const std::string ColorSpace::toString() const +{ + /* Print out a brief name only for standard color sapces. */ + + static const std::vector> colorSpaceNames = { + { ColorSpace::Undefined, "Undefined" }, + { ColorSpace::Raw, "Raw" }, + { ColorSpace::Jpeg, "Jpeg" }, + { ColorSpace::Smpte170m, "Smpte170m" }, + { ColorSpace::Rec709, "Rec709" }, + { ColorSpace::Rec2020, "Rec2020" }, + }; + auto it = std::find_if(colorSpaceNames.begin(), colorSpaceNames.end(), + [this](const auto &item) { + return *this == item.first; + }); + if (it != colorSpaceNames.end()) + return std::string(it->second); + + static const char *primariesNames[] = { + "Undefined", + "Raw", + "Smpte170m", + "Rec709", + "Rec2020", + }; + static const char *encodingNames[] = { + "Undefined", + "Rec601", + "Rec709", + "Rec2020", + }; + static const char *transferFunctionNames[] = { + "Undefined", + "Linear", + "Srgb", + "Rec709", + }; + static const char *rangeNames[] = { + "Undefined", + "Full", + "Limited", + }; + + std::stringstream ss; + ss << std::string(primariesNames[static_cast(primaries)]) << "/" + << std::string(encodingNames[static_cast(ycbcrEncoding)]) << "/" + << std::string(transferFunctionNames[static_cast(transferFunction)]) << "/" + << std::string(rangeNames[static_cast(range)]); + + return ss.str(); +} + +/** + * \var ColorSpace::primaries + * \brief The color primaries + */ + +/** + * \var ColorSpace::ycbcrEncoding + * \brief The Y'CbCr encoding + */ + +/** + * \var ColorSpace::transferFunction + * \brief The transfer function for this color space + */ + +/** + * \var ColorSpace::range + * \brief The pixel range used by this color space + */ + +/** + * \var ColorSpace::Undefined + * \brief A constant representing a fully undefined color space + */ + +/** + * \var ColorSpace::Raw + * \brief A constant representing a raw color space (from a sensor) + */ + +/** + * \var ColorSpace::Jpeg + * \brief A constant representing the JPEG color space used for + * encoding JPEG images (and regarded as being the same as the sRGB + * color space) + */ + +/** + * \var ColorSpace::Smpte170m + * \brief A constant representing the SMPTE170M color space + */ + +/** + * \var ColorSpace::Rec709 + * \brief A constant representing the Rec.709 color space + */ + +/** + * \var ColorSpace::Rec2020 + * \brief A constant representing the Rec.2020 color space + */ + +/** + * \brief Compare color spaces for equality + * \return True if the two color spaces are identical, false otherwise + */ +bool operator==(const ColorSpace &lhs, const ColorSpace &rhs) +{ + return lhs.primaries == rhs.primaries && + lhs.ycbcrEncoding == rhs.ycbcrEncoding && + lhs.transferFunction == rhs.transferFunction && + lhs.range == rhs.range; +} + +/** + * \fn bool operator!=(const ColorSpace &lhs, const ColorSpace &rhs) + * \brief Compare color spaces for inequality + * \return True if the two color spaces are not identical, false otherwise + */ + +} /* namespace libcamera */ diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build index 6727a777..e7371d20 100644 --- a/src/libcamera/meson.build +++ b/src/libcamera/meson.build @@ -8,6 +8,7 @@ libcamera_sources = files([ 'camera_manager.cpp', 'camera_sensor.cpp', 'camera_sensor_properties.cpp', + 'color_space.cpp', 'controls.cpp', 'control_serializer.cpp', 'control_validator.cpp', From patchwork Thu Nov 4 13:58:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 14459 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3A001BDB1C for ; Thu, 4 Nov 2021 13:58:29 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id EBA8560362; Thu, 4 Nov 2021 14:58:27 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="l3E5JKlR"; dkim-atps=neutral Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 68D3960344 for ; Thu, 4 Nov 2021 14:58:24 +0100 (CET) Received: by mail-wm1-x334.google.com with SMTP id z11-20020a1c7e0b000000b0030db7b70b6bso7292594wmc.1 for ; Thu, 04 Nov 2021 06:58:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=nUVlWu72PlyE29kbK+yxhDq/7AfELhGkPOIcJt8NS14=; b=l3E5JKlRgqmxB4ItgFU3li7im056VsVT3kbyu/9+p0QSGqQG9GOc8nOWkAecZdO44O ICdOBYaE5+KJVs4fqG0ha3nA6kLvEcvL3EpnomW0DTWsaRJxZ5bBUJUsIwsoZqMtEvfZ L9rKJ8NfRj0XLDpFeVh+dGfFo8dfURxIKrLps2PNvSercGA0HlCRHvduPm7E/OzP3Sgd bZCiDy4CsSoyKlz95j7jPF6kn/DqKjpaTyFZA99Fscbi0LixaKKy+q7G6Vtw9xKA9HdR KqDS/SiGMqFWgStIxQNcr7C8s0p8NchHAIa2elC2lbwAHypUJFUdGC2WoNAQpgKkjd0o 2xZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=nUVlWu72PlyE29kbK+yxhDq/7AfELhGkPOIcJt8NS14=; b=UTzMRwBi8Q9YkYy7Q6BgMTEQYf91gwuyh3gOSPkYZKxyjlYZkCPyZzXr9ykWi6ozd/ covGBgHhi/XCq1Cx8aoW82pjyhFVKDlcYacpV53ijv7gCLCLnU7HTPxqmvICgqla8lgY EhOaowzHk3OwDJp/LRxBsvjJkp1rLDgS8iU2pWeBlGxSuLeqb/9fofpc1+fJ9/OvrGxS G8AKyNCMun+IrdjQc8b5waPl6/Zx7xMKWB01xnlInDA6VYE2wfS9i92Ksvlv2De24Yh5 9qoU4FNa3SjHDyM9jINy+2BSA/228TuFx9+jjsj2mZP9iSdEjeSP34lFn7cfwbnGfAJm 1nFQ== X-Gm-Message-State: AOAM5336WHbwF8qtpv4T7P9Z7xabJDBZNfQkGRRRRTYmbFSLPvZ/fKVj ALSqAfreFKrO6KG4A/i6mGhUEGztZVTtrg== X-Google-Smtp-Source: ABdhPJz5RoVezjsWUIXmEDcTpdTef/8w8MgwspipUqka7F0LHX0EIgE2fqJ+bvCeF1jubHan9ad6EQ== X-Received: by 2002:a7b:cc8f:: with SMTP id p15mr20505767wma.158.1636034303888; Thu, 04 Nov 2021 06:58:23 -0700 (PDT) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id i20sm2264736wmq.41.2021.11.04.06.58.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 04 Nov 2021 06:58:23 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Thu, 4 Nov 2021 13:58:00 +0000 Message-Id: <20211104135805.5269-3-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211104135805.5269-1-david.plowman@raspberrypi.com> References: <20211104135805.5269-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 2/7] libcamera: Add ColorSpace fields to StreamConfiguration X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This is so that applications can choose appropriate color spaces which will then be passed down to the V4L2 devices. There are two fields: the "requestedColorSpace" which is the one an application wants, and the "actualColorSpace" which is what the underlying hardware can deliver, and which is filled in by the validate() method. Signed-off-by: David Plowman --- include/libcamera/stream.h | 4 ++++ src/libcamera/stream.cpp | 25 +++++++++++++++++++++++++ 2 files changed, 29 insertions(+) diff --git a/include/libcamera/stream.h b/include/libcamera/stream.h index 0c55e716..fe491ff5 100644 --- a/include/libcamera/stream.h +++ b/include/libcamera/stream.h @@ -12,6 +12,7 @@ #include #include +#include #include #include #include @@ -47,6 +48,9 @@ struct StreamConfiguration { unsigned int bufferCount; + ColorSpace requestedColorSpace; + ColorSpace actualColorSpace; + Stream *stream() const { return stream_; } void setStream(Stream *stream) { stream_ = stream; } const StreamFormats &formats() const { return formats_; } diff --git a/src/libcamera/stream.cpp b/src/libcamera/stream.cpp index b421e17e..1ddbbb8c 100644 --- a/src/libcamera/stream.cpp +++ b/src/libcamera/stream.cpp @@ -329,6 +329,31 @@ StreamConfiguration::StreamConfiguration(const StreamFormats &formats) * \brief Requested number of buffers to allocate for the stream */ +/** + * \var StreamConfiguration::requestedColorSpace + * \brief The ColorSpace this stream should have + * + * The generateConfiguration method will generate reasonable default + * values (ColorSpace::Jpeg for stills, ColorSpace::Rec709 for video and + * ColorSpace::Raw for raw streams) but applications are free to overwrite + * this value. + */ + +/** + * \var StreamConfiguration::actualColorSpace + * \brief The ColorSpace the will be used for this stream + * + * This field is filled in by CameraConfiguration::validate(). + * Normally this should match the requestedColorSpace, but it may differ + * if the hardware does not support it. + * + * In general cameras may have different constraints here, for example, + * they may require all output streams to share the same color space. + * Sometimes the fields within this color space may report "Undefined" + * values if the hardware drivers are going to use a color space that + * is not recognised by the ColorSpace class. + */ + /** * \fn StreamConfiguration::stream() * \brief Retrieve the stream associated with the configuration From patchwork Thu Nov 4 13:58:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 14460 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B58FEBDB1C for ; Thu, 4 Nov 2021 13:58:30 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id AA89160348; Thu, 4 Nov 2021 14:58:29 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="MZ5iYReB"; dkim-atps=neutral Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2F8ED60346 for ; Thu, 4 Nov 2021 14:58:25 +0100 (CET) Received: by mail-wr1-x42d.google.com with SMTP id d3so8751836wrh.8 for ; Thu, 04 Nov 2021 06:58:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ron3Qwrj+ZdHUA2wy9RhvrSwsJz/Iu9EfWpAou0UGIc=; b=MZ5iYReBhcfQeBwZSLC80+RjTWvuzU4u1e10phNzRKCVepTwFYhT6bdSXe2nmCYltN 8JTHxqT2LQXcjJ5c7ODlSSfokuXFO/3v+38SJFzrJh0sUcHVr/uZVarrpjJvkcYE1pjg WAGM2Ukk7MgNIIUk+ajskLaKfGg3Dlaav9ry+gVlUoPdOaDmcXq+1G57tMY332BN69A0 kpcfOgJLdL+JFGeeK3KHboIbrkJUhbWIOWquGaqttD77bPjoeIUchieB1LezId4gFkB5 An1kbeDUgMRgDRKCSlvL7Fl2Zckx8SFvb50xJZVXY/8Ch+dR9vk8t9FkrULx9STGGBKF iTHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ron3Qwrj+ZdHUA2wy9RhvrSwsJz/Iu9EfWpAou0UGIc=; b=DbhwKN9dNl69GdilxBc2uFOiNTHeHpJPCldRtqNb+K3vERVGHTGlgwOsjrYsCo2J5b pc/UPtcQq8YgX/8wdK4s2rMlS4oGTpXX673fOwWD+nX9Rb8oEobbyQZBRTsWx4V4JJ8g 5shCt69SqdH3tHtAB73se0t39BAjksWJd0A5crZQ8EJTlXH2V37V6EmnMuRSui0+Swo0 1LSLOJBjZ0Yt5DecxQcli6bU0SvnX2AoOof0FVnuBvDoC07Yd6GeiKiC1OESSEmZymxp LfHWwi0+kZtb/B/Sbk+2EeDfHEO/PXTrlnni6LNuTdelvy5sAG+VZHtlH3lwtGC3gmQ3 J/Hw== X-Gm-Message-State: AOAM530fc1u2zWw1AEUFFhiLFT3ESlV8kLv3A6CPSxRTgKFTRmmdwNGQ WtrEe6UyRv07c+fiGKrENPvaYuqCakM63A== X-Google-Smtp-Source: ABdhPJxOCtTkM4hoJrdWMcH+HTpsucKGzXjRc8g2CvitBO1T1UT3i8sKyFeDhzvpiA1nLNpeayuADw== X-Received: by 2002:a05:6000:186e:: with SMTP id d14mr12820615wri.376.1636034304498; Thu, 04 Nov 2021 06:58:24 -0700 (PDT) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id i20sm2264736wmq.41.2021.11.04.06.58.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 04 Nov 2021 06:58:24 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Thu, 4 Nov 2021 13:58:01 +0000 Message-Id: <20211104135805.5269-4-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211104135805.5269-1-david.plowman@raspberrypi.com> References: <20211104135805.5269-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 3/7] libcamera: Convert between ColorSpace class and V4L2 formats X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" These methods are added to the base V4L2Device class so that they can be shared both by the video device class and subdevices. With the ColorSpace class, the color space and related other fields are stored together, corresponding to a number of fields in the various different V4L2 format structures. Template methods are therefore a convenient implementation. Note that we must explicitly instantiate the templates that will be needed. Signed-off-by: David Plowman --- include/libcamera/internal/v4l2_device.h | 7 + src/libcamera/v4l2_device.cpp | 171 +++++++++++++++++++++++ 2 files changed, 178 insertions(+) diff --git a/include/libcamera/internal/v4l2_device.h b/include/libcamera/internal/v4l2_device.h index f21bc370..24fc4984 100644 --- a/include/libcamera/internal/v4l2_device.h +++ b/include/libcamera/internal/v4l2_device.h @@ -17,6 +17,7 @@ #include #include +#include #include namespace libcamera { @@ -44,6 +45,12 @@ public: void updateControlInfo(); + template + static ColorSpace toColorSpace(const T &v4l2Format); + + template + static int fromColorSpace(const ColorSpace &colorSpace, T &v4l2Format); + protected: V4L2Device(const std::string &deviceNode); ~V4L2Device(); diff --git a/src/libcamera/v4l2_device.cpp b/src/libcamera/v4l2_device.cpp index 9c783c9c..34ec2bf3 100644 --- a/src/libcamera/v4l2_device.cpp +++ b/src/libcamera/v4l2_device.cpp @@ -16,6 +16,8 @@ #include #include +#include + #include #include #include @@ -731,4 +733,173 @@ void V4L2Device::eventAvailable() frameStart.emit(event.u.frame_sync.frame_sequence); } +static const std::map v4l2ToColorSpace = { + { V4L2_COLORSPACE_RAW, ColorSpace::Raw }, + { V4L2_COLORSPACE_JPEG, ColorSpace::Jpeg }, + { V4L2_COLORSPACE_SRGB, ColorSpace::Jpeg }, + { V4L2_COLORSPACE_SMPTE170M, ColorSpace::Smpte170m }, + { V4L2_COLORSPACE_REC709, ColorSpace::Rec709 }, + { V4L2_COLORSPACE_BT2020, ColorSpace::Rec2020 }, +}; + +static const std::map v4l2ToYcbcrEncoding = { + { V4L2_YCBCR_ENC_601, ColorSpace::YcbcrEncoding::Rec601 }, + { V4L2_YCBCR_ENC_709, ColorSpace::YcbcrEncoding::Rec709 }, + { V4L2_YCBCR_ENC_BT2020, ColorSpace::YcbcrEncoding::Rec2020 }, +}; + +static const std::map v4l2ToTransferFunction = { + { V4L2_XFER_FUNC_NONE, ColorSpace::TransferFunction::Linear }, + { V4L2_XFER_FUNC_SRGB, ColorSpace::TransferFunction::Srgb }, + { V4L2_XFER_FUNC_709, ColorSpace::TransferFunction::Rec709 }, +}; + +static const std::map v4l2ToRange = { + { V4L2_QUANTIZATION_FULL_RANGE, ColorSpace::Range::Full }, + { V4L2_QUANTIZATION_LIM_RANGE, ColorSpace::Range::Limited }, +}; + +static const std::vector> colorSpaceToV4l2 = { + { ColorSpace::Raw, V4L2_COLORSPACE_RAW }, + { ColorSpace::Jpeg, V4L2_COLORSPACE_JPEG }, + { ColorSpace::Smpte170m, V4L2_COLORSPACE_SMPTE170M }, + { ColorSpace::Rec709, V4L2_COLORSPACE_REC709 }, + { ColorSpace::Rec2020, V4L2_COLORSPACE_BT2020 }, +}; + +static const std::map primariesToV4l2 = { + { ColorSpace::Primaries::Raw, V4L2_COLORSPACE_RAW }, + { ColorSpace::Primaries::Smpte170m, V4L2_COLORSPACE_SMPTE170M }, + { ColorSpace::Primaries::Rec709, V4L2_COLORSPACE_REC709 }, + { ColorSpace::Primaries::Rec2020, V4L2_COLORSPACE_BT2020 }, +}; + +static const std::map ycbcrEncodingToV4l2 = { + { ColorSpace::YcbcrEncoding::Rec601, V4L2_YCBCR_ENC_601 }, + { ColorSpace::YcbcrEncoding::Rec709, V4L2_YCBCR_ENC_709 }, + { ColorSpace::YcbcrEncoding::Rec2020, V4L2_YCBCR_ENC_BT2020 }, +}; + +static const std::map transferFunctionToV4l2 = { + { ColorSpace::TransferFunction::Linear, V4L2_XFER_FUNC_NONE }, + { ColorSpace::TransferFunction::Srgb, V4L2_XFER_FUNC_SRGB }, + { ColorSpace::TransferFunction::Rec709, V4L2_XFER_FUNC_709 }, +}; + +static const std::map rangeToV4l2 = { + { ColorSpace::Range::Full, V4L2_QUANTIZATION_FULL_RANGE }, + { ColorSpace::Range::Limited, V4L2_QUANTIZATION_LIM_RANGE }, +}; + +/** + * \brief Convert the color space fields in a V4L2 format to a ColorSpace + * \param[in] v4l2Format A V4L2 format containing color space information + * + * The colorspace, ycbcr_enc, xfer_func and quantization fields within a + * V4L2 format structure are converted to a corresponding ColorSpace. + * + * If any V4L2 fields are not recognised those are marked as undefined in + * the ColorSpace, but other fields are still initialised where possible. + * This situation can be detected using the returned value's + * ColorSpace::isFullyDefined() method. + * + * \return The ColorSpace corresponding to the input V4L2 format + */ +template +ColorSpace V4L2Device::toColorSpace(const T &v4l2Format) +{ + ColorSpace colorSpace; + + auto itColor = v4l2ToColorSpace.find(v4l2Format.colorspace); + if (itColor != v4l2ToColorSpace.end()) + colorSpace = itColor->second; + + auto itYcbcrEncoding = v4l2ToYcbcrEncoding.find(v4l2Format.ycbcr_enc); + if (itYcbcrEncoding != v4l2ToYcbcrEncoding.end()) + colorSpace.ycbcrEncoding = itYcbcrEncoding->second; + + auto itTransfer = v4l2ToTransferFunction.find(v4l2Format.xfer_func); + if (itTransfer != v4l2ToTransferFunction.end()) + colorSpace.transferFunction = itTransfer->second; + + auto itRange = v4l2ToRange.find(v4l2Format.quantization); + if (itRange != v4l2ToRange.end()) + colorSpace.range = itRange->second; + + return colorSpace; +} + +template ColorSpace V4L2Device::toColorSpace(const struct v4l2_pix_format &); +template ColorSpace V4L2Device::toColorSpace(const struct v4l2_pix_format_mplane &); +template ColorSpace V4L2Device::toColorSpace(const struct v4l2_mbus_framefmt &); + +/** + * \brief Fill in the color space fields of a V4L2 format from a ColorSpace + * \param[in] colorSpace The ColorSpace to be converted + * \param[out] v4l2Format A V4L2 format containing color space information + * + * The colorspace, ycbcr_enc, xfer_func and quantization fields within a + * V4L2 format structure are filled in from a corresponding ColorSpace. + * + * An error is returned if any of the V4L2 fields do not support the + * value given in the ColorSpace. Such fields are set to the V4L2 + * "default" values, but all other fields are still filled in where + * possible. + * + * \return 0 on success or a negative error code otherwise + */ +template +int V4L2Device::fromColorSpace(const ColorSpace &colorSpace, T &v4l2Format) +{ + int ret = 0; + + v4l2Format.colorspace = V4L2_COLORSPACE_DEFAULT; + v4l2Format.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; + v4l2Format.xfer_func = V4L2_XFER_FUNC_DEFAULT; + v4l2Format.quantization = V4L2_QUANTIZATION_DEFAULT; + + auto itColor = std::find_if(colorSpaceToV4l2.begin(), colorSpaceToV4l2.end(), + [&colorSpace](const auto &item) { + return colorSpace == item.first; + }); + if (itColor != colorSpaceToV4l2.end()) { + v4l2Format.colorspace = itColor->second; + return ret; + } + + /* + * If the colorSpace doesn't precisely match a standard color space, + * then we must choose a V4L2 colorspace with matching primaries. + */ + auto itPrimaries = primariesToV4l2.find(colorSpace.primaries); + if (itPrimaries != primariesToV4l2.end()) + v4l2Format.colorspace = itPrimaries->second; + else + ret = -1; + + auto itYcbcrEncoding = ycbcrEncodingToV4l2.find(colorSpace.ycbcrEncoding); + if (itYcbcrEncoding != ycbcrEncodingToV4l2.end()) + v4l2Format.ycbcr_enc = itYcbcrEncoding->second; + else + ret = -1; + + auto itTransfer = transferFunctionToV4l2.find(colorSpace.transferFunction); + if (itTransfer != transferFunctionToV4l2.end()) + v4l2Format.xfer_func = itTransfer->second; + else + ret = -1; + + auto itRange = rangeToV4l2.find(colorSpace.range); + if (itRange != rangeToV4l2.end()) + v4l2Format.quantization = itRange->second; + else + ret = -1; + + return ret; +} + +template int V4L2Device::fromColorSpace(const ColorSpace &, struct v4l2_pix_format &); +template int V4L2Device::fromColorSpace(const ColorSpace &, struct v4l2_pix_format_mplane &); +template int V4L2Device::fromColorSpace(const ColorSpace &, struct v4l2_mbus_framefmt &); + } /* namespace libcamera */ From patchwork Thu Nov 4 13:58:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 14461 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B5B83BDB1C for ; Thu, 4 Nov 2021 13:58:31 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 881AD6035A; Thu, 4 Nov 2021 14:58:30 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="V7ZvxNWY"; dkim-atps=neutral Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6B2D960128 for ; Thu, 4 Nov 2021 14:58:25 +0100 (CET) Received: by mail-wm1-x333.google.com with SMTP id 71so4582649wma.4 for ; Thu, 04 Nov 2021 06:58:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3jzpKNdq0T31XWyv1nf4OXskRk3Z5TSZ2eDhnU9oflA=; b=V7ZvxNWYv7bvytBs0LTt6iJYfCSxBzkfz4Gn5fwjKGlY0Vxxen1TM/+oG0xg+oodfR YYrTOaMvZlDOEHvKct8gdRA/jHYtQlQ3PiSkArTpeB7lNfnkheQl/Za7XD5aTMUo0vCN DTveOzUnwAzzrqSwaiAornxVtveK2Znr6J54kY41Z6zQy64quMdiVKdj5zBpqqcyLDAH 8gWoMInxK1FR3d1i1T6Q7OKl32IgNIPmM2884ogJz/4m5vw3axAoY4V8vy5Jdnq6PMb+ GHo4Mh6H2AkZU8gwNLXvhB+GH0uTkXseew3/cL9cQxCtGfRyT6VYc+ntig679/z3ix7X tkmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3jzpKNdq0T31XWyv1nf4OXskRk3Z5TSZ2eDhnU9oflA=; b=18GZPUbmXAu7DDsTJ0FGaXqltXvh5CaLrzvVlXgDvKW4zWL7WS3GyK2vg2qRGcOIUA xyGPvDcZlqLz4NfQ7Cwqsh6GXyhnjRMTee4y7kM6/IMnANp0xedob+Dw1G3Pp38ZQnlZ SA1jG0K47ygintxQ5FW6r3TGrQ7amPE8HeP/OIgyJCych/exL+LmreuCuJTX1VxPwS4M ai8icteELQiG5DanykjHQ4ilmdV1lRXztOa0XVU5fIuIFGz8v2EciueUv4I72XThgaQA nQ0nrvNsiT9jxC9dL2/WDJQVVGzbLFkIbAauyC+rOY4RTaio8fQRKBF2sTbk6y3gYUWD hXjQ== X-Gm-Message-State: AOAM531vx33g1RvGRrZPnoOy4cRUzQub0/EvkRXv+3xFRkHlTGMgxy7s N8WuW0VAQ93YZwP6dyFK4ueUq4x7gDMvyQ== X-Google-Smtp-Source: ABdhPJwggjpHIsmc27LmIE3HKfohyY2/t2WTjnEN2iZnhQdbyTBZ3OqgK41YjT1S3Qi52pHrRJbizQ== X-Received: by 2002:a7b:c441:: with SMTP id l1mr22800343wmi.69.1636034304932; Thu, 04 Nov 2021 06:58:24 -0700 (PDT) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id i20sm2264736wmq.41.2021.11.04.06.58.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 04 Nov 2021 06:58:24 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Thu, 4 Nov 2021 13:58:02 +0000 Message-Id: <20211104135805.5269-5-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211104135805.5269-1-david.plowman@raspberrypi.com> References: <20211104135805.5269-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 4/7] libcamera: Support passing ColorSpaces to V4L2 video devices X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The ColorSpace from the StreamConfiguration is now handled appropriately in the V4L2VideoDevice. Signed-off-by: David Plowman Reviewed-by: Kieran Bingham --- include/libcamera/internal/v4l2_videodevice.h | 2 + src/libcamera/v4l2_videodevice.cpp | 67 +++++++++++++++++-- 2 files changed, 65 insertions(+), 4 deletions(-) diff --git a/include/libcamera/internal/v4l2_videodevice.h b/include/libcamera/internal/v4l2_videodevice.h index a1c458e4..6ca749c1 100644 --- a/include/libcamera/internal/v4l2_videodevice.h +++ b/include/libcamera/internal/v4l2_videodevice.h @@ -20,6 +20,7 @@ #include #include +#include #include #include #include @@ -167,6 +168,7 @@ public: V4L2PixelFormat fourcc; Size size; + ColorSpace colorSpace; std::array planes; unsigned int planesCount = 0; diff --git a/src/libcamera/v4l2_videodevice.cpp b/src/libcamera/v4l2_videodevice.cpp index 4f04212d..ccf3d28c 100644 --- a/src/libcamera/v4l2_videodevice.cpp +++ b/src/libcamera/v4l2_videodevice.cpp @@ -370,17 +370,29 @@ bool V4L2BufferCache::Entry::operator==(const FrameBuffer &buffer) const * \brief The plane line stride (in bytes) */ +/** + * \var V4L2DeviceFormat::fourcc + * \brief The fourcc code describing the pixel encoding scheme + * + * The fourcc code, as defined by the V4L2 API with the V4L2_PIX_FMT_* macros, + * that identifies the image format pixel encoding scheme. + */ + /** * \var V4L2DeviceFormat::size * \brief The image size in pixels */ /** - * \var V4L2DeviceFormat::fourcc - * \brief The fourcc code describing the pixel encoding scheme + * \var V4L2DeviceFormat::colorSpace + * \brief The color space of the pixels * - * The fourcc code, as defined by the V4L2 API with the V4L2_PIX_FMT_* macros, - * that identifies the image format pixel encoding scheme. + * When setting or trying a format, passing in "Undefined" fields in the + * ColorSpace is not permitted because the driver will then make an + * arbitrary choice of its own. Choices made by the driver will be + * passed back in the normal way, though note that "Undefined" values can + * be returned if the device has chosen something that the ColorSpace + * class cannot represent. */ /** @@ -879,6 +891,12 @@ int V4L2VideoDevice::getFormatMultiplane(V4L2DeviceFormat *format) format->fourcc = V4L2PixelFormat(pix->pixelformat); format->planesCount = pix->num_planes; + format->colorSpace = toColorSpace(*pix); + if (!format->colorSpace.isFullyDefined()) + LOG(V4L2, Warning) + << "Retrieved undefined color space: " + << format->colorSpace.toString(); + for (unsigned int i = 0; i < format->planesCount; ++i) { format->planes[i].bpl = pix->plane_fmt[i].bytesperline; format->planes[i].size = pix->plane_fmt[i].sizeimage; @@ -893,6 +911,11 @@ int V4L2VideoDevice::trySetFormatMultiplane(V4L2DeviceFormat *format, bool set) struct v4l2_pix_format_mplane *pix = &v4l2Format.fmt.pix_mp; int ret; + if (!format->colorSpace.isFullyDefined()) + LOG(V4L2, Error) + << "Trying to set undefined color space: " + << format->colorSpace.toString(); + v4l2Format.type = bufferType_; pix->width = format->size.width; pix->height = format->size.height; @@ -900,6 +923,12 @@ int V4L2VideoDevice::trySetFormatMultiplane(V4L2DeviceFormat *format, bool set) pix->num_planes = format->planesCount; pix->field = V4L2_FIELD_NONE; + ret = fromColorSpace(format->colorSpace, *pix); + if (ret < 0) + LOG(V4L2, Warning) + << "Setting color space unrecognised by V4L2: " + << format->colorSpace.toString(); + ASSERT(pix->num_planes <= std::size(pix->plane_fmt)); for (unsigned int i = 0; i < pix->num_planes; ++i) { @@ -928,6 +957,12 @@ int V4L2VideoDevice::trySetFormatMultiplane(V4L2DeviceFormat *format, bool set) format->planes[i].size = pix->plane_fmt[i].sizeimage; } + format->colorSpace = toColorSpace(*pix); + if (!format->colorSpace.isFullyDefined()) + LOG(V4L2, Warning) + << "Undefined color space has been set: " + << format->colorSpace.toString(); + return 0; } @@ -951,6 +986,12 @@ int V4L2VideoDevice::getFormatSingleplane(V4L2DeviceFormat *format) format->planes[0].bpl = pix->bytesperline; format->planes[0].size = pix->sizeimage; + format->colorSpace = toColorSpace(*pix); + if (!format->colorSpace.isFullyDefined()) + LOG(V4L2, Warning) + << "Retrieved undefined color space: " + << format->colorSpace.toString(); + return 0; } @@ -960,12 +1001,24 @@ int V4L2VideoDevice::trySetFormatSingleplane(V4L2DeviceFormat *format, bool set) struct v4l2_pix_format *pix = &v4l2Format.fmt.pix; int ret; + if (!format->colorSpace.isFullyDefined()) + LOG(V4L2, Error) + << "Trying to set undefined color space: " + << format->colorSpace.toString(); + v4l2Format.type = bufferType_; pix->width = format->size.width; pix->height = format->size.height; pix->pixelformat = format->fourcc; pix->bytesperline = format->planes[0].bpl; pix->field = V4L2_FIELD_NONE; + + ret = fromColorSpace(format->colorSpace, *pix); + if (ret < 0) + LOG(V4L2, Warning) + << "Set color space unrecognised by V4L2: " + << format->colorSpace.toString(); + ret = ioctl(set ? VIDIOC_S_FMT : VIDIOC_TRY_FMT, &v4l2Format); if (ret) { LOG(V4L2, Error) @@ -985,6 +1038,12 @@ int V4L2VideoDevice::trySetFormatSingleplane(V4L2DeviceFormat *format, bool set) format->planes[0].bpl = pix->bytesperline; format->planes[0].size = pix->sizeimage; + format->colorSpace = toColorSpace(*pix); + if (!format->colorSpace.isFullyDefined()) + LOG(V4L2, Warning) + << "Undefined color space has been set: " + << format->colorSpace.toString(); + return 0; } From patchwork Thu Nov 4 13:58:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 14462 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6EB00C324E for ; Thu, 4 Nov 2021 13:58:32 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 202BB6034F; Thu, 4 Nov 2021 14:58:32 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="T893C1jc"; dkim-atps=neutral Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 04E7460352 for ; Thu, 4 Nov 2021 14:58:26 +0100 (CET) Received: by mail-wm1-x32c.google.com with SMTP id y196so4588360wmc.3 for ; Thu, 04 Nov 2021 06:58:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HsQlhHnuhc87TaSXhA5f4OVRdGAu3zIVZXBgg53nXBM=; b=T893C1jc5/OBi28MfxZnXec0O+Ab2vOFs58ObSFvTaOiWxjS1oMrF0LIiVPRyV1gda EzEgQL4IoDZv8llqkwqmk5Xz1JKerfnUVlfZhd6Xdub5ssMnV7YPpxnlejJ1P9ZPSgz3 KoKSE8X+/tLpU01/+Kj7IhF7hOQbqyGsbScpmYmel9otED6xd+o7mizJc6tYt5A5prPZ SqvFVBsaFhMpO+QtSxQIAWP73L+Kpi8+OPCJ41wKcnq2hyWvYkjLn6P7uKrq9y4e80vb 3K2+iMHRE6tB5lOzbOZuvIW5LllvPnvHmXxLUPysORy4vTMnA+Q9PMEZMV3LYxj/v2sb PTyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HsQlhHnuhc87TaSXhA5f4OVRdGAu3zIVZXBgg53nXBM=; b=OMaeaXue+FcZIwd3Ru5pjWFAjpk6rNlITpQmFba+rIB8qAFB+Q0XgjjtslyD3EeiUB F+XxYHrFK2NL2SUq5JHLitwrLSAjdLxfLpgGvgAfiTXD+C75RttI2kBHthYLoFyWyLUu 2tIiiyXxGqnvA5ZPIP3CQx5xPvuf/Lr+rDIhKNw6DYP41/O97Ksbq9dTqs/Imj5XEyPM oFFwNGUBvFVCGgMjthPHZm410Dpeg/Wve+OWCrzSYzb4vEBIz9wqmR26elXXya85NmjA d5IojsRGQRRGSXSJ3VGA+GULzTDLFRmixwZ7QzXbswlZDmpvR/zdtn3Hfu9qGIW49P0U r+mw== X-Gm-Message-State: AOAM532DobuI3T7iDGxTLTe3LtTuE+mMdXLRGqVGd8qJW57Jusl3YdGH EINuc0zv2reEumEmxXUua0E5HHQqIPhmdQ== X-Google-Smtp-Source: ABdhPJzyTMWuCyBZkFT84bCCTS1nrqpBrpkj+fb7d4leHwdHbMiRmkWuOZQaYRpMcGHluEtUgKr0Jg== X-Received: by 2002:a7b:c744:: with SMTP id w4mr18525399wmk.50.1636034305546; Thu, 04 Nov 2021 06:58:25 -0700 (PDT) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id i20sm2264736wmq.41.2021.11.04.06.58.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 04 Nov 2021 06:58:25 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Thu, 4 Nov 2021 13:58:03 +0000 Message-Id: <20211104135805.5269-6-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211104135805.5269-1-david.plowman@raspberrypi.com> References: <20211104135805.5269-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 5/7] libcamera: Support passing ColorSpaces to V4L2 subdevices X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The ColorSpace from the StreamConfiguration is now handled appropriately in the V4L2Subdevice. Signed-off-by: David Plowman --- include/libcamera/internal/v4l2_subdevice.h | 2 + src/libcamera/camera_sensor.cpp | 1 + src/libcamera/v4l2_subdevice.cpp | 42 ++++++++++++++++++++- 3 files changed, 44 insertions(+), 1 deletion(-) diff --git a/include/libcamera/internal/v4l2_subdevice.h b/include/libcamera/internal/v4l2_subdevice.h index 97b89fb9..f3ab8454 100644 --- a/include/libcamera/internal/v4l2_subdevice.h +++ b/include/libcamera/internal/v4l2_subdevice.h @@ -14,6 +14,7 @@ #include #include +#include #include #include "libcamera/internal/formats.h" @@ -27,6 +28,7 @@ class MediaDevice; struct V4L2SubdeviceFormat { uint32_t mbus_code; Size size; + ColorSpace colorSpace; const std::string toString() const; uint8_t bitsPerPixel() const; diff --git a/src/libcamera/camera_sensor.cpp b/src/libcamera/camera_sensor.cpp index 9fdb8c09..6fcd1c1d 100644 --- a/src/libcamera/camera_sensor.cpp +++ b/src/libcamera/camera_sensor.cpp @@ -613,6 +613,7 @@ V4L2SubdeviceFormat CameraSensor::getFormat(const std::vector &mbu V4L2SubdeviceFormat format{ .mbus_code = bestCode, .size = *bestSize, + .colorSpace = ColorSpace::Raw, }; return format; diff --git a/src/libcamera/v4l2_subdevice.cpp b/src/libcamera/v4l2_subdevice.cpp index 023e2328..b348170f 100644 --- a/src/libcamera/v4l2_subdevice.cpp +++ b/src/libcamera/v4l2_subdevice.cpp @@ -168,6 +168,18 @@ const std::map formatInfoMap = { * \brief The image size in pixels */ +/** + * \var V4L2SubdeviceFormat::colorSpace + * \brief The color space of the pixels + * + * When setting or trying a format, passing in "Undefined" fields in the + * ColorSpace is not recommended because the driver will then make an + * arbitrary choice of its own. Choices made by the driver will be + * passed back in the normal way, though note that "Undefined" values can + * be returned if the device has chosen something that the ColorSpace + * cannot represent. + */ + /** * \brief Assemble and return a string describing the format * \return A string describing the V4L2SubdeviceFormat @@ -400,6 +412,17 @@ int V4L2Subdevice::getFormat(unsigned int pad, V4L2SubdeviceFormat *format, format->size.height = subdevFmt.format.height; format->mbus_code = subdevFmt.format.code; + format->colorSpace = toColorSpace(subdevFmt.format); + /* + * This warning can be ignored on metadata pads. These are normally + * pads other than zero. + * \todo find a way to detect metadata pads and ignore them + */ + if (!format->colorSpace.isFullyDefined()) + LOG(V4L2, Warning) + << "Retrieved undefined color space on pad " << pad + << ": " << format->colorSpace.toString(); + return 0; } @@ -418,6 +441,11 @@ int V4L2Subdevice::getFormat(unsigned int pad, V4L2SubdeviceFormat *format, int V4L2Subdevice::setFormat(unsigned int pad, V4L2SubdeviceFormat *format, Whence whence) { + if (!format->colorSpace.isFullyDefined()) + LOG(V4L2, Error) + << "Trying to set undefined color space: " + << format->colorSpace.toString(); + struct v4l2_subdev_format subdevFmt = {}; subdevFmt.which = whence == ActiveFormat ? V4L2_SUBDEV_FORMAT_ACTIVE : V4L2_SUBDEV_FORMAT_TRY; @@ -427,7 +455,13 @@ int V4L2Subdevice::setFormat(unsigned int pad, V4L2SubdeviceFormat *format, subdevFmt.format.code = format->mbus_code; subdevFmt.format.field = V4L2_FIELD_NONE; - int ret = ioctl(VIDIOC_SUBDEV_S_FMT, &subdevFmt); + int ret = fromColorSpace(format->colorSpace, subdevFmt.format); + if (ret < 0) + LOG(V4L2, Warning) + << "Setting color space unrecognised by V4L2: " + << format->colorSpace.toString(); + + ret = ioctl(VIDIOC_SUBDEV_S_FMT, &subdevFmt); if (ret) { LOG(V4L2, Error) << "Unable to set format on pad " << pad @@ -439,6 +473,12 @@ int V4L2Subdevice::setFormat(unsigned int pad, V4L2SubdeviceFormat *format, format->size.height = subdevFmt.format.height; format->mbus_code = subdevFmt.format.code; + format->colorSpace = toColorSpace(subdevFmt.format); + if (!format->colorSpace.isFullyDefined()) + LOG(V4L2, Warning) + << "Undefined color space has been set: " + << format->colorSpace.toString(); + return 0; } From patchwork Thu Nov 4 13:58:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 14463 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2D198BDB1C for ; Thu, 4 Nov 2021 13:58:35 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C92F760361; Thu, 4 Nov 2021 14:58:34 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="IXYf3K6j"; dkim-atps=neutral Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B1EA26035D for ; Thu, 4 Nov 2021 14:58:26 +0100 (CET) Received: by mail-wm1-x32e.google.com with SMTP id d72-20020a1c1d4b000000b00331140f3dc8so4362043wmd.1 for ; Thu, 04 Nov 2021 06:58:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=px+J59JTE8VA+7mmTv2pDwc8QWJMkP/Tl3k7zA7RZOc=; b=IXYf3K6jDwavl2ZDFSmb3tXf0KRsuVbKLNjGl8xam9XHnObNPyEKIa+kTaTwHF7xts RQaLIvVjPJR9BzJ+555RBC7Ru7J5rv+RzfEi35mVzs7qdMZfVFR8QwhKV75tmMntzsHn CdMelkbUs8NrnRN1WbTrAGAPDhpP8VoURsJh1Pe3bvDvp+FrLokAhTkUUsKCbz828QRb YpOYbPo3sgBcCSu/FAtNQiziWb3Hs8IJJXqP7lzgh+WgexlTzOouPMnM2wkWTl3IMOqK 1RSEMIlqX2c+3mmS6OaUSK85iV2M5p7e1Yv7Hnbgiu1LB2UuFmwvuDlemjaI8ewJ7IGm 3BSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=px+J59JTE8VA+7mmTv2pDwc8QWJMkP/Tl3k7zA7RZOc=; b=aGSxNxOm7e7ZiHU5/1db8Dte4YT+vLtGBauMXrJv++HX0PvBb9BRrjyeHNyhSdULSW oSxEhdZ7JSucEmXA3+bUbH5h1dlRtSmo4JS7ljmieUoybsrNW1Jra9zMMc3sZIPbWx++ Yb+ymuNpSg4GMrKIgm88E/NX5uintPBMCwAWcAsvw6Dvh5gRn0h7PROuAX6/WW5Pb2bi tl84deKSQeS0SffQBTY/0DZuGhN1voimuY6lVOjoXqaR4tfN9DarBzVSHWTGv1h74A3I 64VVt+VnWDnOxn+KxSYJfPc1OL4CKmbZ5/Gp1llJX87cWnum5+pDmdE/rF9i8CyGRWFK By0g== X-Gm-Message-State: AOAM533v6zLAI+cHdZ1qJNEZbFUVghQFek7HUd2200wLyntASnY+h2O2 2IXTO7wygRq8L7wJUJF9QO/uC+ycYtMP5Q== X-Google-Smtp-Source: ABdhPJwA+c23AiLtC1MtRkKctoXHEGgncjNSDRkjmTkjhsZ2/qpiUzyuspsJ8KqZZXIERsNkFVLkTg== X-Received: by 2002:a05:600c:b46:: with SMTP id k6mr3157967wmr.45.1636034306184; Thu, 04 Nov 2021 06:58:26 -0700 (PDT) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id i20sm2264736wmq.41.2021.11.04.06.58.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 04 Nov 2021 06:58:25 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Thu, 4 Nov 2021 13:58:04 +0000 Message-Id: <20211104135805.5269-7-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211104135805.5269-1-david.plowman@raspberrypi.com> References: <20211104135805.5269-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 6/7] libcamera: Add validateColorSpaces to CameraConfiguration class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This method checks that the requested color spaces are sensible and do not contain any undefined enum values. It also initialises the "actual" color space field to the same value as the one requested, in the expectation that the rest of the validate() method will be able to check this. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- include/libcamera/camera.h | 2 ++ src/libcamera/camera.cpp | 51 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 53 insertions(+) diff --git a/include/libcamera/camera.h b/include/libcamera/camera.h index 601ee46e..fdab4410 100644 --- a/include/libcamera/camera.h +++ b/include/libcamera/camera.h @@ -69,6 +69,8 @@ public: protected: CameraConfiguration(); + Status validateColorSpaces(bool sharedColorSpace); + std::vector config_; }; diff --git a/src/libcamera/camera.cpp b/src/libcamera/camera.cpp index 400a7cf0..90e9460b 100644 --- a/src/libcamera/camera.cpp +++ b/src/libcamera/camera.cpp @@ -20,6 +20,7 @@ #include "libcamera/internal/camera.h" #include "libcamera/internal/camera_controls.h" +#include "libcamera/internal/formats.h" #include "libcamera/internal/pipeline_handler.h" /** @@ -314,6 +315,56 @@ std::size_t CameraConfiguration::size() const return config_.size(); } +static bool isRaw(const PixelFormat &pixFmt) +{ + const PixelFormatInfo &info = PixelFormatInfo::info(pixFmt); + return info.isValid() && + info.colourEncoding == PixelFormatInfo::ColourEncodingRAW; +} + +CameraConfiguration::Status CameraConfiguration::validateColorSpaces(bool sharedColorSpace) +{ + Status status = Valid; + + /* Find the largest non-raw stream (if any). */ + int index = -1; + for (unsigned int i = 0; i < config_.size(); i++) { + const StreamConfiguration &cfg = config_[i]; + if (!isRaw(cfg.pixelFormat) && (index < 0 || cfg.size > config_[i].size)) + index = i; + } + + /* + * Here we force raw streams to the correct color space and signal + * an error if we encounter anything undefined. We handle the case + * where all output streams are to share a color space, which we + * choose to be the color space of the largest stream. + */ + for (auto &cfg : config_) { + ColorSpace initialColorSpace = cfg.requestedColorSpace; + + if (isRaw(cfg.pixelFormat)) + cfg.requestedColorSpace = ColorSpace::Raw; + else if (!cfg.requestedColorSpace.isFullyDefined()) { + LOG(Camera, Error) << "Stream has undefined color space"; + cfg.requestedColorSpace = ColorSpace::Jpeg; + } else if (sharedColorSpace) + cfg.requestedColorSpace = config_[index].requestedColorSpace; + + if (cfg.requestedColorSpace != initialColorSpace) + status = Adjusted; + + /* + * We also initialise the actual color space as if the + * hardware can do what we want. But note that the rest + * of the validate() method may change this. + */ + cfg.actualColorSpace = cfg.requestedColorSpace; + } + + return status; +} + /** * \var CameraConfiguration::transform * \brief User-specified transform to be applied to the image From patchwork Thu Nov 4 13:58:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 14464 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A9C6FC324E for ; Thu, 4 Nov 2021 13:58:35 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6292660352; Thu, 4 Nov 2021 14:58:35 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="NRF75VlA"; dkim-atps=neutral Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4AFC36034F for ; Thu, 4 Nov 2021 14:58:27 +0100 (CET) Received: by mail-wm1-x32f.google.com with SMTP id 67-20020a1c1946000000b0030d4c90fa87so4355036wmz.2 for ; Thu, 04 Nov 2021 06:58:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=MWi8mj5S4W6cGhtr7B2BNpqXeeXBc2vqiVreCzJ+MGg=; b=NRF75VlAaoeHCI5komFAHp4swHbQny90tLbCh4pl+GDMbwQS1rc3MMrTVWypiul4VN fHD2+x53Jk4CaqoelioERZYNcakJpk1hOjJsK4TIxtOlH6wz0kE4hdlWnQqNs9hZip3U t7bKZf2WvZGVHAoKi0EO9xxzMdvADCNal2iKg0CCiCsbrXG4oaEmrUkseMX3evCi0/6V p0BIacveiIRCMr7BDmwBQ9h8AyeiNO40R/EbO+zB4Gk+caHGg/YlcyAbGg1vT7Wl/LBR bSs6pv72L3/6DE3qV2E0ODDiwM1y+LSVLFyKLLuB2ik8r3RzSNQ/wFb1vO/cfvf34z48 NiKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=MWi8mj5S4W6cGhtr7B2BNpqXeeXBc2vqiVreCzJ+MGg=; b=AfOKoM0Xxd+V/f1b2rIZJMchzZhwndB4h9JLTpiGJs5vcgcgsOX8yeCkFGHFqJiQ0a ddpn7RdUca2z559jSQz4bvOKXlFw2UL6CTIt4AF+JD+IgdtDn0dP3f8ZJ4fPBXs+PvSa B6xPctzzleIw6h08sE4dMfkvpFBfK1oH9HIvXIVUsJMxSZyXPc/YIMXnVef/XAYwGb9R EbRuSyF8qPK+Tz8N16shR2MCLiTiHXmfjLhd74bMTZ4/BAl7LpqrEb1GegzK7zNFYO+a Ka6Wxc4V6vn1bzPhCyztg5ChHmm/nJX6y5PjHBCdks6KCrCLZRhlXuMxPKKQjZu+Z77m ItXA== X-Gm-Message-State: AOAM532N5Vb+ciXssYBPWOKHC99i1H/owzZhlJw9hDiIc+K/Q1oZBtZ8 FCktBPXOFbXCNUVHyL+90U3fz16kNz+2Ug== X-Google-Smtp-Source: ABdhPJy+rQao/v+ex3rNuP/WSTLIB7o4Q+Ann03LkpPS8XGYLgdYEkZRhW+znZP6UR5m8h8y70KQ1A== X-Received: by 2002:a05:600c:210d:: with SMTP id u13mr23082571wml.82.1636034306844; Thu, 04 Nov 2021 06:58:26 -0700 (PDT) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id i20sm2264736wmq.41.2021.11.04.06.58.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 04 Nov 2021 06:58:26 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Thu, 4 Nov 2021 13:58:05 +0000 Message-Id: <20211104135805.5269-8-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211104135805.5269-1-david.plowman@raspberrypi.com> References: <20211104135805.5269-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 7/7] libcamera: pipeline: raspberrypi: Support color spaces X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The Raspberry Pi pipeline handler now sets color spaces correctly. In generateConfiguration() it sets them to reasonable default values based on the stream role. validate() now calls validateColorSpaces() to ensure that the requested color spaces are sensible, before proceeding to check what the hardware can deliver. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- .../pipeline/raspberrypi/raspberrypi.cpp | 42 +++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 5e1f2273..bbb21e9b 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -93,6 +93,7 @@ V4L2DeviceFormat toV4L2DeviceFormat(const V4L2SubdeviceFormat &format, deviceFormat.fourcc = V4L2PixelFormat::fromPixelFormat(pix); deviceFormat.size = format.size; + deviceFormat.colorSpace = format.colorSpace; return deviceFormat; } @@ -126,6 +127,7 @@ V4L2SubdeviceFormat findBestFormat(const SensorFormats &formatsMap, const Size & { double bestScore = std::numeric_limits::max(), score; V4L2SubdeviceFormat bestFormat; + bestFormat.colorSpace = ColorSpace::Raw; #define PENALTY_AR 1500.0 #define PENALTY_8BIT 2000.0 @@ -332,6 +334,8 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() if (config_.empty()) return Invalid; + status = validateColorSpaces(true); + /* * What if the platform has a non-90 degree rotation? We can't even * "adjust" the configuration and carry on. Alternatively, raising an @@ -494,10 +498,21 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() V4L2DeviceFormat format; format.fourcc = V4L2PixelFormat::fromPixelFormat(cfg.pixelFormat); format.size = cfg.size; + format.colorSpace = cfg.requestedColorSpace; + LOG(RPI, Debug) + << "Try color space " << cfg.requestedColorSpace.toString(); int ret = dev->tryFormat(&format); if (ret) return Invalid; + cfg.actualColorSpace = format.colorSpace; + if (cfg.actualColorSpace != cfg.requestedColorSpace) { + status = Adjusted; + LOG(RPI, Warning) + << "Color space changed from " + << cfg.requestedColorSpace.toString() << " to " + << cfg.actualColorSpace.toString(); + } cfg.stride = format.planes[0].bpl; cfg.frameSize = format.planes[0].size; @@ -522,6 +537,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, PixelFormat pixelFormat; V4L2VideoDevice::Formats fmts; Size size; + ColorSpace colorSpace; if (roles.empty()) return config; @@ -536,6 +552,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, pixelFormat = mbusCodeToPixelFormat(sensorFormat.mbus_code, BayerFormat::Packing::CSI2); ASSERT(pixelFormat.isValid()); + colorSpace = ColorSpace::Raw; bufferCount = 2; rawCount++; break; @@ -543,6 +560,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, case StreamRole::StillCapture: fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::NV12; + colorSpace = ColorSpace::Jpeg; /* Return the largest sensor resolution. */ size = data->sensor_->resolution(); bufferCount = 1; @@ -560,6 +578,8 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, */ fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::YUV420; + /* This will be reasonable for many applications. */ + colorSpace = ColorSpace::Rec709; size = { 1920, 1080 }; bufferCount = 4; outCount++; @@ -568,6 +588,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, case StreamRole::Viewfinder: fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::ARGB8888; + colorSpace = ColorSpace::Jpeg; size = { 800, 600 }; bufferCount = 4; outCount++; @@ -599,6 +620,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, StreamConfiguration cfg(formats); cfg.size = size; cfg.pixelFormat = pixelFormat; + cfg.requestedColorSpace = colorSpace; cfg.bufferCount = bufferCount; config->addConfiguration(cfg); } @@ -703,6 +725,7 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) V4L2PixelFormat fourcc = V4L2PixelFormat::fromPixelFormat(cfg.pixelFormat); format.size = cfg.size; format.fourcc = fourcc; + format.colorSpace = cfg.requestedColorSpace; LOG(RPI, Debug) << "Setting " << stream->name() << " to " << format.toString(); @@ -718,6 +741,23 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) return -EINVAL; } + if (cfg.actualColorSpace != format.colorSpace) { + /* + * We should have been through validate() before so this + * shouldn't be possible, but we mustn't sweep color space + * problems under the carpet. + */ + LOG(RPI, Warning) + << "Unexpected color space change (" + << cfg.actualColorSpace.toString() << " to " + << format.colorSpace.toString() << ") in stream " + << stream->name(); + cfg.actualColorSpace = format.colorSpace; + } + LOG(RPI, Debug) + << "Stream " << stream->name() << " has color space " + << cfg.actualColorSpace.toString(); + cfg.setStream(stream); stream->setExternal(true); @@ -742,6 +782,8 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) format = {}; format.size = maxSize; format.fourcc = V4L2PixelFormat::fromPixelFormat(formats::YUV420); + /* No one asked for output, so the color space doesn't matter. */ + format.colorSpace = ColorSpace::Jpeg; ret = data->isp_[Isp::Output0].dev()->setFormat(&format); if (ret) { LOG(RPI, Error)