From patchwork Thu Nov 18 15:19:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 14626 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D0E2CBF415 for ; Thu, 18 Nov 2021 15:20:31 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 125EC60398; Thu, 18 Nov 2021 16:20:29 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="KD6ss0Lk"; dkim-atps=neutral Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 20D9860231 for ; Thu, 18 Nov 2021 16:20:26 +0100 (CET) Received: by mail-wr1-x42f.google.com with SMTP id a9so12252737wrr.8 for ; Thu, 18 Nov 2021 07:20:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=cDvxr9WaIJzpfgm/V2Z5Htlhe8VMWp/JstxZqXY1wy8=; b=KD6ss0LkGF5Htrk7mr+2hLJTXYuRCwZDQBN1DY174cRrxSd4btpFurtlPSVGAXZY8j btRFN6lmwv1nRwdWJL3SIdoMVG+jrxQAZc+dAVjUDEbyKEvtJWMA+ebjXTxgQ+niaEMi +1z7IvalxnP0xIpC1uzqjQ71/V3wG7iXbQOcMCK0CFleBd/2e8uOExOxc1uHq6bxsKeP 9P9EPewvA8MVyfYY7Mldg96XiP/BfIzfmyC4jvskLnORaqNd//utwtwMab7z9ftAd8/A jW37KORuyycHI+C13ajiyyUSxc+6RjZEZiCkuNveUCrLTIfts9eS6eDCpwSONT2fjcba f0UA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cDvxr9WaIJzpfgm/V2Z5Htlhe8VMWp/JstxZqXY1wy8=; b=Tz8L7aqys0Fm/Ye9TuPrsB40eW+v6vA1STPGNUzcPlB3gyRhjJlshoE6tMe2rhlOdv BkIzLg62Y662dFm7YHSzme2RewJLCSP6UfLmlF3olCeiSqtzcDPisp0xMLvRM60LB0xP 0qV0D3nMy5ChqDm6+SmptA3WQAPRKNwoTyvsWM7c0lWbex2DYgwzTAxaoaR3vQQd2up+ 6ge9NPr9TvfGoINcwe/dWShz2wR6MaH6GNfSTJeecys44SCKgDO8lFtPZLuOfHTQOiI1 6IvMPTcCG8nZ9vu63NUba4/3352xOQfWqyFIBHBU2N+D8nAdj7E/kc7CM9HiS1VUdyFe d5Ew== X-Gm-Message-State: AOAM5318a+9xnn6ya9n8IcR1BuhIvqryiJBtSRj2SlXYnp84wuk4utRZ Ogu43lAw6OAK0/1c4fqWnhvoqA== X-Google-Smtp-Source: ABdhPJyME7/MGYkO+WifBwH0bE8hJQjpqyzwP38oUJHzEynwwLLfao9pzYzknm+qhrjZj9P7X93YMA== X-Received: by 2002:a5d:648e:: with SMTP id o14mr31632365wri.141.1637248825835; Thu, 18 Nov 2021 07:20:25 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id p12sm147367wro.33.2021.11.18.07.20.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 18 Nov 2021 07:20:25 -0800 (PST) From: David Plowman To: laurent.pinchart@ideasonboard.com, kieran.bingham@ideasonboard.com, hverkuil-cisco@xs4all.nl, tfiga@google.com, jacopo@jmondi.org, naush@raspberrypi.com, libcamera-devel@lists.libcamera.org Date: Thu, 18 Nov 2021 15:19:27 +0000 Message-Id: <20211118151933.15627-2-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211118151933.15627-1-david.plowman@raspberrypi.com> References: <20211118151933.15627-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v6 1/7] libcamera: Add ColorSpace class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This class represents a color space by defining its color primaries, YCbCr encoding, the transfer (gamma) function it uses, and whether the output is full or limited range. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck Reviewed-by: Umang Jain --- include/libcamera/color_space.h | 79 +++++++++++ include/libcamera/meson.build | 1 + src/libcamera/color_space.cpp | 243 ++++++++++++++++++++++++++++++++ src/libcamera/meson.build | 1 + 4 files changed, 324 insertions(+) create mode 100644 include/libcamera/color_space.h create mode 100644 src/libcamera/color_space.cpp diff --git a/include/libcamera/color_space.h b/include/libcamera/color_space.h new file mode 100644 index 00000000..44ac077a --- /dev/null +++ b/include/libcamera/color_space.h @@ -0,0 +1,79 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Raspberry Pi (Trading) Limited + * + * color_space.h - color space definitions + */ + +#ifndef __LIBCAMERA_COLOR_SPACE_H__ +#define __LIBCAMERA_COLOR_SPACE_H__ + +#include +#include + +namespace libcamera { + +class ColorSpace +{ +public: + enum class Primaries : int { + Raw, + Smpte170m, + Rec709, + Rec2020, + }; + + enum class YcbcrEncoding : int { + Rec601, + Rec709, + Rec2020, + }; + + enum class TransferFunction : int { + Linear, + Srgb, + Rec709, + }; + + enum class Range : int { + Full, + Limited, + }; + + constexpr ColorSpace(Primaries p, YcbcrEncoding e, TransferFunction t, Range r) + : primaries(p), ycbcrEncoding(e), transferFunction(t), range(r) + { + } + + static const ColorSpace Raw; + static const ColorSpace Jpeg; + static const ColorSpace Srgb; + static const ColorSpace Smpte170m; + static const ColorSpace Rec709; + static const ColorSpace Rec2020; + + Primaries primaries; + YcbcrEncoding ycbcrEncoding; + TransferFunction transferFunction; + Range range; + + const std::string toString() const; + static const std::string toString(const std::optional &colorSpace); +}; + +constexpr ColorSpace ColorSpace::Raw = { Primaries::Raw, YcbcrEncoding::Rec601, TransferFunction::Linear, Range::Full }; +constexpr ColorSpace ColorSpace::Jpeg = { Primaries::Rec709, YcbcrEncoding::Rec601, TransferFunction::Srgb, Range::Full }; +constexpr ColorSpace ColorSpace::Srgb = { Primaries::Rec709, YcbcrEncoding::Rec601, TransferFunction::Srgb, Range::Limited }; +constexpr ColorSpace ColorSpace::Smpte170m = { Primaries::Smpte170m, YcbcrEncoding::Rec601, TransferFunction::Rec709, Range::Limited }; +constexpr ColorSpace ColorSpace::Rec709 = { Primaries::Rec709, YcbcrEncoding::Rec709, TransferFunction::Rec709, Range::Limited }; +constexpr ColorSpace ColorSpace::Rec2020 = { Primaries::Rec2020, YcbcrEncoding::Rec2020, TransferFunction::Rec709, Range::Limited }; + +bool operator==(const ColorSpace &lhs, const ColorSpace &rhs); +static inline bool operator!=(const ColorSpace &lhs, const ColorSpace &rhs) +{ + return !(lhs == rhs); +} + +} /* namespace libcamera */ + +#endif /* __LIBCAMERA_COLOR_SPACE_H__ */ diff --git a/include/libcamera/meson.build b/include/libcamera/meson.build index 7155ff20..131e1740 100644 --- a/include/libcamera/meson.build +++ b/include/libcamera/meson.build @@ -5,6 +5,7 @@ libcamera_include_dir = 'libcamera' / 'libcamera' libcamera_public_headers = files([ 'camera.h', 'camera_manager.h', + 'color_space.h', 'compiler.h', 'controls.h', 'file_descriptor.h', diff --git a/src/libcamera/color_space.cpp b/src/libcamera/color_space.cpp new file mode 100644 index 00000000..1a5fc7a2 --- /dev/null +++ b/src/libcamera/color_space.cpp @@ -0,0 +1,243 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Raspberry Pi (Trading) Limited + * + * color_space.cpp - color spaces. + */ + +#include + +#include +#include +#include + +/** + * \file color_space.h + * \brief Class and enums to represent color spaces + */ + +namespace libcamera { + +/** + * \class ColorSpace + * \brief Class to describe a color space + * + * The ColorSpace class defines the color primaries, the Y'CbCr encoding, + * the transfer function associated with the color space, and the range + * (sometimes also referred to as the quantisation) of the color space. + * + * Certain combinations of these fields form well-known standard color + * spaces such as "JPEG" or "REC709". + * + * In the strictest sense a "color space" formally only refers to the + * color primaries and white point. Here, however, the ColorSpace class + * adopts the common broader usage that includes the transfer function, + * Y'CbCr encoding method and quantisation. + * + * For more information on the specific color spaces described here, please + * see: + * + * sRGB + * JPEG + * SMPTE 170M + * Rec.709 + * Rec.2020 + */ + +/** + * \enum ColorSpace::Primaries + * \brief The color primaries for this color space + * + * \var ColorSpace::Primaries::Raw + * \brief These are raw colors directly from a sensor + * \var ColorSpace::Primaries::Smpte170m + * \brief SMPTE 170M color primaries + * \var ColorSpace::Primaries::Rec709 + * \brief Rec.709 color primaries + * \var ColorSpace::Primaries::Rec2020 + * \brief Rec.2020 color primaries + */ + +/** + * \enum ColorSpace::YcbcrEncoding + * \brief The Y'CbCr encoding + * + * \var ColorSpace::YcbcrEncoding::Rec601 + * \brief Rec.601 Y'CbCr encoding + * \var ColorSpace::YcbcrEncoding::Rec709 + * \brief Rec.709 Y'CbCr encoding + * \var ColorSpace::YcbcrEncoding::Rec2020 + * \brief Rec.2020 Y'CbCr encoding + */ + +/** + * \enum ColorSpace::TransferFunction + * \brief The transfer function used for this color space + * + * \var ColorSpace::TransferFunction::Linear + * \brief This color space uses a linear (identity) transfer function + * \var ColorSpace::TransferFunction::Srgb + * \brief sRGB transfer function + * \var ColorSpace::TransferFunction::Rec709 + * \brief Rec.709 transfer function + */ + +/** + * \enum ColorSpace::Range + * \brief The range (sometimes "quantisation") for this color space + * + * \var ColorSpace::Range::Full + * \brief This color space uses full range pixel values + * \var ColorSpace::Range::Limited + * \brief This color space uses limited range pixel values, being + * 16 to 235 for Y' and 16 to 240 for Cb and Cr (8 bits per sample) + * or 64 to 940 for Y' and 16 to 960 for Cb and Cr (10 bits) + */ + +/** + * \fn ColorSpace::ColorSpace(Primaries p, Encoding e, TransferFunction t, Range r) + * \brief Construct a ColorSpace from explicit values + * \param[in] p The color primaries + * \param[in] e The Y'CbCr encoding + * \param[in] t The transfer function for the color space + * \param[in] r The range of the pixel values in this color space + */ + +/** + * \brief Assemble and return a readable string representation of the + * ColorSpace + * \return A string describing the ColorSpace + */ +const std::string ColorSpace::toString() const +{ + /* Print out a brief name only for standard color sapces. */ + + static const std::vector> colorSpaceNames = { + { ColorSpace::Raw, "Raw" }, + { ColorSpace::Jpeg, "Jpeg" }, + { ColorSpace::Srgb, "Srgb" }, + { ColorSpace::Smpte170m, "Smpte170m" }, + { ColorSpace::Rec709, "Rec709" }, + { ColorSpace::Rec2020, "Rec2020" }, + }; + auto it = std::find_if(colorSpaceNames.begin(), colorSpaceNames.end(), + [this](const auto &item) { + return *this == item.first; + }); + if (it != colorSpaceNames.end()) + return std::string(it->second); + + static const char *primariesNames[] = { + "Raw", + "Smpte170m", + "Rec709", + "Rec2020", + }; + static const char *encodingNames[] = { + "Rec601", + "Rec709", + "Rec2020", + }; + static const char *transferFunctionNames[] = { + "Linear", + "Srgb", + "Rec709", + }; + static const char *rangeNames[] = { + "Full", + "Limited", + }; + + std::stringstream ss; + ss << std::string(primariesNames[static_cast(primaries)]) << "/" + << std::string(encodingNames[static_cast(ycbcrEncoding)]) << "/" + << std::string(transferFunctionNames[static_cast(transferFunction)]) << "/" + << std::string(rangeNames[static_cast(range)]); + + return ss.str(); +} + +/** + * \brief Assemble and return a readable string representation of an + * optional ColorSpace + * \return A string describing the optional ColorSpace + */ +const std::string ColorSpace::toString(const std::optional &colorSpace) +{ + if (!colorSpace) + return "Unknown"; + + return colorSpace->toString(); +} + +/** + * \var ColorSpace::primaries + * \brief The color primaries + */ + +/** + * \var ColorSpace::ycbcrEncoding + * \brief The Y'CbCr encoding + */ + +/** + * \var ColorSpace::transferFunction + * \brief The transfer function for this color space + */ + +/** + * \var ColorSpace::range + * \brief The pixel range used by this color space + */ + +/** + * \var ColorSpace::Undefined + * \brief A constant representing a fully undefined color space + */ + +/** + * \var ColorSpace::Raw + * \brief A constant representing a raw color space (from a sensor) + */ + +/** + * \var ColorSpace::Jpeg + * \brief A constant representing the JPEG color space used for + * encoding JPEG images (and regarded as being the same as the sRGB + * color space) + */ + +/** + * \var ColorSpace::Smpte170m + * \brief A constant representing the SMPTE170M color space + */ + +/** + * \var ColorSpace::Rec709 + * \brief A constant representing the Rec.709 color space + */ + +/** + * \var ColorSpace::Rec2020 + * \brief A constant representing the Rec.2020 color space + */ + +/** + * \brief Compare color spaces for equality + * \return True if the two color spaces are identical, false otherwise + */ +bool operator==(const ColorSpace &lhs, const ColorSpace &rhs) +{ + return lhs.primaries == rhs.primaries && + lhs.ycbcrEncoding == rhs.ycbcrEncoding && + lhs.transferFunction == rhs.transferFunction && + lhs.range == rhs.range; +} + +/** + * \fn bool operator!=(const ColorSpace &lhs, const ColorSpace &rhs) + * \brief Compare color spaces for inequality + * \return True if the two color spaces are not identical, false otherwise + */ + +} /* namespace libcamera */ diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build index 6727a777..e7371d20 100644 --- a/src/libcamera/meson.build +++ b/src/libcamera/meson.build @@ -8,6 +8,7 @@ libcamera_sources = files([ 'camera_manager.cpp', 'camera_sensor.cpp', 'camera_sensor_properties.cpp', + 'color_space.cpp', 'controls.cpp', 'control_serializer.cpp', 'control_validator.cpp', From patchwork Thu Nov 18 15:19:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 14627 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5D2DABF415 for ; Thu, 18 Nov 2021 15:20:33 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0864360371; Thu, 18 Nov 2021 16:20:30 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="D9pAURQ0"; dkim-atps=neutral Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D6F9E6038A for ; Thu, 18 Nov 2021 16:20:26 +0100 (CET) Received: by mail-wr1-x429.google.com with SMTP id u1so12208868wru.13 for ; Thu, 18 Nov 2021 07:20:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kLk2z8qXqfOpGggekOhFU+kISUZcUm+8gTdLG71oWtQ=; b=D9pAURQ0CNj+Xm919iQ+lQNeDQCH3P3++ZpwvVkcTlyGu+YzBJoseGKdFHQD4zYgzp Bb+5lOYEaTmlBfWy8aKCxAqTutzTPbwHIM75kyEC6YiU2REMOMVqQUXjLrB0QPh8ao79 Fxsn1D5Y+xg3SdQmIT0c9UjvGFZtDBxAgIPM6pn5fR9V+dKw0oDXarLcDhkNq/y4pejj 22EDoJtpCVxSM9Qsr//BxCN/5ki785vDapefxotMQMxkEWA/c4Wq+QTwARKilI4ZWqrR w+trMzfM+oiQkIIc7jukkHwSHFkCqLSUqgoVTrAO6UaWBJyeymzacJo2Z1iDfafbXcz6 IpVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kLk2z8qXqfOpGggekOhFU+kISUZcUm+8gTdLG71oWtQ=; b=M2fNUiiTb+s5gXb8MgJDkb8V6KAYdqYuGp0hFYEKKZnzPkwmPvhxsBua+UIcDBtC0e 2csOHz6SAIUsxPr3mo0cKN/DlYskQWFg1GPpBrUTsiNG+Oc2iOr9hj95frS8b89eMVoG nddBawUbwR8uf7GfwKtatxmjIm0FFDWWNRkUC+T0AFCTEFssF8WepzeUiD3hvhzpboTS T7kfaG3cuuh3PCelukvuySEIaXnZIyICeoHdSTszTg8JhfPHxZYHmbIKZ1hIMOXAE43o P4+prpVUMYsxj5to85aRTzagNedEk8Yjgpmr6TvMG2/q7Qu+zdfj5XfsfYEKbwK/btA1 kywg== X-Gm-Message-State: AOAM5328/ixlKclhP4qiOLnLeUTqUoh8M3NjfKx1ohuj81Ex1+YVKwXA 2aMzSWpERpzM6a6BR+ZCjvxSEg== X-Google-Smtp-Source: ABdhPJx/MP1iXD71qEns5O9vMKAyvBo91FlSomT9nJWbyGAmGnPqc9RGqvewbaAWWeJiNZxsTU8MMw== X-Received: by 2002:adf:9bdb:: with SMTP id e27mr32804867wrc.417.1637248826625; Thu, 18 Nov 2021 07:20:26 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id p12sm147367wro.33.2021.11.18.07.20.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 18 Nov 2021 07:20:26 -0800 (PST) From: David Plowman To: laurent.pinchart@ideasonboard.com, kieran.bingham@ideasonboard.com, hverkuil-cisco@xs4all.nl, tfiga@google.com, jacopo@jmondi.org, naush@raspberrypi.com, libcamera-devel@lists.libcamera.org Date: Thu, 18 Nov 2021 15:19:28 +0000 Message-Id: <20211118151933.15627-3-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211118151933.15627-1-david.plowman@raspberrypi.com> References: <20211118151933.15627-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v6 2/7] libcamera: Add ColorSpace fields to StreamConfiguration X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This is so that applications can choose appropriate color spaces which will then be passed down to the V4L2 devices. The ColorSpace field is actually optional. If it is not set you will get the driver's default color space. Signed-off-by: David Plowman Reviewed-by: Umang Jain --- include/libcamera/stream.h | 3 +++ src/libcamera/stream.cpp | 14 ++++++++++++++ 2 files changed, 17 insertions(+) diff --git a/include/libcamera/stream.h b/include/libcamera/stream.h index 0c55e716..bea88eb4 100644 --- a/include/libcamera/stream.h +++ b/include/libcamera/stream.h @@ -12,6 +12,7 @@ #include #include +#include #include #include #include @@ -47,6 +48,8 @@ struct StreamConfiguration { unsigned int bufferCount; + std::optional colorSpace; + Stream *stream() const { return stream_; } void setStream(Stream *stream) { stream_ = stream; } const StreamFormats &formats() const { return formats_; } diff --git a/src/libcamera/stream.cpp b/src/libcamera/stream.cpp index b421e17e..300b3af7 100644 --- a/src/libcamera/stream.cpp +++ b/src/libcamera/stream.cpp @@ -329,6 +329,20 @@ StreamConfiguration::StreamConfiguration(const StreamFormats &formats) * \brief Requested number of buffers to allocate for the stream */ +/** + * \var StreamConfiguration::colorSpace + * \brief The ColorSpace for this stream + * + * A suitable color space may be set here or chosen by an application. + * Alternatively the color space may be left unset, in which case it will + * be up to the driver to choose a color space. + * + * If the system cannot deliver the requested color space, the validate() + * method will overwrite the value here with what it can deliver. Note that + * platforms may enforce extra contraints here, such as requiring all output + * streams to share the same color space. + */ + /** * \fn StreamConfiguration::stream() * \brief Retrieve the stream associated with the configuration From patchwork Thu Nov 18 15:19:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 14628 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2D6C6C324E for ; Thu, 18 Nov 2021 15:20:34 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 70752603BE; Thu, 18 Nov 2021 16:20:33 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="b2Tt3jev"; dkim-atps=neutral Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A38826038A for ; Thu, 18 Nov 2021 16:20:27 +0100 (CET) Received: by mail-wm1-x32f.google.com with SMTP id i12so5666025wmq.4 for ; Thu, 18 Nov 2021 07:20:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=dPpf98qt/hshLduLmlaFBnisRdBSpv5pU9YXKDl3Xjs=; b=b2Tt3jevLf9oa759VttX/WPtaOpnbDh5s1YQ+S4vu+aedyN2TDBDuQk2PvmuZAQtml RacMG6ksPPuI+7AAz6V/ZqJaNQLdCVgCftCmZtGVtfdsO9/vRyz2bJ6mIBelGmIOGEFS HG5dVgUvMvC+oIZGjLuce9mwqTlWe+wWUigTnJK0BjMua0wF/PwQESfOBPeTpo6yt2UI DrKFNDxfkUf/w0Y41k8CNTdwwQk3FXTtDrK56vX9COpH9JSa1j3FB7nc41N+Zg1BEoG2 Th6Cx3RvYG3diaKomVawJRPR5e5PJcV2PixUuqxZkpsroKMW56iBBCt5BMHTlOu5cW3J 6ZGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=dPpf98qt/hshLduLmlaFBnisRdBSpv5pU9YXKDl3Xjs=; b=O8YFz5UAQ8GN2rS690SdKwFUzBV7MOj6Q+RcQsvO8iU1UL0okXV9cI4+sA6ZmZz0VB pDjS1L9IBl8VmJY9oGh1/Ry8atepS4+fBt0mJMmhgNtMwQrniEj9o8KHynO9185M/Jo+ RuaOnfrY1yQJK4aYTAspYZ28NLy6n2obrf1DzAZUF1BJqPWmZQXt0YzOxOjp14LjMnJI s4KXI8mRYYvR94os0OGkmFiRgD2GP1u3SKqQYnWXy9rKILrNfvOLAvedAQJ1XCRNAPWm 0U8q/3pfaqMUW8HPxdmHnk15RY0CRGeFHKRJkUQO1DPyAq7YTjAKZE2VO4C28LjB5wbX Ox5w== X-Gm-Message-State: AOAM531FnWIHB9jRjRAa1dDvRZBNSK23wM2ZSKns6/cuttdPG3SnfDdW Ez+Ev0uQYbbTnkuT/wo6phZabQ== X-Google-Smtp-Source: ABdhPJzzFR0pe5uoUdfZMWv3D0N3BD2HJFfwtBzQpBp+ILSdbj9eFHpKpoNCZxBjeYQWQ6wxuOAD5A== X-Received: by 2002:a05:600c:510d:: with SMTP id o13mr10941598wms.104.1637248827302; Thu, 18 Nov 2021 07:20:27 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id p12sm147367wro.33.2021.11.18.07.20.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 18 Nov 2021 07:20:26 -0800 (PST) From: David Plowman To: laurent.pinchart@ideasonboard.com, kieran.bingham@ideasonboard.com, hverkuil-cisco@xs4all.nl, tfiga@google.com, jacopo@jmondi.org, naush@raspberrypi.com, libcamera-devel@lists.libcamera.org Date: Thu, 18 Nov 2021 15:19:29 +0000 Message-Id: <20211118151933.15627-4-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211118151933.15627-1-david.plowman@raspberrypi.com> References: <20211118151933.15627-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v6 3/7] libcamera: Convert between ColorSpace class and V4L2 formats X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" These methods are added to the base V4L2Device class so that they can be shared both by the video device class and subdevices. With the ColorSpace class, the color space and related other fields are stored together, corresponding to a number of fields in the various different V4L2 format structures. Template methods are therefore a convenient implementation, and we must explicitly instantiate the templates that will be needed. Note that unset color spaces are converted to requests for the device's "default" color space. Signed-off-by: David Plowman --- include/libcamera/internal/v4l2_device.h | 7 + src/libcamera/v4l2_device.cpp | 190 +++++++++++++++++++++++ 2 files changed, 197 insertions(+) diff --git a/include/libcamera/internal/v4l2_device.h b/include/libcamera/internal/v4l2_device.h index f21bc370..89ff6120 100644 --- a/include/libcamera/internal/v4l2_device.h +++ b/include/libcamera/internal/v4l2_device.h @@ -17,6 +17,7 @@ #include #include +#include #include namespace libcamera { @@ -44,6 +45,12 @@ public: void updateControlInfo(); + template + static std::optional toColorSpace(const T &v4l2Format); + + template + static int fromColorSpace(const std::optional &colorSpace, T &v4l2Format); + protected: V4L2Device(const std::string &deviceNode); ~V4L2Device(); diff --git a/src/libcamera/v4l2_device.cpp b/src/libcamera/v4l2_device.cpp index 9c783c9c..4d6985aa 100644 --- a/src/libcamera/v4l2_device.cpp +++ b/src/libcamera/v4l2_device.cpp @@ -16,6 +16,8 @@ #include #include +#include + #include #include #include @@ -731,4 +733,192 @@ void V4L2Device::eventAvailable() frameStart.emit(event.u.frame_sync.frame_sequence); } +static const std::map v4l2ToColorSpace = { + { V4L2_COLORSPACE_RAW, ColorSpace::Raw }, + { V4L2_COLORSPACE_JPEG, ColorSpace::Jpeg }, + { V4L2_COLORSPACE_SRGB, ColorSpace::Srgb }, + { V4L2_COLORSPACE_SMPTE170M, ColorSpace::Smpte170m }, + { V4L2_COLORSPACE_REC709, ColorSpace::Rec709 }, + { V4L2_COLORSPACE_BT2020, ColorSpace::Rec2020 }, +}; + +static const std::map v4l2ToYcbcrEncoding = { + { V4L2_YCBCR_ENC_601, ColorSpace::YcbcrEncoding::Rec601 }, + { V4L2_YCBCR_ENC_709, ColorSpace::YcbcrEncoding::Rec709 }, + { V4L2_YCBCR_ENC_BT2020, ColorSpace::YcbcrEncoding::Rec2020 }, +}; + +static const std::map v4l2ToTransferFunction = { + { V4L2_XFER_FUNC_NONE, ColorSpace::TransferFunction::Linear }, + { V4L2_XFER_FUNC_SRGB, ColorSpace::TransferFunction::Srgb }, + { V4L2_XFER_FUNC_709, ColorSpace::TransferFunction::Rec709 }, +}; + +static const std::map v4l2ToRange = { + { V4L2_QUANTIZATION_FULL_RANGE, ColorSpace::Range::Full }, + { V4L2_QUANTIZATION_LIM_RANGE, ColorSpace::Range::Limited }, +}; + +static const std::vector> colorSpaceToV4l2 = { + { ColorSpace::Raw, V4L2_COLORSPACE_RAW }, + { ColorSpace::Jpeg, V4L2_COLORSPACE_JPEG }, + { ColorSpace::Srgb, V4L2_COLORSPACE_SRGB }, + { ColorSpace::Smpte170m, V4L2_COLORSPACE_SMPTE170M }, + { ColorSpace::Rec709, V4L2_COLORSPACE_REC709 }, + { ColorSpace::Rec2020, V4L2_COLORSPACE_BT2020 }, +}; + +static const std::map primariesToV4l2 = { + { ColorSpace::Primaries::Raw, V4L2_COLORSPACE_RAW }, + { ColorSpace::Primaries::Smpte170m, V4L2_COLORSPACE_SMPTE170M }, + { ColorSpace::Primaries::Rec709, V4L2_COLORSPACE_REC709 }, + { ColorSpace::Primaries::Rec2020, V4L2_COLORSPACE_BT2020 }, +}; + +static const std::map ycbcrEncodingToV4l2 = { + { ColorSpace::YcbcrEncoding::Rec601, V4L2_YCBCR_ENC_601 }, + { ColorSpace::YcbcrEncoding::Rec709, V4L2_YCBCR_ENC_709 }, + { ColorSpace::YcbcrEncoding::Rec2020, V4L2_YCBCR_ENC_BT2020 }, +}; + +static const std::map transferFunctionToV4l2 = { + { ColorSpace::TransferFunction::Linear, V4L2_XFER_FUNC_NONE }, + { ColorSpace::TransferFunction::Srgb, V4L2_XFER_FUNC_SRGB }, + { ColorSpace::TransferFunction::Rec709, V4L2_XFER_FUNC_709 }, +}; + +static const std::map rangeToV4l2 = { + { ColorSpace::Range::Full, V4L2_QUANTIZATION_FULL_RANGE }, + { ColorSpace::Range::Limited, V4L2_QUANTIZATION_LIM_RANGE }, +}; + +/** + * \brief Convert the color space fields in a V4L2 format to a ColorSpace + * \param[in] v4l2Format A V4L2 format containing color space information + * + * The colorspace, ycbcr_enc, xfer_func and quantization fields within a + * V4L2 format structure are converted to a corresponding ColorSpace. + * + * If any V4L2 fields are not recognised then we return an "unset" + * color space. + * + * \return The ColorSpace corresponding to the input V4L2 format + */ +template +std::optional V4L2Device::toColorSpace(const T &v4l2Format) +{ + auto itColor = v4l2ToColorSpace.find(v4l2Format.colorspace); + if (itColor == v4l2ToColorSpace.end()) + return std::nullopt; + + /* This sets all the color space fields to the correct "default" values. */ + ColorSpace colorSpace = itColor->second; + + if (v4l2Format.ycbcr_enc != V4L2_YCBCR_ENC_DEFAULT) { + auto itYcbcrEncoding = v4l2ToYcbcrEncoding.find(v4l2Format.ycbcr_enc); + if (itYcbcrEncoding == v4l2ToYcbcrEncoding.end()) + return std::nullopt; + + colorSpace.ycbcrEncoding = itYcbcrEncoding->second; + } + + if (v4l2Format.xfer_func != V4L2_XFER_FUNC_DEFAULT) { + auto itTransfer = v4l2ToTransferFunction.find(v4l2Format.xfer_func); + if (itTransfer == v4l2ToTransferFunction.end()) + return std::nullopt; + + colorSpace.transferFunction = itTransfer->second; + } + + if (v4l2Format.quantization != V4L2_QUANTIZATION_DEFAULT) { + auto itRange = v4l2ToRange.find(v4l2Format.quantization); + if (itRange == v4l2ToRange.end()) + return std::nullopt; + + colorSpace.range = itRange->second; + } + + return colorSpace; +} + +template std::optional V4L2Device::toColorSpace(const struct v4l2_pix_format &); +template std::optional V4L2Device::toColorSpace(const struct v4l2_pix_format_mplane &); +template std::optional V4L2Device::toColorSpace(const struct v4l2_mbus_framefmt &); + +/** + * \brief Fill in the color space fields of a V4L2 format from a ColorSpace + * \param[in] colorSpace The ColorSpace to be converted + * \param[out] v4l2Format A V4L2 format containing color space information + * + * The colorspace, ycbcr_enc, xfer_func and quantization fields within a + * V4L2 format structure are filled in from a corresponding ColorSpace. + * + * An error is returned if any of the V4L2 fields do not support the + * value given in the ColorSpace. Such fields are set to the V4L2 + * "default" values, but all other fields are still filled in where + * possible. + * + * If the color space is completely unset, "default" V4L2 values are used + * everywhere, so a driver would then choose its preferred color space. + * + * \return 0 on success or a negative error code otherwise + */ +template +int V4L2Device::fromColorSpace(const std::optional &colorSpace, T &v4l2Format) +{ + int ret = 0; + + v4l2Format.colorspace = V4L2_COLORSPACE_DEFAULT; + v4l2Format.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; + v4l2Format.xfer_func = V4L2_XFER_FUNC_DEFAULT; + v4l2Format.quantization = V4L2_QUANTIZATION_DEFAULT; + + if (!colorSpace) + return ret; + + auto itColor = std::find_if(colorSpaceToV4l2.begin(), colorSpaceToV4l2.end(), + [&colorSpace](const auto &item) { + return colorSpace == item.first; + }); + if (itColor != colorSpaceToV4l2.end()) { + v4l2Format.colorspace = itColor->second; + /* Leaving all the other fields as "default" should be fine. */ + return ret; + } + + /* + * If the colorSpace doesn't precisely match a standard color space, + * then we must choose a V4L2 colorspace with matching primaries. + */ + auto itPrimaries = primariesToV4l2.find(colorSpace->primaries); + if (itPrimaries != primariesToV4l2.end()) + v4l2Format.colorspace = itPrimaries->second; + else + ret = -1; + + auto itYcbcrEncoding = ycbcrEncodingToV4l2.find(colorSpace->ycbcrEncoding); + if (itYcbcrEncoding != ycbcrEncodingToV4l2.end()) + v4l2Format.ycbcr_enc = itYcbcrEncoding->second; + else + ret = -1; + + auto itTransfer = transferFunctionToV4l2.find(colorSpace->transferFunction); + if (itTransfer != transferFunctionToV4l2.end()) + v4l2Format.xfer_func = itTransfer->second; + else + ret = -1; + + auto itRange = rangeToV4l2.find(colorSpace->range); + if (itRange != rangeToV4l2.end()) + v4l2Format.quantization = itRange->second; + else + ret = -1; + + return ret; +} + +template int V4L2Device::fromColorSpace(const std::optional &, struct v4l2_pix_format &); +template int V4L2Device::fromColorSpace(const std::optional &, struct v4l2_pix_format_mplane &); +template int V4L2Device::fromColorSpace(const std::optional &, struct v4l2_mbus_framefmt &); + } /* namespace libcamera */ From patchwork Thu Nov 18 15:19:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 14629 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 067B9BF415 for ; Thu, 18 Nov 2021 15:20:36 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B02C4603C2; Thu, 18 Nov 2021 16:20:35 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="LcZPmmT5"; dkim-atps=neutral Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A693860376 for ; Thu, 18 Nov 2021 16:20:28 +0100 (CET) Received: by mail-wr1-x436.google.com with SMTP id b12so12253134wrh.4 for ; Thu, 18 Nov 2021 07:20:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XLKHkKrG/UEEcs/2bX824fXCZkEOH3339Y2W5nVKFIg=; b=LcZPmmT5guaAKEXzQ9fr6NSMmhzsdY2a5/AEvAT4QW6WUJseA/Vm670ie0EPCrQwdb C1dXc6kzL6bIYIIgc14dhKnqyd39pHdwlR3dSTljznj5U0PC/Xo1ChN8NvHwNy/9fvUX zqnrHdEkezHNExAo1ZqApaJTMSLM3P0EEjdDirob7Q6RjfsM4leR9MO4M7CctwXwjNCM QsdntB3j+PupVNLMKRnj8EJlAgH29g35e6Ib4BAQ88QnlG6/O3U5x5ipf69YuzifpMcg 6k5vfk5mccMwR4EW0Go7retcZZYl++tmuxBfhxQH2ntUsmICX9lmFRQSRqs1sBknyu2+ mWjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XLKHkKrG/UEEcs/2bX824fXCZkEOH3339Y2W5nVKFIg=; b=gWkRC/gY2pXuRaDqqASDO9ohOCtKDRhsVTIq9FyHjkauKzoEbdE5kJcJ8vwV/tUFy7 lysh7CNFYyu13xUFiqhmIkMjea3pWemciQhYbj9s2VZGBl63C6IhpdcgiNanzBlB2Roi QRUVZN5mpEnoalQQjNx4ILSobVxFDPUeFZkIueuuSIfyTL59vVeMYN7B9IC6+wDt3xnC 0M4R0ds3BZA3/xBuG3A6Xs44roRr+EuLwnOU2xitPLrAvGPWi4C9SWIto9V0IabQ/qwc MZR+5i354e9vjbM+u0UYDpgHxJnR5KXPpxVfym42Ll3Wfr2BU0qjRxKm2/MDvgG70du+ aF+A== X-Gm-Message-State: AOAM530SD163Joi5aWi4u38JEk0fL23jcFZqMhOlxf5a1QNz5nJ5N/YQ N8MNJJHTu/FnBGLnmnYTg5NxwQ== X-Google-Smtp-Source: ABdhPJxS7qv7lBYa7xwgcTVMyjpOQbcxYZpAYia/uJ97C0TYwe4JsZefwRB1zWiAb6uohDvoX7rj7Q== X-Received: by 2002:a5d:6152:: with SMTP id y18mr32364355wrt.271.1637248828146; Thu, 18 Nov 2021 07:20:28 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id p12sm147367wro.33.2021.11.18.07.20.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 18 Nov 2021 07:20:27 -0800 (PST) From: David Plowman To: laurent.pinchart@ideasonboard.com, kieran.bingham@ideasonboard.com, hverkuil-cisco@xs4all.nl, tfiga@google.com, jacopo@jmondi.org, naush@raspberrypi.com, libcamera-devel@lists.libcamera.org Date: Thu, 18 Nov 2021 15:19:30 +0000 Message-Id: <20211118151933.15627-5-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211118151933.15627-1-david.plowman@raspberrypi.com> References: <20211118151933.15627-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v6 4/7] libcamera: Support passing ColorSpaces to V4L2 video devices X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The ColorSpace from the StreamConfiguration is now handled appropriately in the V4L2VideoDevice. Signed-off-by: David Plowman --- include/libcamera/internal/v4l2_videodevice.h | 2 + src/libcamera/v4l2_videodevice.cpp | 53 +++++++++++++++++-- 2 files changed, 51 insertions(+), 4 deletions(-) diff --git a/include/libcamera/internal/v4l2_videodevice.h b/include/libcamera/internal/v4l2_videodevice.h index a1c458e4..00bc50e7 100644 --- a/include/libcamera/internal/v4l2_videodevice.h +++ b/include/libcamera/internal/v4l2_videodevice.h @@ -20,6 +20,7 @@ #include #include +#include #include #include #include @@ -167,6 +168,7 @@ public: V4L2PixelFormat fourcc; Size size; + std::optional colorSpace; std::array planes; unsigned int planesCount = 0; diff --git a/src/libcamera/v4l2_videodevice.cpp b/src/libcamera/v4l2_videodevice.cpp index 4f04212d..e03ff8b5 100644 --- a/src/libcamera/v4l2_videodevice.cpp +++ b/src/libcamera/v4l2_videodevice.cpp @@ -370,17 +370,27 @@ bool V4L2BufferCache::Entry::operator==(const FrameBuffer &buffer) const * \brief The plane line stride (in bytes) */ +/** + * \var V4L2DeviceFormat::fourcc + * \brief The fourcc code describing the pixel encoding scheme + * + * The fourcc code, as defined by the V4L2 API with the V4L2_PIX_FMT_* macros, + * that identifies the image format pixel encoding scheme. + */ + /** * \var V4L2DeviceFormat::size * \brief The image size in pixels */ /** - * \var V4L2DeviceFormat::fourcc - * \brief The fourcc code describing the pixel encoding scheme + * \var V4L2DeviceFormat::colorSpace + * \brief The color space of the pixels * - * The fourcc code, as defined by the V4L2 API with the V4L2_PIX_FMT_* macros, - * that identifies the image format pixel encoding scheme. + * The color space of the image. When setting the format this may be + * unset, in which case the driver gets to use its default color space. + * After being set, this value should contain the color space that the + * was actually used. */ /** @@ -879,6 +889,10 @@ int V4L2VideoDevice::getFormatMultiplane(V4L2DeviceFormat *format) format->fourcc = V4L2PixelFormat(pix->pixelformat); format->planesCount = pix->num_planes; + format->colorSpace = toColorSpace(*pix); + if (!format->colorSpace) + LOG(V4L2, Error) << "Retrieved unrecognised color space"; + for (unsigned int i = 0; i < format->planesCount; ++i) { format->planes[i].bpl = pix->plane_fmt[i].bytesperline; format->planes[i].size = pix->plane_fmt[i].sizeimage; @@ -893,6 +907,9 @@ int V4L2VideoDevice::trySetFormatMultiplane(V4L2DeviceFormat *format, bool set) struct v4l2_pix_format_mplane *pix = &v4l2Format.fmt.pix_mp; int ret; + if (!format->colorSpace) + LOG(V4L2, Warning) << "Setting undefined color space"; + v4l2Format.type = bufferType_; pix->width = format->size.width; pix->height = format->size.height; @@ -900,6 +917,12 @@ int V4L2VideoDevice::trySetFormatMultiplane(V4L2DeviceFormat *format, bool set) pix->num_planes = format->planesCount; pix->field = V4L2_FIELD_NONE; + ret = fromColorSpace(format->colorSpace, *pix); + if (ret < 0) + LOG(V4L2, Warning) + << "Setting color space unrecognised by V4L2: " + << ColorSpace::toString(format->colorSpace); + ASSERT(pix->num_planes <= std::size(pix->plane_fmt)); for (unsigned int i = 0; i < pix->num_planes; ++i) { @@ -928,6 +951,10 @@ int V4L2VideoDevice::trySetFormatMultiplane(V4L2DeviceFormat *format, bool set) format->planes[i].size = pix->plane_fmt[i].sizeimage; } + format->colorSpace = toColorSpace(*pix); + if (!format->colorSpace) + LOG(V4L2, Warning) << "Unrecognised color space has been set"; + return 0; } @@ -951,6 +978,10 @@ int V4L2VideoDevice::getFormatSingleplane(V4L2DeviceFormat *format) format->planes[0].bpl = pix->bytesperline; format->planes[0].size = pix->sizeimage; + format->colorSpace = toColorSpace(*pix); + if (!format->colorSpace) + LOG(V4L2, Error) << "Retrieved unrecognised color space"; + return 0; } @@ -960,12 +991,22 @@ int V4L2VideoDevice::trySetFormatSingleplane(V4L2DeviceFormat *format, bool set) struct v4l2_pix_format *pix = &v4l2Format.fmt.pix; int ret; + if (!format->colorSpace) + LOG(V4L2, Error) << "Trying to set unrecognised color space"; + v4l2Format.type = bufferType_; pix->width = format->size.width; pix->height = format->size.height; pix->pixelformat = format->fourcc; pix->bytesperline = format->planes[0].bpl; pix->field = V4L2_FIELD_NONE; + + ret = fromColorSpace(format->colorSpace, *pix); + if (ret < 0) + LOG(V4L2, Warning) + << "Set color space unrecognised by V4L2: " + << ColorSpace::toString(format->colorSpace); + ret = ioctl(set ? VIDIOC_S_FMT : VIDIOC_TRY_FMT, &v4l2Format); if (ret) { LOG(V4L2, Error) @@ -985,6 +1026,10 @@ int V4L2VideoDevice::trySetFormatSingleplane(V4L2DeviceFormat *format, bool set) format->planes[0].bpl = pix->bytesperline; format->planes[0].size = pix->sizeimage; + format->colorSpace = toColorSpace(*pix); + if (!format->colorSpace) + LOG(V4L2, Error) << "Undefined color space has been set"; + return 0; } From patchwork Thu Nov 18 15:19:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 14630 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 95BECC324E for ; Thu, 18 Nov 2021 15:20:36 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 382C9603B5; Thu, 18 Nov 2021 16:20:36 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="ACkt+fUo"; dkim-atps=neutral Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 824BD60233 for ; Thu, 18 Nov 2021 16:20:29 +0100 (CET) Received: by mail-wr1-x42d.google.com with SMTP id s13so12280860wrb.3 for ; Thu, 18 Nov 2021 07:20:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=H7T104lkfLZPMGxIBUsm1hoLba7o7e864GWa57cAV/Y=; b=ACkt+fUokvvQHsIy9uFUbhdWJcz3M0oP4GJ8FiQLtjs1vNld/6bXsoHMJaEHDYKwHM 7UGHJ/Q2e2BijL4jCrahi25TfKk2qcPLa/HVcMAZqpIyRJ8vfm8ALpZo5WeIJsj2cank q6SPjRu3uSp2DzqPRf4PrwvPTl7Xblm+MKv0RsVp+4HoOW4gMOnBpIbWf3ZsvxfxDriZ FvnpSbyJIpHrHE3k/tM8sgufybZdsInbcXgC9C9gRFkVkTxX6VXOd7npazenpjBd9W0G 1b84KpEzjYsoTv6iB4W5Hw+KbAlQTEYQwHpyZuGme6fySCqh1E7keik7pnzHvEnGhc1F lc9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=H7T104lkfLZPMGxIBUsm1hoLba7o7e864GWa57cAV/Y=; b=7cBbDZTsxUrkyvbWU//JuZG6qDD2BCpma1fmgTnhDQnGFYD7gccEJjoiW5m9q31oE8 FjXam2qeh2yhNKdhQz3TDNc2rhEwgEjMtP30GCtCskUbvBHzqBqLyxXKrHKVUYFEtZWm 9sosyR39zDax3YJAFtYmG23feavrNuCjSGEF7K24TClFiuM7j6ptrdaq7QXTg6uAsaPs Y7tAdqc6ek/5lVhuhw9sjQIKSYQpXAGqn4k4trS4F/o+U6QpQCpLW6njyxvwnVyshcma ybMsBY+1V6Ddnp2GqWrqSKb9m29ecim75h+ueAUWb9W+2UY7NK/+82pK7uQglWLE2N1E R6xQ== X-Gm-Message-State: AOAM530MmTReJo6cnSyUj9mrjMIeHnnbIvBIwFSvzGPPZ+MqZf3ujjFa GdxsMDPNg7XQAgVtJbNc+NxrNw== X-Google-Smtp-Source: ABdhPJy/9UgkTFGEUY085nCQBqw+HVOEPD7eaAkbIvZ4PMj+/CulRzgev3pKsRc2wCSBMmvQEwgbuQ== X-Received: by 2002:adf:f947:: with SMTP id q7mr31423192wrr.260.1637248829175; Thu, 18 Nov 2021 07:20:29 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id p12sm147367wro.33.2021.11.18.07.20.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 18 Nov 2021 07:20:28 -0800 (PST) From: David Plowman To: laurent.pinchart@ideasonboard.com, kieran.bingham@ideasonboard.com, hverkuil-cisco@xs4all.nl, tfiga@google.com, jacopo@jmondi.org, naush@raspberrypi.com, libcamera-devel@lists.libcamera.org Date: Thu, 18 Nov 2021 15:19:31 +0000 Message-Id: <20211118151933.15627-6-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211118151933.15627-1-david.plowman@raspberrypi.com> References: <20211118151933.15627-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v6 5/7] libcamera: Support passing ColorSpaces to V4L2 subdevices X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The ColorSpace from the StreamConfiguration is now handled appropriately in the V4L2Subdevice. Signed-off-by: David Plowman --- include/libcamera/internal/v4l2_subdevice.h | 2 ++ src/libcamera/camera_sensor.cpp | 1 + src/libcamera/v4l2_subdevice.cpp | 35 ++++++++++++++++++++- 3 files changed, 37 insertions(+), 1 deletion(-) diff --git a/include/libcamera/internal/v4l2_subdevice.h b/include/libcamera/internal/v4l2_subdevice.h index 97b89fb9..a6a0a870 100644 --- a/include/libcamera/internal/v4l2_subdevice.h +++ b/include/libcamera/internal/v4l2_subdevice.h @@ -14,6 +14,7 @@ #include #include +#include #include #include "libcamera/internal/formats.h" @@ -27,6 +28,7 @@ class MediaDevice; struct V4L2SubdeviceFormat { uint32_t mbus_code; Size size; + std::optional colorSpace; const std::string toString() const; uint8_t bitsPerPixel() const; diff --git a/src/libcamera/camera_sensor.cpp b/src/libcamera/camera_sensor.cpp index 9fdb8c09..6fcd1c1d 100644 --- a/src/libcamera/camera_sensor.cpp +++ b/src/libcamera/camera_sensor.cpp @@ -613,6 +613,7 @@ V4L2SubdeviceFormat CameraSensor::getFormat(const std::vector &mbu V4L2SubdeviceFormat format{ .mbus_code = bestCode, .size = *bestSize, + .colorSpace = ColorSpace::Raw, }; return format; diff --git a/src/libcamera/v4l2_subdevice.cpp b/src/libcamera/v4l2_subdevice.cpp index 023e2328..b60ab69a 100644 --- a/src/libcamera/v4l2_subdevice.cpp +++ b/src/libcamera/v4l2_subdevice.cpp @@ -168,6 +168,16 @@ const std::map formatInfoMap = { * \brief The image size in pixels */ +/** + * \var V4L2SubdeviceFormat::colorSpace + * \brief The color space of the pixels + * + * The color space of the image. When setting the format this may be + * unset, in which case the driver gets to use its default color space. + * After being set, this value should contain the color space that the + * was actually used. + */ + /** * \brief Assemble and return a string describing the format * \return A string describing the V4L2SubdeviceFormat @@ -400,6 +410,16 @@ int V4L2Subdevice::getFormat(unsigned int pad, V4L2SubdeviceFormat *format, format->size.height = subdevFmt.format.height; format->mbus_code = subdevFmt.format.code; + format->colorSpace = toColorSpace(subdevFmt.format); + /* + * This warning can be ignored on metadata pads. These are normally + * pads other than zero. + * \todo find a way to detect metadata pads and ignore them + */ + if (!format->colorSpace) + LOG(V4L2, Warning) + << "Retrieved unrecognised color space on pad " << pad; + return 0; } @@ -418,6 +438,9 @@ int V4L2Subdevice::getFormat(unsigned int pad, V4L2SubdeviceFormat *format, int V4L2Subdevice::setFormat(unsigned int pad, V4L2SubdeviceFormat *format, Whence whence) { + if (!format->colorSpace) + LOG(V4L2, Error) << "Setting an undefined color space"; + struct v4l2_subdev_format subdevFmt = {}; subdevFmt.which = whence == ActiveFormat ? V4L2_SUBDEV_FORMAT_ACTIVE : V4L2_SUBDEV_FORMAT_TRY; @@ -427,7 +450,13 @@ int V4L2Subdevice::setFormat(unsigned int pad, V4L2SubdeviceFormat *format, subdevFmt.format.code = format->mbus_code; subdevFmt.format.field = V4L2_FIELD_NONE; - int ret = ioctl(VIDIOC_SUBDEV_S_FMT, &subdevFmt); + int ret = fromColorSpace(format->colorSpace, subdevFmt.format); + if (ret < 0) + LOG(V4L2, Warning) + << "Setting color space unrecognised by V4L2: " + << ColorSpace::toString(format->colorSpace); + + ret = ioctl(VIDIOC_SUBDEV_S_FMT, &subdevFmt); if (ret) { LOG(V4L2, Error) << "Unable to set format on pad " << pad @@ -439,6 +468,10 @@ int V4L2Subdevice::setFormat(unsigned int pad, V4L2SubdeviceFormat *format, format->size.height = subdevFmt.format.height; format->mbus_code = subdevFmt.format.code; + format->colorSpace = toColorSpace(subdevFmt.format); + if (!format->colorSpace) + LOG(V4L2, Warning) << "Unrecognised color space has been set"; + return 0; } From patchwork Thu Nov 18 15:19:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 14631 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3CBB5BF415 for ; Thu, 18 Nov 2021 15:20:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E2E35603C9; Thu, 18 Nov 2021 16:20:36 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="PmhMNfpp"; dkim-atps=neutral Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [IPv6:2a00:1450:4864:20::336]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6C2DC6038A for ; Thu, 18 Nov 2021 16:20:30 +0100 (CET) Received: by mail-wm1-x336.google.com with SMTP id b184-20020a1c1bc1000000b0033140bf8dd5so5073952wmb.5 for ; Thu, 18 Nov 2021 07:20:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=e9vLi2M3N7esEcpYN4gESMpiB1Tm8dGAXoq3VzNR2x4=; b=PmhMNfppy2HSu2E5gW+vG7BLArB4EdsouHEIIjqw3/3uhAYnQBHhsCHNTzfO19VFQM 6auh3Zt93yIh+6xHIM3cOgKRvDhB64s1p3fTF/N8+0wKjKjQvk7JvO5S4b0pJ96V7iJS K00v5RNjnn2ao2sQhi4TS0Oyzu+YbnC9R/YGJEiVz+PpQOOX4PNtM7+cKvKpqqCSsYcH GY+2ayBhvqhY7Dc6Pl3aS9QMvYHC3hU6l30M8GpNWBXsJnM8KmaTqgTzMWkil8BzpE+u Jun9d+vVc6k++6zIiyTrGCLcFGlCJS0NpqRiXveEnIeaHpYJ9URDCCv2wnH6F5dkgKLG f9Mg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=e9vLi2M3N7esEcpYN4gESMpiB1Tm8dGAXoq3VzNR2x4=; b=rMSPljn639hlGD2OKMWEf63qANki2OnRh0KdCJ73LtrQMetbSX3is7KPTxAzJhSUv0 hECU8WXmjOpL3ZsV3OiGYOj3DLVpEY6UOpGf+zEYZab1YvP9K+PZWzt9hOY42xpGAEJC A4KFGcjmdxFbqX1f09WuejUkH00a/tPGo1VWB2wQMW9y3Ql70OPlCF2p7ejnLu7SfsKm J2Vn6fL/25T8tuoeskFNxGg3eajm0pDzJmrO1N6p+ExuJ+SAywgZTokTQ7cxBO8lUN6O x4eIlWGVJGPd6to1Bx63ZfQ9WCi95cxfpd6YXgBACUmasTXKc1PvEDwejMsLfok7h9aJ zjoA== X-Gm-Message-State: AOAM530OTKQqpHsG0ru9Cq7N2VGkUeJNgthn2SR1iCCJ7NEy511cltmL TTtrFVpHgZbPb9lva76b0X228g== X-Google-Smtp-Source: ABdhPJzRbAaHY/mJvm64joUeBIgYmzNLtHqd1my/s8wJO/SBKh0wG/CU0oaHySU7C8LET8LHShAZ4g== X-Received: by 2002:a1c:8015:: with SMTP id b21mr10641552wmd.161.1637248830116; Thu, 18 Nov 2021 07:20:30 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id p12sm147367wro.33.2021.11.18.07.20.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 18 Nov 2021 07:20:29 -0800 (PST) From: David Plowman To: laurent.pinchart@ideasonboard.com, kieran.bingham@ideasonboard.com, hverkuil-cisco@xs4all.nl, tfiga@google.com, jacopo@jmondi.org, naush@raspberrypi.com, libcamera-devel@lists.libcamera.org Date: Thu, 18 Nov 2021 15:19:32 +0000 Message-Id: <20211118151933.15627-7-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211118151933.15627-1-david.plowman@raspberrypi.com> References: <20211118151933.15627-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v6 6/7] libcamera: Add validateColorSpaces to CameraConfiguration class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This method forces raw streams to have the "raw" color space, and also optionally makes all output streams to share the same color space as some platforms may require this. Signed-off-by: David Plowman --- include/libcamera/camera.h | 2 ++ src/libcamera/camera.cpp | 51 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 53 insertions(+) diff --git a/include/libcamera/camera.h b/include/libcamera/camera.h index 601ee46e..fdab4410 100644 --- a/include/libcamera/camera.h +++ b/include/libcamera/camera.h @@ -69,6 +69,8 @@ public: protected: CameraConfiguration(); + Status validateColorSpaces(bool sharedColorSpace); + std::vector config_; }; diff --git a/src/libcamera/camera.cpp b/src/libcamera/camera.cpp index 400a7cf0..c1541685 100644 --- a/src/libcamera/camera.cpp +++ b/src/libcamera/camera.cpp @@ -20,6 +20,7 @@ #include "libcamera/internal/camera.h" #include "libcamera/internal/camera_controls.h" +#include "libcamera/internal/formats.h" #include "libcamera/internal/pipeline_handler.h" /** @@ -314,6 +315,56 @@ std::size_t CameraConfiguration::size() const return config_.size(); } +static bool isRaw(const PixelFormat &pixFmt) +{ + const PixelFormatInfo &info = PixelFormatInfo::info(pixFmt); + return info.isValid() && + info.colourEncoding == PixelFormatInfo::ColourEncodingRAW; +} + +CameraConfiguration::Status CameraConfiguration::validateColorSpaces(bool sharedColorSpace) +{ + Status status = Valid; + + /* Find the largest non-raw stream (if any). */ + int index = -1; + for (unsigned int i = 0; i < config_.size(); i++) { + const StreamConfiguration &cfg = config_[i]; + if (!isRaw(cfg.pixelFormat) && (index < 0 || cfg.size > config_[i].size)) + index = i; + } + + /* + * Here we force raw streams to the correct color space. We handle + * the case where all output streams are to share a color space, + * which will match the color space of the largest (non-raw) stream. + */ + for (auto &cfg : config_) { + std::optional initialColorSpace = cfg.colorSpace; + + if (isRaw(cfg.pixelFormat)) + cfg.colorSpace = ColorSpace::Raw; + else if (sharedColorSpace) { + /* + * When all outputs share a color space, use the biggest + * output if that was set. If that was blank, but this + * stream's is not, then use this stream's color space for + * largest stream. + * (index must be != -1 if there are any non-raw streams.) + */ + if (config_[index].colorSpace) + cfg.colorSpace = config_[index].colorSpace; + else if (cfg.colorSpace) + config_[index].colorSpace = cfg.colorSpace; + } + + if (cfg.colorSpace != initialColorSpace) + status = Adjusted; + } + + return status; +} + /** * \var CameraConfiguration::transform * \brief User-specified transform to be applied to the image From patchwork Thu Nov 18 15:19:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 14632 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E16D2C324F for ; Thu, 18 Nov 2021 15:20:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7CB7C60231; Thu, 18 Nov 2021 16:20:37 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="Y1n1m+94"; dkim-atps=neutral Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3A73960231 for ; Thu, 18 Nov 2021 16:20:31 +0100 (CET) Received: by mail-wm1-x329.google.com with SMTP id c71-20020a1c9a4a000000b0032cdcc8cbafso5084066wme.3 for ; Thu, 18 Nov 2021 07:20:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=zJKMksua6c6xIYVePhIiVLpfcsB/8lKEFhsflNfPxDc=; b=Y1n1m+943lV/X3THWku0FecTuLVN3nsOxp/ssfkLmmDVtY0r2roIj4JrQf5cBVjfo8 8WLO//2NYjm1rdSgoaOGyIaxN6Vog5OtHzmu/vy2histdt5oaOjfbfcwhLkOzDC4Ndq9 xJqJQRAs7m1ioSayUwnLpIjyRELwbvq0EVvTzBZszC0puVV9jBNKgIrRyLqoQ5dksABe TKuU2Rx8feV5MK71J7TMaf2zblqtwMaGlXGWXz7gjXtNjP5Uvt9WGZrLJvCdSKr7ANpK XPeDtQZxhsIiHYp/oEq94gJnX3JL4tdi4N+4qdC8AyKH5QvtISw3qYOxxc0727aCoxzy yXZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zJKMksua6c6xIYVePhIiVLpfcsB/8lKEFhsflNfPxDc=; b=RiJxaJjSMLRXrxJIWrbFKIf3c/a7IGEWsB+n3dRqm1WYPU/TN1x2GAuVB3PY8BLZZf gW7GqNtT+fetG6DuC3Eso0Y/KDkWjcHMaPLz76dooAoVz0JRcr5HoQUYCW0TpLlqVeDm mlIal8FmIxKLUEp2pH1ksT23aOsValb4JlTanp054WFb+RJGhRLqb//zx0AidFqNb3Xc ZOXnk7E0YbDHZH8dq0kPRa1QsVkiNJqmW/eHzt1YSXIhnp4I3gToJLz7EtotpKZ3d8hO nbrhaMfhDkWScfIKkJ5I+ekdMtwRy9FMvAkWL9ohdmH3AgUsjOtzRb4XGuuuamTtLiJc Dsqw== X-Gm-Message-State: AOAM533EVncVpmvqh9G9MMwSdb1gclqbPByUgKyM3mqLnR2rNhY0cVZ5 P9iRpgZrlHL/6SfW3Thb0Zp4iw== X-Google-Smtp-Source: ABdhPJwl+3cpG2F6EqvyQtYon45NsKTdtVczGQI+9OHBrQdQdrXKLRdNhn4NtS+0yuB4CQ30rpZfCA== X-Received: by 2002:a05:600c:224a:: with SMTP id a10mr10815542wmm.154.1637248830874; Thu, 18 Nov 2021 07:20:30 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id p12sm147367wro.33.2021.11.18.07.20.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 18 Nov 2021 07:20:30 -0800 (PST) From: David Plowman To: laurent.pinchart@ideasonboard.com, kieran.bingham@ideasonboard.com, hverkuil-cisco@xs4all.nl, tfiga@google.com, jacopo@jmondi.org, naush@raspberrypi.com, libcamera-devel@lists.libcamera.org Date: Thu, 18 Nov 2021 15:19:33 +0000 Message-Id: <20211118151933.15627-8-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211118151933.15627-1-david.plowman@raspberrypi.com> References: <20211118151933.15627-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v6 7/7] libcamera: pipeline: raspberrypi: Support color spaces X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The Raspberry Pi pipeline handler now sets color spaces correctly. In generateConfiguration() it sets them to reasonable default values based on the stream role. validate() now calls validateColorSpaces() to ensure that the requested color spaces are sensible, before proceeding to check what the hardware can deliver. Signed-off-by: David Plowman --- .../pipeline/raspberrypi/raspberrypi.cpp | 42 +++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 5e1f2273..5f09ba59 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -93,6 +93,7 @@ V4L2DeviceFormat toV4L2DeviceFormat(const V4L2SubdeviceFormat &format, deviceFormat.fourcc = V4L2PixelFormat::fromPixelFormat(pix); deviceFormat.size = format.size; + deviceFormat.colorSpace = format.colorSpace; return deviceFormat; } @@ -126,6 +127,7 @@ V4L2SubdeviceFormat findBestFormat(const SensorFormats &formatsMap, const Size & { double bestScore = std::numeric_limits::max(), score; V4L2SubdeviceFormat bestFormat; + bestFormat.colorSpace = ColorSpace::Raw; #define PENALTY_AR 1500.0 #define PENALTY_8BIT 2000.0 @@ -332,6 +334,8 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() if (config_.empty()) return Invalid; + status = validateColorSpaces(true); + /* * What if the platform has a non-90 degree rotation? We can't even * "adjust" the configuration and carry on. Alternatively, raising an @@ -494,10 +498,21 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() V4L2DeviceFormat format; format.fourcc = V4L2PixelFormat::fromPixelFormat(cfg.pixelFormat); format.size = cfg.size; + format.colorSpace = cfg.colorSpace; + LOG(RPI, Debug) + << "Try color space " << ColorSpace::toString(cfg.colorSpace); int ret = dev->tryFormat(&format); if (ret) return Invalid; + if (!format.colorSpace || cfg.colorSpace != format.colorSpace) { + status = Adjusted; + LOG(RPI, Warning) + << "Color space changed from " + << ColorSpace::toString(cfg.colorSpace) << " to " + << ColorSpace::toString(format.colorSpace); + } + cfg.colorSpace = format.colorSpace; cfg.stride = format.planes[0].bpl; cfg.frameSize = format.planes[0].size; @@ -522,6 +537,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, PixelFormat pixelFormat; V4L2VideoDevice::Formats fmts; Size size; + std::optional colorSpace; if (roles.empty()) return config; @@ -536,6 +552,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, pixelFormat = mbusCodeToPixelFormat(sensorFormat.mbus_code, BayerFormat::Packing::CSI2); ASSERT(pixelFormat.isValid()); + colorSpace = ColorSpace::Raw; bufferCount = 2; rawCount++; break; @@ -543,6 +560,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, case StreamRole::StillCapture: fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::NV12; + colorSpace = ColorSpace::Jpeg; /* Return the largest sensor resolution. */ size = data->sensor_->resolution(); bufferCount = 1; @@ -560,6 +578,8 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, */ fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::YUV420; + /* This will be reasonable for many applications. */ + colorSpace = ColorSpace::Rec709; size = { 1920, 1080 }; bufferCount = 4; outCount++; @@ -568,6 +588,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, case StreamRole::Viewfinder: fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::ARGB8888; + colorSpace = ColorSpace::Jpeg; size = { 800, 600 }; bufferCount = 4; outCount++; @@ -599,6 +620,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, StreamConfiguration cfg(formats); cfg.size = size; cfg.pixelFormat = pixelFormat; + cfg.colorSpace = colorSpace; cfg.bufferCount = bufferCount; config->addConfiguration(cfg); } @@ -703,6 +725,7 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) V4L2PixelFormat fourcc = V4L2PixelFormat::fromPixelFormat(cfg.pixelFormat); format.size = cfg.size; format.fourcc = fourcc; + format.colorSpace = cfg.colorSpace; LOG(RPI, Debug) << "Setting " << stream->name() << " to " << format.toString(); @@ -718,6 +741,23 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) return -EINVAL; } + if (!format.colorSpace || cfg.colorSpace != format.colorSpace) { + /* + * We should have been through validate() before so this + * shouldn't be possible, but we mustn't sweep color space + * problems under the carpet. + */ + LOG(RPI, Warning) + << "Unexpected color space (" + << ColorSpace::toString(cfg.colorSpace) << " to " + << ColorSpace::toString(format.colorSpace) << ") in stream " + << stream->name(); + cfg.colorSpace = format.colorSpace; + } + LOG(RPI, Debug) + << "Stream " << stream->name() << " has color space " + << ColorSpace::toString(cfg.colorSpace); + cfg.setStream(stream); stream->setExternal(true); @@ -742,6 +782,8 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) format = {}; format.size = maxSize; format.fourcc = V4L2PixelFormat::fromPixelFormat(formats::YUV420); + /* No one asked for output, so the color space doesn't matter. */ + format.colorSpace = ColorSpace::Jpeg; ret = data->isp_[Isp::Output0].dev()->setFormat(&format); if (ret) { LOG(RPI, Error)