From patchwork Thu Dec 9 10:12:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 15091 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CC730C324B for ; Thu, 9 Dec 2021 10:13:34 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1ED8460889; Thu, 9 Dec 2021 11:13:33 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="SgOk1daF"; dkim-atps=neutral Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 057E460224 for ; Thu, 9 Dec 2021 11:13:32 +0100 (CET) Received: by mail-wr1-x42f.google.com with SMTP id t9so8757848wrx.7 for ; Thu, 09 Dec 2021 02:13:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=rXCAAkH7rZsll+xp0ceTS/onnCa0D67p71c9GYeoz3c=; b=SgOk1daF+TXVT0fOmONNDf3jB+Ao1v1xGq2dOpi5LbOfbYZkpegbuy3Zt8Olsv0U1m qAftYKFvGbf+3XodnwHShY5OqfBjGPcY+Py9c4Tmep4msx7W0TdoM9Ty5nMxSZIr3jiR H71uTDddQVCcQDd6rQTIu53QNSm1Hw6d8J59VNKJlVFRpqd+77C/SPWEgou5GX3ImXxl ZqgitTG4SShculqs1b08yCVTC6dn9mRskL7k81h53dSPGqU9QjKHzX3phOjnKU4UVApN pOqJX1u2UmKRqQ7V2rBxqcPTEW/ENnIiI10nZSn85XXc5Isk+CIje1EXBR8M7hcpitC4 1Blg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=rXCAAkH7rZsll+xp0ceTS/onnCa0D67p71c9GYeoz3c=; b=ddQ8IJcsZYtT04Zubzk+Hhhrzj4/ThkwAR3AT6u+pU48LToIq3oMKMi6eoRsVbaALk QuDc1t/1+V3//0bsCbxcVtS9id2QkX5kY9RgepM6HBaorxf5k8LCEpz2yxZcjcnpQ6Yg vF4YO1L7JdDRUvj8ii+FJvxjNWC1vd8gZyyY5cOJCt1t+jWsr3lMWKtu6ZaNmlJclfbI gSoEd+wDuNfRv3Mt/2Vb37n2ktfFGVmVdsHMrFvQcCF0p/ZjTAzkMp1kaToY8JvAgnKy 54JGg6mdjxgjk3VF1l3yJ+qrI/A8bXQ7tyD+Vc64okgERXWp9yrQPBDM2QZb6T+dDAef rCYg== X-Gm-Message-State: AOAM531/nezR2MBBvjEULHsnbN/aZXKM0yrjdoo3u6lEYkN/8yAp87ML WHKGf1EBkZUhCbOTlsyJ3gtFTdpVxgYeKRAZ X-Google-Smtp-Source: ABdhPJwJXQUGpWljdQlaDh/zm3ZPp/+7QFrLynTXHVPITCqqPy6ETOI12d1JOELYwhN1ihEUT4j5Kg== X-Received: by 2002:adf:82d3:: with SMTP id 77mr5279150wrc.377.1639044811159; Thu, 09 Dec 2021 02:13:31 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:e4a2:3070:eea4:e434]) by smtp.gmail.com with ESMTPSA id j8sm5079449wrh.16.2021.12.09.02.13.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Dec 2021 02:13:30 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org, Laurent Pinchart , Kieran Bingham , Hans Verkuil , Tomasz Figa , Jacopo Mondi , Naushir Patuck Date: Thu, 9 Dec 2021 10:12:38 +0000 Message-Id: <20211209101245.6187-2-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211209101245.6187-1-david.plowman@raspberrypi.com> References: <20211209101245.6187-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v10 1/8] libcamera: Add ColorSpace class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This class represents a color space by defining its color primaries, the transfer (gamma) function it uses, the YCbCr encoding and whether the output is full or limited range. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck Reviewed-by: Umang Jain Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- include/libcamera/color_space.h | 70 +++++++ include/libcamera/meson.build | 1 + src/libcamera/color_space.cpp | 318 ++++++++++++++++++++++++++++++++ src/libcamera/meson.build | 1 + 4 files changed, 390 insertions(+) create mode 100644 include/libcamera/color_space.h create mode 100644 src/libcamera/color_space.cpp diff --git a/include/libcamera/color_space.h b/include/libcamera/color_space.h new file mode 100644 index 00000000..086c56c1 --- /dev/null +++ b/include/libcamera/color_space.h @@ -0,0 +1,70 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Raspberry Pi (Trading) Limited + * + * color_space.h - color space definitions + */ + +#pragma once + +#include +#include + +namespace libcamera { + +class ColorSpace +{ +public: + enum class Primaries { + Raw, + Smpte170m, + Rec709, + Rec2020, + }; + + enum class TransferFunction { + Linear, + Srgb, + Rec709, + }; + + enum class YcbcrEncoding { + None, + Rec601, + Rec709, + Rec2020, + }; + + enum class Range { + Full, + Limited, + }; + + constexpr ColorSpace(Primaries p, TransferFunction t, YcbcrEncoding e, Range r) + : primaries(p), transferFunction(t), ycbcrEncoding(e), range(r) + { + } + + static const ColorSpace Raw; + static const ColorSpace Jpeg; + static const ColorSpace Srgb; + static const ColorSpace Smpte170m; + static const ColorSpace Rec709; + static const ColorSpace Rec2020; + + Primaries primaries; + TransferFunction transferFunction; + YcbcrEncoding ycbcrEncoding; + Range range; + + std::string toString() const; + static std::string toString(const std::optional &colorSpace); +}; + +bool operator==(const ColorSpace &lhs, const ColorSpace &rhs); +static inline bool operator!=(const ColorSpace &lhs, const ColorSpace &rhs) +{ + return !(lhs == rhs); +} + +} /* namespace libcamera */ diff --git a/include/libcamera/meson.build b/include/libcamera/meson.build index 5f42977c..fd767b11 100644 --- a/include/libcamera/meson.build +++ b/include/libcamera/meson.build @@ -5,6 +5,7 @@ libcamera_include_dir = 'libcamera' / 'libcamera' libcamera_public_headers = files([ 'camera.h', 'camera_manager.h', + 'color_space.h', 'controls.h', 'framebuffer.h', 'framebuffer_allocator.h', diff --git a/src/libcamera/color_space.cpp b/src/libcamera/color_space.cpp new file mode 100644 index 00000000..6c587068 --- /dev/null +++ b/src/libcamera/color_space.cpp @@ -0,0 +1,318 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Raspberry Pi (Trading) Limited + * + * color_space.cpp - color spaces. + */ + +#include + +#include +#include +#include +#include +#include + +/** + * \file color_space.h + * \brief Class and enums to represent color spaces + */ + +namespace libcamera { + +/** + * \class ColorSpace + * \brief Class to describe a color space + * + * The ColorSpace class defines the color primaries, the Y'CbCr encoding, + * the transfer function associated with the color space, and the range + * (sometimes also referred to as the quantisation) of the color space. + * + * Certain combinations of these fields form well-known standard color + * spaces such as "JPEG" or "REC709". + * + * In the strictest sense a "color space" formally only refers to the + * color primaries and white point. Here, however, the ColorSpace class + * adopts the common broader usage that includes the transfer function, + * Y'CbCr encoding method and quantisation. + * + * For more information on the specific color spaces described here, please + * see: + * + * - sRGB + * - JPEG + * - SMPTE 170M + * - Rec.709 + * - Rec.2020 + */ + +/** + * \enum ColorSpace::Primaries + * \brief The color primaries for this color space + * + * \var ColorSpace::Primaries::Raw + * \brief These are raw colors directly from a sensor, the primaries are + * unspecified + * + * \var ColorSpace::Primaries::Smpte170m + * \brief SMPTE 170M color primaries + * + * \var ColorSpace::Primaries::Rec709 + * \brief Rec.709 color primaries + * + * \var ColorSpace::Primaries::Rec2020 + * \brief Rec.2020 color primaries + */ + +/** + * \enum ColorSpace::TransferFunction + * \brief The transfer function used for this color space + * + * \var ColorSpace::TransferFunction::Linear + * \brief This color space uses a linear (identity) transfer function + * + * \var ColorSpace::TransferFunction::Srgb + * \brief sRGB transfer function + * + * \var ColorSpace::TransferFunction::Rec709 + * \brief Rec.709 transfer function + */ + +/** + * \enum ColorSpace::YcbcrEncoding + * \brief The Y'CbCr encoding + * + * \var ColorSpace::YcbcrEncoding::None + * \brief There is no defined Y'CbCr encoding + * + * \var ColorSpace::YcbcrEncoding::Rec601 + * \brief Rec.601 Y'CbCr encoding + * + * \var ColorSpace::YcbcrEncoding::Rec709 + * \brief Rec.709 Y'CbCr encoding + * + * \var ColorSpace::YcbcrEncoding::Rec2020 + * \brief Rec.2020 Y'CbCr encoding + */ + +/** + * \enum ColorSpace::Range + * \brief The range (sometimes "quantisation") for this color space + * + * \var ColorSpace::Range::Full + * \brief This color space uses full range pixel values + * + * \var ColorSpace::Range::Limited + * \brief This color space uses limited range pixel values, being + * 16 to 235 for Y' and 16 to 240 for Cb and Cr (8 bits per sample) + * or 64 to 940 for Y' and 16 to 960 for Cb and Cr (10 bits) + */ + +/** + * \fn ColorSpace::ColorSpace(Primaries p, TransferFunction t, Encoding e, Range r) + * \brief Construct a ColorSpace from explicit values + * \param[in] p The color primaries + * \param[in] t The transfer function for the color space + * \param[in] e The Y'CbCr encoding + * \param[in] r The range of the pixel values in this color space + */ + +/** + * \brief Assemble and return a readable string representation of the + * ColorSpace + * + * If the color space matches a standard ColorSpace (such as ColorSpace::Jpeg) + * then the short name of the color space ("JPEG") is returned. Otherwise + * the four constituent parts of the ColorSpace are assembled into a longer + * string. + * + * \return A string describing the ColorSpace + */ +std::string ColorSpace::toString() const +{ + /* Print out a brief name only for standard color spaces. */ + + static const std::array, 6> colorSpaceNames = { + { + { ColorSpace::Raw, "RAW" }, + { ColorSpace::Jpeg, "JPEG" }, + { ColorSpace::Srgb, "sRGB" }, + { ColorSpace::Smpte170m, "SMPTE170M" }, + { ColorSpace::Rec709, "Rec709" }, + { ColorSpace::Rec2020, "Rec2020" }, + } + }; + auto it = std::find_if(colorSpaceNames.begin(), colorSpaceNames.end(), + [this](const auto &item) { + return *this == item.first; + }); + if (it != colorSpaceNames.end()) + return std::string(it->second); + + /* Assemble a name made of the constituent fields. */ + + static const std::map primariesNames = { + { Primaries::Raw, "Raw" }, + { Primaries::Smpte170m, "Smpte170m" }, + { Primaries::Rec709, "Rec709" }, + { Primaries::Rec2020, "Rec2020" }, + }; + static const std::map transferNames = { + { TransferFunction::Linear, "Linear" }, + { TransferFunction::Srgb, "Srgb" }, + { TransferFunction::Rec709, "Rec709" }, + }; + static const std::map encodingNames = { + { YcbcrEncoding::None, "None" }, + { YcbcrEncoding::Rec601, "Rec601" }, + { YcbcrEncoding::Rec709, "Rec709" }, + { YcbcrEncoding::Rec2020, "Rec2020" }, + }; + static const std::map rangeNames = { + { Range::Full, "Full" }, + { Range::Limited, "Limited" }, + }; + + auto itPrimaries = primariesNames.find(primaries); + std::string primariesName = + itPrimaries == primariesNames.end() ? "Invalid" : itPrimaries->second; + + auto itTransfer = transferNames.find(transferFunction); + std::string transferName = + itTransfer == transferNames.end() ? "Invalid" : itTransfer->second; + + auto itEncoding = encodingNames.find(ycbcrEncoding); + std::string encodingName = + itEncoding == encodingNames.end() ? "Invalid" : itEncoding->second; + + auto itRange = rangeNames.find(range); + std::string rangeName = + itRange == rangeNames.end() ? "Invalid" : itRange->second; + + std::stringstream ss; + ss << primariesName << "/" << transferName << "/" << encodingName << "/" << rangeName; + + return ss.str(); +} + +/** + * \brief Assemble and return a readable string representation of an + * optional ColorSpace + * + * This is a convenience helper to easily obtain a string representation + * for a ColorSpace in parts of the libcamera API where it is stored in a + * std::optional<>. If the ColorSpace is set, this function returns + * \a colorSpace.toString(), otherwise it returns "Unset". + * + * \return A string describing the optional ColorSpace + */ +std::string ColorSpace::toString(const std::optional &colorSpace) +{ + if (!colorSpace) + return "Unset"; + + return colorSpace->toString(); +} + +/** + * \var ColorSpace::primaries + * \brief The color primaries of this color space + */ + +/** + * \var ColorSpace::ycbcrEncoding + * \brief The Y'CbCr encoding used by this color space + */ + +/** + * \var ColorSpace::transferFunction + * \brief The transfer function used by this color space + */ + +/** + * \var ColorSpace::range + * \brief The pixel range used with by color space + */ + +/** + * \brief A constant representing a raw color space (from a sensor) + */ +const ColorSpace ColorSpace::Raw = { + Primaries::Raw, + TransferFunction::Linear, + YcbcrEncoding::None, + Range::Full +}; + +/** + * \brief A constant representing the JPEG color space used for + * encoding JPEG images + */ +const ColorSpace ColorSpace::Jpeg = { + Primaries::Rec709, + TransferFunction::Srgb, + YcbcrEncoding::Rec601, + Range::Full +}; + +/** + * \brief A constant representing the sRGB color space. + * This is identical to the JPEG color space except that the Y'CbCr + * range is limited rather than full. + */ +const ColorSpace ColorSpace::Srgb = { + Primaries::Rec709, + TransferFunction::Srgb, + YcbcrEncoding::Rec601, + Range::Limited +}; + +/** + * \brief A constant representing the SMPTE170M color space + */ +const ColorSpace ColorSpace::Smpte170m = { + Primaries::Smpte170m, + TransferFunction::Rec709, + YcbcrEncoding::Rec601, + Range::Limited +}; + +/** + * \brief A constant representing the Rec.709 color space + */ +const ColorSpace ColorSpace::Rec709 = { + Primaries::Rec709, + TransferFunction::Rec709, + YcbcrEncoding::Rec709, + Range::Limited +}; + +/** + * \brief A constant representing the Rec.2020 color space + */ +const ColorSpace ColorSpace::Rec2020 = { + Primaries::Rec2020, + TransferFunction::Rec709, + YcbcrEncoding::Rec2020, + Range::Limited +}; + +/** + * \brief Compare color spaces for equality + * \return True if the two color spaces are identical, false otherwise + */ +bool operator==(const ColorSpace &lhs, const ColorSpace &rhs) +{ + return lhs.primaries == rhs.primaries && + lhs.transferFunction == rhs.transferFunction && + lhs.ycbcrEncoding == rhs.ycbcrEncoding && + lhs.range == rhs.range; +} + +/** + * \fn bool operator!=(const ColorSpace &lhs, const ColorSpace &rhs) + * \brief Compare color spaces for inequality + * \return True if the two color spaces are not identical, false otherwise + */ + +} /* namespace libcamera */ diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build index 2e54cc04..4045e24e 100644 --- a/src/libcamera/meson.build +++ b/src/libcamera/meson.build @@ -9,6 +9,7 @@ libcamera_sources = files([ 'camera_manager.cpp', 'camera_sensor.cpp', 'camera_sensor_properties.cpp', + 'color_space.cpp', 'controls.cpp', 'control_serializer.cpp', 'control_validator.cpp', From patchwork Thu Dec 9 10:12:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 15092 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 02DC8BF415 for ; Thu, 9 Dec 2021 10:13:35 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A925E6088C; Thu, 9 Dec 2021 11:13:33 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="UZyIbIxa"; dkim-atps=neutral Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 330AD6086A for ; Thu, 9 Dec 2021 11:13:32 +0100 (CET) Received: by mail-wr1-x430.google.com with SMTP id j3so8810099wrp.1 for ; Thu, 09 Dec 2021 02:13:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=+sH0KfN2WD6NxZb8aJTKXBPu6mnk13ViaZCwPQX529M=; b=UZyIbIxa/+2Dl/LyzGV2y3tn66s32bOqhU8HsrVEm+6OZTVVfiOYHgefAV8AYCGsOE zX8CqMlb/QKO/DH4L66suuBw0WMOsNF7jK1QeA6SbIlHgO9mQFa8BXPofE+4XDnQrcIW 4SWoB9a9TB62CVDzsXAZ5hM7s4dwVg3BCSWF1OyxprP08DHA9c0d7wwvLLd1so4lnI2M zitlN/Gx/o7VM2bR9xekUUZlqAXlwndjPuju6uARx463GDJSTXMD3or6ZtBjZYhDtWq9 zYP4EUQcNCe7sKdTe96BRuX4mpmePIzxYWIlwCmeTvNycUD7KfE9TDwQCHSutc4Rp29m rIkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+sH0KfN2WD6NxZb8aJTKXBPu6mnk13ViaZCwPQX529M=; b=MCxVPdIcrig0JDV16SVCyNcCEG3Q6ZdUCX52O0IkNZ7FnQGqM1aWrRAJ5TsZAGa9xA 8MPZwBFV+YPMODPJL6XsaF9svhv7jVprPQHIkKrZiF9qsOUHCsBPfqswhRutQg3x57KC takoIlpCfzoZUoXqJAtWrhv1MILJRs9Q+FphWRMLM0ceKO60rXyQnT2qFbzfFtcYAN0O bJ1NvPgBB82cWY23vP5EzX6/Ia7VIzRCkDoIWnM7ExlIGIZvYSLtEj8AXhbz6oCt6tQY aIdk2lA/q+710d+RIkgAb65YallBArYkVW68/rTJJiPPxCJ0ty6JZrDUCi6rLg1Cq9h+ vZ9Q== X-Gm-Message-State: AOAM531ZMwhBjYpL0uA0DeeYUqjB7CCqgTpKShSLxV/IAMYeHEVCUDEC tXex/ckCg9PLqkStv0HaPTAkdB+zZ0gSP+RK X-Google-Smtp-Source: ABdhPJwNyTbKFhOBDXWv2xdfEwi+PVoNDhQKd3N+QoaG+uuk+rK7mbVUirYxzznWrg1LqNHBwPWb3Q== X-Received: by 2002:adf:a389:: with SMTP id l9mr5316342wrb.121.1639044811691; Thu, 09 Dec 2021 02:13:31 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:e4a2:3070:eea4:e434]) by smtp.gmail.com with ESMTPSA id j8sm5079449wrh.16.2021.12.09.02.13.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Dec 2021 02:13:31 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org, Laurent Pinchart , Kieran Bingham , Hans Verkuil , Tomasz Figa , Jacopo Mondi , Naushir Patuck Date: Thu, 9 Dec 2021 10:12:39 +0000 Message-Id: <20211209101245.6187-3-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211209101245.6187-1-david.plowman@raspberrypi.com> References: <20211209101245.6187-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v10 2/8] libcamera: stream: Add ColorSpace fields to StreamConfiguration X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This is so that applications can choose appropriate color spaces which will then be passed down to the V4L2 devices. The ColorSpace field is actually optional. If it is not set you will get the camera's default color space. Signed-off-by: David Plowman Reviewed-by: Umang Jain Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- include/libcamera/stream.h | 3 +++ src/libcamera/stream.cpp | 20 ++++++++++++++++++++ 2 files changed, 23 insertions(+) diff --git a/include/libcamera/stream.h b/include/libcamera/stream.h index 41ec02b1..f0ae7e62 100644 --- a/include/libcamera/stream.h +++ b/include/libcamera/stream.h @@ -12,6 +12,7 @@ #include #include +#include #include #include #include @@ -47,6 +48,8 @@ struct StreamConfiguration { unsigned int bufferCount; + std::optional colorSpace; + Stream *stream() const { return stream_; } void setStream(Stream *stream) { stream_ = stream; } const StreamFormats &formats() const { return formats_; } diff --git a/src/libcamera/stream.cpp b/src/libcamera/stream.cpp index b421e17e..686e693b 100644 --- a/src/libcamera/stream.cpp +++ b/src/libcamera/stream.cpp @@ -329,6 +329,26 @@ StreamConfiguration::StreamConfiguration(const StreamFormats &formats) * \brief Requested number of buffers to allocate for the stream */ +/** + * \var StreamConfiguration::colorSpace + * \brief The ColorSpace for this stream + * + * This field allows a ColorSpace to be selected for this Stream. + * + * The field is optional and an application can choose to leave it unset. + * Platforms that support the use of color spaces may provide default + * values through the generateConfiguration() method. An application can + * override these when necessary. + * + * If a specific ColorSpace is requested but the Camera cannot deliver it, + * then the StreamConfiguration will be adjusted to a value that can be + * delivered. In this case the validate() method will indicate via its + * return value that the CameraConfiguration has been adjusted. + * + * Note that platforms will typically have different constraints on what + * color spaces can be supported and in what combinations. + */ + /** * \fn StreamConfiguration::stream() * \brief Retrieve the stream associated with the configuration From patchwork Thu Dec 9 10:12:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 15093 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 46E48C324B for ; Thu, 9 Dec 2021 10:13:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E884F608A7; Thu, 9 Dec 2021 11:13:36 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="lU4yRW6H"; dkim-atps=neutral Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 385E260224 for ; Thu, 9 Dec 2021 11:13:34 +0100 (CET) Received: by mail-wm1-x330.google.com with SMTP id g191-20020a1c9dc8000000b0032fbf912885so3654292wme.4 for ; Thu, 09 Dec 2021 02:13:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gsTvC8JzU5P/VtDow4q6d+N6EAyDjwIn0tC9oxF59o8=; b=lU4yRW6H3PFouNP+n9cQJWlxHfLSZjNyPmPPOtIFlYUla5qpuUUVqUdfW92ciilxVm yNd+ZofKdhHDW4RFBGRndSfCeybWyZGdReWSXbrfEPQKEkiZxDiZRaR6eCDH1kyPp3RJ KO8UtjELx1RfFNNmychJibZoylsrfFaZluMT3gRTSHSzgMB4pUMh8jLnNDDB8fz+gdML OnUuVqBxaPGxYia+ls4onhQPpT7HHGUQaFBZVfofw1HCLNy2/PGFkYtwz9DtU69XGkC4 BXZ69G9tulDbyvku2Dp/5BoLzkJ0jsAiTXww8UEvkZDmLybOoz+HEFsfXvcTI3IzcWe1 1cUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gsTvC8JzU5P/VtDow4q6d+N6EAyDjwIn0tC9oxF59o8=; b=m+t0Ew4+ImCP1aEAdm+FuyPi5/QGCuRaMoKuah4tke+v5jc++l4OXJSTntFwrLrv0q yzg0OSYs+6D4Fsmy6nGWs2xv+gHR9JUVU/eBBJapEKGCvCLZGJoeAN2/u4qYqc65MdQJ U/T8rzIf/avlapz2fgC0mzR+2u1mZRDS6SrwjAOCec+PNdxCQ1fpemg955AinITUpg3O AEb4VBIwEJoCf3yNGiBbIRBcGmkbJsGoo1MCHG6SACtBHyQIt5NSkM4/w5qY01cWqmEK yEL2+cTyz5Vi+OiUbnQk/+i9bc1WAuKqWaK2aFC42VApgp2zLu0cze1pPE1hFBBFoXHV t3Aw== X-Gm-Message-State: AOAM533epYE9gu/6jcQ9jvbtTLV8CkdWA7R1vyPEoLD9DO3YUylsrpZq U6BQ0TS4blt2tAZmp3BMWxl433240hjAkiZQ X-Google-Smtp-Source: ABdhPJyz/iq8Bu2q+SMB3mp6zNFzmbzOBj1UKOQgzgpfdgGvh9A/+H6URVaj+x2uArfwlRWHf/0V4A== X-Received: by 2002:a05:600c:3ba3:: with SMTP id n35mr6039286wms.88.1639044813054; Thu, 09 Dec 2021 02:13:33 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:e4a2:3070:eea4:e434]) by smtp.gmail.com with ESMTPSA id j8sm5079449wrh.16.2021.12.09.02.13.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Dec 2021 02:13:32 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org, Laurent Pinchart , Kieran Bingham , Hans Verkuil , Tomasz Figa , Jacopo Mondi , Naushir Patuck Date: Thu, 9 Dec 2021 10:12:40 +0000 Message-Id: <20211209101245.6187-4-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211209101245.6187-1-david.plowman@raspberrypi.com> References: <20211209101245.6187-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v10 3/8] libcamera: video_device: Convert between ColorSpace class and V4L2 formats X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add functions to the V4L2Device class to convert to and from libcamera ColorSpace. These functions are added to the base V4L2Device class so that they can be shared both by the video device class and subdevices. With the ColorSpace class, the color space and related other fields are stored together, corresponding to a number of fields in the various different V4L2 format structures. Template functions are therefore a convenient implementation, and we must explicitly instantiate the templates that will be needed. Note that unset color spaces are converted to requests for the device's "default" color space. Signed-off-by: David Plowman Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- include/libcamera/internal/v4l2_device.h | 8 + src/libcamera/v4l2_device.cpp | 194 +++++++++++++++++++++++ 2 files changed, 202 insertions(+) diff --git a/include/libcamera/internal/v4l2_device.h b/include/libcamera/internal/v4l2_device.h index 8886b750..fe3899e8 100644 --- a/include/libcamera/internal/v4l2_device.h +++ b/include/libcamera/internal/v4l2_device.h @@ -9,6 +9,7 @@ #include #include +#include #include #include @@ -18,6 +19,7 @@ #include #include +#include #include namespace libcamera { @@ -45,6 +47,12 @@ public: void updateControlInfo(); + template + static std::optional toColorSpace(const T &v4l2Format); + + template + static int fromColorSpace(const std::optional &colorSpace, T &v4l2Format); + protected: V4L2Device(const std::string &deviceNode); ~V4L2Device(); diff --git a/src/libcamera/v4l2_device.cpp b/src/libcamera/v4l2_device.cpp index 39f36009..ab7d8791 100644 --- a/src/libcamera/v4l2_device.cpp +++ b/src/libcamera/v4l2_device.cpp @@ -10,11 +10,15 @@ #include #include #include +#include #include #include #include #include #include +#include + +#include #include #include @@ -728,4 +732,194 @@ void V4L2Device::eventAvailable() frameStart.emit(event.u.frame_sync.frame_sequence); } +static const std::map v4l2ToColorSpace = { + { V4L2_COLORSPACE_RAW, ColorSpace::Raw }, + { V4L2_COLORSPACE_JPEG, ColorSpace::Jpeg }, + { V4L2_COLORSPACE_SRGB, ColorSpace::Srgb }, + { V4L2_COLORSPACE_SMPTE170M, ColorSpace::Smpte170m }, + { V4L2_COLORSPACE_REC709, ColorSpace::Rec709 }, + { V4L2_COLORSPACE_BT2020, ColorSpace::Rec2020 }, +}; + +static const std::map v4l2ToYcbcrEncoding = { + { V4L2_YCBCR_ENC_601, ColorSpace::YcbcrEncoding::Rec601 }, + { V4L2_YCBCR_ENC_709, ColorSpace::YcbcrEncoding::Rec709 }, + { V4L2_YCBCR_ENC_BT2020, ColorSpace::YcbcrEncoding::Rec2020 }, +}; + +static const std::map v4l2ToTransferFunction = { + { V4L2_XFER_FUNC_NONE, ColorSpace::TransferFunction::Linear }, + { V4L2_XFER_FUNC_SRGB, ColorSpace::TransferFunction::Srgb }, + { V4L2_XFER_FUNC_709, ColorSpace::TransferFunction::Rec709 }, +}; + +static const std::map v4l2ToRange = { + { V4L2_QUANTIZATION_FULL_RANGE, ColorSpace::Range::Full }, + { V4L2_QUANTIZATION_LIM_RANGE, ColorSpace::Range::Limited }, +}; + +static const std::vector> colorSpaceToV4l2 = { + { ColorSpace::Raw, V4L2_COLORSPACE_RAW }, + { ColorSpace::Jpeg, V4L2_COLORSPACE_JPEG }, + { ColorSpace::Srgb, V4L2_COLORSPACE_SRGB }, + { ColorSpace::Smpte170m, V4L2_COLORSPACE_SMPTE170M }, + { ColorSpace::Rec709, V4L2_COLORSPACE_REC709 }, + { ColorSpace::Rec2020, V4L2_COLORSPACE_BT2020 }, +}; + +static const std::map primariesToV4l2 = { + { ColorSpace::Primaries::Raw, V4L2_COLORSPACE_RAW }, + { ColorSpace::Primaries::Smpte170m, V4L2_COLORSPACE_SMPTE170M }, + { ColorSpace::Primaries::Rec709, V4L2_COLORSPACE_REC709 }, + { ColorSpace::Primaries::Rec2020, V4L2_COLORSPACE_BT2020 }, +}; + +static const std::map ycbcrEncodingToV4l2 = { + { ColorSpace::YcbcrEncoding::Rec601, V4L2_YCBCR_ENC_601 }, + { ColorSpace::YcbcrEncoding::Rec709, V4L2_YCBCR_ENC_709 }, + { ColorSpace::YcbcrEncoding::Rec2020, V4L2_YCBCR_ENC_BT2020 }, +}; + +static const std::map transferFunctionToV4l2 = { + { ColorSpace::TransferFunction::Linear, V4L2_XFER_FUNC_NONE }, + { ColorSpace::TransferFunction::Srgb, V4L2_XFER_FUNC_SRGB }, + { ColorSpace::TransferFunction::Rec709, V4L2_XFER_FUNC_709 }, +}; + +static const std::map rangeToV4l2 = { + { ColorSpace::Range::Full, V4L2_QUANTIZATION_FULL_RANGE }, + { ColorSpace::Range::Limited, V4L2_QUANTIZATION_LIM_RANGE }, +}; + +/** + * \brief Convert the color space fields in a V4L2 format to a ColorSpace + * \param[in] v4l2Format A V4L2 format containing color space information + * + * The colorspace, ycbcr_enc, xfer_func and quantization fields within a + * V4L2 format structure are converted to a corresponding ColorSpace. + * + * If any V4L2 fields are not recognised then we return an "unset" + * color space. + * + * \return The ColorSpace corresponding to the input V4L2 format + * \retval std::nullopt One or more V4L2 color space fields were not recognised + */ +template +std::optional V4L2Device::toColorSpace(const T &v4l2Format) +{ + auto itColor = v4l2ToColorSpace.find(v4l2Format.colorspace); + if (itColor == v4l2ToColorSpace.end()) + return std::nullopt; + + /* This sets all the color space fields to the correct "default" values. */ + ColorSpace colorSpace = itColor->second; + + if (v4l2Format.ycbcr_enc != V4L2_YCBCR_ENC_DEFAULT) { + auto itYcbcrEncoding = v4l2ToYcbcrEncoding.find(v4l2Format.ycbcr_enc); + if (itYcbcrEncoding == v4l2ToYcbcrEncoding.end()) + return std::nullopt; + + colorSpace.ycbcrEncoding = itYcbcrEncoding->second; + } + + if (v4l2Format.xfer_func != V4L2_XFER_FUNC_DEFAULT) { + auto itTransfer = v4l2ToTransferFunction.find(v4l2Format.xfer_func); + if (itTransfer == v4l2ToTransferFunction.end()) + return std::nullopt; + + colorSpace.transferFunction = itTransfer->second; + } + + if (v4l2Format.quantization != V4L2_QUANTIZATION_DEFAULT) { + auto itRange = v4l2ToRange.find(v4l2Format.quantization); + if (itRange == v4l2ToRange.end()) + return std::nullopt; + + colorSpace.range = itRange->second; + } + + return colorSpace; +} + +template std::optional V4L2Device::toColorSpace(const struct v4l2_pix_format &); +template std::optional V4L2Device::toColorSpace(const struct v4l2_pix_format_mplane &); +template std::optional V4L2Device::toColorSpace(const struct v4l2_mbus_framefmt &); + +/** + * \brief Fill in the color space fields of a V4L2 format from a ColorSpace + * \param[in] colorSpace The ColorSpace to be converted + * \param[out] v4l2Format A V4L2 format containing color space information + * + * The colorspace, ycbcr_enc, xfer_func and quantization fields within a + * V4L2 format structure are filled in from a corresponding ColorSpace. + * + * An error is returned if any of the V4L2 fields do not support the + * value given in the ColorSpace. Such fields are set to the V4L2 + * "default" values, but all other fields are still filled in where + * possible. + * + * If the color space is completely unset, "default" V4L2 values are used + * everywhere, so a driver would then choose its preferred color space. + * + * \return 0 on success or a negative error code otherwise + * \retval -EINVAL The ColorSpace does not have a representation using V4L2 enums + */ +template +int V4L2Device::fromColorSpace(const std::optional &colorSpace, T &v4l2Format) +{ + v4l2Format.colorspace = V4L2_COLORSPACE_DEFAULT; + v4l2Format.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; + v4l2Format.xfer_func = V4L2_XFER_FUNC_DEFAULT; + v4l2Format.quantization = V4L2_QUANTIZATION_DEFAULT; + + if (!colorSpace) + return 0; + + auto itColor = std::find_if(colorSpaceToV4l2.begin(), colorSpaceToV4l2.end(), + [&colorSpace](const auto &item) { + return colorSpace == item.first; + }); + if (itColor != colorSpaceToV4l2.end()) { + v4l2Format.colorspace = itColor->second; + /* Leaving all the other fields as "default" should be fine. */ + return 0; + } + + /* + * If the colorSpace doesn't precisely match a standard color space, + * then we must choose a V4L2 colorspace with matching primaries. + */ + int ret = 0; + + auto itPrimaries = primariesToV4l2.find(colorSpace->primaries); + if (itPrimaries != primariesToV4l2.end()) + v4l2Format.colorspace = itPrimaries->second; + else + ret = -EINVAL; + + auto itYcbcrEncoding = ycbcrEncodingToV4l2.find(colorSpace->ycbcrEncoding); + if (itYcbcrEncoding != ycbcrEncodingToV4l2.end()) + v4l2Format.ycbcr_enc = itYcbcrEncoding->second; + else + ret = -EINVAL; + + auto itTransfer = transferFunctionToV4l2.find(colorSpace->transferFunction); + if (itTransfer != transferFunctionToV4l2.end()) + v4l2Format.xfer_func = itTransfer->second; + else + ret = -EINVAL; + + auto itRange = rangeToV4l2.find(colorSpace->range); + if (itRange != rangeToV4l2.end()) + v4l2Format.quantization = itRange->second; + else + ret = -EINVAL; + + return ret; +} + +template int V4L2Device::fromColorSpace(const std::optional &, struct v4l2_pix_format &); +template int V4L2Device::fromColorSpace(const std::optional &, struct v4l2_pix_format_mplane &); +template int V4L2Device::fromColorSpace(const std::optional &, struct v4l2_mbus_framefmt &); + } /* namespace libcamera */ From patchwork Thu Dec 9 10:12:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 15094 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E1F03BF415 for ; Thu, 9 Dec 2021 10:13:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 722B360224; Thu, 9 Dec 2021 11:13:37 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="B+QFMR9h"; dkim-atps=neutral Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 67B456089C for ; Thu, 9 Dec 2021 11:13:34 +0100 (CET) Received: by mail-wr1-x429.google.com with SMTP id d9so8774711wrw.4 for ; Thu, 09 Dec 2021 02:13:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=dNFIU7MwMl9M/WUUAajbXcsy7xJkXDsMafgWJxChUOM=; b=B+QFMR9ha9JliZWZKB9D9qOhUCjgEkJPgTDWZhzHS8rYNls7PDHuURcuDH58rC5OY9 cKJ765AaDunmaMGz4hIPkrqBUv+odyWgRj+fQThFfuVu7EwwS7EOQKgaCybrbkdsI8le X6w18HCbr+l3xaHIA0APMrZy4twLtCb+9Nqc4pE5jS4w4SBIp/RWEPWuT1qyJGbYg7uW 21a0HB78wT1KPUyLu8qKnBnck/KpWTJ7oWFGoh7LpE7C8s3EMmaat/dyMHsLpTu3pajB TzzK/Tfw2d9AE2k1sexoOdXNTCJNkmkifLGToh7yIT4tqVvlKO8zJLSg7Yo7ey/eHT9Y EeMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=dNFIU7MwMl9M/WUUAajbXcsy7xJkXDsMafgWJxChUOM=; b=5wWLgISvnSgnyst1L2k1i2fUmTmEGWKEFYU3DCzH/DktUanSoGANMLmeAkbwt6C0/E 5MzFqZPO5EFqi61/xGi2c2evxZWEsN+gv0tMPW346Ko8uk8xmntdnMsi+WmC6jwNW+ct 6ZBi2hRchziKBY0RcSZSWIF1R9ft4LuhNYqeHaL7bP4YZdnCRx2QZsdCsLva/XlqjtcL DFZ7roOyxNsP5tHC/mzKjDhiAlO3qMzIA5LGmFMooZ4ZDvru1w6bdZ2ar3UPjukxhXS2 TkBZbqJVsjzhWRdBmVIi02KcnIlnlAdANFYxroyStaVEGXNtjoLVlkF2IYHzjrhJWM5k ns6A== X-Gm-Message-State: AOAM531WdQWpw7LmPvpSp44j/kgR5RIfTY0U9QX0z7ttQAR695HUlvKZ 13OECCAhRDuTq0s8DyaFbQ4N1+V0rmWQuZ+r X-Google-Smtp-Source: ABdhPJzB93t4z6/JgUppZUPICDZ6ml9dhvns0cutvX6zKncK+tmznVCpkF1hET2EwWaD0mVpX2rAaA== X-Received: by 2002:adf:fb4f:: with SMTP id c15mr5408424wrs.507.1639044813629; Thu, 09 Dec 2021 02:13:33 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:e4a2:3070:eea4:e434]) by smtp.gmail.com with ESMTPSA id j8sm5079449wrh.16.2021.12.09.02.13.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Dec 2021 02:13:33 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org, Laurent Pinchart , Kieran Bingham , Hans Verkuil , Tomasz Figa , Jacopo Mondi , Naushir Patuck Date: Thu, 9 Dec 2021 10:12:41 +0000 Message-Id: <20211209101245.6187-5-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211209101245.6187-1-david.plowman@raspberrypi.com> References: <20211209101245.6187-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v10 4/8] libcamera: video_device: Support passing ColorSpaces to V4L2 video devices X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The ColorSpace from the StreamConfiguration is now handled appropriately in the V4L2VideoDevice. Signed-off-by: David Plowman Reviewed-by: Laurent Pinchart --- include/libcamera/internal/v4l2_videodevice.h | 3 ++ src/libcamera/v4l2_videodevice.cpp | 32 +++++++++++++++++++ 2 files changed, 35 insertions(+) diff --git a/include/libcamera/internal/v4l2_videodevice.h b/include/libcamera/internal/v4l2_videodevice.h index 9b2ec3af..2d2ccc47 100644 --- a/include/libcamera/internal/v4l2_videodevice.h +++ b/include/libcamera/internal/v4l2_videodevice.h @@ -10,6 +10,7 @@ #include #include #include +#include #include #include #include @@ -21,6 +22,7 @@ #include #include +#include #include #include #include @@ -167,6 +169,7 @@ public: V4L2PixelFormat fourcc; Size size; + std::optional colorSpace; std::array planes; unsigned int planesCount = 0; diff --git a/src/libcamera/v4l2_videodevice.cpp b/src/libcamera/v4l2_videodevice.cpp index b4b89e27..80a8c8e4 100644 --- a/src/libcamera/v4l2_videodevice.cpp +++ b/src/libcamera/v4l2_videodevice.cpp @@ -383,6 +383,21 @@ bool V4L2BufferCache::Entry::operator==(const FrameBuffer &buffer) const * that identifies the image format pixel encoding scheme. */ +/** + * \var V4L2DeviceFormat::colorSpace + * \brief The color space of the pixels + * + * The color space of the image. When setting the format this may be + * unset, in which case the driver gets to use its default color space. + * After being set, this value should contain the color space that + * was actually used. If this value is unset, then the color space chosen + * by the driver could not be represented by the ColorSpace class (and + * should probably be added). + * + * It is up to the pipeline handler or application to check if the + * resulting color space is acceptable. + */ + /** * \var V4L2DeviceFormat::planes * \brief The per-plane memory size information @@ -871,6 +886,7 @@ int V4L2VideoDevice::getFormatMultiplane(V4L2DeviceFormat *format) format->size.height = pix->height; format->fourcc = V4L2PixelFormat(pix->pixelformat); format->planesCount = pix->num_planes; + format->colorSpace = toColorSpace(*pix); for (unsigned int i = 0; i < format->planesCount; ++i) { format->planes[i].bpl = pix->plane_fmt[i].bytesperline; @@ -893,6 +909,12 @@ int V4L2VideoDevice::trySetFormatMultiplane(V4L2DeviceFormat *format, bool set) pix->num_planes = format->planesCount; pix->field = V4L2_FIELD_NONE; + ret = fromColorSpace(format->colorSpace, *pix); + if (ret < 0) + LOG(V4L2, Warning) + << "Setting color space unrecognised by V4L2: " + << ColorSpace::toString(format->colorSpace); + ASSERT(pix->num_planes <= std::size(pix->plane_fmt)); for (unsigned int i = 0; i < pix->num_planes; ++i) { @@ -920,6 +942,7 @@ int V4L2VideoDevice::trySetFormatMultiplane(V4L2DeviceFormat *format, bool set) format->planes[i].bpl = pix->plane_fmt[i].bytesperline; format->planes[i].size = pix->plane_fmt[i].sizeimage; } + format->colorSpace = toColorSpace(*pix); return 0; } @@ -943,6 +966,7 @@ int V4L2VideoDevice::getFormatSingleplane(V4L2DeviceFormat *format) format->planesCount = 1; format->planes[0].bpl = pix->bytesperline; format->planes[0].size = pix->sizeimage; + format->colorSpace = toColorSpace(*pix); return 0; } @@ -959,6 +983,13 @@ int V4L2VideoDevice::trySetFormatSingleplane(V4L2DeviceFormat *format, bool set) pix->pixelformat = format->fourcc; pix->bytesperline = format->planes[0].bpl; pix->field = V4L2_FIELD_NONE; + + ret = fromColorSpace(format->colorSpace, *pix); + if (ret < 0) + LOG(V4L2, Warning) + << "Set color space unrecognised by V4L2: " + << ColorSpace::toString(format->colorSpace); + ret = ioctl(set ? VIDIOC_S_FMT : VIDIOC_TRY_FMT, &v4l2Format); if (ret) { LOG(V4L2, Error) @@ -977,6 +1008,7 @@ int V4L2VideoDevice::trySetFormatSingleplane(V4L2DeviceFormat *format, bool set) format->planesCount = 1; format->planes[0].bpl = pix->bytesperline; format->planes[0].size = pix->sizeimage; + format->colorSpace = toColorSpace(*pix); return 0; } From patchwork Thu Dec 9 10:12:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 15095 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EABFAC3258 for ; Thu, 9 Dec 2021 10:13:38 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 862FE60882; Thu, 9 Dec 2021 11:13:38 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="sG7dYNOk"; dkim-atps=neutral Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CD2B46088B for ; Thu, 9 Dec 2021 11:13:34 +0100 (CET) Received: by mail-wr1-x42c.google.com with SMTP id t18so8714103wrg.11 for ; Thu, 09 Dec 2021 02:13:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XppW0uYN/+TM2EogMDHujEnG3QJ6JaZY9LDGXon35AQ=; b=sG7dYNOkcr32NYdv19RKc+f0z1Z5wU/k9BgnL1kFmGbtpbqjPnJmdvXojzGLdmRL8V TogY2z6KIRdLZOD0xMe6a1lyeYCh1zm1ysc2GlMKX+w+BRCJAS4vtdASGAtIA9ZIfu7L Dr4hehso2Bxa9n7BUjPVuHy0Q7nco02pWnwA9YlTs4XhGGAO9LmMVZJmj7ikgIHL284L LXPKY+fam6VjTN1hSKBWQkR4CUu+4ypQn/m0zb+RgDxL+w5AmaQkWLlAzRMwMHK0SbdO Zcs+JDTjnyaV6TNAmmoO3YWy6ZwOJ85JyhDC8dyEH0mOi7vhtY6VnKjQl/EzUQXjiI8A rgtA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XppW0uYN/+TM2EogMDHujEnG3QJ6JaZY9LDGXon35AQ=; b=414kjcZKCZIQtCQwklLfiT4yggd5Ig9u+N6VrFe0V1ppvgEH80RhRn9Ng6NIuoLWYD LFQaYcFVj08dMOkAJBiwhnXXbBl7TNMM7c+Pfeg9uMrUtsuk47MRVZuy7Q5XzYCALXJc lQzTs2/kg4b4lUwJbrwfgqbaWrdc7VR+A/s6TsjrfCysf6U8gtq3e+nVS3jpYhZMhRQ5 haJezAQosB45KE6RP/QefwxHLcm0YUaLkx6ZzlHaxLfQUa9wBy7VMUgkkfvwZGKO3d7i E2crJq75jHHcQsS8YI3yQJf+ArhkgNLdh0SYlbnl4aYxYQ0wW8Nyt6l3eNCFqowO/OJ2 xvdA== X-Gm-Message-State: AOAM531L+zOxB0nP/uNgJu6yN1mNMTErvW9lWi9V5VtCVS8+wjr87vWm Diz/LLEGFMAniZcnBQliFuw4nbCUnIj1r48v X-Google-Smtp-Source: ABdhPJyG/cn1pfam9n4DYe/VtwBuogFRolDEKj1wnJfGZ5ZJwB4PtuY6ToJV/deaPv4Jcnn69d/m6w== X-Received: by 2002:a05:6000:1201:: with SMTP id e1mr5580352wrx.298.1639044814333; Thu, 09 Dec 2021 02:13:34 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:e4a2:3070:eea4:e434]) by smtp.gmail.com with ESMTPSA id j8sm5079449wrh.16.2021.12.09.02.13.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Dec 2021 02:13:33 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org, Laurent Pinchart , Kieran Bingham , Hans Verkuil , Tomasz Figa , Jacopo Mondi , Naushir Patuck Date: Thu, 9 Dec 2021 10:12:42 +0000 Message-Id: <20211209101245.6187-6-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211209101245.6187-1-david.plowman@raspberrypi.com> References: <20211209101245.6187-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v10 5/8] libcamera: v4l2_subdevice: Add colorSpace field to V4L2SubdeviceFormat X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This adds a ColorSpace field to the V4L2SubdeviceFormat so that we can set and request particular color spaces from V4L2. This commit simply adds the field and fixes some occurrences of brace initializers that would otherwise be broken. A subsequent commit will pass and retrieve the value correctly to/from V4l2 itself. Signed-off-by: David Plowman Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- include/libcamera/internal/v4l2_subdevice.h | 3 +++ src/libcamera/camera_sensor.cpp | 1 + src/libcamera/pipeline/ipu3/cio2.cpp | 7 +++---- src/libcamera/pipeline/simple/simple.cpp | 8 ++++++-- src/libcamera/v4l2_subdevice.cpp | 15 +++++++++++++++ 5 files changed, 28 insertions(+), 6 deletions(-) diff --git a/include/libcamera/internal/v4l2_subdevice.h b/include/libcamera/internal/v4l2_subdevice.h index a6873b67..58d1e511 100644 --- a/include/libcamera/internal/v4l2_subdevice.h +++ b/include/libcamera/internal/v4l2_subdevice.h @@ -8,12 +8,14 @@ #pragma once #include +#include #include #include #include #include +#include #include #include "libcamera/internal/formats.h" @@ -27,6 +29,7 @@ class MediaDevice; struct V4L2SubdeviceFormat { uint32_t mbus_code; Size size; + std::optional colorSpace; const std::string toString() const; uint8_t bitsPerPixel() const; diff --git a/src/libcamera/camera_sensor.cpp b/src/libcamera/camera_sensor.cpp index 7bb39b1e..c3999d35 100644 --- a/src/libcamera/camera_sensor.cpp +++ b/src/libcamera/camera_sensor.cpp @@ -642,6 +642,7 @@ V4L2SubdeviceFormat CameraSensor::getFormat(const std::vector &mbu V4L2SubdeviceFormat format{ .mbus_code = bestCode, .size = *bestSize, + .colorSpace = ColorSpace::Raw, }; return format; diff --git a/src/libcamera/pipeline/ipu3/cio2.cpp b/src/libcamera/pipeline/ipu3/cio2.cpp index 59dda56b..f4e8c663 100644 --- a/src/libcamera/pipeline/ipu3/cio2.cpp +++ b/src/libcamera/pipeline/ipu3/cio2.cpp @@ -322,10 +322,9 @@ V4L2SubdeviceFormat CIO2Device::getSensorFormat(const std::vector return {}; } - V4L2SubdeviceFormat format{ - .mbus_code = bestCode, - .size = bestSize, - }; + V4L2SubdeviceFormat format{}; + format.mbus_code = bestCode; + format.size = bestSize; return format; } diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 701fb4be..a3108fc0 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -457,7 +457,9 @@ int SimpleCameraData::init() * formats on the video node. */ for (unsigned int code : sensor_->mbusCodes()) { - V4L2SubdeviceFormat format{ code, sensor_->resolution() }; + V4L2SubdeviceFormat format{}; + format.mbus_code = code; + format.size = sensor_->resolution(); ret = setupFormats(&format, V4L2Subdevice::TryFormat); if (ret < 0) { @@ -908,7 +910,9 @@ int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c) return ret; const SimpleCameraData::Configuration *pipeConfig = config->pipeConfig(); - V4L2SubdeviceFormat format{ pipeConfig->code, data->sensor_->resolution() }; + V4L2SubdeviceFormat format{}; + format.mbus_code = pipeConfig->code; + format.size = data->sensor_->resolution(); ret = data->setupFormats(&format, V4L2Subdevice::ActiveFormat); if (ret < 0) diff --git a/src/libcamera/v4l2_subdevice.cpp b/src/libcamera/v4l2_subdevice.cpp index 61e15b69..b782325a 100644 --- a/src/libcamera/v4l2_subdevice.cpp +++ b/src/libcamera/v4l2_subdevice.cpp @@ -169,6 +169,21 @@ const std::map formatInfoMap = { * \brief The image size in pixels */ +/** + * \var V4L2SubdeviceFormat::colorSpace + * \brief The color space of the pixels + * + * The color space of the image. When setting the format this may be + * unset, in which case the driver gets to use its default color space. + * After being set, this value should contain the color space that + * was actually used. If this value is unset, then the color space chosen + * by the driver could not be represented by the ColorSpace class (and + * should probably be added). + * + * It is up to the pipeline handler or application to check if the + * resulting color space is acceptable. + */ + /** * \brief Assemble and return a string describing the format * \return A string describing the V4L2SubdeviceFormat From patchwork Thu Dec 9 10:12:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 15096 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BE6AABF415 for ; Thu, 9 Dec 2021 10:13:40 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7A39760224; Thu, 9 Dec 2021 11:13:40 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="VdmnbNCE"; dkim-atps=neutral Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [IPv6:2a00:1450:4864:20::336]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 991FC60876 for ; Thu, 9 Dec 2021 11:13:35 +0100 (CET) Received: by mail-wm1-x336.google.com with SMTP id i12so3744963wmq.4 for ; Thu, 09 Dec 2021 02:13:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=K60rAz66vWi9F057w7dISj11FLoL/9HFVty+RyVxed8=; b=VdmnbNCEdWt9UN4XvxyHAIv1/yCXgX3PDYyefELrvHuCmUeKD1iKjXw006DcTu4CQ7 vxtczPkzmZD7gmdIzHc0LyWDQdyJlrR44AQGkaHcfWnpcH0WScZfo4ObKUWQ5tFLELah 2uqL3PbPRUn6Lt7w0udazDun1hCuo7sJN0sKsR7kxU/gnVtTCuZP+A7Nx1P1FEOw3NPj kQ7oIASQ/dLj5lD+euoEalK+9CxaAy2SlU9n4vy1SB3O6GE0p56TsuXpMsxYQslBAaH0 1Ih8Dm4EdmVePIAb71+2ihina1PCeeUhOfKhh0ArCGCvUM6u8VVnOJLrQ+Zv95gYRCWj KOtA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=K60rAz66vWi9F057w7dISj11FLoL/9HFVty+RyVxed8=; b=0AaeuFLDljJbnFT/26cpcWZSbP60GZIMXELpQhW7xbr9wH83AAkEQjRO2FbFyEwfER 8zl+AxhAiHzh4EobmDgCMblhYQDh8IYK2UZmCUMHezYJvW3ZwfvGsxE0gspqAJApXdwp mccnxj8oX6nknxNuhvzBF1hp6yoV8q2GI9UJ0QUPOYCqyR8qrvmdfVjasiuOIAHRvn5q 3sRk85VGWmDbn4sMtlv4gihv19mBjamdJCkWQpsY83FMnIuDYRZfkTZnPG6Ax0DqbQZW otWifYh1zT5UMZ53aFZWkhZXSCCQZQlKdr9zrn7HxeiIPOaePobqDlVNRjGM1D2QjocM SJZw== X-Gm-Message-State: AOAM533MDVxUc5wFD35v6sktukZgpKXw+1JT9RoGtMyeyPPnrNJPExYC MmRIQQThL2h6YGm5IcpWsjakKXV12JtIRwWu X-Google-Smtp-Source: ABdhPJxWsoxn3YIVcryK2lCY7oepdV8Oh+2rysLQUISS9XCZ7wNZ3AMuNvtGYZG7YQp+ufnJto7fdQ== X-Received: by 2002:a05:600c:190e:: with SMTP id j14mr5831248wmq.75.1639044815166; Thu, 09 Dec 2021 02:13:35 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:e4a2:3070:eea4:e434]) by smtp.gmail.com with ESMTPSA id j8sm5079449wrh.16.2021.12.09.02.13.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Dec 2021 02:13:34 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org, Laurent Pinchart , Kieran Bingham , Hans Verkuil , Tomasz Figa , Jacopo Mondi , Naushir Patuck Date: Thu, 9 Dec 2021 10:12:43 +0000 Message-Id: <20211209101245.6187-7-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211209101245.6187-1-david.plowman@raspberrypi.com> References: <20211209101245.6187-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v10 6/8] libcamera: v4l2_subdevice: Support passing ColorSpaces to V4L2 subdevices X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The ColorSpace from the StreamConfiguration is now handled appropriately in the V4L2Subdevice. Signed-off-by: David Plowman Reviewed-by: Laurent Pinchart --- src/libcamera/v4l2_subdevice.cpp | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/src/libcamera/v4l2_subdevice.cpp b/src/libcamera/v4l2_subdevice.cpp index b782325a..8caa73d6 100644 --- a/src/libcamera/v4l2_subdevice.cpp +++ b/src/libcamera/v4l2_subdevice.cpp @@ -415,6 +415,7 @@ int V4L2Subdevice::getFormat(unsigned int pad, V4L2SubdeviceFormat *format, format->size.width = subdevFmt.format.width; format->size.height = subdevFmt.format.height; format->mbus_code = subdevFmt.format.code; + format->colorSpace = toColorSpace(subdevFmt.format); return 0; } @@ -443,7 +444,13 @@ int V4L2Subdevice::setFormat(unsigned int pad, V4L2SubdeviceFormat *format, subdevFmt.format.code = format->mbus_code; subdevFmt.format.field = V4L2_FIELD_NONE; - int ret = ioctl(VIDIOC_SUBDEV_S_FMT, &subdevFmt); + int ret = fromColorSpace(format->colorSpace, subdevFmt.format); + if (ret < 0) + LOG(V4L2, Warning) + << "Setting color space unrecognised by V4L2: " + << ColorSpace::toString(format->colorSpace); + + ret = ioctl(VIDIOC_SUBDEV_S_FMT, &subdevFmt); if (ret) { LOG(V4L2, Error) << "Unable to set format on pad " << pad @@ -454,6 +461,7 @@ int V4L2Subdevice::setFormat(unsigned int pad, V4L2SubdeviceFormat *format, format->size.width = subdevFmt.format.width; format->size.height = subdevFmt.format.height; format->mbus_code = subdevFmt.format.code; + format->colorSpace = toColorSpace(subdevFmt.format); return 0; } From patchwork Thu Dec 9 10:12:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 15097 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8886DC324B for ; Thu, 9 Dec 2021 10:13:41 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1E7EB6088B; Thu, 9 Dec 2021 11:13:41 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="PmevTKvg"; dkim-atps=neutral Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 73FE36086A for ; Thu, 9 Dec 2021 11:13:36 +0100 (CET) Received: by mail-wm1-x330.google.com with SMTP id o19-20020a1c7513000000b0033a93202467so3661628wmc.2 for ; Thu, 09 Dec 2021 02:13:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=R1YEICgfdgsmW9t4ujB9VSgCIG1ak5d76cb+q5rcH2M=; b=PmevTKvgqticNKUuX1s7g3oqZtnlnX1X0H5nFvPg3jJg3TAORQErKCQyKk0z0YYibV NChP36fJhQ6AK6avwaOFBE0F+18Nrj2S9GNz3egr4FXabi6qwTX8mtjBLcM8ZoWJnQZi Uq1va0qjq08bvzkzZ2HiCnuZc+uU06+X3mXP1rBj20cI1DlwEAT3oTl0160mT15yUfo0 rk21DBIEF7IXMpluLrKFSYAMIsK2CC0yzMPeNjH56S6gxZkvo01GZ/0pKQBgqwff8rjW DMKzELNLBwlnBcnNapJ5TNXCGnP2ZNyNvlHoJWF4avAW0ohc0aPjXSvnl+6A1hzTjRKL 615w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=R1YEICgfdgsmW9t4ujB9VSgCIG1ak5d76cb+q5rcH2M=; b=OoEfORGJosNmnnSJEIu580h9R9Sc0D9PcpGiwK2q5YdwJWbZe/EGwkqA1EXzWFWWGb Y35snhH1ytXL6Flgywqo26UAXr8vJkgh/qy2r1S5jZD1ZpTpF/2ih7Ut2G/feToDqqmp /PPz6MGrM2FIQQCLKKWXdEnHWhBNvRlTHEGQvgZyPUvEhCg3H3mGZroZuuJ+0wlrsZ8l Cq2n9BdlF73o4nLq5QmRfG2iciZcGyTyhIDrDG27qAGpeuH5rRWtXBofflzbyNYZ2tLx nSFVOgVUHWPFNSa/o6g/w+pA/DmvWZotEKblU+bktGHhq83RPn2mx2RVSOqUksYLj+qb G7+g== X-Gm-Message-State: AOAM533tv0eqit+MoHHG+8bIPo3oAzb1+ORYZbAFuT9850tMY0I1lLUu NLpe7tQAsKRq9baj0buJBQE8Fo0dYjlgBSPT X-Google-Smtp-Source: ABdhPJxmgj8JfyawN5UJsPeaYUXA7Qfrw0KrMHONTsJ88pr1+fNMQ1k3bf0r+NDzCZ52XbiLp3puiw== X-Received: by 2002:a1c:208b:: with SMTP id g133mr6002527wmg.128.1639044815908; Thu, 09 Dec 2021 02:13:35 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:e4a2:3070:eea4:e434]) by smtp.gmail.com with ESMTPSA id j8sm5079449wrh.16.2021.12.09.02.13.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Dec 2021 02:13:35 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org, Laurent Pinchart , Kieran Bingham , Hans Verkuil , Tomasz Figa , Jacopo Mondi , Naushir Patuck Date: Thu, 9 Dec 2021 10:12:44 +0000 Message-Id: <20211209101245.6187-8-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211209101245.6187-1-david.plowman@raspberrypi.com> References: <20211209101245.6187-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v10 7/8] libcamera: Add validateColorSpaces to CameraConfiguration class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This method forces raw streams to have the "raw" color space, and also optionally makes all output streams to share the same color space as some platforms may require this. Signed-off-by: David Plowman --- include/libcamera/camera.h | 10 +++++ src/libcamera/camera.cpp | 84 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 94 insertions(+) diff --git a/include/libcamera/camera.h b/include/libcamera/camera.h index a7759ccb..5bb06584 100644 --- a/include/libcamera/camera.h +++ b/include/libcamera/camera.h @@ -13,6 +13,7 @@ #include #include +#include #include #include @@ -69,6 +70,15 @@ public: protected: CameraConfiguration(); + enum class ColorSpaceFlag { + None, + StreamsShareColorSpace, + }; + + using ColorSpaceFlags = Flags; + + Status validateColorSpaces(ColorSpaceFlags flags = ColorSpaceFlag::None); + std::vector config_; }; diff --git a/src/libcamera/camera.cpp b/src/libcamera/camera.cpp index 400a7cf0..87ef8d72 100644 --- a/src/libcamera/camera.cpp +++ b/src/libcamera/camera.cpp @@ -14,12 +14,14 @@ #include #include +#include #include #include #include #include "libcamera/internal/camera.h" #include "libcamera/internal/camera_controls.h" +#include "libcamera/internal/formats.h" #include "libcamera/internal/pipeline_handler.h" /** @@ -314,6 +316,88 @@ std::size_t CameraConfiguration::size() const return config_.size(); } +namespace { + +bool isRaw(const PixelFormat &pixFmt) +{ + const PixelFormatInfo &info = PixelFormatInfo::info(pixFmt); + return info.isValid() && + info.colourEncoding == PixelFormatInfo::ColourEncodingRAW; +} + +} /* namespace */ + +/** + * \enum CameraConfiguration::ColorSpaceFlag + * \brief Specify the behaviour of validateColorSpaces + * \var ColorSpaceFlag::None + * \brief No extra validation of color spaces is required + * \var ColorSpaceFlag::StreamsShareColorSpace + * \brief Non-raw output streams must share the same color space + */ + +/** + * \typedef CameraConfiguration::ColorSpaceFlags + * \brief A bitwise combination of ColorSpaceFlag values + */ + +/** + * \brief Check the color spaces requested for each stream + * \param[in] flags Flags to control the behaviour of this function + * + * This function performs certain consistency checks on the color spaces of + * the streams and may adjust them so that: + * + * - Any raw streams have the Raw color space + * - If shareOutputColorSpaces is set, all output streams are forced to share + * the same color space (this may be a constraint on some platforms). + * + * It is optional for a pipeline handler to use this method. + * + * \return A CameraConfiguration::Status value that describes the validation + * status. + * \retval CameraConfiguration::Invalid The configuration is invalid and can't + * be adjusted. This may only occur in extreme cases such as when the + * configuration is empty. + * \retval CameraConfigutation::Adjusted The configuration has been adjusted + * and is now valid. Parameters may have changed for any stream, and stream + * configurations may have been removed. The caller shall check the + * configuration carefully. + * \retval CameraConfiguration::Valid The configuration was already valid and + * hasn't been adjusted. + */ +CameraConfiguration::Status CameraConfiguration::validateColorSpaces(ColorSpaceFlags flags) +{ + Status status = Valid; + + /* + * Set all raw streams to the Raw color space, and make a note of the largest + * non-raw stream with a defined color space (if there is one). + */ + int index = -1; + for (auto [i, cfg] : utils::enumerate(config_)) { + if (isRaw(cfg.pixelFormat) && cfg.colorSpace != ColorSpace::Raw) { + cfg.colorSpace = ColorSpace::Raw; + status = Adjusted; + } else if (cfg.colorSpace && (index == -1 || cfg.size > config_[i].size)) + index = i; + } + + if (index < 0 || !(flags & ColorSpaceFlag::StreamsShareColorSpace)) + return status; + + /* Make all output color spaces the same, if requested. */ + for (auto &cfg : config_) { + if (!isRaw(cfg.pixelFormat) && + cfg.colorSpace != config_[index].colorSpace) { + cfg.colorSpace = config_[index].colorSpace; + status = Adjusted; + } + } + + return status; +} + /** * \var CameraConfiguration::transform * \brief User-specified transform to be applied to the image From patchwork Thu Dec 9 10:12:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 15098 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0D484C3258 for ; Thu, 9 Dec 2021 10:13:42 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id ABBC06088E; Thu, 9 Dec 2021 11:13:41 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="rWvjyPcj"; dkim-atps=neutral Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3B18B60894 for ; Thu, 9 Dec 2021 11:13:37 +0100 (CET) Received: by mail-wm1-x333.google.com with SMTP id o19-20020a1c7513000000b0033a93202467so3661656wmc.2 for ; Thu, 09 Dec 2021 02:13:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=smk/dJpPZW6o/pfsEEykw+VPU4y38gdL3mRud2K8yx4=; b=rWvjyPcjSWtJUF9jy64Le/Ri4pr52efNSxx5CMuS7kBlYo7w0UA6DCEJ7o6F4oTucG oyhMq2UVoQCumXWVZAkfcsH26MZY8ry6byplKorW3FFOdyjLwUgIb/ehDTsKNx/PQ4ec OWkBWzQZSxBkrTWqreJrx9+m7gbGHLKOk+qW6cjaPmGvdQZC+V7gNMH8kFVM7FrZFE12 ACwP51KUqIGt6X27P9hSkKrHawVznxe7U1hc3W6ZzDrOvdu+na2Kidr2REa/8RdEz5So fHMYmtDIeYshIZ0QxGcO6uSvAYWiS+KUWyHvBnJfUbkUB3BLlrlBdv8HsUZEdsx7KtK7 Q2pw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=smk/dJpPZW6o/pfsEEykw+VPU4y38gdL3mRud2K8yx4=; b=GwKhkSdQXoceBgRVMDyYTc5Codxbd2FRAg3cilfvO+n3vKvZYxBYappbezBJ0h6SsR XH8oBVNfcHF3Qtj1M70QwQGYRe6x/hzRbUQ5OpuMW8gAL6suO7EFP7AMlyBXGk+GF4xO wNvNg6jbVPdB2H/rCb0ptntsRuWEs11YjreL4nlXOoMqOuC3tgySwZu8unqqiyjioFrH 0Eo1XNpWcFZA9o7Tv7eYkpW2RR/L4+ZUyuuPTEQ4tgyFD25c88qH4UZfBC6VbLllr+Uo KMU18oNhcvOo2xxILUUG3FSKJ8l1hWKMbLk1mEfpZGAV46Mgw3j7BB4EhWK7q+dIHVZD Rcjw== X-Gm-Message-State: AOAM533b2X9NNyCKIEE8iwVFCU4Dg9Cgnpj46A78mlKFvtgEPwoIf0yb kjGniVsM9ZO2Rw8S5lJeJ1ijX9uM/2l2Dq8P X-Google-Smtp-Source: ABdhPJxFJa5owrHqje5gRuKNp9osGu80OVHDa8+JzYJMHPXdULAVqH4Li3gB2zZDdCaFOWwqBa4M8g== X-Received: by 2002:a05:600c:2c4a:: with SMTP id r10mr6096688wmg.125.1639044816653; Thu, 09 Dec 2021 02:13:36 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:e4a2:3070:eea4:e434]) by smtp.gmail.com with ESMTPSA id j8sm5079449wrh.16.2021.12.09.02.13.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Dec 2021 02:13:36 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org, Laurent Pinchart , Kieran Bingham , Hans Verkuil , Tomasz Figa , Jacopo Mondi , Naushir Patuck Date: Thu, 9 Dec 2021 10:12:45 +0000 Message-Id: <20211209101245.6187-9-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211209101245.6187-1-david.plowman@raspberrypi.com> References: <20211209101245.6187-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v10 8/8] libcamera: pipeline: raspberrypi: Support color spaces X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The Raspberry Pi pipeline handler now sets color spaces correctly. In generateConfiguration() it sets them to reasonable default values based on the stream role. validate() now calls validateColorSpaces() to ensure that the requested color spaces are sensible, before proceeding to check what the hardware can deliver. Signed-off-by: David Plowman --- .../pipeline/raspberrypi/raspberrypi.cpp | 40 +++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 22c8ee68..368953e1 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -96,6 +96,7 @@ V4L2DeviceFormat toV4L2DeviceFormat(const V4L2SubdeviceFormat &format, deviceFormat.fourcc = V4L2PixelFormat::fromPixelFormat(pix); deviceFormat.size = format.size; + deviceFormat.colorSpace = format.colorSpace; return deviceFormat; } @@ -132,6 +133,7 @@ V4L2SubdeviceFormat findBestFormat(const SensorFormats &formatsMap, const Size & { double bestScore = std::numeric_limits::max(), score; V4L2SubdeviceFormat bestFormat; + bestFormat.colorSpace = ColorSpace::Raw; constexpr float penaltyAr = 1500.0; constexpr float penaltyBitDepth = 500.0; @@ -329,6 +331,8 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() if (config_.empty()) return Invalid; + status = validateColorSpaces(ColorSpaceFlag::StreamsShareColorSpace); + /* * What if the platform has a non-90 degree rotation? We can't even * "adjust" the configuration and carry on. Alternatively, raising an @@ -496,11 +500,25 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() V4L2DeviceFormat format; format.fourcc = V4L2PixelFormat::fromPixelFormat(cfg.pixelFormat); format.size = cfg.size; + format.colorSpace = cfg.colorSpace; + + LOG(RPI, Debug) + << "Try color space " << ColorSpace::toString(cfg.colorSpace); int ret = dev->tryFormat(&format); if (ret) return Invalid; + if (!format.colorSpace || cfg.colorSpace != format.colorSpace) { + status = Adjusted; + LOG(RPI, Warning) + << "Color space changed from " + << ColorSpace::toString(cfg.colorSpace) << " to " + << ColorSpace::toString(format.colorSpace); + } + + cfg.colorSpace = format.colorSpace; + cfg.stride = format.planes[0].bpl; cfg.frameSize = format.planes[0].size; @@ -524,6 +542,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, PixelFormat pixelFormat; V4L2VideoDevice::Formats fmts; Size size; + std::optional colorSpace; if (roles.empty()) return config; @@ -538,6 +557,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, pixelFormat = mbusCodeToPixelFormat(sensorFormat.mbus_code, BayerFormat::Packing::CSI2); ASSERT(pixelFormat.isValid()); + colorSpace = ColorSpace::Raw; bufferCount = 2; rawCount++; break; @@ -545,6 +565,12 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, case StreamRole::StillCapture: fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::NV12; + /* + * Still image codecs usually expect the JPEG color space. + * Even RGB codecs will be fine as the RGB we get with the + * JPEG color space is the same as sRGB. + */ + colorSpace = ColorSpace::Jpeg; /* Return the largest sensor resolution. */ size = data->sensor_->resolution(); bufferCount = 1; @@ -562,6 +588,11 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, */ fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::YUV420; + /* + * Choose a color space appropriate for video recording. + * Rec.709 will be a good default for HD resolutions. + */ + colorSpace = ColorSpace::Rec709; size = { 1920, 1080 }; bufferCount = 4; outCount++; @@ -570,6 +601,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, case StreamRole::Viewfinder: fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::ARGB8888; + colorSpace = ColorSpace::Jpeg; size = { 800, 600 }; bufferCount = 4; outCount++; @@ -612,6 +644,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, StreamConfiguration cfg(formats); cfg.size = size; cfg.pixelFormat = pixelFormat; + cfg.colorSpace = colorSpace; cfg.bufferCount = bufferCount; config->addConfiguration(cfg); } @@ -719,6 +752,7 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) V4L2PixelFormat fourcc = V4L2PixelFormat::fromPixelFormat(cfg.pixelFormat); format.size = cfg.size; format.fourcc = fourcc; + format.colorSpace = cfg.colorSpace; LOG(RPI, Debug) << "Setting " << stream->name() << " to " << format.toString(); @@ -734,6 +768,10 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) return -EINVAL; } + LOG(RPI, Debug) + << "Stream " << stream->name() << " has color space " + << ColorSpace::toString(cfg.colorSpace); + cfg.setStream(stream); stream->setExternal(true); @@ -758,6 +796,8 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) format = {}; format.size = maxSize; format.fourcc = V4L2PixelFormat::fromPixelFormat(formats::YUV420); + /* No one asked for output, so the color space doesn't matter. */ + format.colorSpace = ColorSpace::Jpeg; ret = data->isp_[Isp::Output0].dev()->setFormat(&format); if (ret) { LOG(RPI, Error)