From patchwork Fri Dec 10 14:44:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 15114 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 74F36BF415 for ; Fri, 10 Dec 2021 14:44:41 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id EC621608A0; Fri, 10 Dec 2021 15:44:38 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="ap4twqKz"; dkim-atps=neutral Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C2E0060868 for ; Fri, 10 Dec 2021 15:44:37 +0100 (CET) Received: by mail-wr1-x42c.google.com with SMTP id a18so15315392wrn.6 for ; Fri, 10 Dec 2021 06:44:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=UtqDHjZtUiffgzkJqDsbMYmmZFYYHiKtTASM4gaNBxg=; b=ap4twqKzuJej5c12vzihLAOesXgmEGiulAh3ZdDVlTnD4whJHmDAYcgtq1HdWVpw4O zViWsigETQr/O0FtynUEcBLfJF5UIdnWewU2fcOpq2GWLmG2y/S17nYUUhrkyjU/pNxM 7UNRr6wsNJQiOz16egj92yRBmRQs7Sptu6i1U6KLQ8Ene/lAPCQCvTyMaCDQot1X+zYd 0H3TEba16y+r6VnwSCEpOyJWsLh0z8ehrRaOwT9cxKYw/VNmx7RUivh/Jg53LLEXM+tY lVm2rIgWxhXTtQ++87Y6AXO+jHsJgjoMU9xQIeUxHNx9THc9Prr9m+wNyxmZGAO2j9VM 5h+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=UtqDHjZtUiffgzkJqDsbMYmmZFYYHiKtTASM4gaNBxg=; b=nVtf3QqIGcZ2kHcSdqu3VXlJdXqhmg7/MUg059k/YIXqhqaPASZtvdKJujpRa+kSvF QoRwpfnp2hXB7xA1IuzqjFSv4ynv+3GqlA+N6lKea6f3ncRbVZU0snmR+q3+2VssvvUC awkm1bPhE5WkkE4Qzsdp6NXMw+93YFAYql3AkzzXX6F4YvN4fZ6XmQTZ+J1h3Lu54CP1 AA0FBtcFgisW6Gq1lReGPJ8yScEru1bN2t9DjwNERrA5O9Gus8rN7lif2V0Uh7JnwjrU FJ4mDpNIquuGWESIuWF2CKJ8ixQs94JCUnKCsXLDM4di7qK8OJCcQu43DOFReycniDFc RF5g== X-Gm-Message-State: AOAM532q1qYzvUxTxy+guVXB5gsuifrfkQSABTnQGe5wwM5fYvaUBctH KcA2EPVzv3kmTRZGT9MX5c9edQ== X-Google-Smtp-Source: ABdhPJy3T7QlIUm7BGWKQDGBPvJNWy/V7XMD+uqxbC3NTwZIbDaOnSrZfDs7mIGqatXRAq0c1J+aRg== X-Received: by 2002:a5d:52c3:: with SMTP id r3mr14880761wrv.115.1639147477348; Fri, 10 Dec 2021 06:44:37 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:e4a2:3070:eea4:e434]) by smtp.gmail.com with ESMTPSA id w22sm3000515wmi.27.2021.12.10.06.44.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Dec 2021 06:44:36 -0800 (PST) From: David Plowman To: Laurent Pinchart , Kieran Bingham , Hans Verkuil , Tomasz Figa , Jacopo Mondi , Naushir Patuck , libcamera-devel@lists.libcamera.org Date: Fri, 10 Dec 2021 14:44:17 +0000 Message-Id: <20211210144424.14747-2-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211210144424.14747-1-david.plowman@raspberrypi.com> References: <20211210144424.14747-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v12 1/8] libcamera: Add ColorSpace class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This class represents a color space by defining its color primaries, the transfer (gamma) function it uses, the YCbCr encoding and whether the output is full or limited range. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck Reviewed-by: Umang Jain Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- include/libcamera/color_space.h | 70 +++++++ include/libcamera/meson.build | 1 + src/libcamera/color_space.cpp | 317 ++++++++++++++++++++++++++++++++ src/libcamera/meson.build | 1 + 4 files changed, 389 insertions(+) create mode 100644 include/libcamera/color_space.h create mode 100644 src/libcamera/color_space.cpp diff --git a/include/libcamera/color_space.h b/include/libcamera/color_space.h new file mode 100644 index 00000000..086c56c1 --- /dev/null +++ b/include/libcamera/color_space.h @@ -0,0 +1,70 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Raspberry Pi (Trading) Limited + * + * color_space.h - color space definitions + */ + +#pragma once + +#include +#include + +namespace libcamera { + +class ColorSpace +{ +public: + enum class Primaries { + Raw, + Smpte170m, + Rec709, + Rec2020, + }; + + enum class TransferFunction { + Linear, + Srgb, + Rec709, + }; + + enum class YcbcrEncoding { + None, + Rec601, + Rec709, + Rec2020, + }; + + enum class Range { + Full, + Limited, + }; + + constexpr ColorSpace(Primaries p, TransferFunction t, YcbcrEncoding e, Range r) + : primaries(p), transferFunction(t), ycbcrEncoding(e), range(r) + { + } + + static const ColorSpace Raw; + static const ColorSpace Jpeg; + static const ColorSpace Srgb; + static const ColorSpace Smpte170m; + static const ColorSpace Rec709; + static const ColorSpace Rec2020; + + Primaries primaries; + TransferFunction transferFunction; + YcbcrEncoding ycbcrEncoding; + Range range; + + std::string toString() const; + static std::string toString(const std::optional &colorSpace); +}; + +bool operator==(const ColorSpace &lhs, const ColorSpace &rhs); +static inline bool operator!=(const ColorSpace &lhs, const ColorSpace &rhs) +{ + return !(lhs == rhs); +} + +} /* namespace libcamera */ diff --git a/include/libcamera/meson.build b/include/libcamera/meson.build index 5f42977c..fd767b11 100644 --- a/include/libcamera/meson.build +++ b/include/libcamera/meson.build @@ -5,6 +5,7 @@ libcamera_include_dir = 'libcamera' / 'libcamera' libcamera_public_headers = files([ 'camera.h', 'camera_manager.h', + 'color_space.h', 'controls.h', 'framebuffer.h', 'framebuffer_allocator.h', diff --git a/src/libcamera/color_space.cpp b/src/libcamera/color_space.cpp new file mode 100644 index 00000000..b208d298 --- /dev/null +++ b/src/libcamera/color_space.cpp @@ -0,0 +1,317 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Raspberry Pi (Trading) Limited + * + * color_space.cpp - color spaces. + */ + +#include + +#include +#include +#include +#include +#include + +/** + * \file color_space.h + * \brief Class and enums to represent color spaces + */ + +namespace libcamera { + +/** + * \class ColorSpace + * \brief Class to describe a color space + * + * The ColorSpace class defines the color primaries, the transfer function, + * the Y'CbCr encoding associated with the color space, and the range + * (sometimes also referred to as the quantisation) of the color space. + * + * Certain combinations of these fields form well-known standard color + * spaces such as "JPEG" or "REC709". + * + * In the strictest sense a "color space" formally only refers to the + * color primaries and white point. Here, however, the ColorSpace class + * adopts the common broader usage that includes the transfer function, + * Y'CbCr encoding method and quantisation. + * + * For more information on the specific color spaces described here, please + * see: + * + * - sRGB + * - JPEG + * - SMPTE 170M + * - Rec.709 + * - Rec.2020 + */ + +/** + * \enum ColorSpace::Primaries + * \brief The color primaries for this color space + * + * \var ColorSpace::Primaries::Raw + * \brief These are raw colors directly from a sensor, the primaries are + * unspecified + * + * \var ColorSpace::Primaries::Smpte170m + * \brief SMPTE 170M color primaries + * + * \var ColorSpace::Primaries::Rec709 + * \brief Rec.709 color primaries + * + * \var ColorSpace::Primaries::Rec2020 + * \brief Rec.2020 color primaries + */ + +/** + * \enum ColorSpace::TransferFunction + * \brief The transfer function used for this color space + * + * \var ColorSpace::TransferFunction::Linear + * \brief This color space uses a linear (identity) transfer function + * + * \var ColorSpace::TransferFunction::Srgb + * \brief sRGB transfer function + * + * \var ColorSpace::TransferFunction::Rec709 + * \brief Rec.709 transfer function + */ + +/** + * \enum ColorSpace::YcbcrEncoding + * \brief The Y'CbCr encoding + * + * \var ColorSpace::YcbcrEncoding::None + * \brief There is no defined Y'CbCr encoding (used for non-YUV formats) + * + * \var ColorSpace::YcbcrEncoding::Rec601 + * \brief Rec.601 Y'CbCr encoding + * + * \var ColorSpace::YcbcrEncoding::Rec709 + * \brief Rec.709 Y'CbCr encoding + * + * \var ColorSpace::YcbcrEncoding::Rec2020 + * \brief Rec.2020 Y'CbCr encoding + */ + +/** + * \enum ColorSpace::Range + * \brief The range (sometimes "quantisation") for this color space + * + * \var ColorSpace::Range::Full + * \brief This color space uses full range pixel values + * + * \var ColorSpace::Range::Limited + * \brief This color space uses limited range pixel values, being + * 16 to 235 for Y' and 16 to 240 for Cb and Cr (8 bits per sample) + * or 64 to 940 for Y' and 16 to 960 for Cb and Cr (10 bits) + */ + +/** + * \fn ColorSpace::ColorSpace(Primaries p, TransferFunction t, Encoding e, Range r) + * \brief Construct a ColorSpace from explicit values + * \param[in] p The color primaries + * \param[in] t The transfer function for the color space + * \param[in] e The Y'CbCr encoding + * \param[in] r The range of the pixel values in this color space + */ + +/** + * \brief Assemble and return a readable string representation of the + * ColorSpace + * + * If the color space matches a standard ColorSpace (such as ColorSpace::Jpeg) + * then the short name of the color space ("JPEG") is returned. Otherwise + * the four constituent parts of the ColorSpace are assembled into a longer + * string. + * + * \return A string describing the ColorSpace + */ +std::string ColorSpace::toString() const +{ + /* Print out a brief name only for standard color spaces. */ + + static const std::array, 6> colorSpaceNames = { { + { ColorSpace::Raw, "RAW" }, + { ColorSpace::Jpeg, "JPEG" }, + { ColorSpace::Srgb, "sRGB" }, + { ColorSpace::Smpte170m, "SMPTE170M" }, + { ColorSpace::Rec709, "Rec709" }, + { ColorSpace::Rec2020, "Rec2020" }, + } }; + auto it = std::find_if(colorSpaceNames.begin(), colorSpaceNames.end(), + [this](const auto &item) { + return *this == item.first; + }); + if (it != colorSpaceNames.end()) + return std::string(it->second); + + /* Assemble a name made of the constituent fields. */ + + static const std::map primariesNames = { + { Primaries::Raw, "RAW" }, + { Primaries::Smpte170m, "SMPTE170M" }, + { Primaries::Rec709, "Rec709" }, + { Primaries::Rec2020, "Rec2020" }, + }; + static const std::map transferNames = { + { TransferFunction::Linear, "Linear" }, + { TransferFunction::Srgb, "sRGB" }, + { TransferFunction::Rec709, "Rec709" }, + }; + static const std::map encodingNames = { + { YcbcrEncoding::None, "None" }, + { YcbcrEncoding::Rec601, "Rec601" }, + { YcbcrEncoding::Rec709, "Rec709" }, + { YcbcrEncoding::Rec2020, "Rec2020" }, + }; + static const std::map rangeNames = { + { Range::Full, "Full" }, + { Range::Limited, "Limited" }, + }; + + auto itPrimaries = primariesNames.find(primaries); + std::string primariesName = + itPrimaries == primariesNames.end() ? "Invalid" : itPrimaries->second; + + auto itTransfer = transferNames.find(transferFunction); + std::string transferName = + itTransfer == transferNames.end() ? "Invalid" : itTransfer->second; + + auto itEncoding = encodingNames.find(ycbcrEncoding); + std::string encodingName = + itEncoding == encodingNames.end() ? "Invalid" : itEncoding->second; + + auto itRange = rangeNames.find(range); + std::string rangeName = + itRange == rangeNames.end() ? "Invalid" : itRange->second; + + std::stringstream ss; + ss << primariesName << "/" << transferName << "/" << encodingName << "/" << rangeName; + + return ss.str(); +} + +/** + * \brief Assemble and return a readable string representation of an + * optional ColorSpace + * + * This is a convenience helper to easily obtain a string representation + * for a ColorSpace in parts of the libcamera API where it is stored in a + * std::optional<>. If the ColorSpace is set, this function returns + * \a colorSpace.toString(), otherwise it returns "Unset". + * + * \return A string describing the optional ColorSpace + */ +std::string ColorSpace::toString(const std::optional &colorSpace) +{ + if (!colorSpace) + return "Unset"; + + return colorSpace->toString(); +} + +/** + * \var ColorSpace::primaries + * \brief The color primaries of this color space + */ + +/** + * \var ColorSpace::transferFunction + * \brief The transfer function used by this color space + */ + +/** + * \var ColorSpace::ycbcrEncoding + * \brief The Y'CbCr encoding used by this color space + */ + +/** + * \var ColorSpace::range + * \brief The pixel range used with by color space + */ + +/** + * \brief A constant representing a raw color space (from a sensor) + */ +const ColorSpace ColorSpace::Raw = { + Primaries::Raw, + TransferFunction::Linear, + YcbcrEncoding::None, + Range::Full +}; + +/** + * \brief A constant representing the JPEG color space used for + * encoding JPEG images + */ +const ColorSpace ColorSpace::Jpeg = { + Primaries::Rec709, + TransferFunction::Srgb, + YcbcrEncoding::Rec601, + Range::Full +}; + +/** + * \brief A constant representing the sRGB color space. + * + * This is identical to the JPEG color space except that the Y'CbCr + * range is limited rather than full. + */ +const ColorSpace ColorSpace::Srgb = { + Primaries::Rec709, + TransferFunction::Srgb, + YcbcrEncoding::Rec601, + Range::Limited +}; + +/** + * \brief A constant representing the SMPTE170M color space + */ +const ColorSpace ColorSpace::Smpte170m = { + Primaries::Smpte170m, + TransferFunction::Rec709, + YcbcrEncoding::Rec601, + Range::Limited +}; + +/** + * \brief A constant representing the Rec.709 color space + */ +const ColorSpace ColorSpace::Rec709 = { + Primaries::Rec709, + TransferFunction::Rec709, + YcbcrEncoding::Rec709, + Range::Limited +}; + +/** + * \brief A constant representing the Rec.2020 color space + */ +const ColorSpace ColorSpace::Rec2020 = { + Primaries::Rec2020, + TransferFunction::Rec709, + YcbcrEncoding::Rec2020, + Range::Limited +}; + +/** + * \brief Compare color spaces for equality + * \return True if the two color spaces are identical, false otherwise + */ +bool operator==(const ColorSpace &lhs, const ColorSpace &rhs) +{ + return lhs.primaries == rhs.primaries && + lhs.transferFunction == rhs.transferFunction && + lhs.ycbcrEncoding == rhs.ycbcrEncoding && + lhs.range == rhs.range; +} + +/** + * \fn bool operator!=(const ColorSpace &lhs, const ColorSpace &rhs) + * \brief Compare color spaces for inequality + * \return True if the two color spaces are not identical, false otherwise + */ + +} /* namespace libcamera */ diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build index 2e54cc04..4045e24e 100644 --- a/src/libcamera/meson.build +++ b/src/libcamera/meson.build @@ -9,6 +9,7 @@ libcamera_sources = files([ 'camera_manager.cpp', 'camera_sensor.cpp', 'camera_sensor_properties.cpp', + 'color_space.cpp', 'controls.cpp', 'control_serializer.cpp', 'control_validator.cpp', From patchwork Fri Dec 10 14:44:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 15115 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2D5B3BF415 for ; Fri, 10 Dec 2021 14:44:43 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8FE5D60897; Fri, 10 Dec 2021 15:44:41 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="PFh15tLc"; dkim-atps=neutral Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 540036089A for ; Fri, 10 Dec 2021 15:44:38 +0100 (CET) Received: by mail-wm1-x32e.google.com with SMTP id 77-20020a1c0450000000b0033123de3425so9229049wme.0 for ; Fri, 10 Dec 2021 06:44:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=+sH0KfN2WD6NxZb8aJTKXBPu6mnk13ViaZCwPQX529M=; b=PFh15tLcvGqkun0JNjSyVOvabdRysp3B3lCm7P0RX3vUodzD0x+lUgWcHifSnzgUXt JCJDx3nUB/1MUH7F6Q6AhwNcuXVUD7UgyCHlgmNzdVKf1v7m5/EnNzBkzta1OgUYr2sK oR4ChGLs5SRRGJ+eHB3EizOXzdcamrUMswGeIZf3T/bVdcPi3516A6WmPdx5XumEDC5L oecCWZ9mD9bhoq7MLQ2j4dgCcbQ4Cy7kSeNWtz7EC6eK0lD12O8zLG14B2qnP34PWCBU rrdDbOumjRfAVceaaYwSreQ46uDi6+XYepGlTMoDTe9ju3I8r9vnXIsgrQ1vITxNM8ww BS3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+sH0KfN2WD6NxZb8aJTKXBPu6mnk13ViaZCwPQX529M=; b=LVHvyF8+BBQaLgwS6RZQIDP/cXToVmr8lGcfitjHmkKCcYJLRPmTHj9Qe4MpyOBS6a A5u7PgoFubyHIgP3WRNSBKgtKgZahJ53fKm+JEoOzLWOvABziPJVuPfMAB36mi0LNWud nFefMaIBngkfipZXetyz1+0Ve7SqE6eZUw3+qNiz8drxH5aaXZi7d5o+4aUV91X98/aE zCwewGtxqv/ItFqToJici0OiTQBc0r4f5ugumUb6aw8Gt5Gz16ZvhY9QzOfPZ9QTLeio 1l3EKImkqOoR1xNaaykhzfUisO6vcfHQNRwvNS3mhsGbf6h658yQYXsggUcHyEEqvWmu BlNw== X-Gm-Message-State: AOAM531eI6uhcD25/mzgo2G612XH/2NeaWl8xweT/nUsNvwqJnBf8p95 7jKpd2C6X9wacP7nQbZMDwwB5Q== X-Google-Smtp-Source: ABdhPJyqP9e5TP1Cz3iTlFuY+U68rI/Ub+0S78g17FoNnVJVufOOul0WInw9mB5NBYbYM6JrX1T0ZQ== X-Received: by 2002:a05:600c:4f92:: with SMTP id n18mr16535049wmq.123.1639147477881; Fri, 10 Dec 2021 06:44:37 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:e4a2:3070:eea4:e434]) by smtp.gmail.com with ESMTPSA id w22sm3000515wmi.27.2021.12.10.06.44.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Dec 2021 06:44:37 -0800 (PST) From: David Plowman To: Laurent Pinchart , Kieran Bingham , Hans Verkuil , Tomasz Figa , Jacopo Mondi , Naushir Patuck , libcamera-devel@lists.libcamera.org Date: Fri, 10 Dec 2021 14:44:18 +0000 Message-Id: <20211210144424.14747-3-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211210144424.14747-1-david.plowman@raspberrypi.com> References: <20211210144424.14747-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v12 2/8] libcamera: stream: Add ColorSpace fields to StreamConfiguration X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This is so that applications can choose appropriate color spaces which will then be passed down to the V4L2 devices. The ColorSpace field is actually optional. If it is not set you will get the camera's default color space. Signed-off-by: David Plowman Reviewed-by: Umang Jain Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- include/libcamera/stream.h | 3 +++ src/libcamera/stream.cpp | 20 ++++++++++++++++++++ 2 files changed, 23 insertions(+) diff --git a/include/libcamera/stream.h b/include/libcamera/stream.h index 41ec02b1..f0ae7e62 100644 --- a/include/libcamera/stream.h +++ b/include/libcamera/stream.h @@ -12,6 +12,7 @@ #include #include +#include #include #include #include @@ -47,6 +48,8 @@ struct StreamConfiguration { unsigned int bufferCount; + std::optional colorSpace; + Stream *stream() const { return stream_; } void setStream(Stream *stream) { stream_ = stream; } const StreamFormats &formats() const { return formats_; } diff --git a/src/libcamera/stream.cpp b/src/libcamera/stream.cpp index b421e17e..686e693b 100644 --- a/src/libcamera/stream.cpp +++ b/src/libcamera/stream.cpp @@ -329,6 +329,26 @@ StreamConfiguration::StreamConfiguration(const StreamFormats &formats) * \brief Requested number of buffers to allocate for the stream */ +/** + * \var StreamConfiguration::colorSpace + * \brief The ColorSpace for this stream + * + * This field allows a ColorSpace to be selected for this Stream. + * + * The field is optional and an application can choose to leave it unset. + * Platforms that support the use of color spaces may provide default + * values through the generateConfiguration() method. An application can + * override these when necessary. + * + * If a specific ColorSpace is requested but the Camera cannot deliver it, + * then the StreamConfiguration will be adjusted to a value that can be + * delivered. In this case the validate() method will indicate via its + * return value that the CameraConfiguration has been adjusted. + * + * Note that platforms will typically have different constraints on what + * color spaces can be supported and in what combinations. + */ + /** * \fn StreamConfiguration::stream() * \brief Retrieve the stream associated with the configuration From patchwork Fri Dec 10 14:44:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 15116 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 28C0AC324B for ; Fri, 10 Dec 2021 14:44:44 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 844E4608E1; Fri, 10 Dec 2021 15:44:43 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="MSKCOGFT"; dkim-atps=neutral Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [IPv6:2a00:1450:4864:20::336]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 50916608A1 for ; Fri, 10 Dec 2021 15:44:39 +0100 (CET) Received: by mail-wm1-x336.google.com with SMTP id i12so6992028wmq.4 for ; Fri, 10 Dec 2021 06:44:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=/RkFH9lnjqPpbBEiu0FSDeQShcL5ImUyMbFAFuQTYes=; b=MSKCOGFTOKLz/IwKrlQHk6Hz2pwrs1VVROvEZOnTax1WBkuC8nX2B7xm2wvqF6Eahw qmrCP2dWCBYJXnMmItl2AuosS/2Fu3eqdYDnBVnP+bt6v9UuN2tUGMF8JIdFMhVKDmrX gEJ92k1HOOqdjrufvS/x/Q0budEdzyNHgpogu9us/wyD9XCcR7X51XWdDRgSf8ojCnA6 6HW3eCPu3yZ03WF+Vm9kbjtv5cKpUrpd95tADRwH2l5DapL+huiu+P1BB/PWhBkBXckF JkS3ZlNMpC6bjyRnYWNPdUjxEiG4yi6ukn1CgGmALk4WQsypImBMs1BrPQSpwCI/R6VA kTMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=/RkFH9lnjqPpbBEiu0FSDeQShcL5ImUyMbFAFuQTYes=; b=szsH0ojjIdKSx1d8iTlIMHpCpTmy8gTo9S/zx0CcTII4HzYhhkJbEPkFyd7WBTSq1V hinlz5IZscjMzdxZgUlfyX6P0rwFmqDL7tUYOrkognvgW4LcnLhh7i7cCCUnQXLCa03M QpKxrntaNzLyFQi0ZCOHmbmnOq+8ACtrbjVksdu134VApPgOY1u8m1eqpuh2u4XfZFPq QCOzEm6DVadG3jWPk+uAmT+R8fvmMRFdgj8ur8LArzRtCw6Z+ggQXIMAyXJ94+ukp3m3 8rI5FicUbpTzFlgDnr8klJRFtZi7eWewZ/qbue46aLpVDPlh/RWe10bpkOKBzz0LdWEG AO8w== X-Gm-Message-State: AOAM530ys2GHlA6EE+nt3VkovND/tQMuW+rbac5DfuW0TXgerI48dOwE VqNzSPDDAd+qAntIDjGzciVKgA== X-Google-Smtp-Source: ABdhPJyvnKNQmq5NHq0dvLgXGn7kDcBoyfPGUmx45oSfKsC6pl45ExUpAJhHFZipjcJsfZDtCglZPQ== X-Received: by 2002:a1c:a7c3:: with SMTP id q186mr17717917wme.20.1639147478670; Fri, 10 Dec 2021 06:44:38 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:e4a2:3070:eea4:e434]) by smtp.gmail.com with ESMTPSA id w22sm3000515wmi.27.2021.12.10.06.44.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Dec 2021 06:44:38 -0800 (PST) From: David Plowman To: Laurent Pinchart , Kieran Bingham , Hans Verkuil , Tomasz Figa , Jacopo Mondi , Naushir Patuck , libcamera-devel@lists.libcamera.org Date: Fri, 10 Dec 2021 14:44:19 +0000 Message-Id: <20211210144424.14747-4-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211210144424.14747-1-david.plowman@raspberrypi.com> References: <20211210144424.14747-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v12 3/8] libcamera: video_device: Convert between ColorSpace class and V4L2 formats X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add functions to the V4L2Device class to convert to and from libcamera ColorSpace. These functions are added to the base V4L2Device class so that they can be shared both by the video device class and subdevices. With the ColorSpace class, the color space and related other fields are stored together, corresponding to a number of fields in the various different V4L2 format structures. Template functions are therefore a convenient implementation, and we must explicitly instantiate the templates that will be needed. Note that unset color spaces are converted to requests for the device's "default" color space. Signed-off-by: David Plowman Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- include/libcamera/internal/v4l2_device.h | 8 + src/libcamera/v4l2_device.cpp | 210 +++++++++++++++++++++++ 2 files changed, 218 insertions(+) diff --git a/include/libcamera/internal/v4l2_device.h b/include/libcamera/internal/v4l2_device.h index 8886b750..a52a5f2c 100644 --- a/include/libcamera/internal/v4l2_device.h +++ b/include/libcamera/internal/v4l2_device.h @@ -9,6 +9,7 @@ #include #include +#include #include #include @@ -18,6 +19,7 @@ #include #include +#include #include namespace libcamera { @@ -56,6 +58,12 @@ protected: int fd() const { return fd_.get(); } + template + static std::optional toColorSpace(const T &v4l2Format); + + template + static int fromColorSpace(const std::optional &colorSpace, T &v4l2Format); + private: static ControlType v4l2CtrlType(uint32_t ctrlType); static std::unique_ptr v4l2ControlId(const v4l2_query_ext_ctrl &ctrl); diff --git a/src/libcamera/v4l2_device.cpp b/src/libcamera/v4l2_device.cpp index 39f36009..25ba8c95 100644 --- a/src/libcamera/v4l2_device.cpp +++ b/src/libcamera/v4l2_device.cpp @@ -10,11 +10,15 @@ #include #include #include +#include #include #include #include #include #include +#include + +#include #include #include @@ -728,4 +732,210 @@ void V4L2Device::eventAvailable() frameStart.emit(event.u.frame_sync.frame_sequence); } +static const std::map v4l2ToColorSpace = { + { V4L2_COLORSPACE_RAW, ColorSpace::Raw }, + { V4L2_COLORSPACE_JPEG, ColorSpace::Jpeg }, + { V4L2_COLORSPACE_SRGB, ColorSpace::Srgb }, + { V4L2_COLORSPACE_SMPTE170M, ColorSpace::Smpte170m }, + { V4L2_COLORSPACE_REC709, ColorSpace::Rec709 }, + { V4L2_COLORSPACE_BT2020, ColorSpace::Rec2020 }, +}; + +static const std::map v4l2ToTransferFunction = { + { V4L2_XFER_FUNC_NONE, ColorSpace::TransferFunction::Linear }, + { V4L2_XFER_FUNC_SRGB, ColorSpace::TransferFunction::Srgb }, + { V4L2_XFER_FUNC_709, ColorSpace::TransferFunction::Rec709 }, +}; + +static const std::map v4l2ToYcbcrEncoding = { + { V4L2_YCBCR_ENC_601, ColorSpace::YcbcrEncoding::Rec601 }, + { V4L2_YCBCR_ENC_709, ColorSpace::YcbcrEncoding::Rec709 }, + { V4L2_YCBCR_ENC_BT2020, ColorSpace::YcbcrEncoding::Rec2020 }, +}; + +static const std::map v4l2ToRange = { + { V4L2_QUANTIZATION_FULL_RANGE, ColorSpace::Range::Full }, + { V4L2_QUANTIZATION_LIM_RANGE, ColorSpace::Range::Limited }, +}; + +static const std::vector> colorSpaceToV4l2 = { + { ColorSpace::Raw, V4L2_COLORSPACE_RAW }, + { ColorSpace::Jpeg, V4L2_COLORSPACE_JPEG }, + { ColorSpace::Srgb, V4L2_COLORSPACE_SRGB }, + { ColorSpace::Smpte170m, V4L2_COLORSPACE_SMPTE170M }, + { ColorSpace::Rec709, V4L2_COLORSPACE_REC709 }, + { ColorSpace::Rec2020, V4L2_COLORSPACE_BT2020 }, +}; + +static const std::map primariesToV4l2 = { + { ColorSpace::Primaries::Raw, V4L2_COLORSPACE_RAW }, + { ColorSpace::Primaries::Smpte170m, V4L2_COLORSPACE_SMPTE170M }, + { ColorSpace::Primaries::Rec709, V4L2_COLORSPACE_REC709 }, + { ColorSpace::Primaries::Rec2020, V4L2_COLORSPACE_BT2020 }, +}; + +static const std::map transferFunctionToV4l2 = { + { ColorSpace::TransferFunction::Linear, V4L2_XFER_FUNC_NONE }, + { ColorSpace::TransferFunction::Srgb, V4L2_XFER_FUNC_SRGB }, + { ColorSpace::TransferFunction::Rec709, V4L2_XFER_FUNC_709 }, +}; + +static const std::map ycbcrEncodingToV4l2 = { + { ColorSpace::YcbcrEncoding::Rec601, V4L2_YCBCR_ENC_601 }, + { ColorSpace::YcbcrEncoding::Rec709, V4L2_YCBCR_ENC_709 }, + { ColorSpace::YcbcrEncoding::Rec2020, V4L2_YCBCR_ENC_BT2020 }, +}; + +static const std::map rangeToV4l2 = { + { ColorSpace::Range::Full, V4L2_QUANTIZATION_FULL_RANGE }, + { ColorSpace::Range::Limited, V4L2_QUANTIZATION_LIM_RANGE }, +}; + +/** + * \brief Convert the color space fields in a V4L2 format to a ColorSpace + * \param[in] v4l2Format A V4L2 format containing color space information + * + * The colorspace, ycbcr_enc, xfer_func and quantization fields within a + * V4L2 format structure are converted to a corresponding ColorSpace. + * + * If any V4L2 fields are not recognised then we return an "unset" + * color space. + * + * \return The ColorSpace corresponding to the input V4L2 format + * \retval std::nullopt One or more V4L2 color space fields were not recognised + */ +template +std::optional V4L2Device::toColorSpace(const T &v4l2Format) +{ + auto itColor = v4l2ToColorSpace.find(v4l2Format.colorspace); + if (itColor == v4l2ToColorSpace.end()) + return std::nullopt; + + /* This sets all the color space fields to the correct "default" values. */ + ColorSpace colorSpace = itColor->second; + + if (v4l2Format.xfer_func != V4L2_XFER_FUNC_DEFAULT) { + auto itTransfer = v4l2ToTransferFunction.find(v4l2Format.xfer_func); + if (itTransfer == v4l2ToTransferFunction.end()) + return std::nullopt; + + colorSpace.transferFunction = itTransfer->second; + } + + if (v4l2Format.ycbcr_enc != V4L2_YCBCR_ENC_DEFAULT) { + auto itYcbcrEncoding = v4l2ToYcbcrEncoding.find(v4l2Format.ycbcr_enc); + if (itYcbcrEncoding == v4l2ToYcbcrEncoding.end()) + return std::nullopt; + + colorSpace.ycbcrEncoding = itYcbcrEncoding->second; + } + + if (v4l2Format.quantization != V4L2_QUANTIZATION_DEFAULT) { + auto itRange = v4l2ToRange.find(v4l2Format.quantization); + if (itRange == v4l2ToRange.end()) + return std::nullopt; + + colorSpace.range = itRange->second; + } + + return colorSpace; +} + +template std::optional V4L2Device::toColorSpace(const struct v4l2_pix_format &); +template std::optional V4L2Device::toColorSpace(const struct v4l2_pix_format_mplane &); +template std::optional V4L2Device::toColorSpace(const struct v4l2_mbus_framefmt &); + +/** + * \brief Fill in the color space fields of a V4L2 format from a ColorSpace + * \param[in] colorSpace The ColorSpace to be converted + * \param[out] v4l2Format A V4L2 format containing color space information + * + * The colorspace, ycbcr_enc, xfer_func and quantization fields within a + * V4L2 format structure are filled in from a corresponding ColorSpace. + * + * An error is returned if any of the V4L2 fields do not support the + * value given in the ColorSpace. Such fields are set to the V4L2 + * "default" values, but all other fields are still filled in where + * possible. + * + * If the color space is completely unset, "default" V4L2 values are used + * everywhere, so a driver would then choose its preferred color space. + * + * \return 0 on success or a negative error code otherwise + * \retval -EINVAL The ColorSpace does not have a representation using V4L2 enums + */ +template +int V4L2Device::fromColorSpace(const std::optional &colorSpace, T &v4l2Format) +{ + v4l2Format.colorspace = V4L2_COLORSPACE_DEFAULT; + v4l2Format.xfer_func = V4L2_XFER_FUNC_DEFAULT; + v4l2Format.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; + v4l2Format.quantization = V4L2_QUANTIZATION_DEFAULT; + + if (!colorSpace) + return 0; + + auto itColor = std::find_if(colorSpaceToV4l2.begin(), colorSpaceToV4l2.end(), + [&colorSpace](const auto &item) { + return colorSpace == item.first; + }); + if (itColor != colorSpaceToV4l2.end()) { + v4l2Format.colorspace = itColor->second; + /* Leaving all the other fields as "default" should be fine. */ + return 0; + } + + /* + * If the colorSpace doesn't precisely match a standard color space, + * then we must choose a V4L2 colorspace with matching primaries. + */ + int ret = 0; + + auto itPrimaries = primariesToV4l2.find(colorSpace->primaries); + if (itPrimaries != primariesToV4l2.end()) + v4l2Format.colorspace = itPrimaries->second; + else { + libcamera::LOG(V4L2, Warning) + << "Unrecognised primaries in " + << ColorSpace::toString(colorSpace); + ret = -EINVAL; + } + + auto itTransfer = transferFunctionToV4l2.find(colorSpace->transferFunction); + if (itTransfer != transferFunctionToV4l2.end()) + v4l2Format.xfer_func = itTransfer->second; + else { + libcamera::LOG(V4L2, Warning) + << "Unrecognised transfer function in " + << ColorSpace::toString(colorSpace); + ret = -EINVAL; + } + + auto itYcbcrEncoding = ycbcrEncodingToV4l2.find(colorSpace->ycbcrEncoding); + if (itYcbcrEncoding != ycbcrEncodingToV4l2.end()) + v4l2Format.ycbcr_enc = itYcbcrEncoding->second; + else { + libcamera::LOG(V4L2, Warning) + << "Unrecognised YCbCr encoding in " + << ColorSpace::toString(colorSpace); + ret = -EINVAL; + } + + auto itRange = rangeToV4l2.find(colorSpace->range); + if (itRange != rangeToV4l2.end()) + v4l2Format.quantization = itRange->second; + else { + libcamera::LOG(V4L2, Warning) + << "Unrecognised quantization in " + << ColorSpace::toString(colorSpace); + ret = -EINVAL; + } + + return ret; +} + +template int V4L2Device::fromColorSpace(const std::optional &, struct v4l2_pix_format &); +template int V4L2Device::fromColorSpace(const std::optional &, struct v4l2_pix_format_mplane &); +template int V4L2Device::fromColorSpace(const std::optional &, struct v4l2_mbus_framefmt &); + } /* namespace libcamera */ From patchwork Fri Dec 10 14:44:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 15117 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D4D67BF415 for ; Fri, 10 Dec 2021 14:44:44 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 687466089A; Fri, 10 Dec 2021 15:44:44 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="ZKSRMZ81"; dkim-atps=neutral Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C6377608A2 for ; Fri, 10 Dec 2021 15:44:39 +0100 (CET) Received: by mail-wr1-x434.google.com with SMTP id u17so15331566wrt.3 for ; Fri, 10 Dec 2021 06:44:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=sa0O0O7qhAIzGiKmiVtW9p2VAUFKzxTft78J+gwVYZU=; b=ZKSRMZ812M7va0ljUZfIc2hblrq9h+hm0K+5QLVdq8Mghd9Jq/kJj7UlxAoPeIH6k8 RqxIyeSoolx36FEoMPYJL6Hd2bo9Ze8CSdlA2LvfnjAmuOClePl1MCPu/HXLMPOIvRh4 dP/WZZdrFCja0tvCrATZ8SN/zCcPNib/j7MZXzlcxrM5B2kvsykNjzlMneQg2eMdqMrQ 56mhVsk/myGiIjfRAdF/RL/KrN6+uVjDtMthqI+mSo6Vp/TyVNiXf05Sup0sDRih/9wg hCoT2l1ku6/Yb9VJK5/DmyTxqhIAy2L3juJRLHBe3X3Y7uvnR80sO/r2wxg0qNdMPpEB xVYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=sa0O0O7qhAIzGiKmiVtW9p2VAUFKzxTft78J+gwVYZU=; b=Yob3aYqDedX0d1De7GYuZe0LokRZTeE8+qNL7uU97bVNpjhl2477coN1ygqhzbZ1kt DVuqL+ETmi08jrcExpBOAiT0fMdX9K8gkmHa5QbQfJo1u5wHYlB/OMGEvQsnBu3XNdUH G+v2O25dnuTjtEgkEJIBcrJCuAV2ooyEZKE/Git0VQxSyCc3eRcSQ9sWEZNvpBgnV+7Q vTIc/s65XIdwoT74uiMan/J3GIT6niFNIhtvL1uWSaZ9K5V6eiqWgnD1GXNK0RyUqSTZ ntenmo/0GsVGACNwSfHpHv5sPS+rzMVlNsTSmo+XxwGmj5l+mjwqYEIvnriyNCyxWpWU RiRA== X-Gm-Message-State: AOAM533ErmIaYDrXwINuAMfSfBm8vyH/+xqrquYY6mixpNgbyCTX3B3W iArvw65N3ua5jWcPsl/onVqC+Q== X-Google-Smtp-Source: ABdhPJyjIdO2zfCvNRKOmZFiZ82ca+J1OjQ7GqUWziJH58rfHYWeWqq62sKtFYQENQzUm+8EcX5H4A== X-Received: by 2002:a5d:6244:: with SMTP id m4mr14792682wrv.186.1639147479467; Fri, 10 Dec 2021 06:44:39 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:e4a2:3070:eea4:e434]) by smtp.gmail.com with ESMTPSA id w22sm3000515wmi.27.2021.12.10.06.44.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Dec 2021 06:44:38 -0800 (PST) From: David Plowman To: Laurent Pinchart , Kieran Bingham , Hans Verkuil , Tomasz Figa , Jacopo Mondi , Naushir Patuck , libcamera-devel@lists.libcamera.org Date: Fri, 10 Dec 2021 14:44:20 +0000 Message-Id: <20211210144424.14747-5-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211210144424.14747-1-david.plowman@raspberrypi.com> References: <20211210144424.14747-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v12 4/8] libcamera: video_device: Support passing ColorSpaces to V4L2 video devices X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The ColorSpace from the StreamConfiguration is now handled appropriately in the V4L2VideoDevice. Signed-off-by: David Plowman Reviewed-by: Laurent Pinchart Reviewed-by: Jacopo Mondi --- include/libcamera/internal/v4l2_videodevice.h | 3 +++ src/libcamera/v4l2_videodevice.cpp | 22 +++++++++++++++++++ 2 files changed, 25 insertions(+) diff --git a/include/libcamera/internal/v4l2_videodevice.h b/include/libcamera/internal/v4l2_videodevice.h index 9b2ec3af..2d2ccc47 100644 --- a/include/libcamera/internal/v4l2_videodevice.h +++ b/include/libcamera/internal/v4l2_videodevice.h @@ -10,6 +10,7 @@ #include #include #include +#include #include #include #include @@ -21,6 +22,7 @@ #include #include +#include #include #include #include @@ -167,6 +169,7 @@ public: V4L2PixelFormat fourcc; Size size; + std::optional colorSpace; std::array planes; unsigned int planesCount = 0; diff --git a/src/libcamera/v4l2_videodevice.cpp b/src/libcamera/v4l2_videodevice.cpp index b4b89e27..5f36ee20 100644 --- a/src/libcamera/v4l2_videodevice.cpp +++ b/src/libcamera/v4l2_videodevice.cpp @@ -383,6 +383,21 @@ bool V4L2BufferCache::Entry::operator==(const FrameBuffer &buffer) const * that identifies the image format pixel encoding scheme. */ +/** + * \var V4L2DeviceFormat::colorSpace + * \brief The color space of the pixels + * + * The color space of the image. When setting the format this may be + * unset, in which case the driver gets to use its default color space. + * After being set, this value should contain the color space that + * was actually used. If this value is unset, then the color space chosen + * by the driver could not be represented by the ColorSpace class (and + * should probably be added). + * + * It is up to the pipeline handler or application to check if the + * resulting color space is acceptable. + */ + /** * \var V4L2DeviceFormat::planes * \brief The per-plane memory size information @@ -871,6 +886,7 @@ int V4L2VideoDevice::getFormatMultiplane(V4L2DeviceFormat *format) format->size.height = pix->height; format->fourcc = V4L2PixelFormat(pix->pixelformat); format->planesCount = pix->num_planes; + format->colorSpace = toColorSpace(*pix); for (unsigned int i = 0; i < format->planesCount; ++i) { format->planes[i].bpl = pix->plane_fmt[i].bytesperline; @@ -892,6 +908,7 @@ int V4L2VideoDevice::trySetFormatMultiplane(V4L2DeviceFormat *format, bool set) pix->pixelformat = format->fourcc; pix->num_planes = format->planesCount; pix->field = V4L2_FIELD_NONE; + fromColorSpace(format->colorSpace, *pix); ASSERT(pix->num_planes <= std::size(pix->plane_fmt)); @@ -920,6 +937,7 @@ int V4L2VideoDevice::trySetFormatMultiplane(V4L2DeviceFormat *format, bool set) format->planes[i].bpl = pix->plane_fmt[i].bytesperline; format->planes[i].size = pix->plane_fmt[i].sizeimage; } + format->colorSpace = toColorSpace(*pix); return 0; } @@ -943,6 +961,7 @@ int V4L2VideoDevice::getFormatSingleplane(V4L2DeviceFormat *format) format->planesCount = 1; format->planes[0].bpl = pix->bytesperline; format->planes[0].size = pix->sizeimage; + format->colorSpace = toColorSpace(*pix); return 0; } @@ -959,6 +978,8 @@ int V4L2VideoDevice::trySetFormatSingleplane(V4L2DeviceFormat *format, bool set) pix->pixelformat = format->fourcc; pix->bytesperline = format->planes[0].bpl; pix->field = V4L2_FIELD_NONE; + fromColorSpace(format->colorSpace, *pix); + ret = ioctl(set ? VIDIOC_S_FMT : VIDIOC_TRY_FMT, &v4l2Format); if (ret) { LOG(V4L2, Error) @@ -977,6 +998,7 @@ int V4L2VideoDevice::trySetFormatSingleplane(V4L2DeviceFormat *format, bool set) format->planesCount = 1; format->planes[0].bpl = pix->bytesperline; format->planes[0].size = pix->sizeimage; + format->colorSpace = toColorSpace(*pix); return 0; } From patchwork Fri Dec 10 14:44:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 15118 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8C909C3258 for ; Fri, 10 Dec 2021 14:44:45 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C40EF608E9; Fri, 10 Dec 2021 15:44:44 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="DZSCtoZq"; dkim-atps=neutral Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 897986087E for ; Fri, 10 Dec 2021 15:44:40 +0100 (CET) Received: by mail-wr1-x42d.google.com with SMTP id c4so15282999wrd.9 for ; Fri, 10 Dec 2021 06:44:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XppW0uYN/+TM2EogMDHujEnG3QJ6JaZY9LDGXon35AQ=; b=DZSCtoZqbn3pBKxemm9nj/+6khgiETfGjKwreJW3tnVxGHwBeSmFmtO5+nmMQMEoH1 ihPzR0CzQW507CzKiZLoweUuq372pmbZeEQqT3apCXo990ju5oNqi6hgcobUEmhSWRkB XGtue/mS0I+08p8f+DiZKvJeCVGda9AE5SozibCr5D4rUf+HzUwtEUR2oewld0Yfw8Zs OUmG2AKEiANOPm5jnPsj/mPCpI2by26m9iknP0CxkLyNb7/AlHqyvNnSQa4W0+LsYVxl gjFroQUGGUGLR1vEi4GLddDqX9UNePNIF+KEM3uxCRZ3pLVViHIWMPI37hHjB1M+sE4r 9mZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XppW0uYN/+TM2EogMDHujEnG3QJ6JaZY9LDGXon35AQ=; b=YJhkMaTywn8BdX+LRmHo9Pfavfl4bnz36GKF4+62ZgXwI6nz7jPxfkkzUDehYi0una HNN5BfjErKjMzzCqpLT8/EGVII/idwZ0g+k97YCrbU4vmJawGbVkkbvIPAhyra7vjLPs 3fagVWw6gANtZTTaBjiElqHcpVFFrWFACxTaYPPZwTxPw4H+1DF5ayVe0mFQmQJinAJK 2ealSZfDcdi6RB77i/Hd1vovhCaV+VxojDIXzcWI6HAnCdP+SOFSeM5CQHflh0MfDaRQ sc+JhT8fEJJiVyT9mXlRuRxyHDY0Ls+MBBofTAsLdbH3EpWvrZvFgjCjxxUIm1Z8Oa2P BgQg== X-Gm-Message-State: AOAM531QdK2oF36Oilfp8/woaIc8b+hs1wRYpwdYHm4iJGjmPZTwqK+k IcMSGsLO8fgJfwapqBXiWM7O3g== X-Google-Smtp-Source: ABdhPJyhG3RzsREYd9Y3ikKGXfCGpwoaPiRNysL1PVzSskK5aJRaMVBNTw9ViWnPv8dxQH1H1QZLag== X-Received: by 2002:a05:6000:188c:: with SMTP id a12mr13975574wri.92.1639147480193; Fri, 10 Dec 2021 06:44:40 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:e4a2:3070:eea4:e434]) by smtp.gmail.com with ESMTPSA id w22sm3000515wmi.27.2021.12.10.06.44.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Dec 2021 06:44:39 -0800 (PST) From: David Plowman To: Laurent Pinchart , Kieran Bingham , Hans Verkuil , Tomasz Figa , Jacopo Mondi , Naushir Patuck , libcamera-devel@lists.libcamera.org Date: Fri, 10 Dec 2021 14:44:21 +0000 Message-Id: <20211210144424.14747-6-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211210144424.14747-1-david.plowman@raspberrypi.com> References: <20211210144424.14747-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v12 5/8] libcamera: v4l2_subdevice: Add colorSpace field to V4L2SubdeviceFormat X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This adds a ColorSpace field to the V4L2SubdeviceFormat so that we can set and request particular color spaces from V4L2. This commit simply adds the field and fixes some occurrences of brace initializers that would otherwise be broken. A subsequent commit will pass and retrieve the value correctly to/from V4l2 itself. Signed-off-by: David Plowman Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- include/libcamera/internal/v4l2_subdevice.h | 3 +++ src/libcamera/camera_sensor.cpp | 1 + src/libcamera/pipeline/ipu3/cio2.cpp | 7 +++---- src/libcamera/pipeline/simple/simple.cpp | 8 ++++++-- src/libcamera/v4l2_subdevice.cpp | 15 +++++++++++++++ 5 files changed, 28 insertions(+), 6 deletions(-) diff --git a/include/libcamera/internal/v4l2_subdevice.h b/include/libcamera/internal/v4l2_subdevice.h index a6873b67..58d1e511 100644 --- a/include/libcamera/internal/v4l2_subdevice.h +++ b/include/libcamera/internal/v4l2_subdevice.h @@ -8,12 +8,14 @@ #pragma once #include +#include #include #include #include #include +#include #include #include "libcamera/internal/formats.h" @@ -27,6 +29,7 @@ class MediaDevice; struct V4L2SubdeviceFormat { uint32_t mbus_code; Size size; + std::optional colorSpace; const std::string toString() const; uint8_t bitsPerPixel() const; diff --git a/src/libcamera/camera_sensor.cpp b/src/libcamera/camera_sensor.cpp index 7bb39b1e..c3999d35 100644 --- a/src/libcamera/camera_sensor.cpp +++ b/src/libcamera/camera_sensor.cpp @@ -642,6 +642,7 @@ V4L2SubdeviceFormat CameraSensor::getFormat(const std::vector &mbu V4L2SubdeviceFormat format{ .mbus_code = bestCode, .size = *bestSize, + .colorSpace = ColorSpace::Raw, }; return format; diff --git a/src/libcamera/pipeline/ipu3/cio2.cpp b/src/libcamera/pipeline/ipu3/cio2.cpp index 59dda56b..f4e8c663 100644 --- a/src/libcamera/pipeline/ipu3/cio2.cpp +++ b/src/libcamera/pipeline/ipu3/cio2.cpp @@ -322,10 +322,9 @@ V4L2SubdeviceFormat CIO2Device::getSensorFormat(const std::vector return {}; } - V4L2SubdeviceFormat format{ - .mbus_code = bestCode, - .size = bestSize, - }; + V4L2SubdeviceFormat format{}; + format.mbus_code = bestCode; + format.size = bestSize; return format; } diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 701fb4be..a3108fc0 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -457,7 +457,9 @@ int SimpleCameraData::init() * formats on the video node. */ for (unsigned int code : sensor_->mbusCodes()) { - V4L2SubdeviceFormat format{ code, sensor_->resolution() }; + V4L2SubdeviceFormat format{}; + format.mbus_code = code; + format.size = sensor_->resolution(); ret = setupFormats(&format, V4L2Subdevice::TryFormat); if (ret < 0) { @@ -908,7 +910,9 @@ int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c) return ret; const SimpleCameraData::Configuration *pipeConfig = config->pipeConfig(); - V4L2SubdeviceFormat format{ pipeConfig->code, data->sensor_->resolution() }; + V4L2SubdeviceFormat format{}; + format.mbus_code = pipeConfig->code; + format.size = data->sensor_->resolution(); ret = data->setupFormats(&format, V4L2Subdevice::ActiveFormat); if (ret < 0) diff --git a/src/libcamera/v4l2_subdevice.cpp b/src/libcamera/v4l2_subdevice.cpp index 61e15b69..b782325a 100644 --- a/src/libcamera/v4l2_subdevice.cpp +++ b/src/libcamera/v4l2_subdevice.cpp @@ -169,6 +169,21 @@ const std::map formatInfoMap = { * \brief The image size in pixels */ +/** + * \var V4L2SubdeviceFormat::colorSpace + * \brief The color space of the pixels + * + * The color space of the image. When setting the format this may be + * unset, in which case the driver gets to use its default color space. + * After being set, this value should contain the color space that + * was actually used. If this value is unset, then the color space chosen + * by the driver could not be represented by the ColorSpace class (and + * should probably be added). + * + * It is up to the pipeline handler or application to check if the + * resulting color space is acceptable. + */ + /** * \brief Assemble and return a string describing the format * \return A string describing the V4L2SubdeviceFormat From patchwork Fri Dec 10 14:44:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 15119 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 81CA4BF415 for ; Fri, 10 Dec 2021 14:44:48 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id DEAF0608A8; Fri, 10 Dec 2021 15:44:47 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="Osgcxd1h"; dkim-atps=neutral Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2873060898 for ; Fri, 10 Dec 2021 15:44:41 +0100 (CET) Received: by mail-wr1-x42b.google.com with SMTP id a18so15315688wrn.6 for ; Fri, 10 Dec 2021 06:44:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HmfJ8ac2EKHf4bnrsfrn/IYvdkuy5nb+6QQ8a96WsCY=; b=Osgcxd1h16dTeVKrr8zRN/DxScdKaphCLdzwEtAqE+0KTboj7bcRsFIQWLw0UeB+/B xusGN9AvTzVcGRbsESTMNIB9Pnb0zz+6YrXDoUb73qOAcUSixoTXVOnajp2imoq4WIfk OL6OBsd+wcnMihOZdBWBb+uCQEmIGeRV4BklwH4pmc+QsgTUb4+Aq/7vRvsynAJBfcpp LhR5jl1GDKwgvz72RgVx5PuyCSrrSRADZA4CSWewsDmC7BYb+8+9pwL45+sr6nnZBPim JAdgHpExIdL0Br+M+nEAGQBMjBSDLsAQfLsOLgAvA3uo4kVe7qkD8YRcKMMAz/zaDLHR jT7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HmfJ8ac2EKHf4bnrsfrn/IYvdkuy5nb+6QQ8a96WsCY=; b=bB+3047J9gr1rUyGRVKL1+9D3IXPFumGML+d/cUR6T4DYexCBjldgbO01v6iRGOUaU qlctREyqWgQlFcbZHJzF1Nr/QYYumLuWrFJPBP8Mlr1nONXyzfVh+IGS/ILlPWJ6QRO2 P4g15yfav+4dSQTEBU/cw5iC6jTBWKL7RqcQQFCjAxPcPHiW16nizWaoBS/HI0wkCF/T RPRUE0MpxCjWmPGn9hrtQgxW2GYjys95zMct+PIMLJBEbAZ4D8ONXYXZYRfxq1R6bBpp U8LLZoMgPG1260rvtwgjeT0qDyQI1+ldnNlMqG0wwQuIdX2P55vC0xjNnaVXEqXh7DWw H37g== X-Gm-Message-State: AOAM532bh75sj7TvsSLlCr/nuVD+NpXmwLnvqw2MqGoSfdHMvYIHfQGM CmaqQ64YJ1VF+0yHNBVQT5Wuxg== X-Google-Smtp-Source: ABdhPJyYvQWZayRuF/r2Z8jK6SXmXncn+hOgKnVbSLBDX+p3lvSYoy8Dt0jvaz7px3VCyBqXgmJqvw== X-Received: by 2002:adf:e747:: with SMTP id c7mr14604832wrn.38.1639147480894; Fri, 10 Dec 2021 06:44:40 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:e4a2:3070:eea4:e434]) by smtp.gmail.com with ESMTPSA id w22sm3000515wmi.27.2021.12.10.06.44.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Dec 2021 06:44:40 -0800 (PST) From: David Plowman To: Laurent Pinchart , Kieran Bingham , Hans Verkuil , Tomasz Figa , Jacopo Mondi , Naushir Patuck , libcamera-devel@lists.libcamera.org Date: Fri, 10 Dec 2021 14:44:22 +0000 Message-Id: <20211210144424.14747-7-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211210144424.14747-1-david.plowman@raspberrypi.com> References: <20211210144424.14747-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v12 6/8] libcamera: v4l2_subdevice: Support passing ColorSpaces to V4L2 subdevices X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The ColorSpace from the StreamConfiguration is now handled appropriately in the V4L2Subdevice. Signed-off-by: David Plowman Reviewed-by: Laurent Pinchart Reviewed-by: Jacopo Mondi --- src/libcamera/v4l2_subdevice.cpp | 3 +++ 1 file changed, 3 insertions(+) diff --git a/src/libcamera/v4l2_subdevice.cpp b/src/libcamera/v4l2_subdevice.cpp index b782325a..fa216e85 100644 --- a/src/libcamera/v4l2_subdevice.cpp +++ b/src/libcamera/v4l2_subdevice.cpp @@ -415,6 +415,7 @@ int V4L2Subdevice::getFormat(unsigned int pad, V4L2SubdeviceFormat *format, format->size.width = subdevFmt.format.width; format->size.height = subdevFmt.format.height; format->mbus_code = subdevFmt.format.code; + format->colorSpace = toColorSpace(subdevFmt.format); return 0; } @@ -442,6 +443,7 @@ int V4L2Subdevice::setFormat(unsigned int pad, V4L2SubdeviceFormat *format, subdevFmt.format.height = format->size.height; subdevFmt.format.code = format->mbus_code; subdevFmt.format.field = V4L2_FIELD_NONE; + fromColorSpace(format->colorSpace, subdevFmt.format); int ret = ioctl(VIDIOC_SUBDEV_S_FMT, &subdevFmt); if (ret) { @@ -454,6 +456,7 @@ int V4L2Subdevice::setFormat(unsigned int pad, V4L2SubdeviceFormat *format, format->size.width = subdevFmt.format.width; format->size.height = subdevFmt.format.height; format->mbus_code = subdevFmt.format.code; + format->colorSpace = toColorSpace(subdevFmt.format); return 0; } From patchwork Fri Dec 10 14:44:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 15120 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D648EC324B for ; Fri, 10 Dec 2021 14:44:48 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8CC186089D; Fri, 10 Dec 2021 15:44:48 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="AciMb+2o"; dkim-atps=neutral Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3B098608A8 for ; Fri, 10 Dec 2021 15:44:42 +0100 (CET) Received: by mail-wr1-x435.google.com with SMTP id o13so15252644wrs.12 for ; Fri, 10 Dec 2021 06:44:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=8jCh1Vs4etkIpv7Ue5Cu6UIhqUlddOuq7BkWpeY3clU=; b=AciMb+2o5AASAfGb1wdm+bCABFJX0DGXU2xVbzd8NTK+jd1f5uui6Ri/Lqu1CHIdhw UdCxG9xuCuOOG5nHLiAItUvhgmOzIv3Yraa1X1UwE9v2BOsgmNawZf0aAKl20j4vCmAW ukX+l8KtcIMT6/iv2+K5vDX4T1yp6CTkiA1EyKdHL1ekNLaaScWx/XaYHbnH9thyhzb9 zr6Ro3K7nwYo7sKglnFfcLZRYbKPpe07f7d2M8i7JmcA0btAWToWXWKjYlt9TMsR27KD DC2EhK0ft+UUT05zbht+r+trAvftxOyPo9HCzrL81qxz3qLSiHBEL3ctGUO/I/FJpciQ tFdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8jCh1Vs4etkIpv7Ue5Cu6UIhqUlddOuq7BkWpeY3clU=; b=rCyVIsz9uo+H9oaLVYLLUHY8gXyCkY9IGJlnE4qUk7HFWGqiopuJicsymvYZznip7l jaKAUVL/ALaTznTsYasLEyllmch3IOolYSViBqHd6XYDL6IJ1pcyBSvmmna+pkKao3RM 2zeWbf+Ofb/mJobNqofCut+PcWiVWVAYJ9BOR1zk8M8X5XKEoP4L3v33KP635gMVwVFU B9Aob1RyLohGQuyTcw+VYgFIA6McNKAfzRcyvBP3H/e9J7dQ7orRrcZxL64ZcC5Q3Gfw GHQHfjLl3SDMEHj5xKLZp75HU3IccxdtV6h0eOoRPqWHjlR0MmXc+qKGv49Pv0dqoNi7 2I7A== X-Gm-Message-State: AOAM5314KA61bNbF538TZr19MlntlAKe4h2UaxWOguE1DsLgxGed1TSy lPE7PA1dsHYp4q+XVr6fGqU6EM1qK+ehPA== X-Google-Smtp-Source: ABdhPJzk1BkW6kWbl0xqW1Bc/vK3qsdKK+sWw1yTDawZo5bE8p303RHT9IazwfHLL4/NypSMuoC+Ow== X-Received: by 2002:a5d:6e01:: with SMTP id h1mr13715428wrz.403.1639147481913; Fri, 10 Dec 2021 06:44:41 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:e4a2:3070:eea4:e434]) by smtp.gmail.com with ESMTPSA id w22sm3000515wmi.27.2021.12.10.06.44.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Dec 2021 06:44:41 -0800 (PST) From: David Plowman To: Laurent Pinchart , Kieran Bingham , Hans Verkuil , Tomasz Figa , Jacopo Mondi , Naushir Patuck , libcamera-devel@lists.libcamera.org Date: Fri, 10 Dec 2021 14:44:23 +0000 Message-Id: <20211210144424.14747-8-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211210144424.14747-1-david.plowman@raspberrypi.com> References: <20211210144424.14747-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v12 7/8] libcamera: Add validateColorSpaces to CameraConfiguration class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This function forces raw streams to have the "raw" color space, and also optionally makes all non-raw output streams to share the same color space as some platforms may require this. When sharing color spaces we take the shared value to be the one from the largest of these streams. This choice is ultimately arbitrary, but can be appropriate if smaller output streams are used for image analysis rather than human consumption, when the precise colours may be less important. Signed-off-by: David Plowman Reviewed-by: Laurent Pinchart Reviewed-by: Jacopo Mondi --- include/libcamera/camera.h | 10 +++++ src/libcamera/camera.cpp | 83 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 93 insertions(+) diff --git a/include/libcamera/camera.h b/include/libcamera/camera.h index a7759ccb..5bb06584 100644 --- a/include/libcamera/camera.h +++ b/include/libcamera/camera.h @@ -13,6 +13,7 @@ #include #include +#include #include #include @@ -69,6 +70,15 @@ public: protected: CameraConfiguration(); + enum class ColorSpaceFlag { + None, + StreamsShareColorSpace, + }; + + using ColorSpaceFlags = Flags; + + Status validateColorSpaces(ColorSpaceFlags flags = ColorSpaceFlag::None); + std::vector config_; }; diff --git a/src/libcamera/camera.cpp b/src/libcamera/camera.cpp index 400a7cf0..86d84ac0 100644 --- a/src/libcamera/camera.cpp +++ b/src/libcamera/camera.cpp @@ -14,12 +14,14 @@ #include #include +#include #include #include #include #include "libcamera/internal/camera.h" #include "libcamera/internal/camera_controls.h" +#include "libcamera/internal/formats.h" #include "libcamera/internal/pipeline_handler.h" /** @@ -314,6 +316,87 @@ std::size_t CameraConfiguration::size() const return config_.size(); } +namespace { + +bool isRaw(const PixelFormat &pixFmt) +{ + const PixelFormatInfo &info = PixelFormatInfo::info(pixFmt); + return info.isValid() && + info.colourEncoding == PixelFormatInfo::ColourEncodingRAW; +} + +} /* namespace */ + +/** + * \enum CameraConfiguration::ColorSpaceFlag + * \brief Specify the behaviour of validateColorSpaces + * \var CameraConfiguration::ColorSpaceFlag::None + * \brief No extra validation of color spaces is required + * \var CameraConfiguration::ColorSpaceFlag::StreamsShareColorSpace + * \brief Non-raw output streams must share the same color space + */ + +/** + * \typedef CameraConfiguration::ColorSpaceFlags + * \brief A bitwise combination of ColorSpaceFlag values + */ + +/** + * \brief Check the color spaces requested for each stream + * \param[in] flags Flags to control the behaviour of this function + * + * This function performs certain consistency checks on the color spaces of + * the streams and may adjust them so that: + * + * - Any raw streams have the Raw color space + * - If the StreamsShareColorSpace flag is set, all output streams are forced + * to share the same color space (this may be a constraint on some platforms). + * + * It is optional for a pipeline handler to use this function. + * + * \return A CameraConfiguration::Status value that describes the validation + * status. + * \retval CameraConfigutation::Adjusted The configuration has been adjusted + * and is now valid. The color space of some or all of the streams may bave + * benn changed. The caller shall check the color spaces carefully. + * \retval CameraConfiguration::Valid The configuration was already valid and + * hasn't been adjusted. + */ +CameraConfiguration::Status CameraConfiguration::validateColorSpaces(ColorSpaceFlags flags) +{ + Status status = Valid; + + /* + * Set all raw streams to the Raw color space, and make a note of the largest + * non-raw stream with a defined color space (if there is one). + */ + int index = -1; + for (auto [i, cfg] : utils::enumerate(config_)) { + if (isRaw(cfg.pixelFormat)) { + if (cfg.colorSpace != ColorSpace::Raw) { + cfg.colorSpace = ColorSpace::Raw; + status = Adjusted; + } + } else if (cfg.colorSpace && (index == -1 || cfg.size > config_[i].size)) { + index = i; + } + } + + if (index < 0 || !(flags & ColorSpaceFlag::StreamsShareColorSpace)) + return status; + + /* Make all output color spaces the same, if requested. */ + for (auto &cfg : config_) { + if (!isRaw(cfg.pixelFormat) && + cfg.colorSpace != config_[index].colorSpace) { + cfg.colorSpace = config_[index].colorSpace; + status = Adjusted; + } + } + + return status; +} + /** * \var CameraConfiguration::transform * \brief User-specified transform to be applied to the image From patchwork Fri Dec 10 14:44:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 15121 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9540CC3258 for ; Fri, 10 Dec 2021 14:44:49 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 409876089C; Fri, 10 Dec 2021 15:44:49 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="OE3QI9Vx"; dkim-atps=neutral Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id F240E6089C for ; Fri, 10 Dec 2021 15:44:42 +0100 (CET) Received: by mail-wr1-x42c.google.com with SMTP id u17so15331806wrt.3 for ; Fri, 10 Dec 2021 06:44:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KZ7YYPVoutlpB46wp8p2aCslrMH3MD8i9QfwjpK9yYY=; b=OE3QI9VxqPoo7gygVi6UiKDM6uccEGSFnWqkuZmby10IFcQBimjUZMZak0Jy9Vs2a1 5fv88xX5DCbOBnpKynvDk+FmAhkGpDmbx5GQY1YGJeXop1q2W7yXljfU0ebW9BVF8FUU FbqL9DQfDCzRs9q8lKmKPCR9bYp7LH9Sv/eJI4aZrFafarxjORhwgZGtsoFCB/d+r3cu 95Xw7iJFM2UMZHnZScguZE9GRm7V+RLX7V2YIIivL9afk/C5FhiTNFXkI8OIm+7eo7aR 2N2hxyDKxL9VdMKEfr2bwKF4NCAHt3+Dx1YybuAq02ARG8rWvlYhLKSF6Xu7V2LJa1zd azrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KZ7YYPVoutlpB46wp8p2aCslrMH3MD8i9QfwjpK9yYY=; b=tbaHrxVuDCAKmRNodBHWXezmxcOmKXxd7XOX/2jz9eUzOc59qa1l6PrRClbwhOfOjP wzlkw0t72F4KGrMS2ngbzqIXmTskHwS9LZX0JTIknkKum5zWmrLaPs6jFveQlZlCtIXl qRbL/fDVAafPKMBIUPyv9N2Q656yU/452KGmaeoAQ+BfmQMYvrTo3XQ9K92gEzFKML0T JQT3nLEPd4OO1H2fiF57pJ+mDt/AMXWbJvOm9Ms8v04nzYME4eZSb03u2W+3EFckLZC9 K6bjupjwFS8/Y8oYSY5HV4qxb0immRzcZ7ETAj1Toqq85zf5CcAB9n5Dw4G1f3X4l9ME 1qZQ== X-Gm-Message-State: AOAM532cIvDSK2ldhgM9yG24Uhqu0WqMXvDPXDqgBLuOFUXiZyGfdHYY sdwBHutPbcE5Hgijq8zykThSkA== X-Google-Smtp-Source: ABdhPJxGLb9No8A+YAjh4BKV22+TypXUygtGWrKApAwP4TJDKZNoQOCFLbv8s0F7FRCPdoyjH5A6oQ== X-Received: by 2002:adf:f088:: with SMTP id n8mr13995938wro.411.1639147482694; Fri, 10 Dec 2021 06:44:42 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:e4a2:3070:eea4:e434]) by smtp.gmail.com with ESMTPSA id w22sm3000515wmi.27.2021.12.10.06.44.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Dec 2021 06:44:42 -0800 (PST) From: David Plowman To: Laurent Pinchart , Kieran Bingham , Hans Verkuil , Tomasz Figa , Jacopo Mondi , Naushir Patuck , libcamera-devel@lists.libcamera.org Date: Fri, 10 Dec 2021 14:44:24 +0000 Message-Id: <20211210144424.14747-9-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211210144424.14747-1-david.plowman@raspberrypi.com> References: <20211210144424.14747-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v12 8/8] libcamera: pipeline: raspberrypi: Support color spaces X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The Raspberry Pi pipeline handler now sets color spaces correctly. In generateConfiguration() it sets them to reasonable default values based on the stream role. validate() now calls validateColorSpaces() to ensure that the requested color spaces are sensible, before proceeding to check what the hardware can deliver. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck Reviewed-by: Laurent Pinchart Reviewed-by: Jacopo Mondi --- .../pipeline/raspberrypi/raspberrypi.cpp | 40 +++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 86851ac4..eb74d96c 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -96,6 +96,7 @@ V4L2DeviceFormat toV4L2DeviceFormat(const V4L2SubdeviceFormat &format, deviceFormat.fourcc = V4L2PixelFormat::fromPixelFormat(pix); deviceFormat.size = format.size; + deviceFormat.colorSpace = format.colorSpace; return deviceFormat; } @@ -132,6 +133,7 @@ V4L2SubdeviceFormat findBestFormat(const SensorFormats &formatsMap, const Size & { double bestScore = std::numeric_limits::max(), score; V4L2SubdeviceFormat bestFormat; + bestFormat.colorSpace = ColorSpace::Raw; constexpr float penaltyAr = 1500.0; constexpr float penaltyBitDepth = 500.0; @@ -329,6 +331,8 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() if (config_.empty()) return Invalid; + status = validateColorSpaces(ColorSpaceFlag::StreamsShareColorSpace); + /* * What if the platform has a non-90 degree rotation? We can't even * "adjust" the configuration and carry on. Alternatively, raising an @@ -496,11 +500,25 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() V4L2DeviceFormat format; format.fourcc = V4L2PixelFormat::fromPixelFormat(cfg.pixelFormat); format.size = cfg.size; + format.colorSpace = cfg.colorSpace; + + LOG(RPI, Debug) + << "Try color space " << ColorSpace::toString(cfg.colorSpace); int ret = dev->tryFormat(&format); if (ret) return Invalid; + if (!format.colorSpace || cfg.colorSpace != format.colorSpace) { + status = Adjusted; + LOG(RPI, Debug) + << "Color space changed from " + << ColorSpace::toString(cfg.colorSpace) << " to " + << ColorSpace::toString(format.colorSpace); + } + + cfg.colorSpace = format.colorSpace; + cfg.stride = format.planes[0].bpl; cfg.frameSize = format.planes[0].size; @@ -524,6 +542,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, PixelFormat pixelFormat; V4L2VideoDevice::Formats fmts; Size size; + std::optional colorSpace; if (roles.empty()) return config; @@ -539,6 +558,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, pixelFormat = mbusCodeToPixelFormat(sensorFormat.mbus_code, BayerFormat::Packing::CSI2); ASSERT(pixelFormat.isValid()); + colorSpace = ColorSpace::Raw; bufferCount = 2; rawCount++; break; @@ -546,6 +566,12 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, case StreamRole::StillCapture: fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::NV12; + /* + * Still image codecs usually expect the JPEG color space. + * Even RGB codecs will be fine as the RGB we get with the + * JPEG color space is the same as sRGB. + */ + colorSpace = ColorSpace::Jpeg; /* Return the largest sensor resolution. */ size = sensorSize; bufferCount = 1; @@ -563,6 +589,11 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, */ fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::YUV420; + /* + * Choose a color space appropriate for video recording. + * Rec.709 will be a good default for HD resolutions. + */ + colorSpace = ColorSpace::Rec709; size = { 1920, 1080 }; bufferCount = 4; outCount++; @@ -571,6 +602,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, case StreamRole::Viewfinder: fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::ARGB8888; + colorSpace = ColorSpace::Jpeg; size = { 800, 600 }; bufferCount = 4; outCount++; @@ -617,6 +649,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, StreamConfiguration cfg(formats); cfg.size = size; cfg.pixelFormat = pixelFormat; + cfg.colorSpace = colorSpace; cfg.bufferCount = bufferCount; config->addConfiguration(cfg); } @@ -724,6 +757,7 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) V4L2PixelFormat fourcc = V4L2PixelFormat::fromPixelFormat(cfg.pixelFormat); format.size = cfg.size; format.fourcc = fourcc; + format.colorSpace = cfg.colorSpace; LOG(RPI, Debug) << "Setting " << stream->name() << " to " << format.toString(); @@ -739,6 +773,10 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) return -EINVAL; } + LOG(RPI, Debug) + << "Stream " << stream->name() << " has color space " + << ColorSpace::toString(cfg.colorSpace); + cfg.setStream(stream); stream->setExternal(true); @@ -763,6 +801,8 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) format = {}; format.size = maxSize; format.fourcc = V4L2PixelFormat::fromPixelFormat(formats::YUV420); + /* No one asked for output, so the color space doesn't matter. */ + format.colorSpace = ColorSpace::Jpeg; ret = data->isp_[Isp::Output0].dev()->setFormat(&format); if (ret) { LOG(RPI, Error)