From patchwork Fri Dec 10 11:21:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 15105 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0A0BABF415 for ; Fri, 10 Dec 2021 11:22:08 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A0EFA6089F; Fri, 10 Dec 2021 12:22:07 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="n7Pj6Rks"; dkim-atps=neutral Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com [IPv6:2a00:1450:4864:20::32d]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 35BA76088E for ; Fri, 10 Dec 2021 12:22:04 +0100 (CET) Received: by mail-wm1-x32d.google.com with SMTP id c6-20020a05600c0ac600b0033c3aedd30aso6357393wmr.5 for ; Fri, 10 Dec 2021 03:22:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=GlLAW3bdu7s4XPs+nBmS7UoYh+MwliNpxbCB/j0SLSA=; b=n7Pj6Rks4P9yFHH8Tn1BJFLeQhLsxk+F6EMaTL2l5phvhsXniUSDNOfljZqK+yaiFj 9olzQR4EbRy6KxIdIXwriu+oLduZcgIVpp/0NPHbqQhdaf7qSFHvJLa/qs+1/5zGubzw oNYPOK7uV0rDMZgu4XOFlTjur1J1WoRilUMPB5Lg28pdw6l6jUXmWX2RClyESAPBoqHn rZkR7q5rc0XIPlEuI61GdgEPpyV8t77Y5BM4lxRLxJgg8s3rPj4U0XRHr2mWWcU5FmlA gWN16yJrDtRwVCQC9y6I2y5S8/EShgCCR1DNCbxMMKIst0wUI707esHKYwINh2JuJseH tZSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=GlLAW3bdu7s4XPs+nBmS7UoYh+MwliNpxbCB/j0SLSA=; b=dL4SnvqOIMg9M1ciPVSv9o0b09bMUiiyI60HCAAL/P6dA4dvjCIrVycqdqXKU5cDqu IxGfq73IeV0oW8mxaM3m40s4Y1kTf9Xgp9mO3193HX51k8Ia8j9jewPH/4eguS3F4mde HKnSU4hy8ADW/uHGMxn/7PGlzlNg0ju43usX9vgVXglfrJanMo6BCxDAravoBHRp30h8 V/cYbtIhb/Y2iPqPE52z9dhXjq6Dan/qOOOlk55hHrkF+KdOA743VZ2hOSu4pVgOFXEY DxbBBL9VDyFyj7opvIZCuCuqTvw0Y+qxJv48kpw6mRkBWzq2IsmvN6RkCGylltN0UUVP 7Hug== X-Gm-Message-State: AOAM532Slra6wBIHood6kqfJV7U+jtzOh7FnabNxNikjfSeODmlXArRu rkIIPIA6u9VI0TTZeiCwEz/ILA== X-Google-Smtp-Source: ABdhPJypsLKn+Bl67bBkYUHqpk9BiR8Aby9o9LV3S5ZT0/aiUMPb9Ts86+bs6FEmqNsQvYPHMzsymQ== X-Received: by 2002:a7b:c256:: with SMTP id b22mr15707461wmj.176.1639135323740; Fri, 10 Dec 2021 03:22:03 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:e4a2:3070:eea4:e434]) by smtp.gmail.com with ESMTPSA id z18sm2198469wrq.11.2021.12.10.03.22.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Dec 2021 03:22:03 -0800 (PST) From: David Plowman To: Laurent Pinchart , Kieran Bingham , Hans Verkuil , Tomasz Figa , Jacopo Mondi , Naushir Patuck , libcamera-devel@lists.libcamera.org Date: Fri, 10 Dec 2021 11:21:35 +0000 Message-Id: <20211210112142.18441-2-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211210112142.18441-1-david.plowman@raspberrypi.com> References: <20211210112142.18441-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v11 1/8] libcamera: Add ColorSpace class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This class represents a color space by defining its color primaries, the transfer (gamma) function it uses, the YCbCr encoding and whether the output is full or limited range. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck Reviewed-by: Umang Jain Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- include/libcamera/color_space.h | 70 +++++++ include/libcamera/meson.build | 1 + src/libcamera/color_space.cpp | 317 ++++++++++++++++++++++++++++++++ src/libcamera/meson.build | 1 + 4 files changed, 389 insertions(+) create mode 100644 include/libcamera/color_space.h create mode 100644 src/libcamera/color_space.cpp diff --git a/include/libcamera/color_space.h b/include/libcamera/color_space.h new file mode 100644 index 00000000..086c56c1 --- /dev/null +++ b/include/libcamera/color_space.h @@ -0,0 +1,70 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Raspberry Pi (Trading) Limited + * + * color_space.h - color space definitions + */ + +#pragma once + +#include +#include + +namespace libcamera { + +class ColorSpace +{ +public: + enum class Primaries { + Raw, + Smpte170m, + Rec709, + Rec2020, + }; + + enum class TransferFunction { + Linear, + Srgb, + Rec709, + }; + + enum class YcbcrEncoding { + None, + Rec601, + Rec709, + Rec2020, + }; + + enum class Range { + Full, + Limited, + }; + + constexpr ColorSpace(Primaries p, TransferFunction t, YcbcrEncoding e, Range r) + : primaries(p), transferFunction(t), ycbcrEncoding(e), range(r) + { + } + + static const ColorSpace Raw; + static const ColorSpace Jpeg; + static const ColorSpace Srgb; + static const ColorSpace Smpte170m; + static const ColorSpace Rec709; + static const ColorSpace Rec2020; + + Primaries primaries; + TransferFunction transferFunction; + YcbcrEncoding ycbcrEncoding; + Range range; + + std::string toString() const; + static std::string toString(const std::optional &colorSpace); +}; + +bool operator==(const ColorSpace &lhs, const ColorSpace &rhs); +static inline bool operator!=(const ColorSpace &lhs, const ColorSpace &rhs) +{ + return !(lhs == rhs); +} + +} /* namespace libcamera */ diff --git a/include/libcamera/meson.build b/include/libcamera/meson.build index 5f42977c..fd767b11 100644 --- a/include/libcamera/meson.build +++ b/include/libcamera/meson.build @@ -5,6 +5,7 @@ libcamera_include_dir = 'libcamera' / 'libcamera' libcamera_public_headers = files([ 'camera.h', 'camera_manager.h', + 'color_space.h', 'controls.h', 'framebuffer.h', 'framebuffer_allocator.h', diff --git a/src/libcamera/color_space.cpp b/src/libcamera/color_space.cpp new file mode 100644 index 00000000..85524b96 --- /dev/null +++ b/src/libcamera/color_space.cpp @@ -0,0 +1,317 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Raspberry Pi (Trading) Limited + * + * color_space.cpp - color spaces. + */ + +#include + +#include +#include +#include +#include +#include + +/** + * \file color_space.h + * \brief Class and enums to represent color spaces + */ + +namespace libcamera { + +/** + * \class ColorSpace + * \brief Class to describe a color space + * + * The ColorSpace class defines the color primaries, the Y'CbCr encoding, + * the transfer function associated with the color space, and the range + * (sometimes also referred to as the quantisation) of the color space. + * + * Certain combinations of these fields form well-known standard color + * spaces such as "JPEG" or "REC709". + * + * In the strictest sense a "color space" formally only refers to the + * color primaries and white point. Here, however, the ColorSpace class + * adopts the common broader usage that includes the transfer function, + * Y'CbCr encoding method and quantisation. + * + * For more information on the specific color spaces described here, please + * see: + * + * - sRGB + * - JPEG + * - SMPTE 170M + * - Rec.709 + * - Rec.2020 + */ + +/** + * \enum ColorSpace::Primaries + * \brief The color primaries for this color space + * + * \var ColorSpace::Primaries::Raw + * \brief These are raw colors directly from a sensor, the primaries are + * unspecified + * + * \var ColorSpace::Primaries::Smpte170m + * \brief SMPTE 170M color primaries + * + * \var ColorSpace::Primaries::Rec709 + * \brief Rec.709 color primaries + * + * \var ColorSpace::Primaries::Rec2020 + * \brief Rec.2020 color primaries + */ + +/** + * \enum ColorSpace::TransferFunction + * \brief The transfer function used for this color space + * + * \var ColorSpace::TransferFunction::Linear + * \brief This color space uses a linear (identity) transfer function + * + * \var ColorSpace::TransferFunction::Srgb + * \brief sRGB transfer function + * + * \var ColorSpace::TransferFunction::Rec709 + * \brief Rec.709 transfer function + */ + +/** + * \enum ColorSpace::YcbcrEncoding + * \brief The Y'CbCr encoding + * + * \var ColorSpace::YcbcrEncoding::None + * \brief There is no defined Y'CbCr encoding + * + * \var ColorSpace::YcbcrEncoding::Rec601 + * \brief Rec.601 Y'CbCr encoding + * + * \var ColorSpace::YcbcrEncoding::Rec709 + * \brief Rec.709 Y'CbCr encoding + * + * \var ColorSpace::YcbcrEncoding::Rec2020 + * \brief Rec.2020 Y'CbCr encoding + */ + +/** + * \enum ColorSpace::Range + * \brief The range (sometimes "quantisation") for this color space + * + * \var ColorSpace::Range::Full + * \brief This color space uses full range pixel values + * + * \var ColorSpace::Range::Limited + * \brief This color space uses limited range pixel values, being + * 16 to 235 for Y' and 16 to 240 for Cb and Cr (8 bits per sample) + * or 64 to 940 for Y' and 16 to 960 for Cb and Cr (10 bits) + */ + +/** + * \fn ColorSpace::ColorSpace(Primaries p, TransferFunction t, Encoding e, Range r) + * \brief Construct a ColorSpace from explicit values + * \param[in] p The color primaries + * \param[in] t The transfer function for the color space + * \param[in] e The Y'CbCr encoding + * \param[in] r The range of the pixel values in this color space + */ + +/** + * \brief Assemble and return a readable string representation of the + * ColorSpace + * + * If the color space matches a standard ColorSpace (such as ColorSpace::Jpeg) + * then the short name of the color space ("JPEG") is returned. Otherwise + * the four constituent parts of the ColorSpace are assembled into a longer + * string. + * + * \return A string describing the ColorSpace + */ +std::string ColorSpace::toString() const +{ + /* Print out a brief name only for standard color spaces. */ + + static const std::array, 6> colorSpaceNames = { { + { ColorSpace::Raw, "RAW" }, + { ColorSpace::Jpeg, "JPEG" }, + { ColorSpace::Srgb, "sRGB" }, + { ColorSpace::Smpte170m, "SMPTE170M" }, + { ColorSpace::Rec709, "Rec709" }, + { ColorSpace::Rec2020, "Rec2020" }, + } }; + auto it = std::find_if(colorSpaceNames.begin(), colorSpaceNames.end(), + [this](const auto &item) { + return *this == item.first; + }); + if (it != colorSpaceNames.end()) + return std::string(it->second); + + /* Assemble a name made of the constituent fields. */ + + static const std::map primariesNames = { + { Primaries::Raw, "RAW" }, + { Primaries::Smpte170m, "SMPTE170M" }, + { Primaries::Rec709, "Rec709" }, + { Primaries::Rec2020, "Rec2020" }, + }; + static const std::map transferNames = { + { TransferFunction::Linear, "Linear" }, + { TransferFunction::Srgb, "sRGB" }, + { TransferFunction::Rec709, "Rec709" }, + }; + static const std::map encodingNames = { + { YcbcrEncoding::None, "None" }, + { YcbcrEncoding::Rec601, "Rec601" }, + { YcbcrEncoding::Rec709, "Rec709" }, + { YcbcrEncoding::Rec2020, "Rec2020" }, + }; + static const std::map rangeNames = { + { Range::Full, "Full" }, + { Range::Limited, "Limited" }, + }; + + auto itPrimaries = primariesNames.find(primaries); + std::string primariesName = + itPrimaries == primariesNames.end() ? "Invalid" : itPrimaries->second; + + auto itTransfer = transferNames.find(transferFunction); + std::string transferName = + itTransfer == transferNames.end() ? "Invalid" : itTransfer->second; + + auto itEncoding = encodingNames.find(ycbcrEncoding); + std::string encodingName = + itEncoding == encodingNames.end() ? "Invalid" : itEncoding->second; + + auto itRange = rangeNames.find(range); + std::string rangeName = + itRange == rangeNames.end() ? "Invalid" : itRange->second; + + std::stringstream ss; + ss << primariesName << "/" << transferName << "/" << encodingName << "/" << rangeName; + + return ss.str(); +} + +/** + * \brief Assemble and return a readable string representation of an + * optional ColorSpace + * + * This is a convenience helper to easily obtain a string representation + * for a ColorSpace in parts of the libcamera API where it is stored in a + * std::optional<>. If the ColorSpace is set, this function returns + * \a colorSpace.toString(), otherwise it returns "Unset". + * + * \return A string describing the optional ColorSpace + */ +std::string ColorSpace::toString(const std::optional &colorSpace) +{ + if (!colorSpace) + return "Unset"; + + return colorSpace->toString(); +} + +/** + * \var ColorSpace::primaries + * \brief The color primaries of this color space + */ + +/** + * \var ColorSpace::ycbcrEncoding + * \brief The Y'CbCr encoding used by this color space + */ + +/** + * \var ColorSpace::transferFunction + * \brief The transfer function used by this color space + */ + +/** + * \var ColorSpace::range + * \brief The pixel range used with by color space + */ + +/** + * \brief A constant representing a raw color space (from a sensor) + */ +const ColorSpace ColorSpace::Raw = { + Primaries::Raw, + TransferFunction::Linear, + YcbcrEncoding::None, + Range::Full +}; + +/** + * \brief A constant representing the JPEG color space used for + * encoding JPEG images + */ +const ColorSpace ColorSpace::Jpeg = { + Primaries::Rec709, + TransferFunction::Srgb, + YcbcrEncoding::Rec601, + Range::Full +}; + +/** + * \brief A constant representing the sRGB color space. + * + * This is identical to the JPEG color space except that the Y'CbCr + * range is limited rather than full. + */ +const ColorSpace ColorSpace::Srgb = { + Primaries::Rec709, + TransferFunction::Srgb, + YcbcrEncoding::Rec601, + Range::Limited +}; + +/** + * \brief A constant representing the SMPTE170M color space + */ +const ColorSpace ColorSpace::Smpte170m = { + Primaries::Smpte170m, + TransferFunction::Rec709, + YcbcrEncoding::Rec601, + Range::Limited +}; + +/** + * \brief A constant representing the Rec.709 color space + */ +const ColorSpace ColorSpace::Rec709 = { + Primaries::Rec709, + TransferFunction::Rec709, + YcbcrEncoding::Rec709, + Range::Limited +}; + +/** + * \brief A constant representing the Rec.2020 color space + */ +const ColorSpace ColorSpace::Rec2020 = { + Primaries::Rec2020, + TransferFunction::Rec709, + YcbcrEncoding::Rec2020, + Range::Limited +}; + +/** + * \brief Compare color spaces for equality + * \return True if the two color spaces are identical, false otherwise + */ +bool operator==(const ColorSpace &lhs, const ColorSpace &rhs) +{ + return lhs.primaries == rhs.primaries && + lhs.transferFunction == rhs.transferFunction && + lhs.ycbcrEncoding == rhs.ycbcrEncoding && + lhs.range == rhs.range; +} + +/** + * \fn bool operator!=(const ColorSpace &lhs, const ColorSpace &rhs) + * \brief Compare color spaces for inequality + * \return True if the two color spaces are not identical, false otherwise + */ + +} /* namespace libcamera */ diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build index 2e54cc04..4045e24e 100644 --- a/src/libcamera/meson.build +++ b/src/libcamera/meson.build @@ -9,6 +9,7 @@ libcamera_sources = files([ 'camera_manager.cpp', 'camera_sensor.cpp', 'camera_sensor_properties.cpp', + 'color_space.cpp', 'controls.cpp', 'control_serializer.cpp', 'control_validator.cpp', From patchwork Fri Dec 10 11:21:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 15106 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 58F33C324B for ; Fri, 10 Dec 2021 11:22:09 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 217DD60894; Fri, 10 Dec 2021 12:22:08 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="GKeGqEeu"; dkim-atps=neutral Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A40816088E for ; Fri, 10 Dec 2021 12:22:04 +0100 (CET) Received: by mail-wr1-x434.google.com with SMTP id d9so14376782wrw.4 for ; Fri, 10 Dec 2021 03:22:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=+sH0KfN2WD6NxZb8aJTKXBPu6mnk13ViaZCwPQX529M=; b=GKeGqEeu4r02zZYe5dsi5gaBXsyflqlPT6c3PFmdPpsvn0t7edGGvJ4k1E6q0SqtGo SEfKaVE13RjjQ4d2J9CidSx0XdzXEiQcbyu4XvOtZEHM4numObiVsRmwIKzWPE8F3EeU Gj32dIHu0a2tnCBwwIIOeRt5cepho2UC8rTVfMW4KyUcrGbDC3jOjQaZhyw06VVfFpW8 vB0LKnv0OyFWd5OcDfsnMrlJMAwUWyJd0TMwg/5UtA4q6vT5EZJ8dwExio+R8z2KBYtl TvwrX+Ywkc/gpX+aY0ZkbWafDz1jjvN2SMQqXyN5LgVEqJ9KYBHgI+JJsYHNIrUfuwm0 olxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+sH0KfN2WD6NxZb8aJTKXBPu6mnk13ViaZCwPQX529M=; b=uU3cwSpYxcbXiT1RQ34Ope0qaMLUbS0lKfbVv5XcaiGGGYGRedda9xNjm12krz4BOe tHTfNc71kLP0BohpHbUqfZeROJK/l3lepKSfRtucRJZB/hirKnaHliWALJTX+TlDIMWb mCxbMnF1u+UBuDW/b/+0HCWk+hXnU+0QErYQKIIR+xeLEYvZjE1BBLgo1FYGLJfhrsBH pHTEao2mPYKJvWPZOoI3Poc+RUmK0wA2qGYbhuAOBH3VOpeSQqoZ79ZaZB9Op/X5tggf R0Ooymg/8lgvexD9sQfYc9lLKzpb4IbbSOcypWH/r8cg4tX13DJ/BbHAKHPKMrY4ip3s SMKg== X-Gm-Message-State: AOAM530VKMuXOTlY6f6j3YbxNLREHETChiGU6vOQ4E3Q2+q/pQChG6rH KvTO6dNlfg9L+KOmJHqiKafAMg== X-Google-Smtp-Source: ABdhPJwvBEEYMiXgG9+OlHk3xvU1ggacccQlXfAA7WCNZrOvCALYHzNWwoHT3DrVN5WMS59kwm8fcQ== X-Received: by 2002:a05:6000:1862:: with SMTP id d2mr13608868wri.251.1639135324306; Fri, 10 Dec 2021 03:22:04 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:e4a2:3070:eea4:e434]) by smtp.gmail.com with ESMTPSA id z18sm2198469wrq.11.2021.12.10.03.22.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Dec 2021 03:22:04 -0800 (PST) From: David Plowman To: Laurent Pinchart , Kieran Bingham , Hans Verkuil , Tomasz Figa , Jacopo Mondi , Naushir Patuck , libcamera-devel@lists.libcamera.org Date: Fri, 10 Dec 2021 11:21:36 +0000 Message-Id: <20211210112142.18441-3-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211210112142.18441-1-david.plowman@raspberrypi.com> References: <20211210112142.18441-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v11 2/8] libcamera: stream: Add ColorSpace fields to StreamConfiguration X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This is so that applications can choose appropriate color spaces which will then be passed down to the V4L2 devices. The ColorSpace field is actually optional. If it is not set you will get the camera's default color space. Signed-off-by: David Plowman Reviewed-by: Umang Jain Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- include/libcamera/stream.h | 3 +++ src/libcamera/stream.cpp | 20 ++++++++++++++++++++ 2 files changed, 23 insertions(+) diff --git a/include/libcamera/stream.h b/include/libcamera/stream.h index 41ec02b1..f0ae7e62 100644 --- a/include/libcamera/stream.h +++ b/include/libcamera/stream.h @@ -12,6 +12,7 @@ #include #include +#include #include #include #include @@ -47,6 +48,8 @@ struct StreamConfiguration { unsigned int bufferCount; + std::optional colorSpace; + Stream *stream() const { return stream_; } void setStream(Stream *stream) { stream_ = stream; } const StreamFormats &formats() const { return formats_; } diff --git a/src/libcamera/stream.cpp b/src/libcamera/stream.cpp index b421e17e..686e693b 100644 --- a/src/libcamera/stream.cpp +++ b/src/libcamera/stream.cpp @@ -329,6 +329,26 @@ StreamConfiguration::StreamConfiguration(const StreamFormats &formats) * \brief Requested number of buffers to allocate for the stream */ +/** + * \var StreamConfiguration::colorSpace + * \brief The ColorSpace for this stream + * + * This field allows a ColorSpace to be selected for this Stream. + * + * The field is optional and an application can choose to leave it unset. + * Platforms that support the use of color spaces may provide default + * values through the generateConfiguration() method. An application can + * override these when necessary. + * + * If a specific ColorSpace is requested but the Camera cannot deliver it, + * then the StreamConfiguration will be adjusted to a value that can be + * delivered. In this case the validate() method will indicate via its + * return value that the CameraConfiguration has been adjusted. + * + * Note that platforms will typically have different constraints on what + * color spaces can be supported and in what combinations. + */ + /** * \fn StreamConfiguration::stream() * \brief Retrieve the stream associated with the configuration From patchwork Fri Dec 10 11:21:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 15107 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C56FDBF415 for ; Fri, 10 Dec 2021 11:22:10 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 797826088E; Fri, 10 Dec 2021 12:22:10 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="n2tGcmVD"; dkim-atps=neutral Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7BA7360890 for ; Fri, 10 Dec 2021 12:22:05 +0100 (CET) Received: by mail-wr1-x434.google.com with SMTP id v11so14323083wrw.10 for ; Fri, 10 Dec 2021 03:22:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=w+iu8ypbrVc2RhyRChADW61dJAYHTqFv/toJl3WsmjE=; b=n2tGcmVDPm+Ip3mLX6l3A0yBG/DENjXXAA+O7J2I1nJQ6Ai+TT25C6Bj02xsYdsjlf DjN6r7BjdT+Y+Lok/A+H7cctI07LyWcSlnD/C5GQIOyhyae+DQZ+LOixADVcLYHPngvY TMhcxIPuc3GhpLrDHmRLVZevYTTV1k05/5SdLKBoiCgNHjqBOWw9Zb7jYGTq7RwTWvhH 4EJGyUI7D+44XWZ5GqDTMjGISbVnSOk7EFH2d8VWKskWqrlYFrNPCstfZzxiTkiN5HkI s+XQBx4vsqw6b+JxthbPvOCo4fi25UmtklDFNk/inj90bU/BLQWfRW1Dnxc5D+H0tX0L YQsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=w+iu8ypbrVc2RhyRChADW61dJAYHTqFv/toJl3WsmjE=; b=Bk0ygIj+4y6jbAE/ilsGmvXI2TWI86H0O6acAT3XGtJnYSpdeh0gg/sJsEfoKTfybh QrJKDgAeua6Ik7oK9aIv2LESv2dH3+ciUFj3Mk79XuSvCwVq4ZTr3eW95tByoq0ZOt/D BfYvWEhBFBwwJbctJxlbi9bWeoVuV9HawmPux+Cs3mjIrEjUsMCzEqjLBATfqEvheWTy jHhY5jG5tUnOhH2bf93na8UD4GQBGit5JXQwCLT2bIMZi82aZK1Jxmnuru7fhAsTZBdQ ZoPRpNFoyDTuBGxOCdBoMueaSciWwG2exrA/Gbg0If51yuRaN2fgDG9clO+ZhVCPZZZp g+3g== X-Gm-Message-State: AOAM532URg3speiCWL56i8IkEMkIqbM8munvh9pIHZmSzguonwzmguV3 5w2Zp4WkMYFT6WRKZ2xZ25pD5A== X-Google-Smtp-Source: ABdhPJxQMwCOxv0AQyg/2Je+Q0JnctLAWI6cU3B6r07DMSTWFZaAfok5De5kp3xLqPakf7AP38qo5A== X-Received: by 2002:a5d:68c1:: with SMTP id p1mr12925863wrw.585.1639135325062; Fri, 10 Dec 2021 03:22:05 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:e4a2:3070:eea4:e434]) by smtp.gmail.com with ESMTPSA id z18sm2198469wrq.11.2021.12.10.03.22.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Dec 2021 03:22:04 -0800 (PST) From: David Plowman To: Laurent Pinchart , Kieran Bingham , Hans Verkuil , Tomasz Figa , Jacopo Mondi , Naushir Patuck , libcamera-devel@lists.libcamera.org Date: Fri, 10 Dec 2021 11:21:37 +0000 Message-Id: <20211210112142.18441-4-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211210112142.18441-1-david.plowman@raspberrypi.com> References: <20211210112142.18441-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v11 3/8] libcamera: video_device: Convert between ColorSpace class and V4L2 formats X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add functions to the V4L2Device class to convert to and from libcamera ColorSpace. These functions are added to the base V4L2Device class so that they can be shared both by the video device class and subdevices. With the ColorSpace class, the color space and related other fields are stored together, corresponding to a number of fields in the various different V4L2 format structures. Template functions are therefore a convenient implementation, and we must explicitly instantiate the templates that will be needed. Note that unset color spaces are converted to requests for the device's "default" color space. Signed-off-by: David Plowman Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- include/libcamera/internal/v4l2_device.h | 8 + src/libcamera/v4l2_device.cpp | 194 +++++++++++++++++++++++ 2 files changed, 202 insertions(+) diff --git a/include/libcamera/internal/v4l2_device.h b/include/libcamera/internal/v4l2_device.h index 8886b750..a52a5f2c 100644 --- a/include/libcamera/internal/v4l2_device.h +++ b/include/libcamera/internal/v4l2_device.h @@ -9,6 +9,7 @@ #include #include +#include #include #include @@ -18,6 +19,7 @@ #include #include +#include #include namespace libcamera { @@ -56,6 +58,12 @@ protected: int fd() const { return fd_.get(); } + template + static std::optional toColorSpace(const T &v4l2Format); + + template + static int fromColorSpace(const std::optional &colorSpace, T &v4l2Format); + private: static ControlType v4l2CtrlType(uint32_t ctrlType); static std::unique_ptr v4l2ControlId(const v4l2_query_ext_ctrl &ctrl); diff --git a/src/libcamera/v4l2_device.cpp b/src/libcamera/v4l2_device.cpp index 39f36009..3e8a180f 100644 --- a/src/libcamera/v4l2_device.cpp +++ b/src/libcamera/v4l2_device.cpp @@ -10,11 +10,15 @@ #include #include #include +#include #include #include #include #include #include +#include + +#include #include #include @@ -728,4 +732,194 @@ void V4L2Device::eventAvailable() frameStart.emit(event.u.frame_sync.frame_sequence); } +static const std::map v4l2ToColorSpace = { + { V4L2_COLORSPACE_RAW, ColorSpace::Raw }, + { V4L2_COLORSPACE_JPEG, ColorSpace::Jpeg }, + { V4L2_COLORSPACE_SRGB, ColorSpace::Srgb }, + { V4L2_COLORSPACE_SMPTE170M, ColorSpace::Smpte170m }, + { V4L2_COLORSPACE_REC709, ColorSpace::Rec709 }, + { V4L2_COLORSPACE_BT2020, ColorSpace::Rec2020 }, +}; + +static const std::map v4l2ToTransferFunction = { + { V4L2_XFER_FUNC_NONE, ColorSpace::TransferFunction::Linear }, + { V4L2_XFER_FUNC_SRGB, ColorSpace::TransferFunction::Srgb }, + { V4L2_XFER_FUNC_709, ColorSpace::TransferFunction::Rec709 }, +}; + +static const std::map v4l2ToYcbcrEncoding = { + { V4L2_YCBCR_ENC_601, ColorSpace::YcbcrEncoding::Rec601 }, + { V4L2_YCBCR_ENC_709, ColorSpace::YcbcrEncoding::Rec709 }, + { V4L2_YCBCR_ENC_BT2020, ColorSpace::YcbcrEncoding::Rec2020 }, +}; + +static const std::map v4l2ToRange = { + { V4L2_QUANTIZATION_FULL_RANGE, ColorSpace::Range::Full }, + { V4L2_QUANTIZATION_LIM_RANGE, ColorSpace::Range::Limited }, +}; + +static const std::vector> colorSpaceToV4l2 = { + { ColorSpace::Raw, V4L2_COLORSPACE_RAW }, + { ColorSpace::Jpeg, V4L2_COLORSPACE_JPEG }, + { ColorSpace::Srgb, V4L2_COLORSPACE_SRGB }, + { ColorSpace::Smpte170m, V4L2_COLORSPACE_SMPTE170M }, + { ColorSpace::Rec709, V4L2_COLORSPACE_REC709 }, + { ColorSpace::Rec2020, V4L2_COLORSPACE_BT2020 }, +}; + +static const std::map primariesToV4l2 = { + { ColorSpace::Primaries::Raw, V4L2_COLORSPACE_RAW }, + { ColorSpace::Primaries::Smpte170m, V4L2_COLORSPACE_SMPTE170M }, + { ColorSpace::Primaries::Rec709, V4L2_COLORSPACE_REC709 }, + { ColorSpace::Primaries::Rec2020, V4L2_COLORSPACE_BT2020 }, +}; + +static const std::map transferFunctionToV4l2 = { + { ColorSpace::TransferFunction::Linear, V4L2_XFER_FUNC_NONE }, + { ColorSpace::TransferFunction::Srgb, V4L2_XFER_FUNC_SRGB }, + { ColorSpace::TransferFunction::Rec709, V4L2_XFER_FUNC_709 }, +}; + +static const std::map ycbcrEncodingToV4l2 = { + { ColorSpace::YcbcrEncoding::Rec601, V4L2_YCBCR_ENC_601 }, + { ColorSpace::YcbcrEncoding::Rec709, V4L2_YCBCR_ENC_709 }, + { ColorSpace::YcbcrEncoding::Rec2020, V4L2_YCBCR_ENC_BT2020 }, +}; + +static const std::map rangeToV4l2 = { + { ColorSpace::Range::Full, V4L2_QUANTIZATION_FULL_RANGE }, + { ColorSpace::Range::Limited, V4L2_QUANTIZATION_LIM_RANGE }, +}; + +/** + * \brief Convert the color space fields in a V4L2 format to a ColorSpace + * \param[in] v4l2Format A V4L2 format containing color space information + * + * The colorspace, ycbcr_enc, xfer_func and quantization fields within a + * V4L2 format structure are converted to a corresponding ColorSpace. + * + * If any V4L2 fields are not recognised then we return an "unset" + * color space. + * + * \return The ColorSpace corresponding to the input V4L2 format + * \retval std::nullopt One or more V4L2 color space fields were not recognised + */ +template +std::optional V4L2Device::toColorSpace(const T &v4l2Format) +{ + auto itColor = v4l2ToColorSpace.find(v4l2Format.colorspace); + if (itColor == v4l2ToColorSpace.end()) + return std::nullopt; + + /* This sets all the color space fields to the correct "default" values. */ + ColorSpace colorSpace = itColor->second; + + if (v4l2Format.ycbcr_enc != V4L2_YCBCR_ENC_DEFAULT) { + auto itYcbcrEncoding = v4l2ToYcbcrEncoding.find(v4l2Format.ycbcr_enc); + if (itYcbcrEncoding == v4l2ToYcbcrEncoding.end()) + return std::nullopt; + + colorSpace.ycbcrEncoding = itYcbcrEncoding->second; + } + + if (v4l2Format.xfer_func != V4L2_XFER_FUNC_DEFAULT) { + auto itTransfer = v4l2ToTransferFunction.find(v4l2Format.xfer_func); + if (itTransfer == v4l2ToTransferFunction.end()) + return std::nullopt; + + colorSpace.transferFunction = itTransfer->second; + } + + if (v4l2Format.quantization != V4L2_QUANTIZATION_DEFAULT) { + auto itRange = v4l2ToRange.find(v4l2Format.quantization); + if (itRange == v4l2ToRange.end()) + return std::nullopt; + + colorSpace.range = itRange->second; + } + + return colorSpace; +} + +template std::optional V4L2Device::toColorSpace(const struct v4l2_pix_format &); +template std::optional V4L2Device::toColorSpace(const struct v4l2_pix_format_mplane &); +template std::optional V4L2Device::toColorSpace(const struct v4l2_mbus_framefmt &); + +/** + * \brief Fill in the color space fields of a V4L2 format from a ColorSpace + * \param[in] colorSpace The ColorSpace to be converted + * \param[out] v4l2Format A V4L2 format containing color space information + * + * The colorspace, ycbcr_enc, xfer_func and quantization fields within a + * V4L2 format structure are filled in from a corresponding ColorSpace. + * + * An error is returned if any of the V4L2 fields do not support the + * value given in the ColorSpace. Such fields are set to the V4L2 + * "default" values, but all other fields are still filled in where + * possible. + * + * If the color space is completely unset, "default" V4L2 values are used + * everywhere, so a driver would then choose its preferred color space. + * + * \return 0 on success or a negative error code otherwise + * \retval -EINVAL The ColorSpace does not have a representation using V4L2 enums + */ +template +int V4L2Device::fromColorSpace(const std::optional &colorSpace, T &v4l2Format) +{ + v4l2Format.colorspace = V4L2_COLORSPACE_DEFAULT; + v4l2Format.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; + v4l2Format.xfer_func = V4L2_XFER_FUNC_DEFAULT; + v4l2Format.quantization = V4L2_QUANTIZATION_DEFAULT; + + if (!colorSpace) + return 0; + + auto itColor = std::find_if(colorSpaceToV4l2.begin(), colorSpaceToV4l2.end(), + [&colorSpace](const auto &item) { + return colorSpace == item.first; + }); + if (itColor != colorSpaceToV4l2.end()) { + v4l2Format.colorspace = itColor->second; + /* Leaving all the other fields as "default" should be fine. */ + return 0; + } + + /* + * If the colorSpace doesn't precisely match a standard color space, + * then we must choose a V4L2 colorspace with matching primaries. + */ + int ret = 0; + + auto itPrimaries = primariesToV4l2.find(colorSpace->primaries); + if (itPrimaries != primariesToV4l2.end()) + v4l2Format.colorspace = itPrimaries->second; + else + ret = -EINVAL; + + auto itYcbcrEncoding = ycbcrEncodingToV4l2.find(colorSpace->ycbcrEncoding); + if (itYcbcrEncoding != ycbcrEncodingToV4l2.end()) + v4l2Format.ycbcr_enc = itYcbcrEncoding->second; + else + ret = -EINVAL; + + auto itTransfer = transferFunctionToV4l2.find(colorSpace->transferFunction); + if (itTransfer != transferFunctionToV4l2.end()) + v4l2Format.xfer_func = itTransfer->second; + else + ret = -EINVAL; + + auto itRange = rangeToV4l2.find(colorSpace->range); + if (itRange != rangeToV4l2.end()) + v4l2Format.quantization = itRange->second; + else + ret = -EINVAL; + + return ret; +} + +template int V4L2Device::fromColorSpace(const std::optional &, struct v4l2_pix_format &); +template int V4L2Device::fromColorSpace(const std::optional &, struct v4l2_pix_format_mplane &); +template int V4L2Device::fromColorSpace(const std::optional &, struct v4l2_mbus_framefmt &); + } /* namespace libcamera */ From patchwork Fri Dec 10 11:21:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 15108 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 74081C324B for ; Fri, 10 Dec 2021 11:22:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1A0CB60890; Fri, 10 Dec 2021 12:22:11 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="ib1+ckSw"; dkim-atps=neutral Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2D5876088E for ; Fri, 10 Dec 2021 12:22:06 +0100 (CET) Received: by mail-wm1-x32c.google.com with SMTP id p27-20020a05600c1d9b00b0033bf8532855so6373888wms.3 for ; Fri, 10 Dec 2021 03:22:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=dNFIU7MwMl9M/WUUAajbXcsy7xJkXDsMafgWJxChUOM=; b=ib1+ckSwjnbO4pAmcxGeaXkEJ8tfxlBHi/6Xho4CaHJwXmZpb9vANBKonYeAQxveBN cX6GapcBNB2Xr6VHXnauPj1t0iPWERcTqsVZH9LQPfI1i2PJcJUcQhOtW+yDeueyfVBB WnZncdiYsyCU6kfUZKimOu1xDPFcUStAgN+Yfg6hUfhGtQRkhKe2/+x+0o41l8/edob5 Xdt6a049biFzJygg76ONisNaOPjECXGQOZxRG0djnEAM8Rpw4AjWwt28pxPUHoORv93n XlAaEXNIVEcJb7mOgR5vATsKnzfb1EGTl2t9hsAhZdH/8UAk0LktfIErcu3qm6+AxLhO e/VA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=dNFIU7MwMl9M/WUUAajbXcsy7xJkXDsMafgWJxChUOM=; b=QgLA04HjJ6MgO5Q0THhu7hf5bTbbculN3ftXT9O98Cr940y6jUGoS5ctrdFTRJBWQ5 HoEN5ldAfv8AhDvpO5P6PK4O6ftimIBxYe3EIkgKneURWZbi6U7Ku42+3ovi/wxy40iJ 6sl0aHhGI0gbCPbXzuaIsvW5W3i43WlAb56rvhfI7oE7HQb2pFShMCqUfy8hbqItgJ0K tUYdd6h86zPPAptYPV/GLGML5DwZ5Te3dw4YbZCq/Mn/Mw4KJcIRBMFZ5uB9Zhb/ObEL tLy++VZjkdn88KAYCUTOzrVqOw6m+UbDMDDT5K7/N2Q8xLyxgYhtDZcJx6TtZYe8rzTX DR9w== X-Gm-Message-State: AOAM5331FaBVKJGyTXB3/aXajyOmEod99ZoACK02/tTnZ3luej2uEs26 KpLshiSrsCCyf3byAVYS6q9l/Q== X-Google-Smtp-Source: ABdhPJwyenTbeq+1fZCYd4ypXMro0rGDXe1K22W84Q7GCD33R/aRFuFCip4GmnH3YJLwAgPDGVYHIA== X-Received: by 2002:a05:600c:1d06:: with SMTP id l6mr15069142wms.97.1639135325889; Fri, 10 Dec 2021 03:22:05 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:e4a2:3070:eea4:e434]) by smtp.gmail.com with ESMTPSA id z18sm2198469wrq.11.2021.12.10.03.22.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Dec 2021 03:22:05 -0800 (PST) From: David Plowman To: Laurent Pinchart , Kieran Bingham , Hans Verkuil , Tomasz Figa , Jacopo Mondi , Naushir Patuck , libcamera-devel@lists.libcamera.org Date: Fri, 10 Dec 2021 11:21:38 +0000 Message-Id: <20211210112142.18441-5-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211210112142.18441-1-david.plowman@raspberrypi.com> References: <20211210112142.18441-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v11 4/8] libcamera: video_device: Support passing ColorSpaces to V4L2 video devices X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The ColorSpace from the StreamConfiguration is now handled appropriately in the V4L2VideoDevice. Signed-off-by: David Plowman Reviewed-by: Laurent Pinchart --- include/libcamera/internal/v4l2_videodevice.h | 3 ++ src/libcamera/v4l2_videodevice.cpp | 32 +++++++++++++++++++ 2 files changed, 35 insertions(+) diff --git a/include/libcamera/internal/v4l2_videodevice.h b/include/libcamera/internal/v4l2_videodevice.h index 9b2ec3af..2d2ccc47 100644 --- a/include/libcamera/internal/v4l2_videodevice.h +++ b/include/libcamera/internal/v4l2_videodevice.h @@ -10,6 +10,7 @@ #include #include #include +#include #include #include #include @@ -21,6 +22,7 @@ #include #include +#include #include #include #include @@ -167,6 +169,7 @@ public: V4L2PixelFormat fourcc; Size size; + std::optional colorSpace; std::array planes; unsigned int planesCount = 0; diff --git a/src/libcamera/v4l2_videodevice.cpp b/src/libcamera/v4l2_videodevice.cpp index b4b89e27..80a8c8e4 100644 --- a/src/libcamera/v4l2_videodevice.cpp +++ b/src/libcamera/v4l2_videodevice.cpp @@ -383,6 +383,21 @@ bool V4L2BufferCache::Entry::operator==(const FrameBuffer &buffer) const * that identifies the image format pixel encoding scheme. */ +/** + * \var V4L2DeviceFormat::colorSpace + * \brief The color space of the pixels + * + * The color space of the image. When setting the format this may be + * unset, in which case the driver gets to use its default color space. + * After being set, this value should contain the color space that + * was actually used. If this value is unset, then the color space chosen + * by the driver could not be represented by the ColorSpace class (and + * should probably be added). + * + * It is up to the pipeline handler or application to check if the + * resulting color space is acceptable. + */ + /** * \var V4L2DeviceFormat::planes * \brief The per-plane memory size information @@ -871,6 +886,7 @@ int V4L2VideoDevice::getFormatMultiplane(V4L2DeviceFormat *format) format->size.height = pix->height; format->fourcc = V4L2PixelFormat(pix->pixelformat); format->planesCount = pix->num_planes; + format->colorSpace = toColorSpace(*pix); for (unsigned int i = 0; i < format->planesCount; ++i) { format->planes[i].bpl = pix->plane_fmt[i].bytesperline; @@ -893,6 +909,12 @@ int V4L2VideoDevice::trySetFormatMultiplane(V4L2DeviceFormat *format, bool set) pix->num_planes = format->planesCount; pix->field = V4L2_FIELD_NONE; + ret = fromColorSpace(format->colorSpace, *pix); + if (ret < 0) + LOG(V4L2, Warning) + << "Setting color space unrecognised by V4L2: " + << ColorSpace::toString(format->colorSpace); + ASSERT(pix->num_planes <= std::size(pix->plane_fmt)); for (unsigned int i = 0; i < pix->num_planes; ++i) { @@ -920,6 +942,7 @@ int V4L2VideoDevice::trySetFormatMultiplane(V4L2DeviceFormat *format, bool set) format->planes[i].bpl = pix->plane_fmt[i].bytesperline; format->planes[i].size = pix->plane_fmt[i].sizeimage; } + format->colorSpace = toColorSpace(*pix); return 0; } @@ -943,6 +966,7 @@ int V4L2VideoDevice::getFormatSingleplane(V4L2DeviceFormat *format) format->planesCount = 1; format->planes[0].bpl = pix->bytesperline; format->planes[0].size = pix->sizeimage; + format->colorSpace = toColorSpace(*pix); return 0; } @@ -959,6 +983,13 @@ int V4L2VideoDevice::trySetFormatSingleplane(V4L2DeviceFormat *format, bool set) pix->pixelformat = format->fourcc; pix->bytesperline = format->planes[0].bpl; pix->field = V4L2_FIELD_NONE; + + ret = fromColorSpace(format->colorSpace, *pix); + if (ret < 0) + LOG(V4L2, Warning) + << "Set color space unrecognised by V4L2: " + << ColorSpace::toString(format->colorSpace); + ret = ioctl(set ? VIDIOC_S_FMT : VIDIOC_TRY_FMT, &v4l2Format); if (ret) { LOG(V4L2, Error) @@ -977,6 +1008,7 @@ int V4L2VideoDevice::trySetFormatSingleplane(V4L2DeviceFormat *format, bool set) format->planesCount = 1; format->planes[0].bpl = pix->bytesperline; format->planes[0].size = pix->sizeimage; + format->colorSpace = toColorSpace(*pix); return 0; } From patchwork Fri Dec 10 11:21:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 15109 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1DC9DBF415 for ; Fri, 10 Dec 2021 11:22:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BDCA5608E3; Fri, 10 Dec 2021 12:22:11 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="Mvv0Nj4G"; dkim-atps=neutral Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2D5F46088E for ; Fri, 10 Dec 2021 12:22:07 +0100 (CET) Received: by mail-wm1-x329.google.com with SMTP id i12so6546795wmq.4 for ; Fri, 10 Dec 2021 03:22:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XppW0uYN/+TM2EogMDHujEnG3QJ6JaZY9LDGXon35AQ=; b=Mvv0Nj4GK2BEVkAZ8MQbw6nS/SNjpyfYd2q7vVNY9duqgeb2PRqFGW+U5Wjv2h1fjI dYMHJsaTO7sbwuB5JRx9y0y+nMXAMJhI0VZx84mGY/TQ7vEtNfazH5Brlyl5DREidutc gMuVynssiL8/+qzHmloVshFzkr2ID+taDRh2jef6Cq+Bay5/9l75WD+mlheHfrBAlh1e AGBdRZwJhzkU+z/oPxKVjjk/TCHqVNrAeKNy+/qR9/3BzsnyLa1DEleOzVkLdUaYIB3B ynPwFmm4Im7b6Ok+u6RLAFUZaZmR0LOYpc+LkAUjKScpzTkAEtDc34dX7bAGRjZ4l/Bq NcNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XppW0uYN/+TM2EogMDHujEnG3QJ6JaZY9LDGXon35AQ=; b=uKgIicihoVtD1RLAzWLUyWewRQcywlmmv0RzonP2j9l5GOIAerkUt3cxmvti2rYHZO iS5BcLMYbizdYv2+DixKIjOaxF8bCRmyVGS+qbZmz/i8Iil8Vvu4qzVltSCF+UjDJ4nn MDuYKhkxFW8bKErqEz5ebZ6/6+p+SJRVKTeOtYGjC/QfvawATC1p0FjiYLEncNdAJ12j jLYabQUVoY9UAaDOUCaWKk8Wu2jq6YwuPqik3GJLiawE7Bj9lJhSNmiw13WmsvAMFspm +Co/6RgzqoXWsMpftsw3wC1k66wZcudl43/oOGD8cFywZQllL7JT+9U8pq5doo67dGec yu3g== X-Gm-Message-State: AOAM531IGbMFOFzMdoPv95qA0UoDiAOCpIqHGLqXH3ap3G46Pe8/mOqV oxHnr8ydFMpCuf3mTuX8M9NnZQ== X-Google-Smtp-Source: ABdhPJyYBORRBy9I9gj79LaxbzoNSxLaDqARb86afXjV/8KATPuOKoUh9X+E3SoDqxU6PNtdpxTLmg== X-Received: by 2002:a05:600c:34d6:: with SMTP id d22mr15981786wmq.111.1639135326853; Fri, 10 Dec 2021 03:22:06 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:e4a2:3070:eea4:e434]) by smtp.gmail.com with ESMTPSA id z18sm2198469wrq.11.2021.12.10.03.22.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Dec 2021 03:22:06 -0800 (PST) From: David Plowman To: Laurent Pinchart , Kieran Bingham , Hans Verkuil , Tomasz Figa , Jacopo Mondi , Naushir Patuck , libcamera-devel@lists.libcamera.org Date: Fri, 10 Dec 2021 11:21:39 +0000 Message-Id: <20211210112142.18441-6-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211210112142.18441-1-david.plowman@raspberrypi.com> References: <20211210112142.18441-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v11 5/8] libcamera: v4l2_subdevice: Add colorSpace field to V4L2SubdeviceFormat X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This adds a ColorSpace field to the V4L2SubdeviceFormat so that we can set and request particular color spaces from V4L2. This commit simply adds the field and fixes some occurrences of brace initializers that would otherwise be broken. A subsequent commit will pass and retrieve the value correctly to/from V4l2 itself. Signed-off-by: David Plowman Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- include/libcamera/internal/v4l2_subdevice.h | 3 +++ src/libcamera/camera_sensor.cpp | 1 + src/libcamera/pipeline/ipu3/cio2.cpp | 7 +++---- src/libcamera/pipeline/simple/simple.cpp | 8 ++++++-- src/libcamera/v4l2_subdevice.cpp | 15 +++++++++++++++ 5 files changed, 28 insertions(+), 6 deletions(-) diff --git a/include/libcamera/internal/v4l2_subdevice.h b/include/libcamera/internal/v4l2_subdevice.h index a6873b67..58d1e511 100644 --- a/include/libcamera/internal/v4l2_subdevice.h +++ b/include/libcamera/internal/v4l2_subdevice.h @@ -8,12 +8,14 @@ #pragma once #include +#include #include #include #include #include +#include #include #include "libcamera/internal/formats.h" @@ -27,6 +29,7 @@ class MediaDevice; struct V4L2SubdeviceFormat { uint32_t mbus_code; Size size; + std::optional colorSpace; const std::string toString() const; uint8_t bitsPerPixel() const; diff --git a/src/libcamera/camera_sensor.cpp b/src/libcamera/camera_sensor.cpp index 7bb39b1e..c3999d35 100644 --- a/src/libcamera/camera_sensor.cpp +++ b/src/libcamera/camera_sensor.cpp @@ -642,6 +642,7 @@ V4L2SubdeviceFormat CameraSensor::getFormat(const std::vector &mbu V4L2SubdeviceFormat format{ .mbus_code = bestCode, .size = *bestSize, + .colorSpace = ColorSpace::Raw, }; return format; diff --git a/src/libcamera/pipeline/ipu3/cio2.cpp b/src/libcamera/pipeline/ipu3/cio2.cpp index 59dda56b..f4e8c663 100644 --- a/src/libcamera/pipeline/ipu3/cio2.cpp +++ b/src/libcamera/pipeline/ipu3/cio2.cpp @@ -322,10 +322,9 @@ V4L2SubdeviceFormat CIO2Device::getSensorFormat(const std::vector return {}; } - V4L2SubdeviceFormat format{ - .mbus_code = bestCode, - .size = bestSize, - }; + V4L2SubdeviceFormat format{}; + format.mbus_code = bestCode; + format.size = bestSize; return format; } diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 701fb4be..a3108fc0 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -457,7 +457,9 @@ int SimpleCameraData::init() * formats on the video node. */ for (unsigned int code : sensor_->mbusCodes()) { - V4L2SubdeviceFormat format{ code, sensor_->resolution() }; + V4L2SubdeviceFormat format{}; + format.mbus_code = code; + format.size = sensor_->resolution(); ret = setupFormats(&format, V4L2Subdevice::TryFormat); if (ret < 0) { @@ -908,7 +910,9 @@ int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c) return ret; const SimpleCameraData::Configuration *pipeConfig = config->pipeConfig(); - V4L2SubdeviceFormat format{ pipeConfig->code, data->sensor_->resolution() }; + V4L2SubdeviceFormat format{}; + format.mbus_code = pipeConfig->code; + format.size = data->sensor_->resolution(); ret = data->setupFormats(&format, V4L2Subdevice::ActiveFormat); if (ret < 0) diff --git a/src/libcamera/v4l2_subdevice.cpp b/src/libcamera/v4l2_subdevice.cpp index 61e15b69..b782325a 100644 --- a/src/libcamera/v4l2_subdevice.cpp +++ b/src/libcamera/v4l2_subdevice.cpp @@ -169,6 +169,21 @@ const std::map formatInfoMap = { * \brief The image size in pixels */ +/** + * \var V4L2SubdeviceFormat::colorSpace + * \brief The color space of the pixels + * + * The color space of the image. When setting the format this may be + * unset, in which case the driver gets to use its default color space. + * After being set, this value should contain the color space that + * was actually used. If this value is unset, then the color space chosen + * by the driver could not be represented by the ColorSpace class (and + * should probably be added). + * + * It is up to the pipeline handler or application to check if the + * resulting color space is acceptable. + */ + /** * \brief Assemble and return a string describing the format * \return A string describing the V4L2SubdeviceFormat From patchwork Fri Dec 10 11:21:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 15110 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 94D55C3258 for ; Fri, 10 Dec 2021 11:22:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3F404608E5; Fri, 10 Dec 2021 12:22:12 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="Wrl2E789"; dkim-atps=neutral Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com [IPv6:2a00:1450:4864:20::32a]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DCD366088E for ; Fri, 10 Dec 2021 12:22:07 +0100 (CET) Received: by mail-wm1-x32a.google.com with SMTP id i12so6546811wmq.4 for ; Fri, 10 Dec 2021 03:22:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=K60rAz66vWi9F057w7dISj11FLoL/9HFVty+RyVxed8=; b=Wrl2E789/shUI1lgPdW4mncRL11LlTL/CVwhjNq67rK8hBiuGmasOuKl9shwDTqPrX CXNTkZNVesEzOwLKbuGqm19ZJwltvkR+DB/z8Ekf538Ehvyv4EEGRhs+c2MISzxjGRI0 kT7nEtp8xOte53ZNADhwSdbW1tmj+A6eO2bnaVvlDmHVUqyvz85UXXN1RAlxLVw6LSC+ tTjchHcPeuGiIrqljVA6rxvxtprq/BzwdPhlJtHxi00L8faxm0QUmXOcziCgVupXvFdG FfCllRtlNFdRcSkCw1ItfSAGlKD7eCnjAJTJIFRzxBeKUPl8hhxs/22F4U1FrTu5lUoX jDag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=K60rAz66vWi9F057w7dISj11FLoL/9HFVty+RyVxed8=; b=XBrooQTiOaIWpdtjKvQiU8NY0qreKYKx5pYpsZ3lI//wK6pf9MWTqyM1oWeoVxXbrE D/yOmIm4vvRkIUJCgiMBby0at2FGHJbOq6GYPPPIxRCMOi3xeZZf8gIbFYghubenL1Kv WhMElu0E/zWcJ8Y/eVf5M8FMdUxHe6mSL36SPhgbp8XT31RzQAIdb4dBTd+Nxf8iwkoT EBcErdt6n0oFVo9oj4slBd/q1Jg4lQz0MK/UKP9stK6KvGaRAAkPGN9jmr3TKbMpZfgK C5CbjaF9iuwUwTl+uYNFMUQgfCES5yeCmyLnF9rjYePqqoRBurBm3pv8ksgsGnTRSnPF 9s4w== X-Gm-Message-State: AOAM532Tu4fggJ2421BcTwN7329b8mvWprxBTZ1qRncFz6vmGJwGScvO jKKq/8TVvUtaoJudVWsB5aRtkA== X-Google-Smtp-Source: ABdhPJwc4sJk9+Lsw8/M5jbaGdFQK6n3RkpRNhWiw6LaIxLB3S4D7ImedTsIMWBdhKdGwXxUPzs76w== X-Received: by 2002:a05:600c:4154:: with SMTP id h20mr15518918wmm.189.1639135327516; Fri, 10 Dec 2021 03:22:07 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:e4a2:3070:eea4:e434]) by smtp.gmail.com with ESMTPSA id z18sm2198469wrq.11.2021.12.10.03.22.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Dec 2021 03:22:07 -0800 (PST) From: David Plowman To: Laurent Pinchart , Kieran Bingham , Hans Verkuil , Tomasz Figa , Jacopo Mondi , Naushir Patuck , libcamera-devel@lists.libcamera.org Date: Fri, 10 Dec 2021 11:21:40 +0000 Message-Id: <20211210112142.18441-7-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211210112142.18441-1-david.plowman@raspberrypi.com> References: <20211210112142.18441-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v11 6/8] libcamera: v4l2_subdevice: Support passing ColorSpaces to V4L2 subdevices X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The ColorSpace from the StreamConfiguration is now handled appropriately in the V4L2Subdevice. Signed-off-by: David Plowman Reviewed-by: Laurent Pinchart --- src/libcamera/v4l2_subdevice.cpp | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/src/libcamera/v4l2_subdevice.cpp b/src/libcamera/v4l2_subdevice.cpp index b782325a..8caa73d6 100644 --- a/src/libcamera/v4l2_subdevice.cpp +++ b/src/libcamera/v4l2_subdevice.cpp @@ -415,6 +415,7 @@ int V4L2Subdevice::getFormat(unsigned int pad, V4L2SubdeviceFormat *format, format->size.width = subdevFmt.format.width; format->size.height = subdevFmt.format.height; format->mbus_code = subdevFmt.format.code; + format->colorSpace = toColorSpace(subdevFmt.format); return 0; } @@ -443,7 +444,13 @@ int V4L2Subdevice::setFormat(unsigned int pad, V4L2SubdeviceFormat *format, subdevFmt.format.code = format->mbus_code; subdevFmt.format.field = V4L2_FIELD_NONE; - int ret = ioctl(VIDIOC_SUBDEV_S_FMT, &subdevFmt); + int ret = fromColorSpace(format->colorSpace, subdevFmt.format); + if (ret < 0) + LOG(V4L2, Warning) + << "Setting color space unrecognised by V4L2: " + << ColorSpace::toString(format->colorSpace); + + ret = ioctl(VIDIOC_SUBDEV_S_FMT, &subdevFmt); if (ret) { LOG(V4L2, Error) << "Unable to set format on pad " << pad @@ -454,6 +461,7 @@ int V4L2Subdevice::setFormat(unsigned int pad, V4L2SubdeviceFormat *format, format->size.width = subdevFmt.format.width; format->size.height = subdevFmt.format.height; format->mbus_code = subdevFmt.format.code; + format->colorSpace = toColorSpace(subdevFmt.format); return 0; } From patchwork Fri Dec 10 11:21:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 15111 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0B933C3259 for ; Fri, 10 Dec 2021 11:22:13 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A71996089C; Fri, 10 Dec 2021 12:22:12 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="Oj6PUd0/"; dkim-atps=neutral Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A32056089C for ; Fri, 10 Dec 2021 12:22:08 +0100 (CET) Received: by mail-wr1-x432.google.com with SMTP id u1so14277893wru.13 for ; Fri, 10 Dec 2021 03:22:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=2bNStc3f5yx0+9r6cEGZZIEuYgCV35H3PWXSv/otXK8=; b=Oj6PUd0/l6GDhYXP26/dGOCLjmf00nZqBRVB1jprHDW2gXSuh4KPBH5V4QUPHsNwsy T6AaW9OP74Hf2jpfdezD1fBEk3oaZ+O9mi3bt2t2lpsvjk4VtSq7oC4m/M1131UPM9Y3 L3kzMLTAbPmsPYGcGVDWhse9aV5NjNXZA6wX1YtMb0Xn3owUROXLhhv3Rc8LRCdRSJ6V vAQbZK+gvp/TmnM2IZHJYymADx1r+mQfHOMxwLKtXWXCIjs2yWG0wVv9gHlbugkOTx04 Ed2Ci0Do5jVy/LRWHnsQnPMqimYkaoFK6Pi8XNCnkBoHY1QInkO4PXSihLPCmGT8WlOs UhTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2bNStc3f5yx0+9r6cEGZZIEuYgCV35H3PWXSv/otXK8=; b=PKv/+XJy3cT6dbzUTRHgQGgJHu/NlOc4GtAKJ9146p0FnTriDr6BMp3IYa4MISduZF kWW8ZhKDxSJZOoDDPYevWZQqh6FlEIdvEySId47NCiG2xSuGlrPQGYRMGk20NVDMnKHt h4x+1Pg1UwjODw+cfiTqn/zHuWDI3++lPfk9dbJiCRlziI37d6/eDM3oQ6He6TA/uBPG nJ+cQAFlWnn4KTRcAx3t46eO28BJ2NXCoOmwjEje6N26YSNeQjIgkuxQ5gt4vKvmIUxR s24KkqMOJIDInMySU6xXQtrYMvtoMroHcgLt8t2yEy2xYMYE7Kj4uafhBGWaHcWO4hw2 z59Q== X-Gm-Message-State: AOAM531fIiRoDTRJV7V3MXSUthDVarC2VPS+2bH22LeHDIxelzV/GqgD r8RrNLjB6vfPwNeS4p19sH9C4A== X-Google-Smtp-Source: ABdhPJwiWVRpYlhWu5qtQXZi3is+jdND5+Sp9+8jyI78ekDj2M74WPrLZ4WOtc/EhTJNO5FLv0QFvg== X-Received: by 2002:a5d:6886:: with SMTP id h6mr13314127wru.287.1639135328314; Fri, 10 Dec 2021 03:22:08 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:e4a2:3070:eea4:e434]) by smtp.gmail.com with ESMTPSA id z18sm2198469wrq.11.2021.12.10.03.22.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Dec 2021 03:22:07 -0800 (PST) From: David Plowman To: Laurent Pinchart , Kieran Bingham , Hans Verkuil , Tomasz Figa , Jacopo Mondi , Naushir Patuck , libcamera-devel@lists.libcamera.org Date: Fri, 10 Dec 2021 11:21:41 +0000 Message-Id: <20211210112142.18441-8-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211210112142.18441-1-david.plowman@raspberrypi.com> References: <20211210112142.18441-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v11 7/8] libcamera: Add validateColorSpaces to CameraConfiguration class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This function forces raw streams to have the "raw" color space, and also optionally makes all non-raw output streams to share the same color space as some platforms may require this. When sharing color spaces we take the shared value to be the one from the largest of these streams. This choice is ultimately arbitrary, but can be appropriate if smaller output streams are used for image analysis rather than human consumption, when the precise colours may be less important. Signed-off-by: David Plowman Reviewed-by: Laurent Pinchart --- include/libcamera/camera.h | 10 +++++ src/libcamera/camera.cpp | 80 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 90 insertions(+) diff --git a/include/libcamera/camera.h b/include/libcamera/camera.h index a7759ccb..5bb06584 100644 --- a/include/libcamera/camera.h +++ b/include/libcamera/camera.h @@ -13,6 +13,7 @@ #include #include +#include #include #include @@ -69,6 +70,15 @@ public: protected: CameraConfiguration(); + enum class ColorSpaceFlag { + None, + StreamsShareColorSpace, + }; + + using ColorSpaceFlags = Flags; + + Status validateColorSpaces(ColorSpaceFlags flags = ColorSpaceFlag::None); + std::vector config_; }; diff --git a/src/libcamera/camera.cpp b/src/libcamera/camera.cpp index 400a7cf0..5f8533e8 100644 --- a/src/libcamera/camera.cpp +++ b/src/libcamera/camera.cpp @@ -14,12 +14,14 @@ #include #include +#include #include #include #include #include "libcamera/internal/camera.h" #include "libcamera/internal/camera_controls.h" +#include "libcamera/internal/formats.h" #include "libcamera/internal/pipeline_handler.h" /** @@ -314,6 +316,84 @@ std::size_t CameraConfiguration::size() const return config_.size(); } +namespace { + +bool isRaw(const PixelFormat &pixFmt) +{ + const PixelFormatInfo &info = PixelFormatInfo::info(pixFmt); + return info.isValid() && + info.colourEncoding == PixelFormatInfo::ColourEncodingRAW; +} + +} /* namespace */ + +/** + * \enum CameraConfiguration::ColorSpaceFlag + * \brief Specify the behaviour of validateColorSpaces + * \var CameraConfiguration::ColorSpaceFlag::None + * \brief No extra validation of color spaces is required + * \var CameraConfiguration::ColorSpaceFlag::StreamsShareColorSpace + * \brief Non-raw output streams must share the same color space + */ + +/** + * \typedef CameraConfiguration::ColorSpaceFlags + * \brief A bitwise combination of ColorSpaceFlag values + */ + +/** + * \brief Check the color spaces requested for each stream + * \param[in] flags Flags to control the behaviour of this function + * + * This function performs certain consistency checks on the color spaces of + * the streams and may adjust them so that: + * + * - Any raw streams have the Raw color space + * - If the StreamsShareColorSpace flag is set, all output streams are forced + * to share the same color space (this may be a constraint on some platforms). + * + * It is optional for a pipeline handler to use this function. + * + * \return A CameraConfiguration::Status value that describes the validation + * status. + * \retval CameraConfigutation::Adjusted The configuration has been adjusted + * and is now valid. The color space of some or all of the streams may bave + * benn changed. The caller shall check the color spaces carefully. + * \retval CameraConfiguration::Valid The configuration was already valid and + * hasn't been adjusted. + */ +CameraConfiguration::Status CameraConfiguration::validateColorSpaces(ColorSpaceFlags flags) +{ + Status status = Valid; + + /* + * Set all raw streams to the Raw color space, and make a note of the largest + * non-raw stream with a defined color space (if there is one). + */ + int index = -1; + for (auto [i, cfg] : utils::enumerate(config_)) { + if (isRaw(cfg.pixelFormat) && cfg.colorSpace != ColorSpace::Raw) { + cfg.colorSpace = ColorSpace::Raw; + status = Adjusted; + } else if (cfg.colorSpace && (index == -1 || cfg.size > config_[i].size)) + index = i; + } + + if (index < 0 || !(flags & ColorSpaceFlag::StreamsShareColorSpace)) + return status; + + /* Make all output color spaces the same, if requested. */ + for (auto &cfg : config_) { + if (!isRaw(cfg.pixelFormat) && + cfg.colorSpace != config_[index].colorSpace) { + cfg.colorSpace = config_[index].colorSpace; + status = Adjusted; + } + } + + return status; +} + /** * \var CameraConfiguration::transform * \brief User-specified transform to be applied to the image From patchwork Fri Dec 10 11:21:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 15112 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 86419C324B for ; Fri, 10 Dec 2021 11:22:13 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3C4BF608E8; Fri, 10 Dec 2021 12:22:13 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="kP9r44re"; dkim-atps=neutral Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4CC2D6088E for ; Fri, 10 Dec 2021 12:22:09 +0100 (CET) Received: by mail-wm1-x32c.google.com with SMTP id y196so6545193wmc.3 for ; Fri, 10 Dec 2021 03:22:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Iw1oXJ4y6qHk5Vh9ACkUHIhW6uieykJz0tu6MkgxSbg=; b=kP9r44rexQ2r71m2z9pSpOWMlBQ/SpakNxn9udNZR8c+RpbGW8U5MiJ+sFnJvjCBka OkljS+XtSAPjLkz0ZXcGCu+/pXvO/mewzlJ+rIWpTVOdJxuXK7QCCzpV5jWW9FUHdc1Z aUNOShaZ4ikyL/Ovdl4YR5FLPJW10t/r2Yu/mfY6XfgsgsshHsQj9BPC4sWefQDxG2xq 1dNGuxi5fKq1IZrE3vZoDzgmQveTFIaEkCHdaJTgqMbeuO2peHgehmsHSeIwwoF3QfyI 6z358j1DWtnH0wcLBvX0w0vKzauhfM8Khc773+lj8cmiSL76UaZI5Iya4zOMrmZxziDp wgXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Iw1oXJ4y6qHk5Vh9ACkUHIhW6uieykJz0tu6MkgxSbg=; b=u5mi7RZLKH2PXpUyjPWRZwD4jxCpVQtSpoiIhIzm4YL7bLMWIq5ZkWyjGCF6K3HZ7I fxEni+G4cH8d5nh2AWBdt6Vcb099/gFGiRVqEm5deDUXVnZ/0Y6I5Jx2CHOmt6AhBTA1 MIkGZoYDSN3BzfCaBxiFM69w/lly3l8ZcZXQeL4LYL3YPIBTh8/SVeMPJN9AAAyii6zu vQImStKCITrV6UVqGAFUduY15+98LagY1WF1kuR0p0n3sYkuM2eCkq/0vynZcBiU+cXm V9wafu49NULRydqGrfQuDOm8sY91EdrMkNn6ntaDuMEchPAVTPanJZ6WaKdPPVKOhtSO XJyw== X-Gm-Message-State: AOAM531UUJOrk4ZlSHpfrAKxPMsR5D5nLNqk7rBeEb190s/7rf9V7gjr FlkXe0oUvOJ3dZQ4gSiuXNeLO80mSc+0+A== X-Google-Smtp-Source: ABdhPJyGCaGpfkL82qLEopPdhkhwmi4enN0raSRbBX+PXkXdOhmjzx4x2XfaQ1DaR6I5f335mDvnyA== X-Received: by 2002:a7b:ce8c:: with SMTP id q12mr16007978wmj.91.1639135328970; Fri, 10 Dec 2021 03:22:08 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:e4a2:3070:eea4:e434]) by smtp.gmail.com with ESMTPSA id z18sm2198469wrq.11.2021.12.10.03.22.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Dec 2021 03:22:08 -0800 (PST) From: David Plowman To: Laurent Pinchart , Kieran Bingham , Hans Verkuil , Tomasz Figa , Jacopo Mondi , Naushir Patuck , libcamera-devel@lists.libcamera.org Date: Fri, 10 Dec 2021 11:21:42 +0000 Message-Id: <20211210112142.18441-9-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211210112142.18441-1-david.plowman@raspberrypi.com> References: <20211210112142.18441-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v11 8/8] libcamera: pipeline: raspberrypi: Support color spaces X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The Raspberry Pi pipeline handler now sets color spaces correctly. In generateConfiguration() it sets them to reasonable default values based on the stream role. validate() now calls validateColorSpaces() to ensure that the requested color spaces are sensible, before proceeding to check what the hardware can deliver. Signed-off-by: David Plowman Reviewed-by: Laurent Pinchart --- .../pipeline/raspberrypi/raspberrypi.cpp | 40 +++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 86851ac4..eb74d96c 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -96,6 +96,7 @@ V4L2DeviceFormat toV4L2DeviceFormat(const V4L2SubdeviceFormat &format, deviceFormat.fourcc = V4L2PixelFormat::fromPixelFormat(pix); deviceFormat.size = format.size; + deviceFormat.colorSpace = format.colorSpace; return deviceFormat; } @@ -132,6 +133,7 @@ V4L2SubdeviceFormat findBestFormat(const SensorFormats &formatsMap, const Size & { double bestScore = std::numeric_limits::max(), score; V4L2SubdeviceFormat bestFormat; + bestFormat.colorSpace = ColorSpace::Raw; constexpr float penaltyAr = 1500.0; constexpr float penaltyBitDepth = 500.0; @@ -329,6 +331,8 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() if (config_.empty()) return Invalid; + status = validateColorSpaces(ColorSpaceFlag::StreamsShareColorSpace); + /* * What if the platform has a non-90 degree rotation? We can't even * "adjust" the configuration and carry on. Alternatively, raising an @@ -496,11 +500,25 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() V4L2DeviceFormat format; format.fourcc = V4L2PixelFormat::fromPixelFormat(cfg.pixelFormat); format.size = cfg.size; + format.colorSpace = cfg.colorSpace; + + LOG(RPI, Debug) + << "Try color space " << ColorSpace::toString(cfg.colorSpace); int ret = dev->tryFormat(&format); if (ret) return Invalid; + if (!format.colorSpace || cfg.colorSpace != format.colorSpace) { + status = Adjusted; + LOG(RPI, Debug) + << "Color space changed from " + << ColorSpace::toString(cfg.colorSpace) << " to " + << ColorSpace::toString(format.colorSpace); + } + + cfg.colorSpace = format.colorSpace; + cfg.stride = format.planes[0].bpl; cfg.frameSize = format.planes[0].size; @@ -524,6 +542,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, PixelFormat pixelFormat; V4L2VideoDevice::Formats fmts; Size size; + std::optional colorSpace; if (roles.empty()) return config; @@ -539,6 +558,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, pixelFormat = mbusCodeToPixelFormat(sensorFormat.mbus_code, BayerFormat::Packing::CSI2); ASSERT(pixelFormat.isValid()); + colorSpace = ColorSpace::Raw; bufferCount = 2; rawCount++; break; @@ -546,6 +566,12 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, case StreamRole::StillCapture: fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::NV12; + /* + * Still image codecs usually expect the JPEG color space. + * Even RGB codecs will be fine as the RGB we get with the + * JPEG color space is the same as sRGB. + */ + colorSpace = ColorSpace::Jpeg; /* Return the largest sensor resolution. */ size = sensorSize; bufferCount = 1; @@ -563,6 +589,11 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, */ fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::YUV420; + /* + * Choose a color space appropriate for video recording. + * Rec.709 will be a good default for HD resolutions. + */ + colorSpace = ColorSpace::Rec709; size = { 1920, 1080 }; bufferCount = 4; outCount++; @@ -571,6 +602,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, case StreamRole::Viewfinder: fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::ARGB8888; + colorSpace = ColorSpace::Jpeg; size = { 800, 600 }; bufferCount = 4; outCount++; @@ -617,6 +649,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, StreamConfiguration cfg(formats); cfg.size = size; cfg.pixelFormat = pixelFormat; + cfg.colorSpace = colorSpace; cfg.bufferCount = bufferCount; config->addConfiguration(cfg); } @@ -724,6 +757,7 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) V4L2PixelFormat fourcc = V4L2PixelFormat::fromPixelFormat(cfg.pixelFormat); format.size = cfg.size; format.fourcc = fourcc; + format.colorSpace = cfg.colorSpace; LOG(RPI, Debug) << "Setting " << stream->name() << " to " << format.toString(); @@ -739,6 +773,10 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) return -EINVAL; } + LOG(RPI, Debug) + << "Stream " << stream->name() << " has color space " + << ColorSpace::toString(cfg.colorSpace); + cfg.setStream(stream); stream->setExternal(true); @@ -763,6 +801,8 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) format = {}; format.size = maxSize; format.fourcc = V4L2PixelFormat::fromPixelFormat(formats::YUV420); + /* No one asked for output, so the color space doesn't matter. */ + format.colorSpace = ColorSpace::Jpeg; ret = data->isp_[Isp::Output0].dev()->setFormat(&format); if (ret) { LOG(RPI, Error)