From patchwork Wed Dec 1 15:44:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 14974 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4C4B6BDB13 for ; Wed, 1 Dec 2021 15:45:06 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CA03C607CC; Wed, 1 Dec 2021 16:45:04 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="FU3iqVXD"; dkim-atps=neutral Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BF2BA60744 for ; Wed, 1 Dec 2021 16:45:03 +0100 (CET) Received: by mail-wr1-x42a.google.com with SMTP id c4so53230462wrd.9 for ; Wed, 01 Dec 2021 07:45:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Cp9glBsixRLEKZiH2UGu4p0K9rO+nf2CgExbPffXVV8=; b=FU3iqVXDyBcr1RJWitUBdFv2tp2KpuinScYyXo+io7VvXA12F//aZPJBnfgSB6wgpN Of/hI7Th677WqJMECmi+ly+qwWFzuUrlCANWFaNwPGaHF8Xk3+kO6mmJx8zAjYCRVRbh 0E3DsphaW0ObHi5h6n0AKTRWZB3CumKh6gwXIi1ETlVM8N79xfRtEcYUFPvwG1bYv/Cb lOzW7GCpQjQLXXsxvxynh0A/92fHLC9Mvvd7N53rKjkc8z/ddGvJc2PqXo3dnzG1HrGh gGR4D8O/ooPUrPj69WApUXinRe+bpPmgMWJOHahkzG2rNFxWTzMMoAYOEBtaqoZBkY0A cmrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Cp9glBsixRLEKZiH2UGu4p0K9rO+nf2CgExbPffXVV8=; b=W+ytWEOkgdV3y+vyyK1Bt5eCyQWEi161Rzy2iljO2DLfKiyJ4rsfqiH9lZoA8ZC9WI xKmDEVrryZbLscQyIumATUmvfI11y6JUXOPFNpiYgolOMxDC8w2NtwIYDETcHUCJJT5r M6Aa0OdyUH2LqSweafNodnTJf5c9DgUoHoQM0obBUjoclm1kKc+t1ebjt7yUhJ9FnoOl DM4tXVjsaCSETRbLxAF4qijsWGxBZxI+YItUrb9F9D+wZ4b+fNO25vwYOugbfHJtFV6N yMeGWziPL3xujmByi+c5miuEgkQzoZCmqY8O+Nf9JCjj1HLbsamrLCOwDx6YOVRugF8r XBUw== X-Gm-Message-State: AOAM530rrGcoI1UsJAyJIsFJQbDertIGBGBvZUEkisPs3RSYu+Y0gJsp EPC5/lCm27LyvItuuVFmb1fYazA8GQguN69X X-Google-Smtp-Source: ABdhPJwcMxFlEi7TgK0dPxvVMChCvdqXFq3sHvVmpYbOR+AO8M05Fu5LiKMONCo2vF6ZpkMpJw4wUg== X-Received: by 2002:adf:9d88:: with SMTP id p8mr7942362wre.140.1638373503175; Wed, 01 Dec 2021 07:45:03 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:e4a2:3070:eea4:e434]) by smtp.gmail.com with ESMTPSA id y7sm148209wrw.55.2021.12.01.07.45.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Dec 2021 07:45:02 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org, Laurent Pinchart , Kieran Bingham , Jacopo Mondi , Hans Verkuil , Tomasz Figa , Naushir Patuck Date: Wed, 1 Dec 2021 15:44:27 +0000 Message-Id: <20211201154434.23127-2-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211201154434.23127-1-david.plowman@raspberrypi.com> References: <20211201154434.23127-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v8 1/8] libcamera: Add ColorSpace class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This class represents a color space by defining its color primaries, YCbCr encoding, the transfer (gamma) function it uses, and whether the output is full or limited range. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck Reviewed-by: Umang Jain Reviewed-by: Jacopo Mondi --- include/libcamera/color_space.h | 69 ++++++++ include/libcamera/meson.build | 1 + src/libcamera/color_space.cpp | 305 ++++++++++++++++++++++++++++++++ src/libcamera/meson.build | 1 + 4 files changed, 376 insertions(+) create mode 100644 include/libcamera/color_space.h create mode 100644 src/libcamera/color_space.cpp diff --git a/include/libcamera/color_space.h b/include/libcamera/color_space.h new file mode 100644 index 00000000..0fe6b580 --- /dev/null +++ b/include/libcamera/color_space.h @@ -0,0 +1,69 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Raspberry Pi (Trading) Limited + * + * color_space.h - color space definitions + */ + +#pragma once + +#include +#include + +namespace libcamera { + +class ColorSpace +{ +public: + enum class Primaries { + Raw, + Smpte170m, + Rec709, + Rec2020, + }; + + enum class YcbcrEncoding { + Rec601, + Rec709, + Rec2020, + }; + + enum class TransferFunction { + Linear, + Srgb, + Rec709, + }; + + enum class Range { + Full, + Limited, + }; + + constexpr ColorSpace(Primaries p, YcbcrEncoding e, TransferFunction t, Range r) + : primaries(p), ycbcrEncoding(e), transferFunction(t), range(r) + { + } + + static const ColorSpace Raw; + static const ColorSpace Jpeg; + static const ColorSpace Srgb; + static const ColorSpace Smpte170m; + static const ColorSpace Rec709; + static const ColorSpace Rec2020; + + Primaries primaries; + YcbcrEncoding ycbcrEncoding; + TransferFunction transferFunction; + Range range; + + const std::string toString() const; + static const std::string toString(const std::optional &colorSpace); +}; + +bool operator==(const ColorSpace &lhs, const ColorSpace &rhs); +static inline bool operator!=(const ColorSpace &lhs, const ColorSpace &rhs) +{ + return !(lhs == rhs); +} + +} /* namespace libcamera */ diff --git a/include/libcamera/meson.build b/include/libcamera/meson.build index 7155ff20..131e1740 100644 --- a/include/libcamera/meson.build +++ b/include/libcamera/meson.build @@ -5,6 +5,7 @@ libcamera_include_dir = 'libcamera' / 'libcamera' libcamera_public_headers = files([ 'camera.h', 'camera_manager.h', + 'color_space.h', 'compiler.h', 'controls.h', 'file_descriptor.h', diff --git a/src/libcamera/color_space.cpp b/src/libcamera/color_space.cpp new file mode 100644 index 00000000..5fe4fcc2 --- /dev/null +++ b/src/libcamera/color_space.cpp @@ -0,0 +1,305 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Raspberry Pi (Trading) Limited + * + * color_space.cpp - color spaces. + */ + +#include + +#include +#include +#include +#include +#include + +/** + * \file color_space.h + * \brief Class and enums to represent color spaces + */ + +namespace libcamera { + +/** + * \class ColorSpace + * \brief Class to describe a color space + * + * The ColorSpace class defines the color primaries, the Y'CbCr encoding, + * the transfer function associated with the color space, and the range + * (sometimes also referred to as the quantisation) of the color space. + * + * Certain combinations of these fields form well-known standard color + * spaces such as "JPEG" or "REC709". + * + * In the strictest sense a "color space" formally only refers to the + * color primaries and white point. Here, however, the ColorSpace class + * adopts the common broader usage that includes the transfer function, + * Y'CbCr encoding method and quantisation. + * + * For more information on the specific color spaces described here, please + * see: + * + * sRGB + * JPEG + * SMPTE 170M + * Rec.709 + * Rec.2020 + */ + +/** + * \enum ColorSpace::Primaries + * \brief The color primaries for this color space + * + * \var ColorSpace::Primaries::Raw + * \brief These are raw colors directly from a sensor + * + * \var ColorSpace::Primaries::Smpte170m + * \brief SMPTE 170M color primaries + * + * \var ColorSpace::Primaries::Rec709 + * \brief Rec.709 color primaries + * + * \var ColorSpace::Primaries::Rec2020 + * \brief Rec.2020 color primaries + */ + +/** + * \enum ColorSpace::YcbcrEncoding + * \brief The Y'CbCr encoding + * + * \var ColorSpace::YcbcrEncoding::Rec601 + * \brief Rec.601 Y'CbCr encoding + * + * \var ColorSpace::YcbcrEncoding::Rec709 + * \brief Rec.709 Y'CbCr encoding + * + * \var ColorSpace::YcbcrEncoding::Rec2020 + * \brief Rec.2020 Y'CbCr encoding + */ + +/** + * \enum ColorSpace::TransferFunction + * \brief The transfer function used for this color space + * + * \var ColorSpace::TransferFunction::Linear + * \brief This color space uses a linear (identity) transfer function + * + * \var ColorSpace::TransferFunction::Srgb + * \brief sRGB transfer function + * + * \var ColorSpace::TransferFunction::Rec709 + * \brief Rec.709 transfer function + */ + +/** + * \enum ColorSpace::Range + * \brief The range (sometimes "quantisation") for this color space + * + * \var ColorSpace::Range::Full + * \brief This color space uses full range pixel values + * + * \var ColorSpace::Range::Limited + * \brief This color space uses limited range pixel values, being + * 16 to 235 for Y' and 16 to 240 for Cb and Cr (8 bits per sample) + * or 64 to 940 for Y' and 16 to 960 for Cb and Cr (10 bits) + */ + +/** + * \fn ColorSpace::ColorSpace(Primaries p, Encoding e, TransferFunction t, Range r) + * \brief Construct a ColorSpace from explicit values + * \param[in] p The color primaries + * \param[in] e The Y'CbCr encoding + * \param[in] t The transfer function for the color space + * \param[in] r The range of the pixel values in this color space + */ + +/** + * \brief Assemble and return a readable string representation of the + * ColorSpace + * + * If the color space matches a standard ColorSpace (such as ColorSpace::Jpeg) + * then the short name of the color space ("Jpeg") is returned. Otherwise + * the four constituent parts of the ColorSpace are assembled into a longer + * string. + * + * \return A string describing the ColorSpace + */ +const std::string ColorSpace::toString() const +{ + /* Print out a brief name only for standard color spaces. */ + + static const std::vector> colorSpaceNames = { + { ColorSpace::Raw, "Raw" }, + { ColorSpace::Jpeg, "Jpeg" }, + { ColorSpace::Srgb, "Srgb" }, + { ColorSpace::Smpte170m, "Smpte170m" }, + { ColorSpace::Rec709, "Rec709" }, + { ColorSpace::Rec2020, "Rec2020" }, + }; + auto it = std::find_if(colorSpaceNames.begin(), colorSpaceNames.end(), + [this](const auto &item) { + return *this == item.first; + }); + if (it != colorSpaceNames.end()) + return std::string(it->second); + + /* Assemble a name made of the constituent fields. */ + + static const std::map primariesNames = { + { Primaries::Raw, "Raw" }, + { Primaries::Smpte170m, "Smpte170m" }, + { Primaries::Rec709, "Rec709" }, + { Primaries::Rec2020, "Rec2020" }, + }; + static const std::map encodingNames = { + { YcbcrEncoding::Rec601, "Rec601" }, + { YcbcrEncoding::Rec709, "Rec709" }, + { YcbcrEncoding::Rec2020, "Rec2020" }, + }; + static const std::map transferNames = { + { TransferFunction::Linear, "Linear" }, + { TransferFunction::Srgb, "Srgb" }, + { TransferFunction::Rec709, "Rec709" }, + }; + static const std::map rangeNames = { + { Range::Full, "Full" }, + { Range::Limited, "Limited" }, + }; + + auto itPrimaries = primariesNames.find(primaries); + std::string primariesName = + itPrimaries == primariesNames.end() ? "Invalid" : itPrimaries->second; + + auto itEncoding = encodingNames.find(ycbcrEncoding); + std::string encodingName = + itEncoding == encodingNames.end() ? "Invalid" : itEncoding->second; + + auto itTransfer = transferNames.find(transferFunction); + std::string transferName = + itTransfer == transferNames.end() ? "Invalid" : itTransfer->second; + + auto itRange = rangeNames.find(range); + std::string rangeName = + itRange == rangeNames.end() ? "Invalid" : itRange->second; + + std::stringstream ss; + ss << primariesName << "/" << encodingName << "/" << transferName << "/" << rangeName; + + return ss.str(); +} + +/** + * \brief Assemble and return a readable string representation of an + * optional ColorSpace + * \return A string describing the optional ColorSpace + */ +const std::string ColorSpace::toString(const std::optional &colorSpace) +{ + if (!colorSpace) + return "Unknown"; + + return colorSpace->toString(); +} + +/** + * \var ColorSpace::primaries + * \brief The color primaries of this color space + */ + +/** + * \var ColorSpace::ycbcrEncoding + * \brief The Y'CbCr encoding used with this color space + */ + +/** + * \var ColorSpace::transferFunction + * \brief The transfer function used with this color space + */ + +/** + * \var ColorSpace::range + * \brief The pixel range used with this color space + */ + +/** + * \brief A constant representing a raw color space (from a sensor) + */ +const ColorSpace ColorSpace::Raw = { + Primaries::Raw, + YcbcrEncoding::Rec601, + TransferFunction::Linear, + Range::Full +}; + +/** + * \brief A constant representing the JPEG color space used for + * encoding JPEG images. + */ +const ColorSpace ColorSpace::Jpeg = { + Primaries::Rec709, + YcbcrEncoding::Rec601, + TransferFunction::Srgb, + Range::Full +}; + +/** + * \brief A constant representing the sRGB color space. This is + * identical to the JPEG color space except that the range Y'CbCr + * range is limited rather than full. + */ +const ColorSpace ColorSpace::Srgb = { + Primaries::Rec709, + YcbcrEncoding::Rec601, + TransferFunction::Srgb, + Range::Limited +}; + +/** + * \brief A constant representing the SMPTE170M color space + */ +const ColorSpace ColorSpace::Smpte170m = { + Primaries::Smpte170m, + YcbcrEncoding::Rec601, + TransferFunction::Rec709, + Range::Limited +}; + +/** + * \brief A constant representing the Rec.709 color space + */ +const ColorSpace ColorSpace::Rec709 = { + Primaries::Rec709, + YcbcrEncoding::Rec709, + TransferFunction::Rec709, + Range::Limited +}; + +/** + * \brief A constant representing the Rec.2020 color space + */ +const ColorSpace ColorSpace::Rec2020 = { + Primaries::Rec2020, + YcbcrEncoding::Rec2020, + TransferFunction::Rec709, + Range::Limited +}; + +/** + * \brief Compare color spaces for equality + * \return True if the two color spaces are identical, false otherwise + */ +bool operator==(const ColorSpace &lhs, const ColorSpace &rhs) +{ + return lhs.primaries == rhs.primaries && + lhs.ycbcrEncoding == rhs.ycbcrEncoding && + lhs.transferFunction == rhs.transferFunction && + lhs.range == rhs.range; +} + +/** + * \fn bool operator!=(const ColorSpace &lhs, const ColorSpace &rhs) + * \brief Compare color spaces for inequality + * \return True if the two color spaces are not identical, false otherwise + */ + +} /* namespace libcamera */ diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build index 6727a777..e7371d20 100644 --- a/src/libcamera/meson.build +++ b/src/libcamera/meson.build @@ -8,6 +8,7 @@ libcamera_sources = files([ 'camera_manager.cpp', 'camera_sensor.cpp', 'camera_sensor_properties.cpp', + 'color_space.cpp', 'controls.cpp', 'control_serializer.cpp', 'control_validator.cpp', From patchwork Wed Dec 1 15:44:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 14975 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 78FE8C3250 for ; Wed, 1 Dec 2021 15:45:07 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2E1B9607E3; Wed, 1 Dec 2021 16:45:07 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="qroTas8c"; dkim-atps=neutral Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id AE4F3607BD for ; Wed, 1 Dec 2021 16:45:04 +0100 (CET) Received: by mail-wr1-x430.google.com with SMTP id l16so53220883wrp.11 for ; Wed, 01 Dec 2021 07:45:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=eq/N2vQlsBncc36dYA90+vuvH+H4tIw8Sag0Uta/y1A=; b=qroTas8c94LB82l4oPAZAbO1qisqadcYrQf9CDQy9I0GUt/n8EKAhpj0lc2C3alqKZ JsHMOCr3vq9ndsi4PqFJ1lzNDm6HQGxEVEpd7sU/+Z1UhvTTKRBZ0qWVjWNsO4ocZu5Q 2II5Qgpv2gW9ovuC4wAQ6qf5ocusky5+OHJ8xzDeC8ePg1ofSZalXD5iZfY12FwyCGD7 g/Z0O2RLWItIY7b/h3QB5hDPJi6zLYrsC63PkJPljP5Kz/aTNy7sg14gHGx3kKX2hpWp YiOvBhAegjKtQy8qZED+XpLzWQxqGg8vLnargfbwKc9OQLkOUT972l0EiU2QWBU6MSLe zMqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=eq/N2vQlsBncc36dYA90+vuvH+H4tIw8Sag0Uta/y1A=; b=hnhpv3yMox3My+ZeKCuaA/V9caP2wqJo2Bm5mLF7EsFugnq27T35eG6DPDX4PvDsIp Zd6V1pdhZVRSMD8pAry7Ujsi7LyZ1W0gPnSIARWIf+nSsXNeJYwp3FW+AO9JPJ4LnT59 n7w2m1QOGV1oEm0wSc2896rTsfiGz47PZa6CH0J6UwpIdNMG0u75DMXnZTaeQdlmnYBY gIbXMVmVu1EVFAomwCzIxKSPoRqYvEcxhZmkcCaly5W1bdRDd/7XB1sLKykfdnZ2alnE 105qygs0S3z/pqMb0Y2+89+6mWd8IappG8JcARQRXfQ3J0i/HOEd1nYtm29fG0yWk6iK ZlJw== X-Gm-Message-State: AOAM533UGpo4G2oOtlYmf+bJoTekV/WIq6NCbwlX6uwkP+SdOyvmpHPD ynLZs9mjEH9s3KMhyDHxwXeKl/u5pLAS1wji X-Google-Smtp-Source: ABdhPJwlg5w/bwXue5WouxLniJtL5LkPs+Ar4Csk0cPIBfXGtzPIIreFcKQin5YNlpBtLwtuBvMMbA== X-Received: by 2002:a05:6000:118a:: with SMTP id g10mr7917833wrx.533.1638373504065; Wed, 01 Dec 2021 07:45:04 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:e4a2:3070:eea4:e434]) by smtp.gmail.com with ESMTPSA id y7sm148209wrw.55.2021.12.01.07.45.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Dec 2021 07:45:03 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org, Laurent Pinchart , Kieran Bingham , Jacopo Mondi , Hans Verkuil , Tomasz Figa , Naushir Patuck Date: Wed, 1 Dec 2021 15:44:28 +0000 Message-Id: <20211201154434.23127-3-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211201154434.23127-1-david.plowman@raspberrypi.com> References: <20211201154434.23127-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v8 2/8] libcamera: Add ColorSpace fields to StreamConfiguration X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This is so that applications can choose appropriate color spaces which will then be passed down to the V4L2 devices. The ColorSpace field is actually optional. If it is not set you will get the driver's default color space. Signed-off-by: David Plowman Reviewed-by: Umang Jain Reviewed-by: Jacopo Mondi --- include/libcamera/stream.h | 3 +++ src/libcamera/stream.cpp | 20 ++++++++++++++++++++ 2 files changed, 23 insertions(+) diff --git a/include/libcamera/stream.h b/include/libcamera/stream.h index 41ec02b1..f0ae7e62 100644 --- a/include/libcamera/stream.h +++ b/include/libcamera/stream.h @@ -12,6 +12,7 @@ #include #include +#include #include #include #include @@ -47,6 +48,8 @@ struct StreamConfiguration { unsigned int bufferCount; + std::optional colorSpace; + Stream *stream() const { return stream_; } void setStream(Stream *stream) { stream_ = stream; } const StreamFormats &formats() const { return formats_; } diff --git a/src/libcamera/stream.cpp b/src/libcamera/stream.cpp index b421e17e..686e693b 100644 --- a/src/libcamera/stream.cpp +++ b/src/libcamera/stream.cpp @@ -329,6 +329,26 @@ StreamConfiguration::StreamConfiguration(const StreamFormats &formats) * \brief Requested number of buffers to allocate for the stream */ +/** + * \var StreamConfiguration::colorSpace + * \brief The ColorSpace for this stream + * + * This field allows a ColorSpace to be selected for this Stream. + * + * The field is optional and an application can choose to leave it unset. + * Platforms that support the use of color spaces may provide default + * values through the generateConfiguration() method. An application can + * override these when necessary. + * + * If a specific ColorSpace is requested but the Camera cannot deliver it, + * then the StreamConfiguration will be adjusted to a value that can be + * delivered. In this case the validate() method will indicate via its + * return value that the CameraConfiguration has been adjusted. + * + * Note that platforms will typically have different constraints on what + * color spaces can be supported and in what combinations. + */ + /** * \fn StreamConfiguration::stream() * \brief Retrieve the stream associated with the configuration From patchwork Wed Dec 1 15:44:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 14976 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 66F3DBDB13 for ; Wed, 1 Dec 2021 15:45:09 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 125C7607D2; Wed, 1 Dec 2021 16:45:09 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="P3FZS8A0"; dkim-atps=neutral Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6491B607CB for ; Wed, 1 Dec 2021 16:45:05 +0100 (CET) Received: by mail-wr1-x435.google.com with SMTP id t9so36265822wrx.7 for ; Wed, 01 Dec 2021 07:45:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Ca56FPRLmW1i0SvhL9TsMbr7XLmHJto6AuJCbYbCA+k=; b=P3FZS8A0gAubQqPu09LJTaIWUy8lOme/DIewqm84+Fn0fWks5rWx5CbKmwKXOtcJTo hgb8AxxAU/mMp51Cj/f4yZ7f3oP4de4bXqGFMlJWNJCnDmsBa2GSpyPIPKjVfOlDlZ7Y ctpaBEBJpxuNp6Qu8YXCFFPT8AuIhp487ssg86YFAd3QnhIGa1Dab0P/yx6lyXOkh9KA khunhsJqpSdvcAR1z10jTodZblwsX4Rq6SxJaTHJHpTRQc/qteIBAis2ghrPNlCg9wcu ahTYHVAHW1QslBGrX3feWR5ynfJWnFoMxyPGFqgPnxEyN847w909P3SNt8wju0pk2TJ2 GN7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Ca56FPRLmW1i0SvhL9TsMbr7XLmHJto6AuJCbYbCA+k=; b=i5hWCkOIrja6k7DPXV9iDnm0kepEM4ZUxXZtMAuM8oD+Gs/SuIwA/ysM9tOfhlCPEM Uj+MG1YIVjowM8db82TJJjkPB+KWVWWJx62iGSKA8jP5ugIgOi0HwFDGEq+YIoevhgiA 6RQzqe2B66VRHcWwY7aRyFz5CcwQAVKicKn3uol2DppqtCBxml1XQROgm3WW8DWbXki7 dSPpxDNp/tCT2yiDpAH//sQZvfLYnp31kKTYfvzw1dFuV9g5RUJ502oiLF9i6fjSWHQi 3KVY9xt0kQYtl+k9jwvKFKpgEGq348XLEN6yVJlo9i0rNHwdkDBvSWGbd0yoma1Oa1rN F4Xg== X-Gm-Message-State: AOAM532vmW3jJL9ryKNTNWuQ5+kDkjPFDP19iDSAASqEySWrbX1U6Ckp zUpEOmMfev3zjBMKwB7TdxCErRAdWx1tqIcS X-Google-Smtp-Source: ABdhPJw0Ug83N3tajUzzIXQoTczM+hgOtGFZLl3gEyG8Xq+Z+l0BRr6QnjuayNUseSDW41xW5ZzjmQ== X-Received: by 2002:adf:d1e3:: with SMTP id g3mr7865116wrd.300.1638373504882; Wed, 01 Dec 2021 07:45:04 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:e4a2:3070:eea4:e434]) by smtp.gmail.com with ESMTPSA id y7sm148209wrw.55.2021.12.01.07.45.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Dec 2021 07:45:04 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org, Laurent Pinchart , Kieran Bingham , Jacopo Mondi , Hans Verkuil , Tomasz Figa , Naushir Patuck Date: Wed, 1 Dec 2021 15:44:29 +0000 Message-Id: <20211201154434.23127-4-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211201154434.23127-1-david.plowman@raspberrypi.com> References: <20211201154434.23127-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v8 3/8] libcamera: Convert between ColorSpace class and V4L2 formats X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add functions to the V4L2Device class to convert to and from libcamera ColorSpace. These methods are added to the base V4L2Device class so that they can be shared both by the video device class and subdevices. With the ColorSpace class, the color space and related other fields are stored together, corresponding to a number of fields in the various different V4L2 format structures. Template methods are therefore a convenient implementation, and we must explicitly instantiate the templates that will be needed. Note that unset color spaces are converted to requests for the device's "default" color space. Signed-off-by: David Plowman --- include/libcamera/internal/v4l2_device.h | 7 + src/libcamera/v4l2_device.cpp | 192 +++++++++++++++++++++++ 2 files changed, 199 insertions(+) diff --git a/include/libcamera/internal/v4l2_device.h b/include/libcamera/internal/v4l2_device.h index 7816a290..7f9b0f42 100644 --- a/include/libcamera/internal/v4l2_device.h +++ b/include/libcamera/internal/v4l2_device.h @@ -17,6 +17,7 @@ #include #include +#include #include namespace libcamera { @@ -44,6 +45,12 @@ public: void updateControlInfo(); + template + static std::optional toColorSpace(const T &v4l2Format); + + template + static int fromColorSpace(const std::optional &colorSpace, T &v4l2Format); + protected: V4L2Device(const std::string &deviceNode); ~V4L2Device(); diff --git a/src/libcamera/v4l2_device.cpp b/src/libcamera/v4l2_device.cpp index 9c783c9c..df92fa9e 100644 --- a/src/libcamera/v4l2_device.cpp +++ b/src/libcamera/v4l2_device.cpp @@ -16,6 +16,8 @@ #include #include +#include + #include #include #include @@ -731,4 +733,194 @@ void V4L2Device::eventAvailable() frameStart.emit(event.u.frame_sync.frame_sequence); } +static const std::map v4l2ToColorSpace = { + { V4L2_COLORSPACE_RAW, ColorSpace::Raw }, + { V4L2_COLORSPACE_JPEG, ColorSpace::Jpeg }, + { V4L2_COLORSPACE_SRGB, ColorSpace::Srgb }, + { V4L2_COLORSPACE_SMPTE170M, ColorSpace::Smpte170m }, + { V4L2_COLORSPACE_REC709, ColorSpace::Rec709 }, + { V4L2_COLORSPACE_BT2020, ColorSpace::Rec2020 }, +}; + +static const std::map v4l2ToYcbcrEncoding = { + { V4L2_YCBCR_ENC_601, ColorSpace::YcbcrEncoding::Rec601 }, + { V4L2_YCBCR_ENC_709, ColorSpace::YcbcrEncoding::Rec709 }, + { V4L2_YCBCR_ENC_BT2020, ColorSpace::YcbcrEncoding::Rec2020 }, +}; + +static const std::map v4l2ToTransferFunction = { + { V4L2_XFER_FUNC_NONE, ColorSpace::TransferFunction::Linear }, + { V4L2_XFER_FUNC_SRGB, ColorSpace::TransferFunction::Srgb }, + { V4L2_XFER_FUNC_709, ColorSpace::TransferFunction::Rec709 }, +}; + +static const std::map v4l2ToRange = { + { V4L2_QUANTIZATION_FULL_RANGE, ColorSpace::Range::Full }, + { V4L2_QUANTIZATION_LIM_RANGE, ColorSpace::Range::Limited }, +}; + +static const std::vector> colorSpaceToV4l2 = { + { ColorSpace::Raw, V4L2_COLORSPACE_RAW }, + { ColorSpace::Jpeg, V4L2_COLORSPACE_JPEG }, + { ColorSpace::Srgb, V4L2_COLORSPACE_SRGB }, + { ColorSpace::Smpte170m, V4L2_COLORSPACE_SMPTE170M }, + { ColorSpace::Rec709, V4L2_COLORSPACE_REC709 }, + { ColorSpace::Rec2020, V4L2_COLORSPACE_BT2020 }, +}; + +static const std::map primariesToV4l2 = { + { ColorSpace::Primaries::Raw, V4L2_COLORSPACE_RAW }, + { ColorSpace::Primaries::Smpte170m, V4L2_COLORSPACE_SMPTE170M }, + { ColorSpace::Primaries::Rec709, V4L2_COLORSPACE_REC709 }, + { ColorSpace::Primaries::Rec2020, V4L2_COLORSPACE_BT2020 }, +}; + +static const std::map ycbcrEncodingToV4l2 = { + { ColorSpace::YcbcrEncoding::Rec601, V4L2_YCBCR_ENC_601 }, + { ColorSpace::YcbcrEncoding::Rec709, V4L2_YCBCR_ENC_709 }, + { ColorSpace::YcbcrEncoding::Rec2020, V4L2_YCBCR_ENC_BT2020 }, +}; + +static const std::map transferFunctionToV4l2 = { + { ColorSpace::TransferFunction::Linear, V4L2_XFER_FUNC_NONE }, + { ColorSpace::TransferFunction::Srgb, V4L2_XFER_FUNC_SRGB }, + { ColorSpace::TransferFunction::Rec709, V4L2_XFER_FUNC_709 }, +}; + +static const std::map rangeToV4l2 = { + { ColorSpace::Range::Full, V4L2_QUANTIZATION_FULL_RANGE }, + { ColorSpace::Range::Limited, V4L2_QUANTIZATION_LIM_RANGE }, +}; + +/** + * \brief Convert the color space fields in a V4L2 format to a ColorSpace + * \param[in] v4l2Format A V4L2 format containing color space information + * + * The colorspace, ycbcr_enc, xfer_func and quantization fields within a + * V4L2 format structure are converted to a corresponding ColorSpace. + * + * If any V4L2 fields are not recognised then we return an "unset" + * color space. + * + * \return The ColorSpace corresponding to the input V4L2 format + * \retval std::nullopt One or more V4L2 color space fields were not recognised + */ +template +std::optional V4L2Device::toColorSpace(const T &v4l2Format) +{ + auto itColor = v4l2ToColorSpace.find(v4l2Format.colorspace); + if (itColor == v4l2ToColorSpace.end()) + return std::nullopt; + + /* This sets all the color space fields to the correct "default" values. */ + ColorSpace colorSpace = itColor->second; + + if (v4l2Format.ycbcr_enc != V4L2_YCBCR_ENC_DEFAULT) { + auto itYcbcrEncoding = v4l2ToYcbcrEncoding.find(v4l2Format.ycbcr_enc); + if (itYcbcrEncoding == v4l2ToYcbcrEncoding.end()) + return std::nullopt; + + colorSpace.ycbcrEncoding = itYcbcrEncoding->second; + } + + if (v4l2Format.xfer_func != V4L2_XFER_FUNC_DEFAULT) { + auto itTransfer = v4l2ToTransferFunction.find(v4l2Format.xfer_func); + if (itTransfer == v4l2ToTransferFunction.end()) + return std::nullopt; + + colorSpace.transferFunction = itTransfer->second; + } + + if (v4l2Format.quantization != V4L2_QUANTIZATION_DEFAULT) { + auto itRange = v4l2ToRange.find(v4l2Format.quantization); + if (itRange == v4l2ToRange.end()) + return std::nullopt; + + colorSpace.range = itRange->second; + } + + return colorSpace; +} + +template std::optional V4L2Device::toColorSpace(const struct v4l2_pix_format &); +template std::optional V4L2Device::toColorSpace(const struct v4l2_pix_format_mplane &); +template std::optional V4L2Device::toColorSpace(const struct v4l2_mbus_framefmt &); + +/** + * \brief Fill in the color space fields of a V4L2 format from a ColorSpace + * \param[in] colorSpace The ColorSpace to be converted + * \param[out] v4l2Format A V4L2 format containing color space information + * + * The colorspace, ycbcr_enc, xfer_func and quantization fields within a + * V4L2 format structure are filled in from a corresponding ColorSpace. + * + * An error is returned if any of the V4L2 fields do not support the + * value given in the ColorSpace. Such fields are set to the V4L2 + * "default" values, but all other fields are still filled in where + * possible. + * + * If the color space is completely unset, "default" V4L2 values are used + * everywhere, so a driver would then choose its preferred color space. + * + * \return 0 on success or a negative error code otherwise + * \retval -EINVAL The ColorSpace does not have a representation using V4L2 enums + */ +template +int V4L2Device::fromColorSpace(const std::optional &colorSpace, T &v4l2Format) +{ + v4l2Format.colorspace = V4L2_COLORSPACE_DEFAULT; + v4l2Format.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; + v4l2Format.xfer_func = V4L2_XFER_FUNC_DEFAULT; + v4l2Format.quantization = V4L2_QUANTIZATION_DEFAULT; + + if (!colorSpace) + return 0; + + auto itColor = std::find_if(colorSpaceToV4l2.begin(), colorSpaceToV4l2.end(), + [&colorSpace](const auto &item) { + return colorSpace == item.first; + }); + if (itColor != colorSpaceToV4l2.end()) { + v4l2Format.colorspace = itColor->second; + /* Leaving all the other fields as "default" should be fine. */ + return 0; + } + + /* + * If the colorSpace doesn't precisely match a standard color space, + * then we must choose a V4L2 colorspace with matching primaries. + */ + int ret = 0; + + auto itPrimaries = primariesToV4l2.find(colorSpace->primaries); + if (itPrimaries != primariesToV4l2.end()) + v4l2Format.colorspace = itPrimaries->second; + else + ret = -EINVAL; + + auto itYcbcrEncoding = ycbcrEncodingToV4l2.find(colorSpace->ycbcrEncoding); + if (itYcbcrEncoding != ycbcrEncodingToV4l2.end()) + v4l2Format.ycbcr_enc = itYcbcrEncoding->second; + else + ret = -EINVAL; + + auto itTransfer = transferFunctionToV4l2.find(colorSpace->transferFunction); + if (itTransfer != transferFunctionToV4l2.end()) + v4l2Format.xfer_func = itTransfer->second; + else + ret = -EINVAL; + + auto itRange = rangeToV4l2.find(colorSpace->range); + if (itRange != rangeToV4l2.end()) + v4l2Format.quantization = itRange->second; + else + ret = -EINVAL; + + return ret; +} + +template int V4L2Device::fromColorSpace(const std::optional &, struct v4l2_pix_format &); +template int V4L2Device::fromColorSpace(const std::optional &, struct v4l2_pix_format_mplane &); +template int V4L2Device::fromColorSpace(const std::optional &, struct v4l2_mbus_framefmt &); + } /* namespace libcamera */ From patchwork Wed Dec 1 15:44:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 14977 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B7213C3250 for ; Wed, 1 Dec 2021 15:45:10 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4EB16607DE; Wed, 1 Dec 2021 16:45:10 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="Vtk4QZJl"; dkim-atps=neutral Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 163C160746 for ; Wed, 1 Dec 2021 16:45:06 +0100 (CET) Received: by mail-wr1-x435.google.com with SMTP id u1so53210497wru.13 for ; Wed, 01 Dec 2021 07:45:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=wHD2U/BDIz9i1ybZ6mxu9fSHa6cLozW0JrsZbuoIEVw=; b=Vtk4QZJlvHvPpEOSakoDrBqib5D38hSOVoPvxh1wu6dvtOLPC1AXaKwvuZfY4cmWw+ cxjy73psjMOg4beSzfiFIWipP56WZ4LDwJ1QKrTqLl8u++Oec2aX88PRlCUSVZ3SIr+k NDmq6TXidgeet4V9ZtENqUWVGr/0GORQnTObbSpouFlPcgQpDuitWLT+RMWJ9gYlZjnh 5svusblEiyS67u0HBpyYj5/SnGlyuW2Sjq4JV8epIbSt87hcquh8sn7f0hCCOZtC9cLt 788IdMu+KBQzmrvUGw3Qswos+CEw5KdgM5NqyI5AjfonFxatHiRGNE0hzNvrgx2dyNdI SjGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wHD2U/BDIz9i1ybZ6mxu9fSHa6cLozW0JrsZbuoIEVw=; b=BR9aRfQILIudZT0oRRQPw/u/LZGSZ7RKxYeujTVBcH5lURjraESm4p0If7lGESAVf9 KGrOnbnksgPSFgciv6Yfj/n5H97EYPhognUQUNudrdhSQXi9FKXAN90CzQQ7UXzQSXOB Mk7oDu3oe7Otfxi1pf80Mlfqtw96UPzl/u7LBKW8DrYD1b6iur0s22ayBXXsLg6ADUID VXnbs9LzTA9Xw7VupfXKJhN/VzplZ0ZsxcSNzqOXzXktg1P1MKt8yOn/NMknGNtiJ2OQ Eb0Y1QCBc9LwF2V4Fd9PhrNyrhoxGRdxia8beDUZsDZd/We9/IjN+sptHbmFKbiEokbd IsUw== X-Gm-Message-State: AOAM5319Ko0vCbn4KKVheBRMXw19/EKBU1kBXRyxlrYN9/1K5vwIhGbC 5+7dCNBBEv4znZb/va58MTqdzojSg1xnjuP+ X-Google-Smtp-Source: ABdhPJyK2OXU0PZYRH5YrUGsOSC58KpgSoSPam4SCqv2hIb3K/STrQNFAUccXaDPRnEqduqDgBQblw== X-Received: by 2002:a05:6000:18a7:: with SMTP id b7mr7576214wri.308.1638373505629; Wed, 01 Dec 2021 07:45:05 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:e4a2:3070:eea4:e434]) by smtp.gmail.com with ESMTPSA id y7sm148209wrw.55.2021.12.01.07.45.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Dec 2021 07:45:05 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org, Laurent Pinchart , Kieran Bingham , Jacopo Mondi , Hans Verkuil , Tomasz Figa , Naushir Patuck Date: Wed, 1 Dec 2021 15:44:30 +0000 Message-Id: <20211201154434.23127-5-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211201154434.23127-1-david.plowman@raspberrypi.com> References: <20211201154434.23127-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v8 4/8] libcamera: Support passing ColorSpaces to V4L2 video devices X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The ColorSpace from the StreamConfiguration is now handled appropriately in the V4L2VideoDevice. Signed-off-by: David Plowman --- include/libcamera/internal/v4l2_videodevice.h | 2 ++ src/libcamera/v4l2_videodevice.cpp | 32 +++++++++++++++++++ 2 files changed, 34 insertions(+) diff --git a/include/libcamera/internal/v4l2_videodevice.h b/include/libcamera/internal/v4l2_videodevice.h index 4a44b7fd..602f40fb 100644 --- a/include/libcamera/internal/v4l2_videodevice.h +++ b/include/libcamera/internal/v4l2_videodevice.h @@ -20,6 +20,7 @@ #include #include +#include #include #include #include @@ -167,6 +168,7 @@ public: V4L2PixelFormat fourcc; Size size; + std::optional colorSpace; std::array planes; unsigned int planesCount = 0; diff --git a/src/libcamera/v4l2_videodevice.cpp b/src/libcamera/v4l2_videodevice.cpp index 4f04212d..c8f78602 100644 --- a/src/libcamera/v4l2_videodevice.cpp +++ b/src/libcamera/v4l2_videodevice.cpp @@ -383,6 +383,21 @@ bool V4L2BufferCache::Entry::operator==(const FrameBuffer &buffer) const * that identifies the image format pixel encoding scheme. */ +/** + * \var V4L2DeviceFormat::colorSpace + * \brief The color space of the pixels + * + * The color space of the image. When setting the format this may be + * unset, in which case the driver gets to use its default color space. + * After being set, this value should contain the color space that + * was actually used. If this value is unset, then the color space chosen + * by the driver could not be represented by the ColorSpace class (and + * should probably be added). + * + * It is up to the pipeline handler or application to check if the + * resulting color space is acceptable. + */ + /** * \var V4L2DeviceFormat::planes * \brief The per-plane memory size information @@ -878,6 +893,7 @@ int V4L2VideoDevice::getFormatMultiplane(V4L2DeviceFormat *format) format->size.height = pix->height; format->fourcc = V4L2PixelFormat(pix->pixelformat); format->planesCount = pix->num_planes; + format->colorSpace = toColorSpace(*pix); for (unsigned int i = 0; i < format->planesCount; ++i) { format->planes[i].bpl = pix->plane_fmt[i].bytesperline; @@ -900,6 +916,12 @@ int V4L2VideoDevice::trySetFormatMultiplane(V4L2DeviceFormat *format, bool set) pix->num_planes = format->planesCount; pix->field = V4L2_FIELD_NONE; + ret = fromColorSpace(format->colorSpace, *pix); + if (ret < 0) + LOG(V4L2, Warning) + << "Setting color space unrecognised by V4L2: " + << ColorSpace::toString(format->colorSpace); + ASSERT(pix->num_planes <= std::size(pix->plane_fmt)); for (unsigned int i = 0; i < pix->num_planes; ++i) { @@ -927,6 +949,7 @@ int V4L2VideoDevice::trySetFormatMultiplane(V4L2DeviceFormat *format, bool set) format->planes[i].bpl = pix->plane_fmt[i].bytesperline; format->planes[i].size = pix->plane_fmt[i].sizeimage; } + format->colorSpace = toColorSpace(*pix); return 0; } @@ -950,6 +973,7 @@ int V4L2VideoDevice::getFormatSingleplane(V4L2DeviceFormat *format) format->planesCount = 1; format->planes[0].bpl = pix->bytesperline; format->planes[0].size = pix->sizeimage; + format->colorSpace = toColorSpace(*pix); return 0; } @@ -966,6 +990,13 @@ int V4L2VideoDevice::trySetFormatSingleplane(V4L2DeviceFormat *format, bool set) pix->pixelformat = format->fourcc; pix->bytesperline = format->planes[0].bpl; pix->field = V4L2_FIELD_NONE; + + ret = fromColorSpace(format->colorSpace, *pix); + if (ret < 0) + LOG(V4L2, Warning) + << "Set color space unrecognised by V4L2: " + << ColorSpace::toString(format->colorSpace); + ret = ioctl(set ? VIDIOC_S_FMT : VIDIOC_TRY_FMT, &v4l2Format); if (ret) { LOG(V4L2, Error) @@ -984,6 +1015,7 @@ int V4L2VideoDevice::trySetFormatSingleplane(V4L2DeviceFormat *format, bool set) format->planesCount = 1; format->planes[0].bpl = pix->bytesperline; format->planes[0].size = pix->sizeimage; + format->colorSpace = toColorSpace(*pix); return 0; } From patchwork Wed Dec 1 15:44:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 14978 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5F8B5BDB13 for ; Wed, 1 Dec 2021 15:45:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CBE50607CC; Wed, 1 Dec 2021 16:45:10 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="RnGhUj9/"; dkim-atps=neutral Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E14AF607DE for ; Wed, 1 Dec 2021 16:45:06 +0100 (CET) Received: by mail-wm1-x329.google.com with SMTP id p27-20020a05600c1d9b00b0033bf8532855so1477561wms.3 for ; Wed, 01 Dec 2021 07:45:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=qxoMJAMEnsNt1Akqwe6Jje7fWqWl6ifflaUOF436ktw=; b=RnGhUj9/tOuii5ctHPr1GYOvX7nvvAB9vPFMHO0uA7B46Gnum4vNx+VM7ayHGhcfkR 4mSSfa6u+aFCeV1DmG6vMEwx6HZ117HybTiyqZf8gtGv/uBbsgKCAeUrP+R0s7v3ILQl SgjmJ7BXXtzoqG8ness0TIVniBAjmezfA0rMQomCAA08AmpfjrdkpUUPnIY1AxJoUS+G yQxg/PxC/rdEZxYxY0Ee3s75+wiScIwHr6ngKUE5E0a9enxRYF6Tu+sSeBr14uetoGFe ff8Frs4j/rWVjNU2kmQivbQRPuWdzFGWAIay4pVPm75+P9nd2ZHEcHArWGbOEQ5YrJPF 3Qrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=qxoMJAMEnsNt1Akqwe6Jje7fWqWl6ifflaUOF436ktw=; b=O/akgXcg7Os+AP6vvWA6m4rS0wMQi3PzBMy5YVyoCdfLElRwZrD4c4LS4ICpK7Wcoi SoS4QRLxmxTGi0jy0+5hKt6f5YvNS36KgAhSzB5rIJ5iFaahUb9qwOt6eB715FPK0g2J eFhGb4sVctWBTg1raq9S4bYOCEGoyGnfkgiPmpSSUHEYzqJK7bqmLhnhJMYbPfaXG8of P0QYz0aiss5LSQnMLDj36c4Pc9mfly4xLInb8F/RKO+uYH3UHF2eBiIX31ewnxIE6nDg 1RKYQTXqpf9oUVxl1sFSh7e30DbVggPwykPcefoO+cT23c3xzEmTgMW+IkNw+AimLn6p uMkA== X-Gm-Message-State: AOAM5336bZervA+yFPYHuC5/YKvV1j3zvFqCL/Y+90KQIs4YXVtGX3UM +Og4gLCI06MkmH8+vrRgRrNX3i5dxTcJLwBL X-Google-Smtp-Source: ABdhPJycHHoIczHQ36k4uzjPCGiKToot2FktPoCIvZQwnouJmZAlgRW/9+RCDhZzQlLjNkfzKCK61g== X-Received: by 2002:a1c:1c1:: with SMTP id 184mr8002016wmb.1.1638373506349; Wed, 01 Dec 2021 07:45:06 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:e4a2:3070:eea4:e434]) by smtp.gmail.com with ESMTPSA id y7sm148209wrw.55.2021.12.01.07.45.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Dec 2021 07:45:05 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org, Laurent Pinchart , Kieran Bingham , Jacopo Mondi , Hans Verkuil , Tomasz Figa , Naushir Patuck Date: Wed, 1 Dec 2021 15:44:31 +0000 Message-Id: <20211201154434.23127-6-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211201154434.23127-1-david.plowman@raspberrypi.com> References: <20211201154434.23127-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v8 5/8] libcamera: Add colorSpace field to V4L2SubdeviceFormat X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This adds a ColorSpace field to the V4L2SubdeviceFormat so that we can set and request particular color spaces from V4L2. This commit simply adds the field and fixes some occurrences of brace initializers that would otherwise be broken. A subsequent commit will pass and retrieve the value correctly to/from V4l2 itself. Signed-off-by: David Plowman --- include/libcamera/internal/v4l2_subdevice.h | 2 ++ src/libcamera/camera_sensor.cpp | 1 + src/libcamera/pipeline/ipu3/cio2.cpp | 7 +++---- src/libcamera/pipeline/simple/simple.cpp | 8 ++++++-- src/libcamera/v4l2_subdevice.cpp | 15 +++++++++++++++ 5 files changed, 27 insertions(+), 6 deletions(-) diff --git a/include/libcamera/internal/v4l2_subdevice.h b/include/libcamera/internal/v4l2_subdevice.h index 484fcfdd..e6fa451b 100644 --- a/include/libcamera/internal/v4l2_subdevice.h +++ b/include/libcamera/internal/v4l2_subdevice.h @@ -14,6 +14,7 @@ #include #include +#include #include #include "libcamera/internal/formats.h" @@ -27,6 +28,7 @@ class MediaDevice; struct V4L2SubdeviceFormat { uint32_t mbus_code; Size size; + std::optional colorSpace; const std::string toString() const; uint8_t bitsPerPixel() const; diff --git a/src/libcamera/camera_sensor.cpp b/src/libcamera/camera_sensor.cpp index 9fdb8c09..6fcd1c1d 100644 --- a/src/libcamera/camera_sensor.cpp +++ b/src/libcamera/camera_sensor.cpp @@ -613,6 +613,7 @@ V4L2SubdeviceFormat CameraSensor::getFormat(const std::vector &mbu V4L2SubdeviceFormat format{ .mbus_code = bestCode, .size = *bestSize, + .colorSpace = ColorSpace::Raw, }; return format; diff --git a/src/libcamera/pipeline/ipu3/cio2.cpp b/src/libcamera/pipeline/ipu3/cio2.cpp index 59dda56b..f4e8c663 100644 --- a/src/libcamera/pipeline/ipu3/cio2.cpp +++ b/src/libcamera/pipeline/ipu3/cio2.cpp @@ -322,10 +322,9 @@ V4L2SubdeviceFormat CIO2Device::getSensorFormat(const std::vector return {}; } - V4L2SubdeviceFormat format{ - .mbus_code = bestCode, - .size = bestSize, - }; + V4L2SubdeviceFormat format{}; + format.mbus_code = bestCode; + format.size = bestSize; return format; } diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 701fb4be..a3108fc0 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -457,7 +457,9 @@ int SimpleCameraData::init() * formats on the video node. */ for (unsigned int code : sensor_->mbusCodes()) { - V4L2SubdeviceFormat format{ code, sensor_->resolution() }; + V4L2SubdeviceFormat format{}; + format.mbus_code = code; + format.size = sensor_->resolution(); ret = setupFormats(&format, V4L2Subdevice::TryFormat); if (ret < 0) { @@ -908,7 +910,9 @@ int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c) return ret; const SimpleCameraData::Configuration *pipeConfig = config->pipeConfig(); - V4L2SubdeviceFormat format{ pipeConfig->code, data->sensor_->resolution() }; + V4L2SubdeviceFormat format{}; + format.mbus_code = pipeConfig->code; + format.size = data->sensor_->resolution(); ret = data->setupFormats(&format, V4L2Subdevice::ActiveFormat); if (ret < 0) diff --git a/src/libcamera/v4l2_subdevice.cpp b/src/libcamera/v4l2_subdevice.cpp index 023e2328..66e08333 100644 --- a/src/libcamera/v4l2_subdevice.cpp +++ b/src/libcamera/v4l2_subdevice.cpp @@ -168,6 +168,21 @@ const std::map formatInfoMap = { * \brief The image size in pixels */ +/** + * \var V4L2SubdeviceFormat::colorSpace + * \brief The color space of the pixels + * + * The color space of the image. When setting the format this may be + * unset, in which case the driver gets to use its default color space. + * After being set, this value should contain the color space that + * was actually used. If this value is unset, then the color space chosen + * by the driver could not be represented by the ColorSpace class (and + * should probably be added). + * + * It is up to the pipeline handler or application to check if the + * resulting color space is acceptable. + */ + /** * \brief Assemble and return a string describing the format * \return A string describing the V4L2SubdeviceFormat From patchwork Wed Dec 1 15:44:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 14979 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1E85AC3251 for ; Wed, 1 Dec 2021 15:45:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 50A10607C5; Wed, 1 Dec 2021 16:45:11 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="MqYFmHu8"; dkim-atps=neutral Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com [IPv6:2a00:1450:4864:20::433]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 837FF607DA for ; Wed, 1 Dec 2021 16:45:07 +0100 (CET) Received: by mail-wr1-x433.google.com with SMTP id a18so53329372wrn.6 for ; Wed, 01 Dec 2021 07:45:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ZHv2BE5qFSbf6FhZH1c6JuntQk94zpM3K7P0xT4gJ58=; b=MqYFmHu896DyNgClkTvpPw1lmZ5gDJ/nWnsY54FUQAaXxcLqiApRaex/Uc8q9g+sgU AnohFjrJOWHBGCpBuuLqz41VmxmniVjBuYBqV94F6QNqQHsXdcIKf13pZuCUlulBMkn7 29eehjrtV7u2hHz0hchNaSLnYf1e3PATvUlWZv52pm+/6N/rKfGKdcH4Qu7J8Zi0KEB4 jL3O8Vhe+v7bzD/H51NYFVwmeyAoBUhE6/koy37f/PoAJdQ7i6MFAktBklZsLDh3vsai Kl1Ot1bVHzskh9If7Yc0GVxgvrBmDDzXKUQOW3rp5loisyaPTcu9kxYCoxmf2J97X9SF /DEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ZHv2BE5qFSbf6FhZH1c6JuntQk94zpM3K7P0xT4gJ58=; b=LnbvhYhvOsbW344+Y8sAVAbSTLzWvNASSNx/3pTnf7U3+5ssMeFIliKQvZvXXNpEiT KCvKcc8iYBGBj7idFMD2HY76LYjlpgYfD2ggK5dUXKluqmZBuJGNJ5u4k5ID8JcwzkA8 74Vo0S19fbxMtxgSGmJi3uG3U86KTDoMKhfmY46QzdeG3NMLf5WvWpOtRHL4FiaGmB+R IFPG6GNhz4NebAtvMPBt1WxG3ZuFvPSrQe6iGDuKckOu0jfCE0PXg4yOVndwmDpNwmBL lKSEkozR2qFdPeP01cHBYetRZIKmQKoi4qnWqNN9W8nqMkjNAzkEaX9TVsffW0pG2xt4 itgQ== X-Gm-Message-State: AOAM530ZzwQTVbxAEKLLEgRXjOwWOSGlCH6jgNCL9mz79MxpeRyF9acG MwMYUvCrCOWbH7kAEl7plCheAKLxmUC3b+ai X-Google-Smtp-Source: ABdhPJzllcTQVaPtOFwNq5GeDOat9k+glRl3C6jlzFsj0bTitr/6xptJtc2MYCD3O8z9+9BtPbtPBQ== X-Received: by 2002:adf:df0d:: with SMTP id y13mr7754214wrl.176.1638373507117; Wed, 01 Dec 2021 07:45:07 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:e4a2:3070:eea4:e434]) by smtp.gmail.com with ESMTPSA id y7sm148209wrw.55.2021.12.01.07.45.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Dec 2021 07:45:06 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org, Laurent Pinchart , Kieran Bingham , Jacopo Mondi , Hans Verkuil , Tomasz Figa , Naushir Patuck Date: Wed, 1 Dec 2021 15:44:32 +0000 Message-Id: <20211201154434.23127-7-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211201154434.23127-1-david.plowman@raspberrypi.com> References: <20211201154434.23127-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v8 6/8] libcamera: Support passing ColorSpaces to V4L2 subdevices X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The ColorSpace from the StreamConfiguration is now handled appropriately in the V4L2Subdevice. Signed-off-by: David Plowman --- src/libcamera/v4l2_subdevice.cpp | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/src/libcamera/v4l2_subdevice.cpp b/src/libcamera/v4l2_subdevice.cpp index 66e08333..ee450c31 100644 --- a/src/libcamera/v4l2_subdevice.cpp +++ b/src/libcamera/v4l2_subdevice.cpp @@ -414,6 +414,7 @@ int V4L2Subdevice::getFormat(unsigned int pad, V4L2SubdeviceFormat *format, format->size.width = subdevFmt.format.width; format->size.height = subdevFmt.format.height; format->mbus_code = subdevFmt.format.code; + format->colorSpace = toColorSpace(subdevFmt.format); return 0; } @@ -442,7 +443,13 @@ int V4L2Subdevice::setFormat(unsigned int pad, V4L2SubdeviceFormat *format, subdevFmt.format.code = format->mbus_code; subdevFmt.format.field = V4L2_FIELD_NONE; - int ret = ioctl(VIDIOC_SUBDEV_S_FMT, &subdevFmt); + int ret = fromColorSpace(format->colorSpace, subdevFmt.format); + if (ret < 0) + LOG(V4L2, Warning) + << "Setting color space unrecognised by V4L2: " + << ColorSpace::toString(format->colorSpace); + + ret = ioctl(VIDIOC_SUBDEV_S_FMT, &subdevFmt); if (ret) { LOG(V4L2, Error) << "Unable to set format on pad " << pad @@ -453,6 +460,7 @@ int V4L2Subdevice::setFormat(unsigned int pad, V4L2SubdeviceFormat *format, format->size.width = subdevFmt.format.width; format->size.height = subdevFmt.format.height; format->mbus_code = subdevFmt.format.code; + format->colorSpace = toColorSpace(subdevFmt.format); return 0; } From patchwork Wed Dec 1 15:44:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 14980 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C606CC3252 for ; Wed, 1 Dec 2021 15:45:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6572560820; Wed, 1 Dec 2021 16:45:12 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="O4Tizd0u"; dkim-atps=neutral Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7BCD460746 for ; Wed, 1 Dec 2021 16:45:08 +0100 (CET) Received: by mail-wr1-x42e.google.com with SMTP id t9so36266397wrx.7 for ; Wed, 01 Dec 2021 07:45:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hdblI0WBXeyTejRYwVT6qZCUGz4IKvH5clh91jOGakA=; b=O4Tizd0ufv8BPCCjyRZQwG0k7nQK+CIq2C8R6lANccuvIxA4dew6KWhWaijcnTiDJq xTObLvyCzJonU+4Z3fAJhtfia1Y7kxqwJ4/hTVw727tUQmwxNfrnXhwtCeqIsOkid8r4 /9P3cCOWmQxAk5YyIVXhwvZjwaWBDneEzXfkiO2q4633qfpZ1gxfx+HUtqprCEXrSAbU aw/jIEpkkdu8fmRKlzOZpmQ5j/RzEeOVsBCKPZkJ+HvTVoC0+Qf1xWRU8o99jwuMzjna KHuxG8E3KTIxKTkI/cy8XEHOyK4MtX0oEPImhdWsRe+f0e9L4gJ1fI0XpYEHnD0RxCcX meVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hdblI0WBXeyTejRYwVT6qZCUGz4IKvH5clh91jOGakA=; b=BqxTk4/J1Vw2+vk6IdJT7ehJXvu4GA0yRmXbJpW7t/FDJXGf2TlLxKpaqhwXjxtlOs 2VB3iGRWSiEA1q8M0tMZkf1yGWSdEp2wr28WEqJfPl5iutTdSN/n1F9uhVe8quyuZH2g dv15zldsegpnWxwPQD5FzO8SvF4gk6TBilp6+P32i9ACB7vYuR9Nfxxpnc6bzk6vS5X4 qly+l9qVqd44sJyZRM0Kj4uuEhKF01DUDe6+eEaI/R3kIanV0OQg91YxDh/oQmwtIVwH 2mli2d8MJdIxh02/uL/xfjph8NCnT3ay+NnoCt2Qp6Xdj0SOibscWxT+rVjjWHaemdCZ IsjQ== X-Gm-Message-State: AOAM530LVOMTQL1j0beF3zBHNxKj0STCR+joj0s/Iyd6mrSFmITVfxUk CgsNl3ugHjxTE1HbPl7lWVMCTfezTUy7jUkY X-Google-Smtp-Source: ABdhPJyT5wRzPpQCPtL9exMksCcsQ65Dmyqxf4Az/sFoep42HO+20b1XEsVCrCqO9FJQlAtkg+IgUQ== X-Received: by 2002:a5d:648e:: with SMTP id o14mr7477279wri.141.1638373508039; Wed, 01 Dec 2021 07:45:08 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:e4a2:3070:eea4:e434]) by smtp.gmail.com with ESMTPSA id y7sm148209wrw.55.2021.12.01.07.45.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Dec 2021 07:45:07 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org, Laurent Pinchart , Kieran Bingham , Jacopo Mondi , Hans Verkuil , Tomasz Figa , Naushir Patuck Date: Wed, 1 Dec 2021 15:44:33 +0000 Message-Id: <20211201154434.23127-8-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211201154434.23127-1-david.plowman@raspberrypi.com> References: <20211201154434.23127-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v8 7/8] libcamera: Add validateColorSpaces to CameraConfiguration class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This method forces raw streams to have the "raw" color space, and also optionally makes all output streams to share the same color space as some platforms may require this. Signed-off-by: David Plowman Reviewed-by: Jacopo Mondi --- include/libcamera/camera.h | 2 ++ src/libcamera/camera.cpp | 69 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 71 insertions(+) diff --git a/include/libcamera/camera.h b/include/libcamera/camera.h index a7759ccb..fdcb66ab 100644 --- a/include/libcamera/camera.h +++ b/include/libcamera/camera.h @@ -69,6 +69,8 @@ public: protected: CameraConfiguration(); + Status validateColorSpaces(bool shareOutputColorSpaces); + std::vector config_; }; diff --git a/src/libcamera/camera.cpp b/src/libcamera/camera.cpp index 400a7cf0..a6a93dd0 100644 --- a/src/libcamera/camera.cpp +++ b/src/libcamera/camera.cpp @@ -14,12 +14,14 @@ #include #include +#include #include #include #include #include "libcamera/internal/camera.h" #include "libcamera/internal/camera_controls.h" +#include "libcamera/internal/formats.h" #include "libcamera/internal/pipeline_handler.h" /** @@ -314,6 +316,73 @@ std::size_t CameraConfiguration::size() const return config_.size(); } +static bool isRaw(const PixelFormat &pixFmt) +{ + const PixelFormatInfo &info = PixelFormatInfo::info(pixFmt); + return info.isValid() && + info.colourEncoding == PixelFormatInfo::ColourEncodingRAW; +} + +/** + * \brief Check the color spaces requested for each stream + * \param[in] shareOutputColorSpaces Force all output streams to share the same + * color space + * + * This function performs certain consistency checks on the color spaces of + * the streams and may adjust them so that: + * + * - Any raw streams have the Raw color space + * - If shareOutputColorSpaces is set, all output streams are forced to share + * the same color space (this may be a constraint on some platforms). + * + * It is optional for a pipeline handler to use this method. + * + * \return A CameraConfiguration::Status value that describes the validation + * status. + * \retval CameraConfiguration::Invalid The configuration is invalid and can't + * be adjusted. This may only occur in extreme cases such as when the + * configuration is empty. + * \retval CameraConfigutation::Adjusted The configuration has been adjusted + * and is now valid. Parameters may have changed for any stream, and stream + * configurations may have been removed. The caller shall check the + * configuration carefully. + * \retval CameraConfiguration::Valid The configuration was already valid and + * hasn't been adjusted. + */ +CameraConfiguration::Status CameraConfiguration::validateColorSpaces(bool shareOutputColorSpaces) +{ + Status status = Valid; + + /* + * Set all raw streams to the Raw color space, and make a note of the largest + * non-raw stream with a defined color space (if there is one). + */ + int index = -1; + for (auto [i, cfg] : utils::enumerate(config_)) { + if (isRaw(cfg.pixelFormat)) { + if (cfg.colorSpace != ColorSpace::Raw) { + cfg.colorSpace = ColorSpace::Raw; + status = Adjusted; + } + else if (cfg.colorSpace && (index == -1 || cfg.size > config_[i].size)) + index = i; + } + + if (index < 0 || !shareOutputColorSpaces) + return status; + + /* Make all output color spaces the same, if requested. */ + for (auto &cfg : config_) { + if (!isRaw(cfg.pixelFormat) && + cfg.colorSpace != config_[index].colorSpace) { + cfg.colorSpace = config_[index].colorSpace; + status = Adjusted; + } + } + + return status; +} + /** * \var CameraConfiguration::transform * \brief User-specified transform to be applied to the image From patchwork Wed Dec 1 15:44:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 14981 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 26F36C3250 for ; Wed, 1 Dec 2021 15:45:14 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CC605607BD; Wed, 1 Dec 2021 16:45:13 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="egZLE11i"; dkim-atps=neutral Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 44E3E60820 for ; Wed, 1 Dec 2021 16:45:09 +0100 (CET) Received: by mail-wm1-x32c.google.com with SMTP id o19-20020a1c7513000000b0033a93202467so1478776wmc.2 for ; Wed, 01 Dec 2021 07:45:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=VJw90Ju80i+WaMP0k5HDW9tQHz/oLLZEeXbCbFwltEk=; b=egZLE11ijGhRUxvpsICptPjzDRmOqV7byRQKeHG69VPEclD5NJdq5uUTZLWode4W3l WR8JQeWjlsJcFoObqu/lgM8wMIoxF1RriFtew5Sll9opzDculqo0mlTygQbB1Z1Jjkkr Uq+68914b6W6m0zpAOr1lOLysQu6iYhjrNYTXgb7vObUNOCKtuJw/VkCUfsRrSQjcZ2P GqbaZI8IjE1fnWhREYlRdCc/tTwANthobrROejZOmlKZ7w1XKX/dOrltmAEbAZKPgkDa 6OMnm6SM2WLov/LOQxHlOBgjWdESgLKoi2C78rzDmVVYw9kUb07gl/SkOoyX+pylMRT1 SroA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=VJw90Ju80i+WaMP0k5HDW9tQHz/oLLZEeXbCbFwltEk=; b=4t4P36pSlGG4sIj28J5YI5ayPnNFvs0xLNbcPgkd+ODljKE7WIIvA8bhDLBg6EqnUM y6ZFjhv03bgHdi9sTVk09kYf1cWcdTVWI4CLepXwLt7E+39pFd0ef9/mt2KjXzC+0Y6A 4kiLJ0jUA/fMzQ3VWZXuW5Z942Ta+fCnSuwIIp86FfRrv0HxzL8q9+VYWTenJMyGv+Pm +zjtiLgmcptA7aqCj8f7o+QqT2ONTa7jdLNgeqSWEeR284Ax4hQ0vcU7sTJfaI3hM5E3 DirDIizDwl1XNiMhtZas80Mz72FiCzgguAkf7K7k1mzWfDGBV7MtA1FeTcowN3R20W5E cUcg== X-Gm-Message-State: AOAM533korm/abMnODt/SQ479PbqmrwBt7S797zcPul+BTZ/j94bgUCd qYgwhXDvS6SNnlYhLyTNzYmndnjB2g1qK2TB X-Google-Smtp-Source: ABdhPJzfigTm4sVxp4bmGACoE3u9kKir946KgdiZHxqlYLwAJWM23sg02WAvudmgh/5OscGfeEUPOg== X-Received: by 2002:a1c:f20e:: with SMTP id s14mr8198200wmc.186.1638373508671; Wed, 01 Dec 2021 07:45:08 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:e4a2:3070:eea4:e434]) by smtp.gmail.com with ESMTPSA id y7sm148209wrw.55.2021.12.01.07.45.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Dec 2021 07:45:08 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org, Laurent Pinchart , Kieran Bingham , Jacopo Mondi , Hans Verkuil , Tomasz Figa , Naushir Patuck Date: Wed, 1 Dec 2021 15:44:34 +0000 Message-Id: <20211201154434.23127-9-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211201154434.23127-1-david.plowman@raspberrypi.com> References: <20211201154434.23127-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v8 8/8] libcamera: pipeline: raspberrypi: Support color spaces X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The Raspberry Pi pipeline handler now sets color spaces correctly. In generateConfiguration() it sets them to reasonable default values based on the stream role. validate() now calls validateColorSpaces() to ensure that the requested color spaces are sensible, before proceeding to check what the hardware can deliver. Signed-off-by: David Plowman --- .../pipeline/raspberrypi/raspberrypi.cpp | 43 +++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index ad526a8b..19e7addb 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -93,6 +93,7 @@ V4L2DeviceFormat toV4L2DeviceFormat(const V4L2SubdeviceFormat &format, deviceFormat.fourcc = V4L2PixelFormat::fromPixelFormat(pix); deviceFormat.size = format.size; + deviceFormat.colorSpace = format.colorSpace; return deviceFormat; } @@ -129,6 +130,7 @@ V4L2SubdeviceFormat findBestFormat(const SensorFormats &formatsMap, const Size & { double bestScore = std::numeric_limits::max(), score; V4L2SubdeviceFormat bestFormat; + bestFormat.colorSpace = ColorSpace::Raw; #define PENALTY_AR 1500.0 #define PENALTY_8BIT 2000.0 @@ -335,6 +337,8 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() if (config_.empty()) return Invalid; + status = validateColorSpaces(true); + /* * What if the platform has a non-90 degree rotation? We can't even * "adjust" the configuration and carry on. Alternatively, raising an @@ -497,11 +501,23 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() V4L2DeviceFormat format; format.fourcc = V4L2PixelFormat::fromPixelFormat(cfg.pixelFormat); format.size = cfg.size; + format.colorSpace = cfg.colorSpace; + LOG(RPI, Debug) + << "Try color space " << ColorSpace::toString(cfg.colorSpace); int ret = dev->tryFormat(&format); if (ret) return Invalid; + if (!format.colorSpace || cfg.colorSpace != format.colorSpace) { + status = Adjusted; + LOG(RPI, Warning) + << "Color space changed from " + << ColorSpace::toString(cfg.colorSpace) << " to " + << ColorSpace::toString(format.colorSpace); + } + cfg.colorSpace = format.colorSpace; + cfg.stride = format.planes[0].bpl; cfg.frameSize = format.planes[0].size; @@ -525,6 +541,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, PixelFormat pixelFormat; V4L2VideoDevice::Formats fmts; Size size; + std::optional colorSpace; if (roles.empty()) return config; @@ -539,6 +556,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, pixelFormat = mbusCodeToPixelFormat(sensorFormat.mbus_code, BayerFormat::Packing::CSI2); ASSERT(pixelFormat.isValid()); + colorSpace = ColorSpace::Raw; bufferCount = 2; rawCount++; break; @@ -546,6 +564,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, case StreamRole::StillCapture: fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::NV12; + colorSpace = ColorSpace::Jpeg; /* Return the largest sensor resolution. */ size = data->sensor_->resolution(); bufferCount = 1; @@ -563,6 +582,8 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, */ fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::YUV420; + /* This will be reasonable for many applications. */ + colorSpace = ColorSpace::Rec709; size = { 1920, 1080 }; bufferCount = 4; outCount++; @@ -571,6 +592,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, case StreamRole::Viewfinder: fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::ARGB8888; + colorSpace = ColorSpace::Jpeg; size = { 800, 600 }; bufferCount = 4; outCount++; @@ -602,6 +624,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, StreamConfiguration cfg(formats); cfg.size = size; cfg.pixelFormat = pixelFormat; + cfg.colorSpace = colorSpace; cfg.bufferCount = bufferCount; config->addConfiguration(cfg); } @@ -706,6 +729,7 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) V4L2PixelFormat fourcc = V4L2PixelFormat::fromPixelFormat(cfg.pixelFormat); format.size = cfg.size; format.fourcc = fourcc; + format.colorSpace = cfg.colorSpace; LOG(RPI, Debug) << "Setting " << stream->name() << " to " << format.toString(); @@ -721,6 +745,23 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) return -EINVAL; } + if (!format.colorSpace || cfg.colorSpace != format.colorSpace) { + /* + * We should have been through validate() before so this + * shouldn't be possible, but we mustn't sweep color space + * problems under the carpet. + */ + LOG(RPI, Warning) + << "Unexpected color space (" + << ColorSpace::toString(cfg.colorSpace) << " to " + << ColorSpace::toString(format.colorSpace) << ") in stream " + << stream->name(); + cfg.colorSpace = format.colorSpace; + } + LOG(RPI, Debug) + << "Stream " << stream->name() << " has color space " + << ColorSpace::toString(cfg.colorSpace); + cfg.setStream(stream); stream->setExternal(true); @@ -745,6 +786,8 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) format = {}; format.size = maxSize; format.fourcc = V4L2PixelFormat::fromPixelFormat(formats::YUV420); + /* No one asked for output, so the color space doesn't matter. */ + format.colorSpace = ColorSpace::Jpeg; ret = data->isp_[Isp::Output0].dev()->setFormat(&format); if (ret) { LOG(RPI, Error)