From patchwork Wed Oct 20 11:08:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 14194 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id AA324BDB1C for ; Wed, 20 Oct 2021 11:08:33 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 686A868F63; Wed, 20 Oct 2021 13:08:31 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="EW4ecQoI"; dkim-atps=neutral Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E94976023A for ; Wed, 20 Oct 2021 13:08:28 +0200 (CEST) Received: by mail-wm1-x329.google.com with SMTP id o24-20020a05600c511800b0030d9da600aeso9268434wms.4 for ; Wed, 20 Oct 2021 04:08:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3wsXnte+0apF9covIJAO5fYmIQ9gPUg+RchAtukBl9Y=; b=EW4ecQoIwArS+wV478/XwBmGUrr4At/oyXgVkfEfaPoAB8MG4kz9Zidi8JCULftKZl K2aSY5Frfo2p5YoJe22CJYDdddISKKe9LJ/kUapa8REPg56C/CEcH5/jlLTb8gPe0OCf vwKjoWybwpxBHfWn48OM8b0sTU6v6I8OhM6KtC3ssd6MQJLzKP7UFxSXD+2TElw8c+Rr EvFt0eiwHiidJphJiY4d7/R0lDzY7QCNYEWKTcHFkto+fv7nDkKNlsNIpb3o20MCrDXy oa8erEyn9AFh56efr/zOt4sIoCT2vCO++h5+49Atc1ykjUkALgbdgAdIQZNXwPDSeHTf FayA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3wsXnte+0apF9covIJAO5fYmIQ9gPUg+RchAtukBl9Y=; b=jPC60ZOjU9J2HxF3KsbKaLvXRBVdIWmx3DWLbpUVWw8JmIA/jGyZke1v4q35wbvWgp axg+sFyiY8lFGa26KERTHu8jV2kP4xUj2RidgVAT92pmzAXptwiiDUCkNdvz7duWPvQl o8a+PIl6xuoaVovzPZ4Vyde1y9gP8qr91pYiSUTfb8Gv9jFGciIZ1MiyYJGIEvCWyGaP yvB6I2JSqJfUci3ESmn4o1I6oM4gm2fo4vSEnT4gOtt8X21dl/E9suREZvfu9KmWLwyd dBWfNfS+07eRJyLcfXX/b2lEP62sM6CmxnzxMI6ysXJkcBLFTwCUA638hLZc3HKpSnsf g9kg== X-Gm-Message-State: AOAM533ZaMi29TeVXXTEXo9ZX7hgKTJaMAZl/ykn3WQnZzT7FAyWYUS4 OXF7EMFGmctUHqO8a0TImpKBIDBnUEU/Tw== X-Google-Smtp-Source: ABdhPJxap4UavBh0CSB08n4Npxqh1Slpf4sQSlDWEVIJprpbAI8sIRlbhFPCVDORwvAtLEMJ03oO/Q== X-Received: by 2002:adf:a556:: with SMTP id j22mr50336297wrb.431.1634728108216; Wed, 20 Oct 2021 04:08:28 -0700 (PDT) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id f20sm1929654wmq.38.2021.10.20.04.08.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 Oct 2021 04:08:27 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Wed, 20 Oct 2021 12:08:19 +0100 Message-Id: <20211020110825.12902-2-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211020110825.12902-1-david.plowman@raspberrypi.com> References: <20211020110825.12902-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 1/7] libcamera: Add ColorSpace class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This class represents a color space by defining its color primaries, YCbCr encoding, the transfer (gamma) function it uses, and whether the output is full or limited range. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- include/libcamera/color_space.h | 88 +++++++++++ include/libcamera/meson.build | 1 + src/libcamera/color_space.cpp | 256 ++++++++++++++++++++++++++++++++ src/libcamera/meson.build | 1 + 4 files changed, 346 insertions(+) create mode 100644 include/libcamera/color_space.h create mode 100644 src/libcamera/color_space.cpp diff --git a/include/libcamera/color_space.h b/include/libcamera/color_space.h new file mode 100644 index 00000000..2af9da31 --- /dev/null +++ b/include/libcamera/color_space.h @@ -0,0 +1,88 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Raspberry Pi (Trading) Limited + * + * color_space.h - color space definitions + */ + +#ifndef __LIBCAMERA_COLOR_SPACE_H__ +#define __LIBCAMERA_COLOR_SPACE_H__ + +#include + +namespace libcamera { + +class ColorSpace +{ +public: + enum class Primaries : int { + Undefined, + Raw, + Smpte170m, + Rec709, + Rec2020, + }; + + enum class YcbcrEncoding : int { + Undefined, + Rec601, + Rec709, + Rec2020, + }; + + enum class TransferFunction : int { + Undefined, + Linear, + Srgb, + Rec709, + }; + + enum class Range : int { + Undefined, + Full, + Limited, + }; + + constexpr ColorSpace() + : ColorSpace(Primaries::Undefined, YcbcrEncoding::Undefined, TransferFunction::Undefined, Range::Undefined) + { + } + + constexpr ColorSpace(Primaries p, YcbcrEncoding e, TransferFunction t, Range r) + : primaries(p), ycbcrEncoding(e), transferFunction(t), range(r) + { + } + + static const ColorSpace Undefined; + static const ColorSpace Raw; + static const ColorSpace Jpeg; + static const ColorSpace Smpte170m; + static const ColorSpace Rec709; + static const ColorSpace Rec2020; + + Primaries primaries; + YcbcrEncoding ycbcrEncoding; + TransferFunction transferFunction; + Range range; + + bool isFullyDefined() const; + + const std::string toString() const; +}; + +constexpr ColorSpace ColorSpace::Undefined = { Primaries::Undefined, YcbcrEncoding::Undefined, TransferFunction::Undefined, Range::Undefined }; +constexpr ColorSpace ColorSpace::Raw = { Primaries::Raw, YcbcrEncoding::Rec601, TransferFunction::Linear, Range::Full }; +constexpr ColorSpace ColorSpace::Jpeg = { Primaries::Rec709, YcbcrEncoding::Rec601, TransferFunction::Srgb, Range::Full }; +constexpr ColorSpace ColorSpace::Smpte170m = { Primaries::Smpte170m, YcbcrEncoding::Rec601, TransferFunction::Rec709, Range::Limited }; +constexpr ColorSpace ColorSpace::Rec709 = { Primaries::Rec709, YcbcrEncoding::Rec709, TransferFunction::Rec709, Range::Limited }; +constexpr ColorSpace ColorSpace::Rec2020 = { Primaries::Rec2020, YcbcrEncoding::Rec2020, TransferFunction::Rec709, Range::Limited }; + +bool operator==(const ColorSpace &lhs, const ColorSpace &rhs); +static inline bool operator!=(const ColorSpace &lhs, const ColorSpace &rhs) +{ + return !(lhs == rhs); +} + +} /* namespace libcamera */ + +#endif /* __LIBCAMERA_COLOR_SPACE_H__ */ diff --git a/include/libcamera/meson.build b/include/libcamera/meson.build index 7155ff20..131e1740 100644 --- a/include/libcamera/meson.build +++ b/include/libcamera/meson.build @@ -5,6 +5,7 @@ libcamera_include_dir = 'libcamera' / 'libcamera' libcamera_public_headers = files([ 'camera.h', 'camera_manager.h', + 'color_space.h', 'compiler.h', 'controls.h', 'file_descriptor.h', diff --git a/src/libcamera/color_space.cpp b/src/libcamera/color_space.cpp new file mode 100644 index 00000000..aec4107f --- /dev/null +++ b/src/libcamera/color_space.cpp @@ -0,0 +1,256 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Raspberry Pi (Trading) Limited + * + * color_space.cpp - color spaces. + */ + +#include + +#include +#include +#include + +/** + * \file color_space.h + * \brief Class and enums to represent color spaces + */ + +namespace libcamera { + +/** + * \class ColorSpace + * \brief Class to describe a color space + * + * The ColorSpace class defines the color primaries, the Y'CbCr encoding, + * the transfer function associated with the color space, and the range + * (sometimes also referred to as the quantisation) of the color space. + * + * Certain combinations of these fields form well-known standard color + * spaces such as "JPEG" or "REC709". Applications must not request color + * spaces with undefined fields, but the "Undefined" value may be + * returned if the camera drivers decide to use a color space that is + * not recognised by the ColorSpace class. + * + * For more information on the specific color spaces described here, please + * see: + * + * sRGB and JPEG +>* SMPTE 170M + * Rec.709 + * Rec.2020 + */ + +/** + * \enum ColorSpace::Primaries + * \brief The color primaries for this color space + * + * \var ColorSpace::Primaries::Undefined + * \brief The color primaries are undefined + * \var ColorSpace::Primaries::Raw + * \brief These are raw colors directly from a sensor + * \var ColorSpace::Primaries::Smpte170m + * \brief SMPTE 170M color primaries + * \var ColorSpace::Primaries::Rec709 + * \brief Rec.709 color primaries + * \var ColorSpace::Primaries::Rec2020 + * \brief Rec.2020 color primaries + */ + +/** + * \enum ColorSpace::YcbcrEncoding + * \brief The Y'CbCr encoding + * + * \var ColorSpace::YcbcrEncoding::Undefined + * \brief The Y'CbCr encoding is undefined + * \var ColorSpace::YcbcrEncoding::Rec601 + * \brief Rec.601 Y'CbCr encoding + * \var ColorSpace::YcbcrEncoding::Rec709 + * \brief Rec.709 Y'CbCr encoding + * \var ColorSpace::YcbcrEncoding::Rec2020 + * \brief Rec.2020 Y'CbCr encoding + */ + +/** + * \enum ColorSpace::TransferFunction + * \brief The transfer function used for this color space + * + * \var ColorSpace::TransferFunction::Undefined + * \brief The transfer function is not specified + * \var ColorSpace::TransferFunction::Linear + * \brief This color space uses a linear (identity) transfer function + * \var ColorSpace::TransferFunction::Srgb + * \brief sRGB transfer function + * \var ColorSpace::TransferFunction::Rec709 + * \brief Rec709 transfer function + */ + +/** + * \enum ColorSpace::Range + * \brief The range (sometimes "quantisation") for this color space + * + * \var ColorSpace::Range::Undefined + * \brief The range is not specified + * \var ColorSpace::Range::Full + * \brief This color space uses full range pixel values + * \var ColorSpace::Range::Limited + * \brief This color space uses limited range pixel values, being + * 16 to 235 for Y' and 16 to 240 for Cb and Cr + */ + +/** + * \fn ColorSpace::ColorSpace(Encoding e, TransferFunction t, Range r) + * \brief Construct a ColorSpace from explicit values + * \param[in] e The Y'CbCr encoding + * \param[in] t The transfer function for the color space + * \param[in] r The range of the pixel values in this color space + */ + +/** + * \fn ColorSpace::ColorSpace() + * \brief Construct a color space with undefined encoding, transfer function + * and range + */ + +/** + * \brief Check if all the fields of the color space are defined + * \return Return true if all the fields of the color space are defined, + * otherwise false + */ +bool ColorSpace::isFullyDefined() const +{ + return primaries != Primaries::Undefined && + ycbcrEncoding != YcbcrEncoding::Undefined && + transferFunction != TransferFunction::Undefined && + range != Range::Undefined; +} + +/** + * \brief Assemble and return a readable string representation of the + * ColorSpace + * \return A string describing the ColorSpace + */ +const std::string ColorSpace::toString() const +{ + /* Print out a brief name only for standard color sapces. */ + + static const std::vector> colorSpaceNames = { + { ColorSpace::Undefined, "Undefined" }, + { ColorSpace::Raw, "Raw" }, + { ColorSpace::Jpeg, "Jpeg" }, + { ColorSpace::Smpte170m, "Smpte170m" }, + { ColorSpace::Rec709, "Rec709" }, + { ColorSpace::Rec2020, "Rec2020" }, + }; + auto it = std::find_if(colorSpaceNames.begin(), colorSpaceNames.end(), + [this](const auto &item) { + return *this == item.first; + }); + if (it != colorSpaceNames.end()) + return std::string(it->second); + + static const char *primariesNames[] = { + "Undefined", + "Raw", + "Smpte170m", + "Rec709", + "Rec2020", + }; + static const char *encodingNames[] = { + "Undefined", + "Rec601", + "Rec709", + "Rec2020", + }; + static const char *transferFunctionNames[] = { + "Undefined", + "Linear", + "Srgb", + "Rec709", + }; + static const char *rangeNames[] = { + "Undefined", + "Full", + "Limited", + }; + + std::stringstream ss; + ss << std::string(primariesNames[static_cast(primaries)]) << "/" + << std::string(encodingNames[static_cast(ycbcrEncoding)]) << "/" + << std::string(transferFunctionNames[static_cast(transferFunction)]) << "/" + << std::string(rangeNames[static_cast(range)]); + + return ss.str(); +} + +/** + * \var ColorSpace::primaries + * \brief The color primaries + */ + +/** + * \var ColorSpace::ycbcrEncoding + * \brief The Y'CbCr encoding + */ + +/** + * \var ColorSpace::transferFunction + * \brief The transfer function for this color space + */ + +/** + * \var ColorSpace::range + * \brief The pixel range used by this color space + */ + +/** + * \var ColorSpace::Undefined + * \brief A constant representing a fully undefined color space + */ + +/** + * \var ColorSpace::Raw + * \brief A constant representing a raw color space (from a sensor) + */ + +/** + * \var ColorSpace::Jpeg + * \brief A constant representing the JPEG color space used for + * encoding JPEG images (and regarded as being the same as the sRGB + * color space) + */ + +/** + * \var ColorSpace::Smpte170m + * \brief A constant representing the SMPTE170M color space + */ + +/** + * \var ColorSpace::Rec709 + * \brief A constant representing the Rec.709 color space + */ + +/** + * \var ColorSpace::Rec2020 + * \brief A constant representing the Rec.2020 color space + */ + +/** + * \brief Compare color spaces for equality + * \return True if the two color spaces are identical, false otherwise + */ +bool operator==(const ColorSpace &lhs, const ColorSpace &rhs) +{ + return lhs.primaries == rhs.primaries && + lhs.ycbcrEncoding == rhs.ycbcrEncoding && + lhs.transferFunction == rhs.transferFunction && + lhs.range == rhs.range; +} + +/** + * \fn bool operator!=(const ColorSpace &lhs, const ColorSpace &rhs) + * \brief Compare color spaces for inequality + * \return True if the two color spaces are not identical, false otherwise + */ + +} /* namespace libcamera */ diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build index 243dd3c1..8dc5d39d 100644 --- a/src/libcamera/meson.build +++ b/src/libcamera/meson.build @@ -8,6 +8,7 @@ libcamera_sources = files([ 'camera_manager.cpp', 'camera_sensor.cpp', 'camera_sensor_properties.cpp', + 'color_space.cpp', 'controls.cpp', 'control_serializer.cpp', 'control_validator.cpp', From patchwork Wed Oct 20 11:08:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 14195 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 217D2BDB1C for ; Wed, 20 Oct 2021 11:08:35 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 19D1768F5B; Wed, 20 Oct 2021 13:08:32 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="KmTrBTsC"; dkim-atps=neutral Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [IPv6:2a00:1450:4864:20::336]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 23D7368F58 for ; Wed, 20 Oct 2021 13:08:29 +0200 (CEST) Received: by mail-wm1-x336.google.com with SMTP id g2so18090148wme.4 for ; Wed, 20 Oct 2021 04:08:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=nUVlWu72PlyE29kbK+yxhDq/7AfELhGkPOIcJt8NS14=; b=KmTrBTsC6CaK6kGd5df/WRSVJooLx48YzdvXVCrLYDvaIXElorwpCXtCTxvHsDZPLV solbmWcxJifES77xHy0E7qfD7fqVKE5Tx9rC5VrHkaIB1IRyhGbeiH57AlDUvmLRlg9S d7l83XMnywzBtQ//tlq10NqPLgQx1G/ua6uTWw2xbT6BILUkP6T9qbkTC8kyMAX+JYN9 uZLKu1yxztYhBdakyDUfJfN1946phBF5g/CPSObEVfAJzfee6mEv4Xg5zJnyqpzWZpYg u/DQiPkEny7To0f2XxrCQuLovpaH7tcEa6CbjeLfn9LcbXmuBUnwrgkjmuA+P9CLodaS HFjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=nUVlWu72PlyE29kbK+yxhDq/7AfELhGkPOIcJt8NS14=; b=e+2mvxgWD4z7e23oD3Tw0s4mq6nefxLn48kb+crIeR4hpsl2T+xWfRZ3sQCwTpokIA jlyXSUSndkKFpT14LcvKLDpf5LMLVSaWJuIiNtTGhqKhynVMGUf5I8nlemCslY13+v4y Rkx/ud93oOSOdnZdv3Kz/Twh4WsR9qhODGQkyTI6dYGrSSC7LgKZkn1VXI7vcES3ctPW +bs4T1HLzLAy4kDjkyGZYJX8w3KVBRQyOGR5twDlb2ZsJOFuqIg0njhfzo4RB58YfZpq suhuE/eua1BVy8bYjerOXdAalmdzMR3/MA0mt2o1C2T6omV3Z64eac4CHan+icsdbWjZ PaCw== X-Gm-Message-State: AOAM532u4eYF2XTjaYpp3C+6wLhTGAvaNGetaPUA4BbnxjhJynUFPJT5 AqUYHbZlPKHGIXnn7Xb2QEeHhubFMjxOFQ== X-Google-Smtp-Source: ABdhPJz4Y1+1ud+c4h2HWUAtVPpVoEiOEs4Rm9WJ552WWh5awsREnIl3MfRvDkq4aCgd8P6pipVVqQ== X-Received: by 2002:adf:ca09:: with SMTP id o9mr49890012wrh.303.1634728108715; Wed, 20 Oct 2021 04:08:28 -0700 (PDT) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id f20sm1929654wmq.38.2021.10.20.04.08.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 Oct 2021 04:08:28 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Wed, 20 Oct 2021 12:08:20 +0100 Message-Id: <20211020110825.12902-3-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211020110825.12902-1-david.plowman@raspberrypi.com> References: <20211020110825.12902-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 2/7] libcamera: Add ColorSpace fields to StreamConfiguration X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This is so that applications can choose appropriate color spaces which will then be passed down to the V4L2 devices. There are two fields: the "requestedColorSpace" which is the one an application wants, and the "actualColorSpace" which is what the underlying hardware can deliver, and which is filled in by the validate() method. Signed-off-by: David Plowman --- include/libcamera/stream.h | 4 ++++ src/libcamera/stream.cpp | 25 +++++++++++++++++++++++++ 2 files changed, 29 insertions(+) diff --git a/include/libcamera/stream.h b/include/libcamera/stream.h index 0c55e716..fe491ff5 100644 --- a/include/libcamera/stream.h +++ b/include/libcamera/stream.h @@ -12,6 +12,7 @@ #include #include +#include #include #include #include @@ -47,6 +48,9 @@ struct StreamConfiguration { unsigned int bufferCount; + ColorSpace requestedColorSpace; + ColorSpace actualColorSpace; + Stream *stream() const { return stream_; } void setStream(Stream *stream) { stream_ = stream; } const StreamFormats &formats() const { return formats_; } diff --git a/src/libcamera/stream.cpp b/src/libcamera/stream.cpp index b421e17e..1ddbbb8c 100644 --- a/src/libcamera/stream.cpp +++ b/src/libcamera/stream.cpp @@ -329,6 +329,31 @@ StreamConfiguration::StreamConfiguration(const StreamFormats &formats) * \brief Requested number of buffers to allocate for the stream */ +/** + * \var StreamConfiguration::requestedColorSpace + * \brief The ColorSpace this stream should have + * + * The generateConfiguration method will generate reasonable default + * values (ColorSpace::Jpeg for stills, ColorSpace::Rec709 for video and + * ColorSpace::Raw for raw streams) but applications are free to overwrite + * this value. + */ + +/** + * \var StreamConfiguration::actualColorSpace + * \brief The ColorSpace the will be used for this stream + * + * This field is filled in by CameraConfiguration::validate(). + * Normally this should match the requestedColorSpace, but it may differ + * if the hardware does not support it. + * + * In general cameras may have different constraints here, for example, + * they may require all output streams to share the same color space. + * Sometimes the fields within this color space may report "Undefined" + * values if the hardware drivers are going to use a color space that + * is not recognised by the ColorSpace class. + */ + /** * \fn StreamConfiguration::stream() * \brief Retrieve the stream associated with the configuration From patchwork Wed Oct 20 11:08:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 14196 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D8CF7BDB1C for ; Wed, 20 Oct 2021 11:08:35 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D18FD68F5C; Wed, 20 Oct 2021 13:08:33 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="VBn33umX"; dkim-atps=neutral Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C54326023A for ; Wed, 20 Oct 2021 13:08:29 +0200 (CEST) Received: by mail-wm1-x32c.google.com with SMTP id d198-20020a1c1dcf000000b00322f53b9b89so9308294wmd.0 for ; Wed, 20 Oct 2021 04:08:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=BuIekqX960ITwcFo2R4vuTMKBt/WbY00sR8GY6vpIrs=; b=VBn33umXoT1iOATO06nxyDnyEaSW8vG7Ux3WXJqjYTzLLysOhrgNx9fIZq36UDUI67 iYz7Nh41poSWZeumepEZ56TV6tau51tixNX8QwCFHzzwfYor1oeFXfWVywfBzIZtpT9V MvkSIecoURyzD/8/lv2fTIMlRxMl6bszpLZZydzZ0MiawekiBiDIp0QfEpweeZ1VWNnW QNY3Tq5203iGrS+PH5DYXm+5K8FmyLBs312tIYFktczwQ9aU9o82B78TZmbA6ukd7n24 JQrBU6uP3QhkGX8dSJ9ZwmifwMXyN1qKtsqD1ytoeoK9G12f2shoqjBJZMm1vNe4D7/d knkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BuIekqX960ITwcFo2R4vuTMKBt/WbY00sR8GY6vpIrs=; b=mmtxcjesBVXS8rRzRCjR7xaTW9ch7ru7xRURyQeNxoOuofXaBcZ8nN6A3/ePG3VcoT RPW9JMaAp0dmY8L0fM6RiXKN5mDpPcgz+smXso9dXfTH7qvxuBbby+BCKXWAZt9A8djS 41RcfqZUciFbiq4oa8Be//KVmorKkutD54ST8e7lY89Eanj7QDZY94Mc/hm5ABm3wABp UgNkOjQI3RZcpNeW3wvGjddACdzIhCuIm0bCQxUnX3fq4tGChKPwBjIkXRyn2zO3J/tF W/v7okkPsu3azBOwqoev7U6SwqFcAy1QhQuzlBfrLJp2jhL6+jPtwCRMa7akd21SF90J HvlQ== X-Gm-Message-State: AOAM532/jg05TJq58GFFhGvv25sxNUMGXgkoE87A2jewLSCS++l3jF5S 2kmn7ON/SfOstVsMsXmDOyqTgLp45CmwkA== X-Google-Smtp-Source: ABdhPJzclAzP+ttP3iwPVvrIEP+CmmhgtRZTNPU+YwX6IcrX4miLYUZpTGbKrJA+leNp9UE0C7pEcg== X-Received: by 2002:a1c:5417:: with SMTP id i23mr12686350wmb.17.1634728109309; Wed, 20 Oct 2021 04:08:29 -0700 (PDT) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id f20sm1929654wmq.38.2021.10.20.04.08.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 Oct 2021 04:08:28 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Wed, 20 Oct 2021 12:08:21 +0100 Message-Id: <20211020110825.12902-4-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211020110825.12902-1-david.plowman@raspberrypi.com> References: <20211020110825.12902-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 3/7] libcamera: Convert between ColorSpace class and V4L2 formats X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" These methods are added to the base V4L2Device class so that they can be shared both by the video device class and subdevices. With the ColorSpace class, the color space and related other fields are stored together, corresponding to a number of fields in the various different V4L2 format structures. Template methods are therefore a convenient implementation. Note that we must explicitly instantiate the templates that will be needed. Signed-off-by: David Plowman --- include/libcamera/internal/v4l2_device.h | 7 + src/libcamera/v4l2_device.cpp | 171 +++++++++++++++++++++++ 2 files changed, 178 insertions(+) diff --git a/include/libcamera/internal/v4l2_device.h b/include/libcamera/internal/v4l2_device.h index f21bc370..77ea30f6 100644 --- a/include/libcamera/internal/v4l2_device.h +++ b/include/libcamera/internal/v4l2_device.h @@ -17,6 +17,7 @@ #include #include +#include #include namespace libcamera { @@ -44,6 +45,12 @@ public: void updateControlInfo(); + template + static ColorSpace v4l2ToColorSpace(const T &v4l2Format); + + template + static int colorSpaceToV4l2(const ColorSpace &colorSpace, T &v4l2Format); + protected: V4L2Device(const std::string &deviceNode); ~V4L2Device(); diff --git a/src/libcamera/v4l2_device.cpp b/src/libcamera/v4l2_device.cpp index 9c783c9c..771dc096 100644 --- a/src/libcamera/v4l2_device.cpp +++ b/src/libcamera/v4l2_device.cpp @@ -16,6 +16,8 @@ #include #include +#include + #include #include #include @@ -731,4 +733,173 @@ void V4L2Device::eventAvailable() frameStart.emit(event.u.frame_sync.frame_sequence); } +static const std::map v4l2ToColorSpaceTable = { + { V4L2_COLORSPACE_RAW, ColorSpace::Raw }, + { V4L2_COLORSPACE_JPEG, ColorSpace::Jpeg }, + { V4L2_COLORSPACE_SRGB, ColorSpace::Jpeg }, + { V4L2_COLORSPACE_SMPTE170M, ColorSpace::Smpte170m }, + { V4L2_COLORSPACE_REC709, ColorSpace::Rec709 }, + { V4L2_COLORSPACE_BT2020, ColorSpace::Rec2020 }, +}; + +static const std::map v4l2ToYcbcrEncodingTable = { + { V4L2_YCBCR_ENC_601, ColorSpace::YcbcrEncoding::Rec601 }, + { V4L2_YCBCR_ENC_709, ColorSpace::YcbcrEncoding::Rec709 }, + { V4L2_YCBCR_ENC_BT2020, ColorSpace::YcbcrEncoding::Rec2020 }, +}; + +static const std::map v4l2ToTransferFunctionTable = { + { V4L2_XFER_FUNC_NONE, ColorSpace::TransferFunction::Linear }, + { V4L2_XFER_FUNC_SRGB, ColorSpace::TransferFunction::Srgb }, + { V4L2_XFER_FUNC_709, ColorSpace::TransferFunction::Rec709 }, +}; + +static const std::map v4l2ToRangeTable = { + { V4L2_QUANTIZATION_FULL_RANGE, ColorSpace::Range::Full }, + { V4L2_QUANTIZATION_LIM_RANGE, ColorSpace::Range::Limited }, +}; + +static const std::vector> colorSpaceToV4l2Table = { + { ColorSpace::Raw, V4L2_COLORSPACE_RAW }, + { ColorSpace::Jpeg, V4L2_COLORSPACE_JPEG }, + { ColorSpace::Smpte170m, V4L2_COLORSPACE_SMPTE170M }, + { ColorSpace::Rec709, V4L2_COLORSPACE_REC709 }, + { ColorSpace::Rec2020, V4L2_COLORSPACE_BT2020 }, +}; + +static const std::map primariesToV4l2Table = { + { ColorSpace::Primaries::Raw, V4L2_COLORSPACE_RAW }, + { ColorSpace::Primaries::Smpte170m, V4L2_COLORSPACE_SMPTE170M }, + { ColorSpace::Primaries::Rec709, V4L2_COLORSPACE_REC709 }, + { ColorSpace::Primaries::Rec2020, V4L2_COLORSPACE_BT2020 }, +}; + +static const std::map ycbcrEncodingToV4l2Table = { + { ColorSpace::YcbcrEncoding::Rec601, V4L2_YCBCR_ENC_601 }, + { ColorSpace::YcbcrEncoding::Rec709, V4L2_YCBCR_ENC_709 }, + { ColorSpace::YcbcrEncoding::Rec2020, V4L2_YCBCR_ENC_BT2020 }, +}; + +static const std::map transferFunctionToV4l2Table = { + { ColorSpace::TransferFunction::Linear, V4L2_XFER_FUNC_NONE }, + { ColorSpace::TransferFunction::Srgb, V4L2_XFER_FUNC_SRGB }, + { ColorSpace::TransferFunction::Rec709, V4L2_XFER_FUNC_709 }, +}; + +static const std::map rangeToV4l2Table = { + { ColorSpace::Range::Full, V4L2_QUANTIZATION_FULL_RANGE }, + { ColorSpace::Range::Limited, V4L2_QUANTIZATION_LIM_RANGE }, +}; + +/** + * \brief Convert the color space fields in a V4L2 format to a ColorSpace + * \param[in] v4l2Format A V4L2 format containing color space information + * + * The colorspace, ycbcr_enc, xfer_func and quantization fields within a + * V4L2 format structure are converted to a corresponding ColorSpace. + * + * If any V4L2 fields are not recognised those are marked as undefined in + * the ColorSpace, but other fields are still initialised where possible. + * This situation can be detected using the returned value's + * ColorSpace::isFullyDefined() method. + * + * \return The ColorSpace corresponding to the input V4L2 format + */ +template +ColorSpace V4L2Device::v4l2ToColorSpace(const T &v4l2Format) +{ + ColorSpace colorSpace; + + auto itColor = v4l2ToColorSpaceTable.find(v4l2Format.colorspace); + if (itColor != v4l2ToColorSpaceTable.end()) + colorSpace = itColor->second; + + auto itYcbcrEncoding = v4l2ToYcbcrEncodingTable.find(v4l2Format.ycbcr_enc); + if (itYcbcrEncoding != v4l2ToYcbcrEncodingTable.end()) + colorSpace.ycbcrEncoding = itYcbcrEncoding->second; + + auto itTransfer = v4l2ToTransferFunctionTable.find(v4l2Format.xfer_func); + if (itTransfer != v4l2ToTransferFunctionTable.end()) + colorSpace.transferFunction = itTransfer->second; + + auto itRange = v4l2ToRangeTable.find(v4l2Format.quantization); + if (itRange != v4l2ToRangeTable.end()) + colorSpace.range = itRange->second; + + return colorSpace; +} + +template ColorSpace V4L2Device::v4l2ToColorSpace(const struct v4l2_pix_format &); +template ColorSpace V4L2Device::v4l2ToColorSpace(const struct v4l2_pix_format_mplane &); +template ColorSpace V4L2Device::v4l2ToColorSpace(const struct v4l2_mbus_framefmt &); + +/** + * \brief Fill in the color space fields of a V4L2 format from a ColorSpace + * \param[in] colorSpace The ColorSpace to be converted + * \param[out] v4l2Format A V4L2 format containing color space information + * + * The colorspace, ycbcr_enc, xfer_func and quantization fields within a + * V4L2 format structure are filled in from a corresponding ColorSpace. + * + * An error is returned if any of the V4L2 fields do not support the + * value given in the ColorSpace. Such fields are set to the V4L2 + * "default" values, but all other fields are still filled in where + * possible. + * + * \return 0 on success or a negative error code otherwise + */ +template +int V4L2Device::colorSpaceToV4l2(const ColorSpace &colorSpace, T &v4l2Format) +{ + int ret = 0; + + v4l2Format.colorspace = V4L2_COLORSPACE_DEFAULT; + v4l2Format.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; + v4l2Format.xfer_func = V4L2_XFER_FUNC_DEFAULT; + v4l2Format.quantization = V4L2_QUANTIZATION_DEFAULT; + + auto itColor = std::find_if(colorSpaceToV4l2Table.begin(), colorSpaceToV4l2Table.end(), + [&colorSpace](const auto &item) { + return colorSpace == item.first; + }); + if (itColor != colorSpaceToV4l2Table.end()) { + v4l2Format.colorspace = itColor->second; + return ret; + } + + /* + * If the colorSpace doesn't precisely match a standard color space, + * then we must choose a V4L2 colorspace with matching primaries. + */ + auto itPrimaries = primariesToV4l2Table.find(colorSpace.primaries); + if (itPrimaries != primariesToV4l2Table.end()) + v4l2Format.colorspace = itPrimaries->second; + else + ret = -1; + + auto itYcbcrEncoding = ycbcrEncodingToV4l2Table.find(colorSpace.ycbcrEncoding); + if (itYcbcrEncoding != ycbcrEncodingToV4l2Table.end()) + v4l2Format.ycbcr_enc = itYcbcrEncoding->second; + else + ret = -1; + + auto itTransfer = transferFunctionToV4l2Table.find(colorSpace.transferFunction); + if (itTransfer != transferFunctionToV4l2Table.end()) + v4l2Format.xfer_func = itTransfer->second; + else + ret = -1; + + auto itRange = rangeToV4l2Table.find(colorSpace.range); + if (itRange != rangeToV4l2Table.end()) + v4l2Format.quantization = itRange->second; + else + ret = -1; + + return ret; +} + +template int V4L2Device::colorSpaceToV4l2(const ColorSpace &, struct v4l2_pix_format &); +template int V4L2Device::colorSpaceToV4l2(const ColorSpace &, struct v4l2_pix_format_mplane &); +template int V4L2Device::colorSpaceToV4l2(const ColorSpace &, struct v4l2_mbus_framefmt &); + } /* namespace libcamera */ From patchwork Wed Oct 20 11:08:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 14197 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1F2E3C324E for ; Wed, 20 Oct 2021 11:08:36 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3A93868F6D; Wed, 20 Oct 2021 13:08:34 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="kkqQDXkN"; dkim-atps=neutral Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8767768F60 for ; Wed, 20 Oct 2021 13:08:30 +0200 (CEST) Received: by mail-wm1-x334.google.com with SMTP id o24-20020a05600c511800b0030d9da600aeso9268614wms.4 for ; Wed, 20 Oct 2021 04:08:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=PmSettw5PZPQKCMg1GXcyVLAY5F5dDI8K7YMj39hOXU=; b=kkqQDXkNoNzRKqzd3EONW9P2UpV4v4fxDeapviwEre0wn72lSOi+CfQAveTU3jym0B sLfY8a3WP57s66oARuBTxMFNkyztaucQNdB1ynMk0BYNIMNFnguFn8rJMYTZuvFwWCvq GcC2iY3/bnr/N+v2waHSs73+CSTA8hnv5s0n/LS3sKcv6N2MSvDstXSCheQixDcnkuLE 8ACwBLzGImnDxOpVVrW73uIhKIFx/eDixLevYiv7GIzzkbGF1L0/j7zs5TauYqzdoiWf wXnBz/JqtJr3gPy/uwsVXKJzVjTSk0Oke/pIArSTDQT1YSR66VjdqRPPNMcf+jxuw8iY k/BA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=PmSettw5PZPQKCMg1GXcyVLAY5F5dDI8K7YMj39hOXU=; b=SgXYDcjzDKZfhPXfiQZGr2f7ruEpesl0pOAEzLB16W9CO/wFYyQALM+x0TNRHiYktb EEC7uoA96Vl/t5iPwypB4QY2TIbWC5u123oydnYgyBcESO7a9INIqkqnhoTzs6inIECn o8mLg0o4yOqUKWKHtix60vj0Ot5yZZfE+ybU6JBCwQiPgy8mA+32vP8NgkkD4absYUbF 1nemmokYN19yT8Zan8k39hVQCEAr3M3NYd38q5eAlDR5bEvH3XUXa4k1MO8iPyAKzNbA 00tgmSmX8K4fOI7udj4KuBJiiCitWy2S+v+8gcT2CiDbZLVf8BkmVrIS0I0bSweip4q/ qNHg== X-Gm-Message-State: AOAM530j3chXhpKmk+le5nzKQisMd1ScRdogXwyVMAXTfTbjbF6g3CA9 xiDRpHRRsiYVVOrEnEp0tpL8hDcrXFQseg== X-Google-Smtp-Source: ABdhPJzDBxB+yRm43whssvSG72Ab49iOxzXVLJ0IBo0n/kirXlsReck+Kyp52NY8XnNs6BJvn2ieFg== X-Received: by 2002:a1c:cc09:: with SMTP id h9mr13084187wmb.191.1634728109947; Wed, 20 Oct 2021 04:08:29 -0700 (PDT) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id f20sm1929654wmq.38.2021.10.20.04.08.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 Oct 2021 04:08:29 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Wed, 20 Oct 2021 12:08:22 +0100 Message-Id: <20211020110825.12902-5-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211020110825.12902-1-david.plowman@raspberrypi.com> References: <20211020110825.12902-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 4/7] libcamera: Support passing ColorSpaces to V4L2 video devices X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The ColorSpace from the StreamConfiguration is now handled appropriately in the V4L2VideoDevice. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- include/libcamera/internal/v4l2_videodevice.h | 2 + src/libcamera/v4l2_videodevice.cpp | 69 +++++++++++++++++-- 2 files changed, 66 insertions(+), 5 deletions(-) diff --git a/include/libcamera/internal/v4l2_videodevice.h b/include/libcamera/internal/v4l2_videodevice.h index efe34d47..34cc9cdd 100644 --- a/include/libcamera/internal/v4l2_videodevice.h +++ b/include/libcamera/internal/v4l2_videodevice.h @@ -20,6 +20,7 @@ #include #include +#include #include #include #include @@ -163,6 +164,7 @@ public: V4L2PixelFormat fourcc; Size size; + ColorSpace colorSpace; std::array planes; unsigned int planesCount = 0; diff --git a/src/libcamera/v4l2_videodevice.cpp b/src/libcamera/v4l2_videodevice.cpp index ba5f88cd..9e57e257 100644 --- a/src/libcamera/v4l2_videodevice.cpp +++ b/src/libcamera/v4l2_videodevice.cpp @@ -364,11 +364,6 @@ bool V4L2BufferCache::Entry::operator==(const FrameBuffer &buffer) const * \brief The plane line stride (in bytes) */ -/** - * \var V4L2DeviceFormat::size - * \brief The image size in pixels - */ - /** * \var V4L2DeviceFormat::fourcc * \brief The fourcc code describing the pixel encoding scheme @@ -377,6 +372,23 @@ bool V4L2BufferCache::Entry::operator==(const FrameBuffer &buffer) const * that identifies the image format pixel encoding scheme. */ +/** + * \var V4L2DeviceFormat::size + * \brief The image size in pixels + */ + +/** + * \var V4L2DeviceFormat::colorSpace + * \brief The color space of the pixels + * + * When setting or trying a format, passing in "Undefined" fields in the + * ColorSpace is not permitted because the driver will then make an + * arbitrary choice of its own. Choices made by the driver will be + * passed back in the normal way, though note that "Undefined" values can + * be returned if the device has chosen something that the ColorSpace + * class cannot represent. + */ + /** * \var V4L2DeviceFormat::planes * \brief The per-plane memory size information @@ -873,6 +885,12 @@ int V4L2VideoDevice::getFormatMultiplane(V4L2DeviceFormat *format) format->fourcc = V4L2PixelFormat(pix->pixelformat); format->planesCount = pix->num_planes; + format->colorSpace = v4l2ToColorSpace(*pix); + if (!format->colorSpace.isFullyDefined()) + LOG(V4L2, Warning) + << "Retrieved undefined color space: " + << format->colorSpace.toString(); + for (unsigned int i = 0; i < format->planesCount; ++i) { format->planes[i].bpl = pix->plane_fmt[i].bytesperline; format->planes[i].size = pix->plane_fmt[i].sizeimage; @@ -887,6 +905,11 @@ int V4L2VideoDevice::trySetFormatMultiplane(V4L2DeviceFormat *format, bool set) struct v4l2_pix_format_mplane *pix = &v4l2Format.fmt.pix_mp; int ret; + if (!format->colorSpace.isFullyDefined()) + LOG(V4L2, Error) + << "Trying to set undefined color space: " + << format->colorSpace.toString(); + v4l2Format.type = bufferType_; pix->width = format->size.width; pix->height = format->size.height; @@ -894,6 +917,12 @@ int V4L2VideoDevice::trySetFormatMultiplane(V4L2DeviceFormat *format, bool set) pix->num_planes = format->planesCount; pix->field = V4L2_FIELD_NONE; + ret = colorSpaceToV4l2(format->colorSpace, *pix); + if (ret < 0) + LOG(V4L2, Warning) + << "Setting color space unrecognised by V4L2: " + << format->colorSpace.toString(); + ASSERT(pix->num_planes <= std::size(pix->plane_fmt)); for (unsigned int i = 0; i < pix->num_planes; ++i) { @@ -922,6 +951,12 @@ int V4L2VideoDevice::trySetFormatMultiplane(V4L2DeviceFormat *format, bool set) format->planes[i].size = pix->plane_fmt[i].sizeimage; } + format->colorSpace = v4l2ToColorSpace(*pix); + if (!format->colorSpace.isFullyDefined()) + LOG(V4L2, Warning) + << "Undefined color space has been set: " + << format->colorSpace.toString(); + return 0; } @@ -945,6 +980,12 @@ int V4L2VideoDevice::getFormatSingleplane(V4L2DeviceFormat *format) format->planes[0].bpl = pix->bytesperline; format->planes[0].size = pix->sizeimage; + format->colorSpace = v4l2ToColorSpace(*pix); + if (!format->colorSpace.isFullyDefined()) + LOG(V4L2, Warning) + << "Retrieved undefined color space: " + << format->colorSpace.toString(); + return 0; } @@ -954,12 +995,24 @@ int V4L2VideoDevice::trySetFormatSingleplane(V4L2DeviceFormat *format, bool set) struct v4l2_pix_format *pix = &v4l2Format.fmt.pix; int ret; + if (!format->colorSpace.isFullyDefined()) + LOG(V4L2, Error) + << "Trying to set undefined color space: " + << format->colorSpace.toString(); + v4l2Format.type = bufferType_; pix->width = format->size.width; pix->height = format->size.height; pix->pixelformat = format->fourcc; pix->bytesperline = format->planes[0].bpl; pix->field = V4L2_FIELD_NONE; + + ret = colorSpaceToV4l2(format->colorSpace, *pix); + if (ret < 0) + LOG(V4L2, Warning) + << "Set color space unrecognised by V4L2: " + << format->colorSpace.toString(); + ret = ioctl(set ? VIDIOC_S_FMT : VIDIOC_TRY_FMT, &v4l2Format); if (ret) { LOG(V4L2, Error) @@ -979,6 +1032,12 @@ int V4L2VideoDevice::trySetFormatSingleplane(V4L2DeviceFormat *format, bool set) format->planes[0].bpl = pix->bytesperline; format->planes[0].size = pix->sizeimage; + format->colorSpace = v4l2ToColorSpace(*pix); + if (!format->colorSpace.isFullyDefined()) + LOG(V4L2, Warning) + << "Undefined color space has been set: " + << format->colorSpace.toString(); + return 0; } From patchwork Wed Oct 20 11:08:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 14198 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C9EFCBDB1C for ; Wed, 20 Oct 2021 11:08:36 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8C4DD68F6E; Wed, 20 Oct 2021 13:08:36 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="l7axcNvu"; dkim-atps=neutral Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1175D68F57 for ; Wed, 20 Oct 2021 13:08:31 +0200 (CEST) Received: by mail-wm1-x330.google.com with SMTP id g39so13933242wmp.3 for ; Wed, 20 Oct 2021 04:08:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=6J3bCUodDtUau6XCsO6hTiATOA9Y8C5RyfpkX5Sa+sE=; b=l7axcNvuyqIJkluGDkXypyWd4i2QeQpaho5HxSv9RkZQ6mvQUh8gRhJH2MiFyxB4gA TrAvqS0r46/B5a6/D1zE7fjzrZ1+z5dKkDyaIcJXo9Q5ju7hSeF6/zXwpNCTf83MdDa3 uENTTJgY0LGk3mmtpc5NBBNjLqUCA+9nNdXQASg6g4WuIovwG6ddBEr6uhWJ5mSKq5OB 8c1dDqoFmIRDe8Th6sWBqc6IFL2uOvcnf90FansYl6w7JJjmhulzRYWhTx1m+SQT1HwF tY7t6m8C1qjvZb+K3Q46FLrQHR6BHY/0pI6qPRnnxn/gFJ2SJuiL0pisOKzJ5Gr1tely RTxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6J3bCUodDtUau6XCsO6hTiATOA9Y8C5RyfpkX5Sa+sE=; b=cMv0NzQc+oZRMd47oSMmfw8W3h92ZWQvIbLncHWW0/8PutNcOVUkEPHr5iPbKI8U8u vqs69QSeGyMX1mLnlSSCcvvaWDuqwi0zt1QTmjTtojvHW2x4HcD0FLyjhrnMEVRhdOi7 uhyWNxqY4e8m6q7pEVy+EGsm5vBsmBtqWCpsJ0t0tPHJC8vYO1tCgolKtuOoU1xtx4cm sVpQRxOQUCR5QLrgER2LhbpGO16q/MPScgh2lXjIE0MSBpC/de2/NZRhlAMQJE2qjHwd ZiXrR5eRxQqDcg8w3fdrq78tslHYn6OxWkzMYbDutn1zUju3KH4MZ86MfyIsbgeY2V2o 47Yw== X-Gm-Message-State: AOAM533hZ1HlGMPOxKHz/Z16/XbTRTUh7HUyrDbUV6YjV6mZk9ccl7G8 YJG+Z9FVSe+cGkAXE6nz4gWXAXTXYgdbNg== X-Google-Smtp-Source: ABdhPJwI2dNpUrb96EcAm67nTepnmLcFDVqJtRsHHP7uLzb3G6vWehj7KqlZvn6YZE08EsEAnYBgnw== X-Received: by 2002:a05:6000:2c8:: with SMTP id o8mr50591909wry.430.1634728110570; Wed, 20 Oct 2021 04:08:30 -0700 (PDT) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id f20sm1929654wmq.38.2021.10.20.04.08.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 Oct 2021 04:08:30 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Wed, 20 Oct 2021 12:08:23 +0100 Message-Id: <20211020110825.12902-6-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211020110825.12902-1-david.plowman@raspberrypi.com> References: <20211020110825.12902-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 5/7] libcamera: Support passing ColorSpaces to V4L2 subdevices X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The ColorSpace from the StreamConfiguration is now handled appropriately in the V4L2Subdevice. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- include/libcamera/internal/v4l2_subdevice.h | 2 ++ src/libcamera/camera_sensor.cpp | 1 + src/libcamera/v4l2_subdevice.cpp | 37 ++++++++++++++++++++- 3 files changed, 39 insertions(+), 1 deletion(-) diff --git a/include/libcamera/internal/v4l2_subdevice.h b/include/libcamera/internal/v4l2_subdevice.h index 97b89fb9..f3ab8454 100644 --- a/include/libcamera/internal/v4l2_subdevice.h +++ b/include/libcamera/internal/v4l2_subdevice.h @@ -14,6 +14,7 @@ #include #include +#include #include #include "libcamera/internal/formats.h" @@ -27,6 +28,7 @@ class MediaDevice; struct V4L2SubdeviceFormat { uint32_t mbus_code; Size size; + ColorSpace colorSpace; const std::string toString() const; uint8_t bitsPerPixel() const; diff --git a/src/libcamera/camera_sensor.cpp b/src/libcamera/camera_sensor.cpp index 9fdb8c09..6fcd1c1d 100644 --- a/src/libcamera/camera_sensor.cpp +++ b/src/libcamera/camera_sensor.cpp @@ -613,6 +613,7 @@ V4L2SubdeviceFormat CameraSensor::getFormat(const std::vector &mbu V4L2SubdeviceFormat format{ .mbus_code = bestCode, .size = *bestSize, + .colorSpace = ColorSpace::Raw, }; return format; diff --git a/src/libcamera/v4l2_subdevice.cpp b/src/libcamera/v4l2_subdevice.cpp index 023e2328..8538e66c 100644 --- a/src/libcamera/v4l2_subdevice.cpp +++ b/src/libcamera/v4l2_subdevice.cpp @@ -168,6 +168,18 @@ const std::map formatInfoMap = { * \brief The image size in pixels */ +/** + * \var V4L2SubdeviceFormat::colorSpace + * \brief The color space of the pixels + * + * When setting or trying a format, passing in "Undefined" fields in the + * ColorSpace is not recommended because the driver will then make an + * arbitrary choice of its own. Choices made by the driver will be + * passed back in the normal way, though note that "Undefined" values can + * be returned if the device has chosen something that the ColorSpace + * cannot represent. + */ + /** * \brief Assemble and return a string describing the format * \return A string describing the V4L2SubdeviceFormat @@ -400,6 +412,12 @@ int V4L2Subdevice::getFormat(unsigned int pad, V4L2SubdeviceFormat *format, format->size.height = subdevFmt.format.height; format->mbus_code = subdevFmt.format.code; + format->colorSpace = v4l2ToColorSpace(subdevFmt.format); + if (!format->colorSpace.isFullyDefined()) + LOG(V4L2, Warning) + << "Retrieved undefined color space: " + << format->colorSpace.toString(); + return 0; } @@ -418,6 +436,11 @@ int V4L2Subdevice::getFormat(unsigned int pad, V4L2SubdeviceFormat *format, int V4L2Subdevice::setFormat(unsigned int pad, V4L2SubdeviceFormat *format, Whence whence) { + if (!format->colorSpace.isFullyDefined()) + LOG(V4L2, Error) + << "Trying to set undefined color space: " + << format->colorSpace.toString(); + struct v4l2_subdev_format subdevFmt = {}; subdevFmt.which = whence == ActiveFormat ? V4L2_SUBDEV_FORMAT_ACTIVE : V4L2_SUBDEV_FORMAT_TRY; @@ -427,7 +450,13 @@ int V4L2Subdevice::setFormat(unsigned int pad, V4L2SubdeviceFormat *format, subdevFmt.format.code = format->mbus_code; subdevFmt.format.field = V4L2_FIELD_NONE; - int ret = ioctl(VIDIOC_SUBDEV_S_FMT, &subdevFmt); + int ret = colorSpaceToV4l2(format->colorSpace, subdevFmt.format); + if (ret < 0) + LOG(V4L2, Warning) + << "Setting color space unrecognised by V4L2: " + << format->colorSpace.toString(); + + ret = ioctl(VIDIOC_SUBDEV_S_FMT, &subdevFmt); if (ret) { LOG(V4L2, Error) << "Unable to set format on pad " << pad @@ -439,6 +468,12 @@ int V4L2Subdevice::setFormat(unsigned int pad, V4L2SubdeviceFormat *format, format->size.height = subdevFmt.format.height; format->mbus_code = subdevFmt.format.code; + format->colorSpace = v4l2ToColorSpace(subdevFmt.format); + if (!format->colorSpace.isFullyDefined()) + LOG(V4L2, Warning) + << "Undefined color space has been set: " + << format->colorSpace.toString(); + return 0; } From patchwork Wed Oct 20 11:08:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 14199 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3DD83C324E for ; Wed, 20 Oct 2021 11:08:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id ED60868F71; Wed, 20 Oct 2021 13:08:36 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="EryKXKyt"; dkim-atps=neutral Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A5DE668F64 for ; Wed, 20 Oct 2021 13:08:31 +0200 (CEST) Received: by mail-wm1-x331.google.com with SMTP id 67-20020a1c1946000000b0030d4c90fa87so9268930wmz.2 for ; Wed, 20 Oct 2021 04:08:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=/DkWsC49pTR4LtQbKMXqaDsUcUfiAkrci28YuUbz4pg=; b=EryKXKytDszuoNQQe4JsV9U3GFRqmxL0OPKwdOyPqzXrq45p6U3dPWCLdhOO5UTwfG q1taqsHbf/FyJGVN5ZFCJrOc9iQvmSUcQrsoIGpa9yRPYKbDL/k3HEAB0/GikVWeJwR9 9sl/cB2oUiMqL9MvnQt2XGTFhlZjKGQLhjWefhkmLIy3XKoqI/A3Dz+2ImqW2vKZv+YF bl72cwe80o6is3KNBVnMVOpuq8FQDVt5EOi0ZhkzdTpxQO4tPSXG9JlPw3Biau26LUMN Qgcqf4BSvYZWJ1ifzHsEt5AiIv0UcTZD23zFsgOjBBbkl7qc+pwGvTz5mbnaPGF5gW4d THGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=/DkWsC49pTR4LtQbKMXqaDsUcUfiAkrci28YuUbz4pg=; b=tzwSGmlwexbVrBtnNadi+R0WqnRvAQNqO6SOvz4afTgqk/Ds6dmA/nZ1J76orU9Zsz IoQTuRCl6CDCfRwGkDLWM+efSqn+VDzIeB4/T1u2/cQh6phejg55Tn6/P/LWih+wSNRs ksJZWvGjlHxrY7KpwvCy07ACjIfgKKVb1nxxThhZ2uQ+r8Iw0gON7J2EKfUY18JEakol 8iPwbFS3cE/g+ny9t+PwrJ2m77TBH6Fi/MBnx97iH0212tzdiLMksvypkI3ZBAKhhLiY M5hbOeaptcDBI+aWsMcohQ3NAJ/sAdX++u13MRXo1Y5Ho3InaXbmW0QAPwl1iORE/8/y 6Niw== X-Gm-Message-State: AOAM532o0P+QQY3d3VBXx5JRmEG+0Mqmb25+WRIxwysuhmhsn8xHF0Vm 94RN9rMlE2NRXAjI4kgUKj1jkLa8JiIExA== X-Google-Smtp-Source: ABdhPJzDsF7AyeXeP+Id459Q+EGISvZ0qi3DR44d4F0DhcnEDscCfUUVIhKpZVvho2f2OZRO1xxKLQ== X-Received: by 2002:a1c:f008:: with SMTP id a8mr12857449wmb.140.1634728111145; Wed, 20 Oct 2021 04:08:31 -0700 (PDT) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id f20sm1929654wmq.38.2021.10.20.04.08.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 Oct 2021 04:08:30 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Wed, 20 Oct 2021 12:08:24 +0100 Message-Id: <20211020110825.12902-7-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211020110825.12902-1-david.plowman@raspberrypi.com> References: <20211020110825.12902-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 6/7] libcamera: Add validateColorSpaces to CameraConfiguration class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This method checks that the requested color spaces are sensible and do not contain any undefined enum values. It also initialises the "actual" color space field to the same value as the one requested, in the expectation that the rest of the validate() method will be able to check this. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- include/libcamera/camera.h | 2 ++ src/libcamera/camera.cpp | 53 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 55 insertions(+) diff --git a/include/libcamera/camera.h b/include/libcamera/camera.h index 601ee46e..fdab4410 100644 --- a/include/libcamera/camera.h +++ b/include/libcamera/camera.h @@ -69,6 +69,8 @@ public: protected: CameraConfiguration(); + Status validateColorSpaces(bool sharedColorSpace); + std::vector config_; }; diff --git a/src/libcamera/camera.cpp b/src/libcamera/camera.cpp index 71809bcd..79451ef4 100644 --- a/src/libcamera/camera.cpp +++ b/src/libcamera/camera.cpp @@ -19,6 +19,7 @@ #include #include "libcamera/internal/camera.h" +#include "libcamera/internal/formats.h" #include "libcamera/internal/pipeline_handler.h" /** @@ -313,6 +314,58 @@ std::size_t CameraConfiguration::size() const return config_.size(); } +static bool isRaw(const PixelFormat &pixFmt) +{ + const PixelFormatInfo &info = PixelFormatInfo::info(pixFmt); + return info.isValid() && + info.colourEncoding == PixelFormatInfo::ColourEncodingRAW; +} + +CameraConfiguration::Status CameraConfiguration::validateColorSpaces(bool sharedColorSpace) +{ + Status status = Valid; + + /* Find the largest non-raw stream (if any). */ + int index = -1; + for (unsigned int i = 0; i < config_.size(); i++) { + const StreamConfiguration &cfg = config_[i]; + if (!isRaw(cfg.pixelFormat) && + (index < 0 || + cfg.size.width * cfg.size.height > config_[i].size.width * config_[i].size.height)) + index = i; + } + + /* + * Here we force raw streams to the correct color space and signal + * an error if we encounter anything undefined. We handle the case + * where all output streams are to share a color space, which we + * choose to be the color space of the largest stream. + */ + for (auto &cfg : config_) { + ColorSpace initialColorSpace = cfg.requestedColorSpace; + + if (isRaw(cfg.pixelFormat)) + cfg.requestedColorSpace = ColorSpace::Raw; + else if (!cfg.requestedColorSpace.isFullyDefined()) { + LOG(Camera, Error) << "Stream has undefined color space"; + cfg.requestedColorSpace = ColorSpace::Jpeg; + } else if (sharedColorSpace) + cfg.requestedColorSpace = config_[index].requestedColorSpace; + + if (cfg.requestedColorSpace != initialColorSpace) + status = Adjusted; + + /* + * We also initialise the actual color space as if the + * hardware can do what we want. But note that the rest + * of the validate() method may change this. + */ + cfg.actualColorSpace = cfg.requestedColorSpace; + } + + return status; +} + /** * \var CameraConfiguration::transform * \brief User-specified transform to be applied to the image From patchwork Wed Oct 20 11:08:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 14200 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BE392C324F for ; Wed, 20 Oct 2021 11:08:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6ABC268F74; Wed, 20 Oct 2021 13:08:37 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="qe/MpdsA"; dkim-atps=neutral Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 61AD868F5E for ; Wed, 20 Oct 2021 13:08:32 +0200 (CEST) Received: by mail-wm1-x32f.google.com with SMTP id 186-20020a1c01c3000000b0030d8315b593so639165wmb.5 for ; Wed, 20 Oct 2021 04:08:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=nqj/cQjosr/odg0+Bg8Q+LeBCBQcpdKOTyrPJkxixCs=; b=qe/MpdsABGaMHKvggm2TRjbIaouNVVfPEoY5ZdCCOU1ie9KUZUcQ8Wzcn7YdYqirIu 948Lcl3xAVfTvyaRzt8kpkERF+ovIVzschuTyE6NymWR/IHDMsPMvabvmdL/JMAl7dOF Ey8BvzRdrj6nrnERAAi1u9IhUp8LpSQOxRZPTwhtcoTdeB7gdl0FDIUHTJyiigEQioVK m3kI4EAWEkZW7uK8I5R5FtSSZc2InZoxWxjiyif2tYJ38GdaTCYe2AhiBYH47dnAkIM5 0ERKTQSPWzaCcB0E8e+00DA987WOVt5wEwMPgQikslW2W0uak6FRypSfOiohLtoWcVhC xfVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=nqj/cQjosr/odg0+Bg8Q+LeBCBQcpdKOTyrPJkxixCs=; b=BKq6lDoxPloeOSqiNCZkR4zFNBnBbaBOPh1beE7Byzb5ZDMSV8D7Exn3C9FIKwtxbS KkRsOlQZbfMTug36Fz8IIXhy0YZrUhBUo2RHpadeRk63RcCBff7WbwxNMJuoxWGMJrdF Clb7zP6kn394MOjTsqvh1cJrEdMkLT18rVpJqxmSJBluMwnSgT9ZG7Q13kmH+DMAg0oQ GC1z63Wfq+htR0jNgFx3v2zA1vPXvsb63+1eBgQIumUU8gfJyq3PGj2iyBZyWyDBTWWs In2wfIMPtyed3VE6pQTvhupx585H/lFUas5AvoMRLm0d071qPQQ929xijgJyTRppDViS imNw== X-Gm-Message-State: AOAM530j7oK3aYkEPlVv04ZdTgZKHxWXRYiRxWh3pSuaDE/40hFAyQRF KsikJpoY2i0j44zESanfuIQJK6wFmcjdbQ== X-Google-Smtp-Source: ABdhPJzM5yQ7TLMQNieurpSbb6JOkI8FrGNn4luqig+T9RSyIstiDp4dx6EqtsJoyAD6Aj3MbuppfA== X-Received: by 2002:a1c:2395:: with SMTP id j143mr12974805wmj.107.1634728111885; Wed, 20 Oct 2021 04:08:31 -0700 (PDT) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id f20sm1929654wmq.38.2021.10.20.04.08.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 Oct 2021 04:08:31 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Wed, 20 Oct 2021 12:08:25 +0100 Message-Id: <20211020110825.12902-8-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211020110825.12902-1-david.plowman@raspberrypi.com> References: <20211020110825.12902-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 7/7] libcamera: pipeline: raspberrypi: Support color spaces X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The Raspberry Pi pipeline handler now sets color spaces correctly. In generateConfiguration() it sets them to reasonable default values based on the stream role. validate() now calls validateColorSpaces() to ensure that the requested color spaces are sensible, before proceeding to check what the hardware can deliver. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- .../pipeline/raspberrypi/raspberrypi.cpp | 42 +++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 1634ca98..22db54ce 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -288,6 +288,8 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() if (config_.empty()) return Invalid; + status = validateColorSpaces(true); + /* * What if the platform has a non-90 degree rotation? We can't even * "adjust" the configuration and carry on. Alternatively, raising an @@ -354,6 +356,7 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() */ V4L2VideoDevice::Formats fmts = data_->unicam_[Unicam::Image].dev()->formats(); V4L2DeviceFormat sensorFormat = findBestMode(fmts, cfg.size); + sensorFormat.colorSpace = ColorSpace::Raw; int ret = data_->unicam_[Unicam::Image].dev()->tryFormat(&sensorFormat); if (ret) return Invalid; @@ -449,10 +452,21 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() V4L2DeviceFormat format; format.fourcc = V4L2PixelFormat::fromPixelFormat(cfg.pixelFormat); format.size = cfg.size; + format.colorSpace = cfg.requestedColorSpace; + LOG(RPI, Debug) + << "Try color space " << cfg.requestedColorSpace.toString(); int ret = dev->tryFormat(&format); if (ret) return Invalid; + cfg.actualColorSpace = format.colorSpace; + if (cfg.actualColorSpace != cfg.requestedColorSpace) { + status = Adjusted; + LOG(RPI, Warning) + << "Color space changed from " + << cfg.requestedColorSpace.toString() << " to " + << cfg.actualColorSpace.toString(); + } cfg.stride = format.planes[0].bpl; cfg.frameSize = format.planes[0].size; @@ -477,6 +491,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, PixelFormat pixelFormat; V4L2VideoDevice::Formats fmts; Size size; + ColorSpace colorSpace; if (roles.empty()) return config; @@ -491,6 +506,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, sensorFormat = findBestMode(fmts, size); pixelFormat = sensorFormat.fourcc.toPixelFormat(); ASSERT(pixelFormat.isValid()); + colorSpace = ColorSpace::Raw; bufferCount = 2; rawCount++; break; @@ -498,6 +514,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, case StreamRole::StillCapture: fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::NV12; + colorSpace = ColorSpace::Jpeg; /* Return the largest sensor resolution. */ size = data->sensor_->resolution(); bufferCount = 1; @@ -515,6 +532,8 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, */ fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::YUV420; + /* This will be reasonable for many applications. */ + colorSpace = ColorSpace::Rec709; size = { 1920, 1080 }; bufferCount = 4; outCount++; @@ -523,6 +542,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, case StreamRole::Viewfinder: fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::ARGB8888; + colorSpace = ColorSpace::Jpeg; size = { 800, 600 }; bufferCount = 4; outCount++; @@ -554,6 +574,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, StreamConfiguration cfg(formats); cfg.size = size; cfg.pixelFormat = pixelFormat; + cfg.requestedColorSpace = colorSpace; cfg.bufferCount = bufferCount; config->addConfiguration(cfg); } @@ -601,6 +622,7 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) /* First calculate the best sensor mode we can use based on the user request. */ V4L2VideoDevice::Formats fmts = data->unicam_[Unicam::Image].dev()->formats(); V4L2DeviceFormat sensorFormat = findBestMode(fmts, rawStream ? sensorSize : maxSize); + sensorFormat.colorSpace = ColorSpace::Raw; /* * Unicam image output format. The ISP input format gets set at start, @@ -650,6 +672,7 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) V4L2PixelFormat fourcc = V4L2PixelFormat::fromPixelFormat(cfg.pixelFormat); format.size = cfg.size; format.fourcc = fourcc; + format.colorSpace = cfg.requestedColorSpace; LOG(RPI, Debug) << "Setting " << stream->name() << " to " << format.toString(); @@ -665,6 +688,23 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) return -EINVAL; } + if (cfg.actualColorSpace != format.colorSpace) { + /* + * We should have been through validate() before so this + * shouldn't be possible, but we mustn't sweep color space + * problems under the carpet. + */ + LOG(RPI, Warning) + << "Unexpected color space change (" + << cfg.actualColorSpace.toString() << " to " + << format.colorSpace.toString() << ") in stream " + << stream->name(); + cfg.actualColorSpace = format.colorSpace; + } + LOG(RPI, Debug) + << "Stream " << stream->name() << " has color space " + << cfg.actualColorSpace.toString(); + cfg.setStream(stream); stream->setExternal(true); @@ -689,6 +729,8 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) format = {}; format.size = maxSize; format.fourcc = V4L2PixelFormat::fromPixelFormat(formats::YUV420); + /* No one asked for output, so the color space doesn't matter. */ + format.colorSpace = ColorSpace::Jpeg; ret = data->isp_[Isp::Output0].dev()->setFormat(&format); if (ret) { LOG(RPI, Error)