From patchwork Mon Nov 15 16:11:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 14607 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id ECDD4BF415 for ; Mon, 15 Nov 2021 16:13:02 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 009D960398; Mon, 15 Nov 2021 17:12:59 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="TCaBzHXs"; dkim-atps=neutral Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EE2E36036C for ; Mon, 15 Nov 2021 17:12:57 +0100 (CET) Received: by mail-wr1-x435.google.com with SMTP id r8so31681242wra.7 for ; Mon, 15 Nov 2021 08:12:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=tlUI2V+eGxAjDG+xyKZcONVDwUS7KK1blhVHHWSuJ+I=; b=TCaBzHXssEWC0SXCME6BAa2Pj8P/zTII9AyOGm/d3w5kmq3T/lLkUB5y4+Lst0Ncsp TFEx+4h/btu/ECYNSo6+FCdOa8KOjNyHhUP8xUzzqSSXdHkZwy1A/MtVFCxPadwOmQ8t g3VhF5lyxLGe8k4DrocTdAoS5+H3br4NX/N+qIF706gHK7cGo9YBBk75n8bEb734llgJ g3GwWDiq5NeU5FGQCT73ZIXHC0ZzDCDi0JZS6Xb7CVVCTJXyxVK8yc44j0uPwmEK65+j uAbhYdjJLqNzQwina3wgRW31ktsIpJjD44ZpDf8EiJyKH5TnL6tPyn3X5gUQBAFhsCbw wcMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tlUI2V+eGxAjDG+xyKZcONVDwUS7KK1blhVHHWSuJ+I=; b=mEZ96chVH/b7NBoowYT/tKKjp97VBADQSMvP8yRht5gpGW0bC4zcjR2Rb9mLoVWOif iXDSsMmm2SvUNIcsmagDTDb2eraEFDIxw3+9ninf+zNJRUUL1AhXmQAf1+NVeiBntKwK 8XwSgZuGoQr1JIo4qveHj5akID2exfDH8Jn1d6abCDFOQ280RXTL2aT+1hCSJB6n6kmC cqVMCv3r8CK/E+KvqicMVg+rHkdj50u5jqrbceEf40D4EhcsAzn4oYTHV8z5D/l5Ahgf 3ghxLrozgvJC9LL7EhKyrp88is3tJ64AMOgRFTvea9fxln4IZwQDmjIXDeDENYTSc9+J RHEw== X-Gm-Message-State: AOAM532L3CizkkLo75yrjq5P2hllDNWX9/vN9cTTA88YaN1CcLc1o9nv 5v7XvdJgLjfCJe4GesHMKb/snA== X-Google-Smtp-Source: ABdhPJx+ahr5NB8rojJMsUXFt7L+galG1ve8weyW+AhM2+lKj7o7pt+OGDohvL8aaId1SVV0gICs5A== X-Received: by 2002:a05:6000:18a7:: with SMTP id b7mr186746wri.308.1636992777608; Mon, 15 Nov 2021 08:12:57 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id j40sm16315615wms.16.2021.11.15.08.12.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Nov 2021 08:12:57 -0800 (PST) From: David Plowman To: hverkuil-cisco@xs4all.nl, laurent.pinchart@ideasonboard.com, kieran.bingham@ideasonboard.com, jacopo@jmondi.org, tfiga@google.com, naush@raspberrypi.com, libcamera-devel@lists.libcamera.org Date: Mon, 15 Nov 2021 16:11:59 +0000 Message-Id: <20211115161205.24335-2-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211115161205.24335-1-david.plowman@raspberrypi.com> References: <20211115161205.24335-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 1/7] libcamera: Add ColorSpace class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This class represents a color space by defining its color primaries, YCbCr encoding, the transfer (gamma) function it uses, and whether the output is full or limited range. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- include/libcamera/color_space.h | 88 +++++++++++ include/libcamera/meson.build | 1 + src/libcamera/color_space.cpp | 257 ++++++++++++++++++++++++++++++++ src/libcamera/meson.build | 1 + 4 files changed, 347 insertions(+) create mode 100644 include/libcamera/color_space.h create mode 100644 src/libcamera/color_space.cpp diff --git a/include/libcamera/color_space.h b/include/libcamera/color_space.h new file mode 100644 index 00000000..2af9da31 --- /dev/null +++ b/include/libcamera/color_space.h @@ -0,0 +1,88 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Raspberry Pi (Trading) Limited + * + * color_space.h - color space definitions + */ + +#ifndef __LIBCAMERA_COLOR_SPACE_H__ +#define __LIBCAMERA_COLOR_SPACE_H__ + +#include + +namespace libcamera { + +class ColorSpace +{ +public: + enum class Primaries : int { + Undefined, + Raw, + Smpte170m, + Rec709, + Rec2020, + }; + + enum class YcbcrEncoding : int { + Undefined, + Rec601, + Rec709, + Rec2020, + }; + + enum class TransferFunction : int { + Undefined, + Linear, + Srgb, + Rec709, + }; + + enum class Range : int { + Undefined, + Full, + Limited, + }; + + constexpr ColorSpace() + : ColorSpace(Primaries::Undefined, YcbcrEncoding::Undefined, TransferFunction::Undefined, Range::Undefined) + { + } + + constexpr ColorSpace(Primaries p, YcbcrEncoding e, TransferFunction t, Range r) + : primaries(p), ycbcrEncoding(e), transferFunction(t), range(r) + { + } + + static const ColorSpace Undefined; + static const ColorSpace Raw; + static const ColorSpace Jpeg; + static const ColorSpace Smpte170m; + static const ColorSpace Rec709; + static const ColorSpace Rec2020; + + Primaries primaries; + YcbcrEncoding ycbcrEncoding; + TransferFunction transferFunction; + Range range; + + bool isFullyDefined() const; + + const std::string toString() const; +}; + +constexpr ColorSpace ColorSpace::Undefined = { Primaries::Undefined, YcbcrEncoding::Undefined, TransferFunction::Undefined, Range::Undefined }; +constexpr ColorSpace ColorSpace::Raw = { Primaries::Raw, YcbcrEncoding::Rec601, TransferFunction::Linear, Range::Full }; +constexpr ColorSpace ColorSpace::Jpeg = { Primaries::Rec709, YcbcrEncoding::Rec601, TransferFunction::Srgb, Range::Full }; +constexpr ColorSpace ColorSpace::Smpte170m = { Primaries::Smpte170m, YcbcrEncoding::Rec601, TransferFunction::Rec709, Range::Limited }; +constexpr ColorSpace ColorSpace::Rec709 = { Primaries::Rec709, YcbcrEncoding::Rec709, TransferFunction::Rec709, Range::Limited }; +constexpr ColorSpace ColorSpace::Rec2020 = { Primaries::Rec2020, YcbcrEncoding::Rec2020, TransferFunction::Rec709, Range::Limited }; + +bool operator==(const ColorSpace &lhs, const ColorSpace &rhs); +static inline bool operator!=(const ColorSpace &lhs, const ColorSpace &rhs) +{ + return !(lhs == rhs); +} + +} /* namespace libcamera */ + +#endif /* __LIBCAMERA_COLOR_SPACE_H__ */ diff --git a/include/libcamera/meson.build b/include/libcamera/meson.build index 7155ff20..131e1740 100644 --- a/include/libcamera/meson.build +++ b/include/libcamera/meson.build @@ -5,6 +5,7 @@ libcamera_include_dir = 'libcamera' / 'libcamera' libcamera_public_headers = files([ 'camera.h', 'camera_manager.h', + 'color_space.h', 'compiler.h', 'controls.h', 'file_descriptor.h', diff --git a/src/libcamera/color_space.cpp b/src/libcamera/color_space.cpp new file mode 100644 index 00000000..dfd9fa1d --- /dev/null +++ b/src/libcamera/color_space.cpp @@ -0,0 +1,257 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Raspberry Pi (Trading) Limited + * + * color_space.cpp - color spaces. + */ + +#include + +#include +#include +#include + +/** + * \file color_space.h + * \brief Class and enums to represent color spaces + */ + +namespace libcamera { + +/** + * \class ColorSpace + * \brief Class to describe a color space + * + * The ColorSpace class defines the color primaries, the Y'CbCr encoding, + * the transfer function associated with the color space, and the range + * (sometimes also referred to as the quantisation) of the color space. + * + * Certain combinations of these fields form well-known standard color + * spaces such as "JPEG" or "REC709". Applications must not request color + * spaces with undefined fields, but the "Undefined" value may be + * returned if the camera drivers decide to use a color space that is + * not recognised by the ColorSpace class. + * + * For more information on the specific color spaces described here, please + * see: + * + * sRGB and JPEG +>* SMPTE 170M + * Rec.709 + * Rec.2020 + */ + +/** + * \enum ColorSpace::Primaries + * \brief The color primaries for this color space + * + * \var ColorSpace::Primaries::Undefined + * \brief The color primaries are undefined + * \var ColorSpace::Primaries::Raw + * \brief These are raw colors directly from a sensor + * \var ColorSpace::Primaries::Smpte170m + * \brief SMPTE 170M color primaries + * \var ColorSpace::Primaries::Rec709 + * \brief Rec.709 color primaries + * \var ColorSpace::Primaries::Rec2020 + * \brief Rec.2020 color primaries + */ + +/** + * \enum ColorSpace::YcbcrEncoding + * \brief The Y'CbCr encoding + * + * \var ColorSpace::YcbcrEncoding::Undefined + * \brief The Y'CbCr encoding is undefined + * \var ColorSpace::YcbcrEncoding::Rec601 + * \brief Rec.601 Y'CbCr encoding + * \var ColorSpace::YcbcrEncoding::Rec709 + * \brief Rec.709 Y'CbCr encoding + * \var ColorSpace::YcbcrEncoding::Rec2020 + * \brief Rec.2020 Y'CbCr encoding + */ + +/** + * \enum ColorSpace::TransferFunction + * \brief The transfer function used for this color space + * + * \var ColorSpace::TransferFunction::Undefined + * \brief The transfer function is not specified + * \var ColorSpace::TransferFunction::Linear + * \brief This color space uses a linear (identity) transfer function + * \var ColorSpace::TransferFunction::Srgb + * \brief sRGB transfer function + * \var ColorSpace::TransferFunction::Rec709 + * \brief Rec709 transfer function + */ + +/** + * \enum ColorSpace::Range + * \brief The range (sometimes "quantisation") for this color space + * + * \var ColorSpace::Range::Undefined + * \brief The range is not specified + * \var ColorSpace::Range::Full + * \brief This color space uses full range pixel values + * \var ColorSpace::Range::Limited + * \brief This color space uses limited range pixel values, being + * 16 to 235 for Y' and 16 to 240 for Cb and Cr (8 bits per sample) + * or 64 to 940 for Y' and 16 to 960 for Cb and Cr (10 bits) + */ + +/** + * \fn ColorSpace::ColorSpace(Encoding e, TransferFunction t, Range r) + * \brief Construct a ColorSpace from explicit values + * \param[in] e The Y'CbCr encoding + * \param[in] t The transfer function for the color space + * \param[in] r The range of the pixel values in this color space + */ + +/** + * \fn ColorSpace::ColorSpace() + * \brief Construct a color space with undefined encoding, transfer function + * and range + */ + +/** + * \brief Check if all the fields of the color space are defined + * \return Return true if all the fields of the color space are defined, + * otherwise false + */ +bool ColorSpace::isFullyDefined() const +{ + return primaries != Primaries::Undefined && + ycbcrEncoding != YcbcrEncoding::Undefined && + transferFunction != TransferFunction::Undefined && + range != Range::Undefined; +} + +/** + * \brief Assemble and return a readable string representation of the + * ColorSpace + * \return A string describing the ColorSpace + */ +const std::string ColorSpace::toString() const +{ + /* Print out a brief name only for standard color sapces. */ + + static const std::vector> colorSpaceNames = { + { ColorSpace::Undefined, "Undefined" }, + { ColorSpace::Raw, "Raw" }, + { ColorSpace::Jpeg, "Jpeg" }, + { ColorSpace::Smpte170m, "Smpte170m" }, + { ColorSpace::Rec709, "Rec709" }, + { ColorSpace::Rec2020, "Rec2020" }, + }; + auto it = std::find_if(colorSpaceNames.begin(), colorSpaceNames.end(), + [this](const auto &item) { + return *this == item.first; + }); + if (it != colorSpaceNames.end()) + return std::string(it->second); + + static const char *primariesNames[] = { + "Undefined", + "Raw", + "Smpte170m", + "Rec709", + "Rec2020", + }; + static const char *encodingNames[] = { + "Undefined", + "Rec601", + "Rec709", + "Rec2020", + }; + static const char *transferFunctionNames[] = { + "Undefined", + "Linear", + "Srgb", + "Rec709", + }; + static const char *rangeNames[] = { + "Undefined", + "Full", + "Limited", + }; + + std::stringstream ss; + ss << std::string(primariesNames[static_cast(primaries)]) << "/" + << std::string(encodingNames[static_cast(ycbcrEncoding)]) << "/" + << std::string(transferFunctionNames[static_cast(transferFunction)]) << "/" + << std::string(rangeNames[static_cast(range)]); + + return ss.str(); +} + +/** + * \var ColorSpace::primaries + * \brief The color primaries + */ + +/** + * \var ColorSpace::ycbcrEncoding + * \brief The Y'CbCr encoding + */ + +/** + * \var ColorSpace::transferFunction + * \brief The transfer function for this color space + */ + +/** + * \var ColorSpace::range + * \brief The pixel range used by this color space + */ + +/** + * \var ColorSpace::Undefined + * \brief A constant representing a fully undefined color space + */ + +/** + * \var ColorSpace::Raw + * \brief A constant representing a raw color space (from a sensor) + */ + +/** + * \var ColorSpace::Jpeg + * \brief A constant representing the JPEG color space used for + * encoding JPEG images (and regarded as being the same as the sRGB + * color space) + */ + +/** + * \var ColorSpace::Smpte170m + * \brief A constant representing the SMPTE170M color space + */ + +/** + * \var ColorSpace::Rec709 + * \brief A constant representing the Rec.709 color space + */ + +/** + * \var ColorSpace::Rec2020 + * \brief A constant representing the Rec.2020 color space + */ + +/** + * \brief Compare color spaces for equality + * \return True if the two color spaces are identical, false otherwise + */ +bool operator==(const ColorSpace &lhs, const ColorSpace &rhs) +{ + return lhs.primaries == rhs.primaries && + lhs.ycbcrEncoding == rhs.ycbcrEncoding && + lhs.transferFunction == rhs.transferFunction && + lhs.range == rhs.range; +} + +/** + * \fn bool operator!=(const ColorSpace &lhs, const ColorSpace &rhs) + * \brief Compare color spaces for inequality + * \return True if the two color spaces are not identical, false otherwise + */ + +} /* namespace libcamera */ diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build index 6727a777..e7371d20 100644 --- a/src/libcamera/meson.build +++ b/src/libcamera/meson.build @@ -8,6 +8,7 @@ libcamera_sources = files([ 'camera_manager.cpp', 'camera_sensor.cpp', 'camera_sensor_properties.cpp', + 'color_space.cpp', 'controls.cpp', 'control_serializer.cpp', 'control_validator.cpp', From patchwork Mon Nov 15 16:12:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 14608 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5B642BF415 for ; Mon, 15 Nov 2021 16:13:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 037276036C; Mon, 15 Nov 2021 17:13:05 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="XVMawex1"; dkim-atps=neutral Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6B5526038A for ; Mon, 15 Nov 2021 17:12:59 +0100 (CET) Received: by mail-wr1-x42e.google.com with SMTP id a9so5564227wrr.8 for ; Mon, 15 Nov 2021 08:12:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=nUVlWu72PlyE29kbK+yxhDq/7AfELhGkPOIcJt8NS14=; b=XVMawex1WpM22FegeF6CknBX6PO3gxVlU5o/PF1oQtiDQucLhRtkcoOGOfMRHqUSPC WHR0/wg4xKHmcvVxU3/fctvePWjm+XjBtgTyuOMol1VD70073WRiYbJL8ojlZAzibn8N kFp98sg4g5AyX0HbUAUIPktYluKKd8zeKivSGwGGXqeIbqjkiAszjZPzjiZdHHC7YdRa RPnDbgtAPsfP+VqzgtJ22cs0Cy1ZqvzFWr96L9fDOYawU13+NaP6RyjtKaLrCMKPGUJB PL8k4OuLIYJh2rD5KctEIFOdyWF4L6SAlulOKOw8Wdw2Ny3DMCvYqBBXbGJVVMGyphyX F7cg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=nUVlWu72PlyE29kbK+yxhDq/7AfELhGkPOIcJt8NS14=; b=EdAiHo0IATwrecs2df8+0h9Km/nAz83Dq8UlX59vatiCbJefICe/24kTIREarbUn+3 7/ETZUzLm69uENGVRAtXVumUIEOw7yKDxSWJ/FNikDK07yN/OY5WOFqlXnS1Zr2cwc1s LpVDYu023NttZl4REygzFSUlN3EVTFzlmpOXEDGOR4PLALpmPdMi6nvPz0GEkY5sCdJw elbN2iyIsMNDDc74gC1nNSV67i3i94msOAh8u3HLPEkzFCEO7joERLFZWnkpuCYRiuxz ipYI9MQHgdPRUUCyaFk0qY/RdHYEqIRuN1+JNWzsZaBQtuTATm7UGGeCqHkzta+oYlf3 Hg0g== X-Gm-Message-State: AOAM531GuDfpPnrHPvd6P097xl+eQ12CcVWl1UpZmm9B1g8yPH0Q6USi JDMrH3INzpgRNlGAOhQYDJHJdA== X-Google-Smtp-Source: ABdhPJzFb0ubWNGB/FhLothieQB55Oanm48khGe1NurwGjcCN4DLho+q9T1NK1PIvLD3L8gp0cwoKg== X-Received: by 2002:a5d:68cb:: with SMTP id p11mr186357wrw.262.1636992779201; Mon, 15 Nov 2021 08:12:59 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id j40sm16315615wms.16.2021.11.15.08.12.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Nov 2021 08:12:57 -0800 (PST) From: David Plowman To: hverkuil-cisco@xs4all.nl, laurent.pinchart@ideasonboard.com, kieran.bingham@ideasonboard.com, jacopo@jmondi.org, tfiga@google.com, naush@raspberrypi.com, libcamera-devel@lists.libcamera.org Date: Mon, 15 Nov 2021 16:12:00 +0000 Message-Id: <20211115161205.24335-3-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211115161205.24335-1-david.plowman@raspberrypi.com> References: <20211115161205.24335-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 2/7] libcamera: Add ColorSpace fields to StreamConfiguration X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This is so that applications can choose appropriate color spaces which will then be passed down to the V4L2 devices. There are two fields: the "requestedColorSpace" which is the one an application wants, and the "actualColorSpace" which is what the underlying hardware can deliver, and which is filled in by the validate() method. Signed-off-by: David Plowman --- include/libcamera/stream.h | 4 ++++ src/libcamera/stream.cpp | 25 +++++++++++++++++++++++++ 2 files changed, 29 insertions(+) diff --git a/include/libcamera/stream.h b/include/libcamera/stream.h index 0c55e716..fe491ff5 100644 --- a/include/libcamera/stream.h +++ b/include/libcamera/stream.h @@ -12,6 +12,7 @@ #include #include +#include #include #include #include @@ -47,6 +48,9 @@ struct StreamConfiguration { unsigned int bufferCount; + ColorSpace requestedColorSpace; + ColorSpace actualColorSpace; + Stream *stream() const { return stream_; } void setStream(Stream *stream) { stream_ = stream; } const StreamFormats &formats() const { return formats_; } diff --git a/src/libcamera/stream.cpp b/src/libcamera/stream.cpp index b421e17e..1ddbbb8c 100644 --- a/src/libcamera/stream.cpp +++ b/src/libcamera/stream.cpp @@ -329,6 +329,31 @@ StreamConfiguration::StreamConfiguration(const StreamFormats &formats) * \brief Requested number of buffers to allocate for the stream */ +/** + * \var StreamConfiguration::requestedColorSpace + * \brief The ColorSpace this stream should have + * + * The generateConfiguration method will generate reasonable default + * values (ColorSpace::Jpeg for stills, ColorSpace::Rec709 for video and + * ColorSpace::Raw for raw streams) but applications are free to overwrite + * this value. + */ + +/** + * \var StreamConfiguration::actualColorSpace + * \brief The ColorSpace the will be used for this stream + * + * This field is filled in by CameraConfiguration::validate(). + * Normally this should match the requestedColorSpace, but it may differ + * if the hardware does not support it. + * + * In general cameras may have different constraints here, for example, + * they may require all output streams to share the same color space. + * Sometimes the fields within this color space may report "Undefined" + * values if the hardware drivers are going to use a color space that + * is not recognised by the ColorSpace class. + */ + /** * \fn StreamConfiguration::stream() * \brief Retrieve the stream associated with the configuration From patchwork Mon Nov 15 16:12:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 14609 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E708FBF415 for ; Mon, 15 Nov 2021 16:13:07 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9D83B60378; Mon, 15 Nov 2021 17:13:07 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="SVDT0CdL"; dkim-atps=neutral Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3DC32603B5 for ; Mon, 15 Nov 2021 17:13:00 +0100 (CET) Received: by mail-wm1-x329.google.com with SMTP id z200so14467944wmc.1 for ; Mon, 15 Nov 2021 08:13:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ron3Qwrj+ZdHUA2wy9RhvrSwsJz/Iu9EfWpAou0UGIc=; b=SVDT0CdLxXi8aCC3BPK82DDg81ZHfsyAZxZTW2Pmog+vuN8WOGcKcl4p9s7OEEjZ4L u2TzLWq34nGUpXcvLDeCmqTpW6aNctll7s8zOZE5qRNKHqGXOHm5UIaoLlIu/o3zV/TS z4uACIyNlQsdccPMKPyk1I+efQ2Y5X1xffXVU2aUEm5sE+p5/TbhoFT0sKhPEyZ/MDkS xSf4yQzvhQjBNdTilUE1K9YoYgAEA45p/2euspUw0hU2jev3uG3ykVVhlu4kLR9yZj8o YqUmhIjlWW305jYeGreJjmObxomqIY/y8O+Fm76DPd2lzsDd3wu/xBQ6WmzOPw1PayhV dpUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ron3Qwrj+ZdHUA2wy9RhvrSwsJz/Iu9EfWpAou0UGIc=; b=CqZJ4zebQFr/v9KPEsteBMZwp0pMnMPr0Gra/LqfsgX/7yvqBWWTWhzAggwaTXXP9f fK/YA7P7uzHn6g42bv/lr0ghI1zP3KjskiEEaiAh0juHIQV7fLXpguE0+32aR7GqepZs 1NoFqErEqQhxsdHSu+vNcoWC36MX19lIBv7qaX2tV3+So/CpFVV9kfTqLxJs+c2dTEzn b8XgwaePFEVHkq2JsH+TO7yvaI2tNv7w+AR3nfUT9TImhmwD1J38PwQo7JLdOYIX6Q7R JuQbwAsVTJO+YH8w43A1c1yMqQ0cd9VREPykLbPa+Gi0REgprrUnhbm9qT3T65H2Bth8 mFpA== X-Gm-Message-State: AOAM530e0bs+rfVQiUiWvYy6npHLvrxfDyoC+5pTM0CRcXuGy/4YjJyC rurtC3OhKQv7p3eR/4XL+GAa3Q== X-Google-Smtp-Source: ABdhPJyt34JW8D7D2kDHVxQmDUYI2FPOw8LtftUURiG9ktdpOijzgfXTalEEUHAmkhy7i3AbWapMZg== X-Received: by 2002:a7b:cb55:: with SMTP id v21mr43652039wmj.83.1636992779865; Mon, 15 Nov 2021 08:12:59 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id j40sm16315615wms.16.2021.11.15.08.12.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Nov 2021 08:12:59 -0800 (PST) From: David Plowman To: hverkuil-cisco@xs4all.nl, laurent.pinchart@ideasonboard.com, kieran.bingham@ideasonboard.com, jacopo@jmondi.org, tfiga@google.com, naush@raspberrypi.com, libcamera-devel@lists.libcamera.org Date: Mon, 15 Nov 2021 16:12:01 +0000 Message-Id: <20211115161205.24335-4-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211115161205.24335-1-david.plowman@raspberrypi.com> References: <20211115161205.24335-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 3/7] libcamera: Convert between ColorSpace class and V4L2 formats X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" These methods are added to the base V4L2Device class so that they can be shared both by the video device class and subdevices. With the ColorSpace class, the color space and related other fields are stored together, corresponding to a number of fields in the various different V4L2 format structures. Template methods are therefore a convenient implementation. Note that we must explicitly instantiate the templates that will be needed. Signed-off-by: David Plowman --- include/libcamera/internal/v4l2_device.h | 7 + src/libcamera/v4l2_device.cpp | 171 +++++++++++++++++++++++ 2 files changed, 178 insertions(+) diff --git a/include/libcamera/internal/v4l2_device.h b/include/libcamera/internal/v4l2_device.h index f21bc370..24fc4984 100644 --- a/include/libcamera/internal/v4l2_device.h +++ b/include/libcamera/internal/v4l2_device.h @@ -17,6 +17,7 @@ #include #include +#include #include namespace libcamera { @@ -44,6 +45,12 @@ public: void updateControlInfo(); + template + static ColorSpace toColorSpace(const T &v4l2Format); + + template + static int fromColorSpace(const ColorSpace &colorSpace, T &v4l2Format); + protected: V4L2Device(const std::string &deviceNode); ~V4L2Device(); diff --git a/src/libcamera/v4l2_device.cpp b/src/libcamera/v4l2_device.cpp index 9c783c9c..34ec2bf3 100644 --- a/src/libcamera/v4l2_device.cpp +++ b/src/libcamera/v4l2_device.cpp @@ -16,6 +16,8 @@ #include #include +#include + #include #include #include @@ -731,4 +733,173 @@ void V4L2Device::eventAvailable() frameStart.emit(event.u.frame_sync.frame_sequence); } +static const std::map v4l2ToColorSpace = { + { V4L2_COLORSPACE_RAW, ColorSpace::Raw }, + { V4L2_COLORSPACE_JPEG, ColorSpace::Jpeg }, + { V4L2_COLORSPACE_SRGB, ColorSpace::Jpeg }, + { V4L2_COLORSPACE_SMPTE170M, ColorSpace::Smpte170m }, + { V4L2_COLORSPACE_REC709, ColorSpace::Rec709 }, + { V4L2_COLORSPACE_BT2020, ColorSpace::Rec2020 }, +}; + +static const std::map v4l2ToYcbcrEncoding = { + { V4L2_YCBCR_ENC_601, ColorSpace::YcbcrEncoding::Rec601 }, + { V4L2_YCBCR_ENC_709, ColorSpace::YcbcrEncoding::Rec709 }, + { V4L2_YCBCR_ENC_BT2020, ColorSpace::YcbcrEncoding::Rec2020 }, +}; + +static const std::map v4l2ToTransferFunction = { + { V4L2_XFER_FUNC_NONE, ColorSpace::TransferFunction::Linear }, + { V4L2_XFER_FUNC_SRGB, ColorSpace::TransferFunction::Srgb }, + { V4L2_XFER_FUNC_709, ColorSpace::TransferFunction::Rec709 }, +}; + +static const std::map v4l2ToRange = { + { V4L2_QUANTIZATION_FULL_RANGE, ColorSpace::Range::Full }, + { V4L2_QUANTIZATION_LIM_RANGE, ColorSpace::Range::Limited }, +}; + +static const std::vector> colorSpaceToV4l2 = { + { ColorSpace::Raw, V4L2_COLORSPACE_RAW }, + { ColorSpace::Jpeg, V4L2_COLORSPACE_JPEG }, + { ColorSpace::Smpte170m, V4L2_COLORSPACE_SMPTE170M }, + { ColorSpace::Rec709, V4L2_COLORSPACE_REC709 }, + { ColorSpace::Rec2020, V4L2_COLORSPACE_BT2020 }, +}; + +static const std::map primariesToV4l2 = { + { ColorSpace::Primaries::Raw, V4L2_COLORSPACE_RAW }, + { ColorSpace::Primaries::Smpte170m, V4L2_COLORSPACE_SMPTE170M }, + { ColorSpace::Primaries::Rec709, V4L2_COLORSPACE_REC709 }, + { ColorSpace::Primaries::Rec2020, V4L2_COLORSPACE_BT2020 }, +}; + +static const std::map ycbcrEncodingToV4l2 = { + { ColorSpace::YcbcrEncoding::Rec601, V4L2_YCBCR_ENC_601 }, + { ColorSpace::YcbcrEncoding::Rec709, V4L2_YCBCR_ENC_709 }, + { ColorSpace::YcbcrEncoding::Rec2020, V4L2_YCBCR_ENC_BT2020 }, +}; + +static const std::map transferFunctionToV4l2 = { + { ColorSpace::TransferFunction::Linear, V4L2_XFER_FUNC_NONE }, + { ColorSpace::TransferFunction::Srgb, V4L2_XFER_FUNC_SRGB }, + { ColorSpace::TransferFunction::Rec709, V4L2_XFER_FUNC_709 }, +}; + +static const std::map rangeToV4l2 = { + { ColorSpace::Range::Full, V4L2_QUANTIZATION_FULL_RANGE }, + { ColorSpace::Range::Limited, V4L2_QUANTIZATION_LIM_RANGE }, +}; + +/** + * \brief Convert the color space fields in a V4L2 format to a ColorSpace + * \param[in] v4l2Format A V4L2 format containing color space information + * + * The colorspace, ycbcr_enc, xfer_func and quantization fields within a + * V4L2 format structure are converted to a corresponding ColorSpace. + * + * If any V4L2 fields are not recognised those are marked as undefined in + * the ColorSpace, but other fields are still initialised where possible. + * This situation can be detected using the returned value's + * ColorSpace::isFullyDefined() method. + * + * \return The ColorSpace corresponding to the input V4L2 format + */ +template +ColorSpace V4L2Device::toColorSpace(const T &v4l2Format) +{ + ColorSpace colorSpace; + + auto itColor = v4l2ToColorSpace.find(v4l2Format.colorspace); + if (itColor != v4l2ToColorSpace.end()) + colorSpace = itColor->second; + + auto itYcbcrEncoding = v4l2ToYcbcrEncoding.find(v4l2Format.ycbcr_enc); + if (itYcbcrEncoding != v4l2ToYcbcrEncoding.end()) + colorSpace.ycbcrEncoding = itYcbcrEncoding->second; + + auto itTransfer = v4l2ToTransferFunction.find(v4l2Format.xfer_func); + if (itTransfer != v4l2ToTransferFunction.end()) + colorSpace.transferFunction = itTransfer->second; + + auto itRange = v4l2ToRange.find(v4l2Format.quantization); + if (itRange != v4l2ToRange.end()) + colorSpace.range = itRange->second; + + return colorSpace; +} + +template ColorSpace V4L2Device::toColorSpace(const struct v4l2_pix_format &); +template ColorSpace V4L2Device::toColorSpace(const struct v4l2_pix_format_mplane &); +template ColorSpace V4L2Device::toColorSpace(const struct v4l2_mbus_framefmt &); + +/** + * \brief Fill in the color space fields of a V4L2 format from a ColorSpace + * \param[in] colorSpace The ColorSpace to be converted + * \param[out] v4l2Format A V4L2 format containing color space information + * + * The colorspace, ycbcr_enc, xfer_func and quantization fields within a + * V4L2 format structure are filled in from a corresponding ColorSpace. + * + * An error is returned if any of the V4L2 fields do not support the + * value given in the ColorSpace. Such fields are set to the V4L2 + * "default" values, but all other fields are still filled in where + * possible. + * + * \return 0 on success or a negative error code otherwise + */ +template +int V4L2Device::fromColorSpace(const ColorSpace &colorSpace, T &v4l2Format) +{ + int ret = 0; + + v4l2Format.colorspace = V4L2_COLORSPACE_DEFAULT; + v4l2Format.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; + v4l2Format.xfer_func = V4L2_XFER_FUNC_DEFAULT; + v4l2Format.quantization = V4L2_QUANTIZATION_DEFAULT; + + auto itColor = std::find_if(colorSpaceToV4l2.begin(), colorSpaceToV4l2.end(), + [&colorSpace](const auto &item) { + return colorSpace == item.first; + }); + if (itColor != colorSpaceToV4l2.end()) { + v4l2Format.colorspace = itColor->second; + return ret; + } + + /* + * If the colorSpace doesn't precisely match a standard color space, + * then we must choose a V4L2 colorspace with matching primaries. + */ + auto itPrimaries = primariesToV4l2.find(colorSpace.primaries); + if (itPrimaries != primariesToV4l2.end()) + v4l2Format.colorspace = itPrimaries->second; + else + ret = -1; + + auto itYcbcrEncoding = ycbcrEncodingToV4l2.find(colorSpace.ycbcrEncoding); + if (itYcbcrEncoding != ycbcrEncodingToV4l2.end()) + v4l2Format.ycbcr_enc = itYcbcrEncoding->second; + else + ret = -1; + + auto itTransfer = transferFunctionToV4l2.find(colorSpace.transferFunction); + if (itTransfer != transferFunctionToV4l2.end()) + v4l2Format.xfer_func = itTransfer->second; + else + ret = -1; + + auto itRange = rangeToV4l2.find(colorSpace.range); + if (itRange != rangeToV4l2.end()) + v4l2Format.quantization = itRange->second; + else + ret = -1; + + return ret; +} + +template int V4L2Device::fromColorSpace(const ColorSpace &, struct v4l2_pix_format &); +template int V4L2Device::fromColorSpace(const ColorSpace &, struct v4l2_pix_format_mplane &); +template int V4L2Device::fromColorSpace(const ColorSpace &, struct v4l2_mbus_framefmt &); + } /* namespace libcamera */ From patchwork Mon Nov 15 16:12:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 14610 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8B57AC324E for ; Mon, 15 Nov 2021 16:13:08 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 111066036C; Mon, 15 Nov 2021 17:13:08 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="mvdIQrCX"; dkim-atps=neutral Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1DCA260378 for ; Mon, 15 Nov 2021 17:13:01 +0100 (CET) Received: by mail-wm1-x334.google.com with SMTP id r9-20020a7bc089000000b00332f4abf43fso177805wmh.0 for ; Mon, 15 Nov 2021 08:13:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3jzpKNdq0T31XWyv1nf4OXskRk3Z5TSZ2eDhnU9oflA=; b=mvdIQrCXL/yZe6pnmvRy+iXbwl1nl/NTwTjw2kn3QgD2ZskZFRwtUqO/BhyRxX3nz+ YdG0KtpTKCyCmG3qaIuPOMR0c6IJsW9bCRi2QT+tYbePTKyWl7VlZx+ap3PmN2+jX6LN +pa4D2fojI8xNFvV9HA9D3iVk6pBShkH6Sd1v8JjlHSy2WtYEYYI9Iy34BS2ZheSEMiA MNSDnoF/BXqIFvbUe4GLhiNtNbnxPEkBdE7KVkhrx4NL8yX3XrT+RCeCsAWMUo6vtvPE flphFKNGsVCBOHpiReNwwPCgWZvmYsDS+3ndT0fKd63/9XUPNVZJf/f6he8ZTDVs+rqQ 3qzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3jzpKNdq0T31XWyv1nf4OXskRk3Z5TSZ2eDhnU9oflA=; b=2OiwbLc8Bx+kH9cnRlEyDwXW6hwCvxfCqFOq15aP3xyYL9hGA2miiASrtIN/Jlgmq0 q1PLPTzZM2he+ZSmlbqk68WND7Inzlg0M+gdqZf0WFSh0FLproJ/pNsbFwvRlRa//jod FVS4xWfLg9pvDGHBR0jMe7VVeXVmkIx/9ixYuVmjUs5gAaKRdD4282ItmePoPWLYaBA7 WuLSF69FBm05fQ5tMMKIWFNF5zabrEihk1pGbFygGE4gBL5Uj1/cK0l2qhJ+W1Al8AkH 04nrsK1kfUe27DCPBQEdt3iWDNZTz19GZEw+O75IJ8/vCgtInjLjFaIsri4pgA8cpWCq cDRQ== X-Gm-Message-State: AOAM530yvoZuLoERFe/pDd4b0pkg7g4w0yM5+oUrPU+3cpfSpU/RrAdH BX4Zxme5mK14xF1sYOi6yz7u7w== X-Google-Smtp-Source: ABdhPJypEF3n80Zva1HokmHj0j/FgyLvwk0y2/wqcaHgmKpDGLy1STTNgLkpYFUyz88RRSQcmS1hMw== X-Received: by 2002:a7b:c102:: with SMTP id w2mr60289070wmi.151.1636992780675; Mon, 15 Nov 2021 08:13:00 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id j40sm16315615wms.16.2021.11.15.08.12.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Nov 2021 08:13:00 -0800 (PST) From: David Plowman To: hverkuil-cisco@xs4all.nl, laurent.pinchart@ideasonboard.com, kieran.bingham@ideasonboard.com, jacopo@jmondi.org, tfiga@google.com, naush@raspberrypi.com, libcamera-devel@lists.libcamera.org Date: Mon, 15 Nov 2021 16:12:02 +0000 Message-Id: <20211115161205.24335-5-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211115161205.24335-1-david.plowman@raspberrypi.com> References: <20211115161205.24335-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 4/7] libcamera: Support passing ColorSpaces to V4L2 video devices X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The ColorSpace from the StreamConfiguration is now handled appropriately in the V4L2VideoDevice. Signed-off-by: David Plowman --- include/libcamera/internal/v4l2_videodevice.h | 2 + src/libcamera/v4l2_videodevice.cpp | 67 +++++++++++++++++-- 2 files changed, 65 insertions(+), 4 deletions(-) diff --git a/include/libcamera/internal/v4l2_videodevice.h b/include/libcamera/internal/v4l2_videodevice.h index a1c458e4..6ca749c1 100644 --- a/include/libcamera/internal/v4l2_videodevice.h +++ b/include/libcamera/internal/v4l2_videodevice.h @@ -20,6 +20,7 @@ #include #include +#include #include #include #include @@ -167,6 +168,7 @@ public: V4L2PixelFormat fourcc; Size size; + ColorSpace colorSpace; std::array planes; unsigned int planesCount = 0; diff --git a/src/libcamera/v4l2_videodevice.cpp b/src/libcamera/v4l2_videodevice.cpp index 4f04212d..ccf3d28c 100644 --- a/src/libcamera/v4l2_videodevice.cpp +++ b/src/libcamera/v4l2_videodevice.cpp @@ -370,17 +370,29 @@ bool V4L2BufferCache::Entry::operator==(const FrameBuffer &buffer) const * \brief The plane line stride (in bytes) */ +/** + * \var V4L2DeviceFormat::fourcc + * \brief The fourcc code describing the pixel encoding scheme + * + * The fourcc code, as defined by the V4L2 API with the V4L2_PIX_FMT_* macros, + * that identifies the image format pixel encoding scheme. + */ + /** * \var V4L2DeviceFormat::size * \brief The image size in pixels */ /** - * \var V4L2DeviceFormat::fourcc - * \brief The fourcc code describing the pixel encoding scheme + * \var V4L2DeviceFormat::colorSpace + * \brief The color space of the pixels * - * The fourcc code, as defined by the V4L2 API with the V4L2_PIX_FMT_* macros, - * that identifies the image format pixel encoding scheme. + * When setting or trying a format, passing in "Undefined" fields in the + * ColorSpace is not permitted because the driver will then make an + * arbitrary choice of its own. Choices made by the driver will be + * passed back in the normal way, though note that "Undefined" values can + * be returned if the device has chosen something that the ColorSpace + * class cannot represent. */ /** @@ -879,6 +891,12 @@ int V4L2VideoDevice::getFormatMultiplane(V4L2DeviceFormat *format) format->fourcc = V4L2PixelFormat(pix->pixelformat); format->planesCount = pix->num_planes; + format->colorSpace = toColorSpace(*pix); + if (!format->colorSpace.isFullyDefined()) + LOG(V4L2, Warning) + << "Retrieved undefined color space: " + << format->colorSpace.toString(); + for (unsigned int i = 0; i < format->planesCount; ++i) { format->planes[i].bpl = pix->plane_fmt[i].bytesperline; format->planes[i].size = pix->plane_fmt[i].sizeimage; @@ -893,6 +911,11 @@ int V4L2VideoDevice::trySetFormatMultiplane(V4L2DeviceFormat *format, bool set) struct v4l2_pix_format_mplane *pix = &v4l2Format.fmt.pix_mp; int ret; + if (!format->colorSpace.isFullyDefined()) + LOG(V4L2, Error) + << "Trying to set undefined color space: " + << format->colorSpace.toString(); + v4l2Format.type = bufferType_; pix->width = format->size.width; pix->height = format->size.height; @@ -900,6 +923,12 @@ int V4L2VideoDevice::trySetFormatMultiplane(V4L2DeviceFormat *format, bool set) pix->num_planes = format->planesCount; pix->field = V4L2_FIELD_NONE; + ret = fromColorSpace(format->colorSpace, *pix); + if (ret < 0) + LOG(V4L2, Warning) + << "Setting color space unrecognised by V4L2: " + << format->colorSpace.toString(); + ASSERT(pix->num_planes <= std::size(pix->plane_fmt)); for (unsigned int i = 0; i < pix->num_planes; ++i) { @@ -928,6 +957,12 @@ int V4L2VideoDevice::trySetFormatMultiplane(V4L2DeviceFormat *format, bool set) format->planes[i].size = pix->plane_fmt[i].sizeimage; } + format->colorSpace = toColorSpace(*pix); + if (!format->colorSpace.isFullyDefined()) + LOG(V4L2, Warning) + << "Undefined color space has been set: " + << format->colorSpace.toString(); + return 0; } @@ -951,6 +986,12 @@ int V4L2VideoDevice::getFormatSingleplane(V4L2DeviceFormat *format) format->planes[0].bpl = pix->bytesperline; format->planes[0].size = pix->sizeimage; + format->colorSpace = toColorSpace(*pix); + if (!format->colorSpace.isFullyDefined()) + LOG(V4L2, Warning) + << "Retrieved undefined color space: " + << format->colorSpace.toString(); + return 0; } @@ -960,12 +1001,24 @@ int V4L2VideoDevice::trySetFormatSingleplane(V4L2DeviceFormat *format, bool set) struct v4l2_pix_format *pix = &v4l2Format.fmt.pix; int ret; + if (!format->colorSpace.isFullyDefined()) + LOG(V4L2, Error) + << "Trying to set undefined color space: " + << format->colorSpace.toString(); + v4l2Format.type = bufferType_; pix->width = format->size.width; pix->height = format->size.height; pix->pixelformat = format->fourcc; pix->bytesperline = format->planes[0].bpl; pix->field = V4L2_FIELD_NONE; + + ret = fromColorSpace(format->colorSpace, *pix); + if (ret < 0) + LOG(V4L2, Warning) + << "Set color space unrecognised by V4L2: " + << format->colorSpace.toString(); + ret = ioctl(set ? VIDIOC_S_FMT : VIDIOC_TRY_FMT, &v4l2Format); if (ret) { LOG(V4L2, Error) @@ -985,6 +1038,12 @@ int V4L2VideoDevice::trySetFormatSingleplane(V4L2DeviceFormat *format, bool set) format->planes[0].bpl = pix->bytesperline; format->planes[0].size = pix->sizeimage; + format->colorSpace = toColorSpace(*pix); + if (!format->colorSpace.isFullyDefined()) + LOG(V4L2, Warning) + << "Undefined color space has been set: " + << format->colorSpace.toString(); + return 0; } From patchwork Mon Nov 15 16:12:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 14611 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B8372BF415 for ; Mon, 15 Nov 2021 16:13:09 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 18E0D60394; Mon, 15 Nov 2021 17:13:09 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="SqRfhdN/"; dkim-atps=neutral Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com [IPv6:2a00:1450:4864:20::32d]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 849196037A for ; Mon, 15 Nov 2021 17:13:01 +0100 (CET) Received: by mail-wm1-x32d.google.com with SMTP id c71-20020a1c9a4a000000b0032cdcc8cbafso211789wme.3 for ; Mon, 15 Nov 2021 08:13:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HsQlhHnuhc87TaSXhA5f4OVRdGAu3zIVZXBgg53nXBM=; b=SqRfhdN/Vzo+apIhpnwUtHMCHyfqYHmrVfIVWXXhTqhlyvW9RclIapiTZ2E99AEaDk bjHnSGNAqJumylCGDOABx4pPEXUJeUYWK3xbY17lzmm1JJJbN1t+ag26+M7LsdRz+1cq a+aMPqrypaWtq5LxGACIec8AXzioE0iEh2+hczAAKnj2zF2GTVGdEBThyGdnwqYfKJYM HnRWeUleFf+LmHTH2krRLQTPy/NwuNvsGW5FcB5wbLohZFsb2nwxvNyapeNZ0iRaJVlU lrdBlqKsTMhr4IvyLN9o2ghzjbAld0FmoU/7ezemhikRa9XYLojK41Z/w57PrNFIJit9 IsGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HsQlhHnuhc87TaSXhA5f4OVRdGAu3zIVZXBgg53nXBM=; b=3usD1QUlFOlJ6QtZufztEvCkCSHyZgjmczyn3/iOcKyMR0ln00bHxq3vtdDIWWLZoK OPfxcpZPQsUthT1xlbfuGYt4i+vzGiHagJVwqG9VBx64vjpIKSZrc4NRri/aod0tlNJk hHICHAwQR0rOh4H7qJzVsMQoa+9DXDMGC1dWPuzepo8CfzieY11J0JSaMkdq8jIPjBFj t0tjPnVaS4wbrNkJ2a79Sf0EMd3N+A7dUmmFnhnX+1Zhlq/OYHleTMeSHNoZFPVtkDYV J6hGGlwW36d777htbe2nykxZJZpabuwY6h9jLXkd0t84KUhD1CAPBnTse8KGNXU+8FC2 bYtg== X-Gm-Message-State: AOAM530vLxoFNrvtcHvtH3i9ze83LOwgx6yRFhr8u5YFbT+qVqbw1Ee8 LCAESEtPTrOotXfDKYPHLI9xxA== X-Google-Smtp-Source: ABdhPJy6OM2KCpf4NIE0S3d8AvryM1KFBAWM68cQCd9HiM4gKMY9jPZR0IXDK3UfFcGh7XRgbq4KgQ== X-Received: by 2002:a05:600c:2118:: with SMTP id u24mr60624693wml.0.1636992781268; Mon, 15 Nov 2021 08:13:01 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id j40sm16315615wms.16.2021.11.15.08.13.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Nov 2021 08:13:00 -0800 (PST) From: David Plowman To: hverkuil-cisco@xs4all.nl, laurent.pinchart@ideasonboard.com, kieran.bingham@ideasonboard.com, jacopo@jmondi.org, tfiga@google.com, naush@raspberrypi.com, libcamera-devel@lists.libcamera.org Date: Mon, 15 Nov 2021 16:12:03 +0000 Message-Id: <20211115161205.24335-6-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211115161205.24335-1-david.plowman@raspberrypi.com> References: <20211115161205.24335-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 5/7] libcamera: Support passing ColorSpaces to V4L2 subdevices X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The ColorSpace from the StreamConfiguration is now handled appropriately in the V4L2Subdevice. Signed-off-by: David Plowman --- include/libcamera/internal/v4l2_subdevice.h | 2 + src/libcamera/camera_sensor.cpp | 1 + src/libcamera/v4l2_subdevice.cpp | 42 ++++++++++++++++++++- 3 files changed, 44 insertions(+), 1 deletion(-) diff --git a/include/libcamera/internal/v4l2_subdevice.h b/include/libcamera/internal/v4l2_subdevice.h index 97b89fb9..f3ab8454 100644 --- a/include/libcamera/internal/v4l2_subdevice.h +++ b/include/libcamera/internal/v4l2_subdevice.h @@ -14,6 +14,7 @@ #include #include +#include #include #include "libcamera/internal/formats.h" @@ -27,6 +28,7 @@ class MediaDevice; struct V4L2SubdeviceFormat { uint32_t mbus_code; Size size; + ColorSpace colorSpace; const std::string toString() const; uint8_t bitsPerPixel() const; diff --git a/src/libcamera/camera_sensor.cpp b/src/libcamera/camera_sensor.cpp index 9fdb8c09..6fcd1c1d 100644 --- a/src/libcamera/camera_sensor.cpp +++ b/src/libcamera/camera_sensor.cpp @@ -613,6 +613,7 @@ V4L2SubdeviceFormat CameraSensor::getFormat(const std::vector &mbu V4L2SubdeviceFormat format{ .mbus_code = bestCode, .size = *bestSize, + .colorSpace = ColorSpace::Raw, }; return format; diff --git a/src/libcamera/v4l2_subdevice.cpp b/src/libcamera/v4l2_subdevice.cpp index 023e2328..b348170f 100644 --- a/src/libcamera/v4l2_subdevice.cpp +++ b/src/libcamera/v4l2_subdevice.cpp @@ -168,6 +168,18 @@ const std::map formatInfoMap = { * \brief The image size in pixels */ +/** + * \var V4L2SubdeviceFormat::colorSpace + * \brief The color space of the pixels + * + * When setting or trying a format, passing in "Undefined" fields in the + * ColorSpace is not recommended because the driver will then make an + * arbitrary choice of its own. Choices made by the driver will be + * passed back in the normal way, though note that "Undefined" values can + * be returned if the device has chosen something that the ColorSpace + * cannot represent. + */ + /** * \brief Assemble and return a string describing the format * \return A string describing the V4L2SubdeviceFormat @@ -400,6 +412,17 @@ int V4L2Subdevice::getFormat(unsigned int pad, V4L2SubdeviceFormat *format, format->size.height = subdevFmt.format.height; format->mbus_code = subdevFmt.format.code; + format->colorSpace = toColorSpace(subdevFmt.format); + /* + * This warning can be ignored on metadata pads. These are normally + * pads other than zero. + * \todo find a way to detect metadata pads and ignore them + */ + if (!format->colorSpace.isFullyDefined()) + LOG(V4L2, Warning) + << "Retrieved undefined color space on pad " << pad + << ": " << format->colorSpace.toString(); + return 0; } @@ -418,6 +441,11 @@ int V4L2Subdevice::getFormat(unsigned int pad, V4L2SubdeviceFormat *format, int V4L2Subdevice::setFormat(unsigned int pad, V4L2SubdeviceFormat *format, Whence whence) { + if (!format->colorSpace.isFullyDefined()) + LOG(V4L2, Error) + << "Trying to set undefined color space: " + << format->colorSpace.toString(); + struct v4l2_subdev_format subdevFmt = {}; subdevFmt.which = whence == ActiveFormat ? V4L2_SUBDEV_FORMAT_ACTIVE : V4L2_SUBDEV_FORMAT_TRY; @@ -427,7 +455,13 @@ int V4L2Subdevice::setFormat(unsigned int pad, V4L2SubdeviceFormat *format, subdevFmt.format.code = format->mbus_code; subdevFmt.format.field = V4L2_FIELD_NONE; - int ret = ioctl(VIDIOC_SUBDEV_S_FMT, &subdevFmt); + int ret = fromColorSpace(format->colorSpace, subdevFmt.format); + if (ret < 0) + LOG(V4L2, Warning) + << "Setting color space unrecognised by V4L2: " + << format->colorSpace.toString(); + + ret = ioctl(VIDIOC_SUBDEV_S_FMT, &subdevFmt); if (ret) { LOG(V4L2, Error) << "Unable to set format on pad " << pad @@ -439,6 +473,12 @@ int V4L2Subdevice::setFormat(unsigned int pad, V4L2SubdeviceFormat *format, format->size.height = subdevFmt.format.height; format->mbus_code = subdevFmt.format.code; + format->colorSpace = toColorSpace(subdevFmt.format); + if (!format->colorSpace.isFullyDefined()) + LOG(V4L2, Warning) + << "Undefined color space has been set: " + << format->colorSpace.toString(); + return 0; } From patchwork Mon Nov 15 16:12:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 14612 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9641EC324E for ; Mon, 15 Nov 2021 16:13:10 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2F075603C4; Mon, 15 Nov 2021 17:13:10 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="lA6ZNaof"; dkim-atps=neutral Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com [IPv6:2a00:1450:4864:20::332]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 361356036C for ; Mon, 15 Nov 2021 17:13:02 +0100 (CET) Received: by mail-wm1-x332.google.com with SMTP id i8-20020a7bc948000000b0030db7b70b6bso16261036wml.1 for ; Mon, 15 Nov 2021 08:13:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=px+J59JTE8VA+7mmTv2pDwc8QWJMkP/Tl3k7zA7RZOc=; b=lA6ZNaofLCd2+JjkqogeuZmcmMVGLhUM0+6CRZ/fT5UjGckn0jQdjkOGAEWqRKXR/E IXPN/VF0pA9gBQ93pdcytpcDFrCNjlsiIIHZt/vf16z+aY/Ba32DsQptFaM/fOU6jMwo vOoaK2QAdwUgSHU9fqAR/PIa4Pm1pLTcEcapGVJTl9svYQK8chEM2QjU02VzkehD+BhD Cj0DHn3EhadopDhW3zbj4QhsIYqBpNzRWOPDlKHulDxjbGBXBkivDTIImUmdsQE3rX0w UGl+iJ/9oeet8igx9xqgTe7+CBcPhdYbnxNcPhFbUhV2Xv7pndDCKU63FwPzH+xPV2Ey RwcA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=px+J59JTE8VA+7mmTv2pDwc8QWJMkP/Tl3k7zA7RZOc=; b=yfSNita43E3M4trlgtKhPShpTToqLhWruydQr+2BuHqT6UvzUL5U5Dcd3O3srLZCvq NAkKMi39A8ZWRdgB94el2/HWsG7SAZH8NVlCs9IW/dk/EOcAHAr0nV7AOA7Gx1bRwXZ+ eHQ267KNljGmt5VgvlVdNd/tPmFLTp1HpEJwTMbrj5cH+0H6SuK3sgshrPqR13ZebB6j tkeL9YvV8/gRS7W9xgdWJZfdWa4KhWhM8NvGdwVOWA0zf/8CPgnqsC1yowQhOeQbYZXB x4IvAh33OV0Co/UGbZ7ba9D5YicXFkAU+bkSoCi+xW9uhl86/utGUGh9O6KoobX5jjyP IEPg== X-Gm-Message-State: AOAM5327/dUILYCspqVS7kEA+VzJjGFVwEAlYGJrb8S6b/yiCxFkijZJ oEhdv+cJW27BV2flIsOQxC31KA== X-Google-Smtp-Source: ABdhPJyptZyIeKPnPgLThjbBMaTNmFu5bpz/Rvnt+zx3TsroBHz4L0QZtNMkI5iITapygenVuwP19g== X-Received: by 2002:a1c:7ed3:: with SMTP id z202mr42913048wmc.110.1636992781945; Mon, 15 Nov 2021 08:13:01 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id j40sm16315615wms.16.2021.11.15.08.13.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Nov 2021 08:13:01 -0800 (PST) From: David Plowman To: hverkuil-cisco@xs4all.nl, laurent.pinchart@ideasonboard.com, kieran.bingham@ideasonboard.com, jacopo@jmondi.org, tfiga@google.com, naush@raspberrypi.com, libcamera-devel@lists.libcamera.org Date: Mon, 15 Nov 2021 16:12:04 +0000 Message-Id: <20211115161205.24335-7-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211115161205.24335-1-david.plowman@raspberrypi.com> References: <20211115161205.24335-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 6/7] libcamera: Add validateColorSpaces to CameraConfiguration class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This method checks that the requested color spaces are sensible and do not contain any undefined enum values. It also initialises the "actual" color space field to the same value as the one requested, in the expectation that the rest of the validate() method will be able to check this. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- include/libcamera/camera.h | 2 ++ src/libcamera/camera.cpp | 51 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 53 insertions(+) diff --git a/include/libcamera/camera.h b/include/libcamera/camera.h index 601ee46e..fdab4410 100644 --- a/include/libcamera/camera.h +++ b/include/libcamera/camera.h @@ -69,6 +69,8 @@ public: protected: CameraConfiguration(); + Status validateColorSpaces(bool sharedColorSpace); + std::vector config_; }; diff --git a/src/libcamera/camera.cpp b/src/libcamera/camera.cpp index 400a7cf0..90e9460b 100644 --- a/src/libcamera/camera.cpp +++ b/src/libcamera/camera.cpp @@ -20,6 +20,7 @@ #include "libcamera/internal/camera.h" #include "libcamera/internal/camera_controls.h" +#include "libcamera/internal/formats.h" #include "libcamera/internal/pipeline_handler.h" /** @@ -314,6 +315,56 @@ std::size_t CameraConfiguration::size() const return config_.size(); } +static bool isRaw(const PixelFormat &pixFmt) +{ + const PixelFormatInfo &info = PixelFormatInfo::info(pixFmt); + return info.isValid() && + info.colourEncoding == PixelFormatInfo::ColourEncodingRAW; +} + +CameraConfiguration::Status CameraConfiguration::validateColorSpaces(bool sharedColorSpace) +{ + Status status = Valid; + + /* Find the largest non-raw stream (if any). */ + int index = -1; + for (unsigned int i = 0; i < config_.size(); i++) { + const StreamConfiguration &cfg = config_[i]; + if (!isRaw(cfg.pixelFormat) && (index < 0 || cfg.size > config_[i].size)) + index = i; + } + + /* + * Here we force raw streams to the correct color space and signal + * an error if we encounter anything undefined. We handle the case + * where all output streams are to share a color space, which we + * choose to be the color space of the largest stream. + */ + for (auto &cfg : config_) { + ColorSpace initialColorSpace = cfg.requestedColorSpace; + + if (isRaw(cfg.pixelFormat)) + cfg.requestedColorSpace = ColorSpace::Raw; + else if (!cfg.requestedColorSpace.isFullyDefined()) { + LOG(Camera, Error) << "Stream has undefined color space"; + cfg.requestedColorSpace = ColorSpace::Jpeg; + } else if (sharedColorSpace) + cfg.requestedColorSpace = config_[index].requestedColorSpace; + + if (cfg.requestedColorSpace != initialColorSpace) + status = Adjusted; + + /* + * We also initialise the actual color space as if the + * hardware can do what we want. But note that the rest + * of the validate() method may change this. + */ + cfg.actualColorSpace = cfg.requestedColorSpace; + } + + return status; +} + /** * \var CameraConfiguration::transform * \brief User-specified transform to be applied to the image From patchwork Mon Nov 15 16:12:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 14613 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7267BBF415 for ; Mon, 15 Nov 2021 16:13:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0F50C603EE; Mon, 15 Nov 2021 17:13:11 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="pkafFUku"; dkim-atps=neutral Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DE06D603BE for ; Mon, 15 Nov 2021 17:13:02 +0100 (CET) Received: by mail-wm1-x32c.google.com with SMTP id o29so14465858wms.2 for ; Mon, 15 Nov 2021 08:13:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=MWi8mj5S4W6cGhtr7B2BNpqXeeXBc2vqiVreCzJ+MGg=; b=pkafFUkuNsZmry6XV2Z3L1635rN0NUm46EEalWr8Y4ft1buSpHbllgzIv/HpD3O8YL c519zKZaSlJHMuMIkoFw3OOO3Ky+wRpY8gJUUq1DxjNvVq2LJV/N48jiTmqy8pU8wv9I TzTpIf6iZoiWDVvGIFq7YuxTBS3G1TL5dlDi44WFtENXDsQEGvwzIrnaZoAThTSIm61X KBmuBJwgcl0rcnoKZUZ+/u67XOmDQ3ibUsj+vw+0A0s+C4QPtS17nDP4MIFmEodWfNCw pJ1uIhdPYCQCIKsE/jl/63Okv9KxrwCHVXZasR3rdwsZuHAwmJaNJVU06enapOgrh8l1 AEbQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=MWi8mj5S4W6cGhtr7B2BNpqXeeXBc2vqiVreCzJ+MGg=; b=5suObWhDYUlYq2/Es69cVpnsgeVCd63mZP3JJyxGNggwc+oMHNx9+pPU834lZrArns 9wLyEq8MJ9554rG3tduk22ZxI83XhCUC15RKAUq50fnKMMKxJk8cOGV3Q1xfo7BpRzQg GS1X6Xec8gWeKnLYzjQlc+/dEfd/AJuQEnAB5iMrCzRdkoGmzE057zjXbeX6RBVjHvSk +aKVsPBYGr3PK/cPRPAoceMeNbzyETwJan8IcUfjOfBWof33eHTHuQ7Oyt7BKUjEMi7Z 7QNp1B0MhDnPshvFOZOjSJqoDTunARG2uketpSKYRXharw+9T4Az06eXv7UCR9eKLNyf B1oQ== X-Gm-Message-State: AOAM530zFq3hlZ4AzMge34q1mwVCD9rjfebOAXrkxw7KIfBYWmB1JRnN yZ20d/VU22pw2dCRQYMWbZJZhQuNuoQZEilg X-Google-Smtp-Source: ABdhPJwOwf5AXDcB5BwQalHOCOiBNJBq3ftV4r+n3CsV9NpA6zPVut7C2g5Cf66XJDvKZZro+4QMwA== X-Received: by 2002:a1c:cc09:: with SMTP id h9mr45736360wmb.191.1636992782633; Mon, 15 Nov 2021 08:13:02 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id j40sm16315615wms.16.2021.11.15.08.13.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Nov 2021 08:13:02 -0800 (PST) From: David Plowman To: hverkuil-cisco@xs4all.nl, laurent.pinchart@ideasonboard.com, kieran.bingham@ideasonboard.com, jacopo@jmondi.org, tfiga@google.com, naush@raspberrypi.com, libcamera-devel@lists.libcamera.org Date: Mon, 15 Nov 2021 16:12:05 +0000 Message-Id: <20211115161205.24335-8-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211115161205.24335-1-david.plowman@raspberrypi.com> References: <20211115161205.24335-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 7/7] libcamera: pipeline: raspberrypi: Support color spaces X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The Raspberry Pi pipeline handler now sets color spaces correctly. In generateConfiguration() it sets them to reasonable default values based on the stream role. validate() now calls validateColorSpaces() to ensure that the requested color spaces are sensible, before proceeding to check what the hardware can deliver. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- .../pipeline/raspberrypi/raspberrypi.cpp | 42 +++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 5e1f2273..bbb21e9b 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -93,6 +93,7 @@ V4L2DeviceFormat toV4L2DeviceFormat(const V4L2SubdeviceFormat &format, deviceFormat.fourcc = V4L2PixelFormat::fromPixelFormat(pix); deviceFormat.size = format.size; + deviceFormat.colorSpace = format.colorSpace; return deviceFormat; } @@ -126,6 +127,7 @@ V4L2SubdeviceFormat findBestFormat(const SensorFormats &formatsMap, const Size & { double bestScore = std::numeric_limits::max(), score; V4L2SubdeviceFormat bestFormat; + bestFormat.colorSpace = ColorSpace::Raw; #define PENALTY_AR 1500.0 #define PENALTY_8BIT 2000.0 @@ -332,6 +334,8 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() if (config_.empty()) return Invalid; + status = validateColorSpaces(true); + /* * What if the platform has a non-90 degree rotation? We can't even * "adjust" the configuration and carry on. Alternatively, raising an @@ -494,10 +498,21 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() V4L2DeviceFormat format; format.fourcc = V4L2PixelFormat::fromPixelFormat(cfg.pixelFormat); format.size = cfg.size; + format.colorSpace = cfg.requestedColorSpace; + LOG(RPI, Debug) + << "Try color space " << cfg.requestedColorSpace.toString(); int ret = dev->tryFormat(&format); if (ret) return Invalid; + cfg.actualColorSpace = format.colorSpace; + if (cfg.actualColorSpace != cfg.requestedColorSpace) { + status = Adjusted; + LOG(RPI, Warning) + << "Color space changed from " + << cfg.requestedColorSpace.toString() << " to " + << cfg.actualColorSpace.toString(); + } cfg.stride = format.planes[0].bpl; cfg.frameSize = format.planes[0].size; @@ -522,6 +537,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, PixelFormat pixelFormat; V4L2VideoDevice::Formats fmts; Size size; + ColorSpace colorSpace; if (roles.empty()) return config; @@ -536,6 +552,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, pixelFormat = mbusCodeToPixelFormat(sensorFormat.mbus_code, BayerFormat::Packing::CSI2); ASSERT(pixelFormat.isValid()); + colorSpace = ColorSpace::Raw; bufferCount = 2; rawCount++; break; @@ -543,6 +560,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, case StreamRole::StillCapture: fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::NV12; + colorSpace = ColorSpace::Jpeg; /* Return the largest sensor resolution. */ size = data->sensor_->resolution(); bufferCount = 1; @@ -560,6 +578,8 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, */ fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::YUV420; + /* This will be reasonable for many applications. */ + colorSpace = ColorSpace::Rec709; size = { 1920, 1080 }; bufferCount = 4; outCount++; @@ -568,6 +588,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, case StreamRole::Viewfinder: fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::ARGB8888; + colorSpace = ColorSpace::Jpeg; size = { 800, 600 }; bufferCount = 4; outCount++; @@ -599,6 +620,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, StreamConfiguration cfg(formats); cfg.size = size; cfg.pixelFormat = pixelFormat; + cfg.requestedColorSpace = colorSpace; cfg.bufferCount = bufferCount; config->addConfiguration(cfg); } @@ -703,6 +725,7 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) V4L2PixelFormat fourcc = V4L2PixelFormat::fromPixelFormat(cfg.pixelFormat); format.size = cfg.size; format.fourcc = fourcc; + format.colorSpace = cfg.requestedColorSpace; LOG(RPI, Debug) << "Setting " << stream->name() << " to " << format.toString(); @@ -718,6 +741,23 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) return -EINVAL; } + if (cfg.actualColorSpace != format.colorSpace) { + /* + * We should have been through validate() before so this + * shouldn't be possible, but we mustn't sweep color space + * problems under the carpet. + */ + LOG(RPI, Warning) + << "Unexpected color space change (" + << cfg.actualColorSpace.toString() << " to " + << format.colorSpace.toString() << ") in stream " + << stream->name(); + cfg.actualColorSpace = format.colorSpace; + } + LOG(RPI, Debug) + << "Stream " << stream->name() << " has color space " + << cfg.actualColorSpace.toString(); + cfg.setStream(stream); stream->setExternal(true); @@ -742,6 +782,8 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) format = {}; format.size = maxSize; format.fourcc = V4L2PixelFormat::fromPixelFormat(formats::YUV420); + /* No one asked for output, so the color space doesn't matter. */ + format.colorSpace = ColorSpace::Jpeg; ret = data->isp_[Isp::Output0].dev()->setFormat(&format); if (ret) { LOG(RPI, Error)