From patchwork Mon Sep 27 12:33:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 13951 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 32AACBDC71 for ; Mon, 27 Sep 2021 12:33:36 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id EA85769197; Mon, 27 Sep 2021 14:33:35 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="fKGmSx2w"; dkim-atps=neutral Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E2B0E6918A for ; Mon, 27 Sep 2021 14:33:32 +0200 (CEST) Received: by mail-wr1-x42e.google.com with SMTP id v17so8250924wrv.9 for ; Mon, 27 Sep 2021 05:33:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=tmZBQE4orKD4XTYhvhaHMaOkQ+/79Rt9g3dMNGENyY0=; b=fKGmSx2wTs5HnfNGvUGfksD7LUrjfqFF7n13SX4FHIZP7N+uwQ39Kfr4MlUXPnA0cS 4//eUUgv4hHteuioVa+fjkkVbYbAfdhRm250kOnT/WTpTnB39V6ZuNlwFkVQh9+KvcBm od1guBgtRphBt6Gny/pZjZrUnvuEUyqFSAmA/RyihbDt4uzyYamD9yf3zE16e/ciB6ya eYdlWfGEIX8J/f1j7gSa2QwVUC7dfaLyCERUlL9b98JwHAFyAOeBmnbsiVktjU5tYI2t 6ECWMi+TddG3z4J627avV0tiLuWQGjBlEfAQprWsy+KwMOpf7aZ1P4wx+F6Ote7YkUHO v3fw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tmZBQE4orKD4XTYhvhaHMaOkQ+/79Rt9g3dMNGENyY0=; b=F3WJVjNT9M5wB0aQky6ntqiFtesBz6zetQhuqG3xBrjWhnj4ZOpbeq3S5eoa9hR7XA NaNMsNNeku4PIZoN6omRhc/Q0lJz3pv8asmCuMok7IyGkCj8subO4gCvxqbRJjDm5SGc u2fDQ/dM7hQNHkl/Lehsss3dm484GmxtqQ3dUztfx4ONtApfiQhG5pM40ZBt5fYxuHlS roysmy3FxWWsE/zYrgQXyNPvS3syHyf6PpyFZtbxZlcVUOLrBPHEHmCxEviyEqi5dmV1 eTH3M1aNh1IR7kUblGm8bSd4DiFqV56QaoM5r5903BE5aiWr/2/XE4UmfkHReE/z1wXw Nuzw== X-Gm-Message-State: AOAM531xGXflvtNvOH+uZCiyGm/jTKCSVlF3ZkPMEnkVi08xB0I6eltu IFYpdrGfp5V8tfo60X9Zz7G39Fgm3R/7HkXM X-Google-Smtp-Source: ABdhPJymWdPSaGlm+hqlWL30GVRgnDqYwovZlEBSjFU6/UcDrmbDbvt7wkknGXsFz7fG8gwtZbqcuQ== X-Received: by 2002:a05:6000:124f:: with SMTP id j15mr4444113wrx.366.1632746012205; Mon, 27 Sep 2021 05:33:32 -0700 (PDT) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id r9sm16285110wru.2.2021.09.27.05.33.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Sep 2021 05:33:31 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Mon, 27 Sep 2021 13:33:25 +0100 Message-Id: <20210927123327.14554-2-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210927123327.14554-1-david.plowman@raspberrypi.com> References: <20210927123327.14554-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 1/3] libcamera: Add ColorSpace class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This class represents a colour space by defining its YCbCr encoding, the transfer (gamma) function is uses, and whether the output is full or limited range. Signed-off-by: David Plowman --- include/libcamera/color_space.h | 83 +++++++++++++ include/libcamera/meson.build | 1 + src/libcamera/color_space.cpp | 207 ++++++++++++++++++++++++++++++++ src/libcamera/meson.build | 1 + 4 files changed, 292 insertions(+) create mode 100644 include/libcamera/color_space.h create mode 100644 src/libcamera/color_space.cpp diff --git a/include/libcamera/color_space.h b/include/libcamera/color_space.h new file mode 100644 index 00000000..9cd10503 --- /dev/null +++ b/include/libcamera/color_space.h @@ -0,0 +1,83 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Raspberry Pi (Trading) Limited + * + * color_space.h - color space definitions + */ + +#ifndef __LIBCAMERA_COLOR_SPACE_H__ +#define __LIBCAMERA_COLOR_SPACE_H__ + +#include + +namespace libcamera { + +class ColorSpace +{ +public: + enum class Encoding : int { + UNDEFINED, + RAW, + REC601, + REC709, + REC2020, + VIDEO, + }; + + enum class TransferFunction : int { + UNDEFINED, + IDENTITY, + SRGB, + REC709, + }; + + enum class Range : int { + UNDEFINED, + FULL, + LIMITED, + }; + + constexpr ColorSpace(Encoding e, TransferFunction t, Range r) + : encoding(e), transferFunction(t), range(r) + { + } + + constexpr ColorSpace() + : ColorSpace(Encoding::UNDEFINED, TransferFunction::UNDEFINED, Range::UNDEFINED) + { + } + + static const ColorSpace UNDEFINED; + static const ColorSpace RAW; + static const ColorSpace JFIF; + static const ColorSpace SMPTE170M; + static const ColorSpace REC709; + static const ColorSpace REC2020; + static const ColorSpace VIDEO; + + Encoding encoding; + TransferFunction transferFunction; + Range range; + + bool isFullyDefined() const; + + const std::string toString() const; +}; + +constexpr ColorSpace ColorSpace::UNDEFINED = { Encoding::UNDEFINED, TransferFunction::UNDEFINED, Range::UNDEFINED }; +constexpr ColorSpace ColorSpace::RAW = { Encoding::RAW, TransferFunction::IDENTITY, Range::FULL }; +constexpr ColorSpace ColorSpace::JFIF = { Encoding::REC601, TransferFunction::SRGB, Range::FULL }; +constexpr ColorSpace ColorSpace::SMPTE170M = { Encoding::REC601, TransferFunction::REC709, Range::LIMITED }; +constexpr ColorSpace ColorSpace::REC709 = { Encoding::REC709, TransferFunction::REC709, Range::LIMITED }; +constexpr ColorSpace ColorSpace::REC2020 = { Encoding::REC2020, TransferFunction::REC709, Range::LIMITED }; +constexpr ColorSpace ColorSpace::VIDEO = { Encoding::VIDEO, TransferFunction::REC709, Range::LIMITED }; + +bool operator==(const ColorSpace &lhs, const ColorSpace &rhs); +static inline bool operator!=(const ColorSpace &lhs, const ColorSpace &rhs) +{ + return !(lhs == rhs); +} + +} /* namespace libcamera */ + +#endif /* __LIBCAMERA_COLOR_SPACE_H__ */ diff --git a/include/libcamera/meson.build b/include/libcamera/meson.build index 5b25ef84..7a8a04e5 100644 --- a/include/libcamera/meson.build +++ b/include/libcamera/meson.build @@ -3,6 +3,7 @@ libcamera_public_headers = files([ 'camera.h', 'camera_manager.h', + 'color_space.h', 'compiler.h', 'controls.h', 'file_descriptor.h', diff --git a/src/libcamera/color_space.cpp b/src/libcamera/color_space.cpp new file mode 100644 index 00000000..d1ccb4cc --- /dev/null +++ b/src/libcamera/color_space.cpp @@ -0,0 +1,207 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Raspberry Pi (Trading) Limited + * + * color_space.cpp - color spaces. + */ + +#include + +#include + +/** + * \file color_space.h + * \brief Class and enums to represent colour spaces. + */ + +namespace libcamera { + +/** + * \class ColorSpace + * \brief Class to describe a color space. + * + * The color space class defines the encodings of the color primaries, the + * transfer function associated with the color space, and the range (sometimes + * also referred to as the quantisation) of the color space. + * + * Certain combinations of these fields form well-known standard color spaces, + * such as "JFIF" or "REC709", though there is flexibility to leave some or all + * of them undefined too. + */ + +/** + * \enum ColorSpace::Encoding + * \brief The encoding used for the color primaries. + * + * \var ColorSpace::Encoding::UNDEFINED + * \brief The encoding for the colour primaries is not specified. + * \var ColorSpace::Encoding::RAW + * \brief These are raw colours from the sensor. + * \var ColorSpace::Encoding::REC601 + * \brief REC601 colour primaries. + * \var ColorSpace::Encoding::REC709 + * \brief Rec709 colour primaries. + * \var ColorSpace::Encoding::REC2020 + * \brief REC2020 colour primaries. + * \var ColorSpace::Encoding::VIDEO + * \brief A place-holder for video streams which will be resolved to one + * of REC601, REC709 or REC2020 once the video resolution is known. + */ + +/** + * \enum ColorSpace::TransferFunction + * \brief The transfer function used for this colour space. + * + * \var ColorSpace::TransferFunction::UNDEFINED + * \brief The transfer function is not specified. + * \var ColorSpace::TransferFunction::IDENTITY + * \brief This color space uses an identity transfer function. + * \var ColorSpace::TransferFunction::SRGB + * \brief sRGB transfer function. + * \var ColorSpace::TransferFunction::REC709 + * \brief Rec709 transfer function. + */ + +/** + * \enum ColorSpace::Range + * \brief The range (sometimes "quantisation") for this color space. + * + * \var ColorSpace::Range::UNDEFINED + * \brief The range is not specified. + * \var ColorSpace::Range::FULL + * \brief This color space uses full range pixel values. + * \var ColorSpace::Range::LIMITED + * \brief This color space uses limited range pixel values. + */ + +/** + * \fn ColorSpace::ColorSpace(Encoding e, TransferFunction t, Range r) + * \brief Construct a ColorSpace from explicit values + * \param[in] e The encoding for the color primaries + * \param[in] t The transfer function for the color space + * \param[in] r The range of the pixel values in this color space + */ + +/** + * \fn ColorSpace::ColorSpace() + * \brief Construct a color space with undefined encoding, transfer function + * and range + */ + +/** + * \brief Return true if all the fields of the color space are defined, otherwise false. + */ +bool ColorSpace::isFullyDefined() const +{ + return encoding != Encoding::UNDEFINED && + transferFunction != TransferFunction::UNDEFINED && + range != Range::UNDEFINED; +} + +/** + * \brief Assemble and return a readable string representation of the + * ColorSpace + * \return A string describing the ColorSpace + */ +const std::string ColorSpace::toString() const +{ + static const char *encodings[] = { + "UNDEFINED", + "RAW", + "REC601", + "REC709", + "REC2020", + }; + static const char *transferFunctions[] = { + "UNDEFINED", + "IDENTITY", + "SRGB", + "REC709", + }; + static const char *ranges[] = { + "UNDEFINED", + "FULL", + "LIMITED", + }; + + std::stringstream ss; + ss << std::string(encodings[static_cast(encoding)]) << "+" + << std::string(transferFunctions[static_cast(transferFunction)]) << "+" + << std::string(ranges[static_cast(range)]); + + return ss.str(); +} + +/** + * \var ColorSpace::encoding + * \brief The encoding of the color primaries + */ + +/** + * \var ColorSpace::transferFunction + * \brief The transfer function for this color space. + */ + +/** + * \var ColorSpace::range + * \brief The pixel range used by this color space. + */ + +/** + * \var ColorSpace::UNDEFINED + * \brief A constant representing a fully undefined color space. + */ + +/** + * \var ColorSpace::RAW + * \brief A constant representing a raw color space (from a sensor). + */ + +/** + * \var ColorSpace::JFIF + * \brief A constant representing the JFIF color space usually used for + * encoding JPEG images. + */ + +/** + * \var ColorSpace::SMPTE170M + * \brief A constant representing the SMPTE170M color space (sometimes also + * referred to as "full range BT601"). + */ + +/** + * \var ColorSpace::REC709 + * \brief A constant representing the REC709 color space. + */ + +/** + * \var ColorSpace::REC2020 + * \brief A constant representing the REC2020 color space. + */ + +/** + * \var ColorSpace::VIDEO + * \brief A constant that video streams can use to indicate the "default" + * color space for a video of this resolution, once that is is known. For + * exmample, SD streams would interpret this as SMPTE170M, HD streams as + * REC709 and ultra HD as REC2020. + */ + +/** + * \brief Compare color spaces for equality + * \return True if the two color spaces are identical, false otherwise + */ +bool operator==(const ColorSpace &lhs, const ColorSpace &rhs) +{ + return lhs.encoding == rhs.encoding && + lhs.transferFunction == rhs.transferFunction && + lhs.range == rhs.range; +} + +/** + * \fn bool operator!=(const ColorSpace &lhs, const ColorSpace &rhs) + * \brief Compare color spaces for inequality + * \return True if the two color spaces are not identical, false otherwise + */ + +} /* namespace libcamera */ diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build index bf82d38b..88dfbf24 100644 --- a/src/libcamera/meson.build +++ b/src/libcamera/meson.build @@ -8,6 +8,7 @@ libcamera_sources = files([ 'camera_manager.cpp', 'camera_sensor.cpp', 'camera_sensor_properties.cpp', + 'color_space.cpp', 'controls.cpp', 'control_serializer.cpp', 'control_validator.cpp', From patchwork Mon Sep 27 12:33:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 13952 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0DA6BBDC71 for ; Mon, 27 Sep 2021 12:33:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 49F416919A; Mon, 27 Sep 2021 14:33:36 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="B562Qh2v"; dkim-atps=neutral Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3119769189 for ; Mon, 27 Sep 2021 14:33:33 +0200 (CEST) Received: by mail-wr1-x42d.google.com with SMTP id i23so51759090wrb.2 for ; Mon, 27 Sep 2021 05:33:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=G0rebdG51e0aDi44uhZ3M4+B0tGBPEoMAQ34J//YoUY=; b=B562Qh2vnmIBz49G6sIaDqMLGWIFenrGfTJecWtuCYiO1M5m1AZ9ng4YTesLz1EdOK /hH0qXFpdSqOaaqI7gF3QWJy5DjrZLpvXsjPdaXxx19BqsWvOPyJXkET5wVEsIVYGoaM 0OYr+LaSV761bjKNRfyUZjGBCTzc1r3cvcJFTSfVfvuIgtqH4eve3EQfSmBbCj75p+03 pDyroIoHzxh/jNgPtjsm+Lt8xZrT3MiJx6I0R50Jf9HweLxz+RTfpDGlDLdsB4k7Lxug HE81HlCmqAkG0hPgFFo0cfZ1z2UnOFftlD1JnoHeRxDyAUm0oaf/Pfc2y87qS1tsR+Vh 1PgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=G0rebdG51e0aDi44uhZ3M4+B0tGBPEoMAQ34J//YoUY=; b=hDGBxwMcuWoDk22k9YlnphxBLU7r51/TW4AfYGq4EViEIr+kkm5nT0Urq/Ehs+hAJq Rss6PB0XQWqExmXzlolRssGo3JAxKWYkJxk2ffAfog8BCO+WEcNWo+rYPZ/u5vaVoexd 59Ds72UyoN4UFs/Bs3+zfFTbPPZp2BiA971AFeWvDLPV2nblR70duPYNHT2IGpDrNmgP ll4RKYL4+P0V7ey+kqJ16krXTKdT8hWFmfC0e/yD1/p3RjfxxOSxH2xLAwmYHj1pROlp bIu2m5j+gNmwqHLEoDBnTGtWMAU5NUXHnrmr81uWkg7kjTFAwzpx6DecQaIYcqjWzgJx BJRQ== X-Gm-Message-State: AOAM533IXS6m8BUfDo+8NbZSzc+tCzXHy+e0P5uM/6qzkBhQ3ml5WqiL g/wh26Dkf5FC4P+dX+OU0/wFwDfaEInAKiLs X-Google-Smtp-Source: ABdhPJy/3Fnr4diaeeMeGBg7ZnLU26BMUMZ+O1jWF4+HwbeUIZ9cSQ4Ed94+MQ0yeKgtqBoGdujlYw== X-Received: by 2002:a5d:64e6:: with SMTP id g6mr26718336wri.151.1632746012617; Mon, 27 Sep 2021 05:33:32 -0700 (PDT) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id r9sm16285110wru.2.2021.09.27.05.33.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Sep 2021 05:33:32 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Mon, 27 Sep 2021 13:33:26 +0100 Message-Id: <20210927123327.14554-3-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210927123327.14554-1-david.plowman@raspberrypi.com> References: <20210927123327.14554-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 2/3] libcamera: Support passing ColorSpaces to V4L2 drivers X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The ColorSpace class is added to the StreamConfiguration, and is now passed to V4L2 devices where it is handled appropriately. Note how this means that the colour space is configured per-stream (though platforms may restrict this). Signed-off-by: David Plowman --- include/libcamera/internal/v4l2_videodevice.h | 2 + include/libcamera/stream.h | 3 + src/libcamera/v4l2_videodevice.cpp | 119 ++++++++++++++++++ 3 files changed, 124 insertions(+) diff --git a/include/libcamera/internal/v4l2_videodevice.h b/include/libcamera/internal/v4l2_videodevice.h index efe34d47..34cc9cdd 100644 --- a/include/libcamera/internal/v4l2_videodevice.h +++ b/include/libcamera/internal/v4l2_videodevice.h @@ -20,6 +20,7 @@ #include #include +#include #include #include #include @@ -163,6 +164,7 @@ public: V4L2PixelFormat fourcc; Size size; + ColorSpace colorSpace; std::array planes; unsigned int planesCount = 0; diff --git a/include/libcamera/stream.h b/include/libcamera/stream.h index 0c55e716..131f7733 100644 --- a/include/libcamera/stream.h +++ b/include/libcamera/stream.h @@ -12,6 +12,7 @@ #include #include +#include #include #include #include @@ -47,6 +48,8 @@ struct StreamConfiguration { unsigned int bufferCount; + ColorSpace colorSpace; + Stream *stream() const { return stream_; } void setStream(Stream *stream) { stream_ = stream; } const StreamFormats &formats() const { return formats_; } diff --git a/src/libcamera/v4l2_videodevice.cpp b/src/libcamera/v4l2_videodevice.cpp index ba5f88cd..3e6c8b87 100644 --- a/src/libcamera/v4l2_videodevice.cpp +++ b/src/libcamera/v4l2_videodevice.cpp @@ -369,6 +369,11 @@ bool V4L2BufferCache::Entry::operator==(const FrameBuffer &buffer) const * \brief The image size in pixels */ +/** + * \var V4L2DeviceFormat::colorSpace + * \brief The color space of the pixels + */ + /** * \var V4L2DeviceFormat::fourcc * \brief The fourcc code describing the pixel encoding scheme @@ -752,6 +757,114 @@ int V4L2VideoDevice::getFormat(V4L2DeviceFormat *format) return getFormatSingleplane(format); } +static const std::vector> colorSpaceToV4l2 = { + { ColorSpace::RAW, V4L2_COLORSPACE_RAW }, + { ColorSpace::JFIF, V4L2_COLORSPACE_JPEG }, + { ColorSpace::SMPTE170M, V4L2_COLORSPACE_SMPTE170M }, + { ColorSpace::REC709, V4L2_COLORSPACE_REC709 }, + { ColorSpace::REC2020, V4L2_COLORSPACE_BT2020 }, +}; + +static const std::map encodingToV4l2 = { + { ColorSpace::Encoding::REC601, V4L2_YCBCR_ENC_601 }, + { ColorSpace::Encoding::REC709, V4L2_YCBCR_ENC_709 }, + { ColorSpace::Encoding::REC2020, V4L2_YCBCR_ENC_BT2020 }, +}; + +static const std::map transferFunctionToV4l2 = { + { ColorSpace::TransferFunction::IDENTITY, V4L2_XFER_FUNC_NONE }, + { ColorSpace::TransferFunction::SRGB, V4L2_XFER_FUNC_SRGB }, + { ColorSpace::TransferFunction::REC709, V4L2_XFER_FUNC_709 }, +}; + +static const std::map rangeToV4l2 = { + { ColorSpace::Range::FULL, V4L2_QUANTIZATION_FULL_RANGE }, + { ColorSpace::Range::LIMITED, V4L2_QUANTIZATION_LIM_RANGE }, +}; + +static const std::map v4l2ToColorSpace = { + { V4L2_COLORSPACE_RAW, ColorSpace::RAW }, + { V4L2_COLORSPACE_JPEG, ColorSpace::JFIF }, + { V4L2_COLORSPACE_SRGB, ColorSpace::JFIF }, + { V4L2_COLORSPACE_SMPTE170M, ColorSpace::SMPTE170M }, + { V4L2_COLORSPACE_REC709, ColorSpace::REC709 }, + { V4L2_COLORSPACE_BT2020, ColorSpace::REC2020 }, +}; + +static const std::map v4l2ToEncoding = { + { V4L2_YCBCR_ENC_601, ColorSpace::Encoding::REC601 }, + { V4L2_YCBCR_ENC_709, ColorSpace::Encoding::REC709 }, + { V4L2_YCBCR_ENC_BT2020, ColorSpace::Encoding::REC2020 }, +}; + +static const std::map v4l2ToTransferFunction = { + { V4L2_XFER_FUNC_NONE, ColorSpace::TransferFunction::IDENTITY }, + { V4L2_XFER_FUNC_SRGB, ColorSpace::TransferFunction::SRGB }, + { V4L2_XFER_FUNC_709, ColorSpace::TransferFunction::REC709 }, +}; + +static const std::map v4l2ToRange = { + { V4L2_QUANTIZATION_FULL_RANGE, ColorSpace::Range::FULL }, + { V4L2_QUANTIZATION_LIM_RANGE, ColorSpace::Range::LIMITED }, +}; + +template +static void setColorSpace(const ColorSpace &colorSpace, T &v4l2PixFormat) +{ + if (!colorSpace.isFullyDefined()) + LOG(V4L2, Warning) << "Setting non-fully defined colour space" + << colorSpace.toString(); + + v4l2PixFormat.colorspace = V4L2_COLORSPACE_DEFAULT; + v4l2PixFormat.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; + v4l2PixFormat.xfer_func = V4L2_XFER_FUNC_DEFAULT; + v4l2PixFormat.quantization = V4L2_QUANTIZATION_DEFAULT; + + auto itColor = std::find_if(colorSpaceToV4l2.begin(), colorSpaceToV4l2.end(), + [&colorSpace](const std::pair &item) { return colorSpace == item.first; }); + if (itColor != colorSpaceToV4l2.end()) + v4l2PixFormat.colorspace = itColor->second; + + auto itEncoding = encodingToV4l2.find(colorSpace.encoding); + if (itEncoding != encodingToV4l2.end()) + v4l2PixFormat.ycbcr_enc = itEncoding->second; + + auto itTransfer = transferFunctionToV4l2.find(colorSpace.transferFunction); + if (itTransfer != transferFunctionToV4l2.end()) + v4l2PixFormat.xfer_func = itTransfer->second; + + auto itRange = rangeToV4l2.find(colorSpace.range); + if (itRange != rangeToV4l2.end()) + v4l2PixFormat.quantization = itRange->second; +} + +template +static ColorSpace getColorSpace(const T &v4l2PixFormat) +{ + ColorSpace colorSpace; + + auto itColor = v4l2ToColorSpace.find(v4l2PixFormat.colorspace); + if (itColor != v4l2ToColorSpace.end()) + colorSpace = itColor->second; + + auto itEncoding = v4l2ToEncoding.find(v4l2PixFormat.ycbcr_enc); + if (itEncoding != v4l2ToEncoding.end()) + colorSpace.encoding = itEncoding->second; + + auto itTransfer = v4l2ToTransferFunction.find(v4l2PixFormat.xfer_func); + if (itTransfer != v4l2ToTransferFunction.end()) + colorSpace.transferFunction = itTransfer->second; + + auto itRange = v4l2ToRange.find(v4l2PixFormat.quantization); + if (itRange != v4l2ToRange.end()) + colorSpace.range = itRange->second; + + if (!colorSpace.isFullyDefined()) + LOG(V4L2, Warning) << "Returning non-fully defined colour space" + << colorSpace.toString(); + return colorSpace; +} + /** * \brief Try an image format on the V4L2 video device * \param[inout] format The image format to test applicability to the video device @@ -871,6 +984,7 @@ int V4L2VideoDevice::getFormatMultiplane(V4L2DeviceFormat *format) format->size.width = pix->width; format->size.height = pix->height; format->fourcc = V4L2PixelFormat(pix->pixelformat); + format->colorSpace = getColorSpace(*pix); format->planesCount = pix->num_planes; for (unsigned int i = 0; i < format->planesCount; ++i) { @@ -893,6 +1007,7 @@ int V4L2VideoDevice::trySetFormatMultiplane(V4L2DeviceFormat *format, bool set) pix->pixelformat = format->fourcc; pix->num_planes = format->planesCount; pix->field = V4L2_FIELD_NONE; + setColorSpace(format->colorSpace, *pix); ASSERT(pix->num_planes <= std::size(pix->plane_fmt)); @@ -921,6 +1036,7 @@ int V4L2VideoDevice::trySetFormatMultiplane(V4L2DeviceFormat *format, bool set) format->planes[i].bpl = pix->plane_fmt[i].bytesperline; format->planes[i].size = pix->plane_fmt[i].sizeimage; } + format->colorSpace = getColorSpace(*pix); return 0; } @@ -941,6 +1057,7 @@ int V4L2VideoDevice::getFormatSingleplane(V4L2DeviceFormat *format) format->size.width = pix->width; format->size.height = pix->height; format->fourcc = V4L2PixelFormat(pix->pixelformat); + format->colorSpace = getColorSpace(*pix); format->planesCount = 1; format->planes[0].bpl = pix->bytesperline; format->planes[0].size = pix->sizeimage; @@ -960,6 +1077,7 @@ int V4L2VideoDevice::trySetFormatSingleplane(V4L2DeviceFormat *format, bool set) pix->pixelformat = format->fourcc; pix->bytesperline = format->planes[0].bpl; pix->field = V4L2_FIELD_NONE; + setColorSpace(format->colorSpace, *pix); ret = ioctl(set ? VIDIOC_S_FMT : VIDIOC_TRY_FMT, &v4l2Format); if (ret) { LOG(V4L2, Error) @@ -978,6 +1096,7 @@ int V4L2VideoDevice::trySetFormatSingleplane(V4L2DeviceFormat *format, bool set) format->planesCount = 1; format->planes[0].bpl = pix->bytesperline; format->planes[0].size = pix->sizeimage; + format->colorSpace = getColorSpace(*pix); return 0; } From patchwork Mon Sep 27 12:33:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 13953 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EE36DBDC71 for ; Mon, 27 Sep 2021 12:33:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2B4C46919B; Mon, 27 Sep 2021 14:33:37 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="G1lB8Gg/"; dkim-atps=neutral Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D6FD569191 for ; Mon, 27 Sep 2021 14:33:33 +0200 (CEST) Received: by mail-wr1-x430.google.com with SMTP id t18so51726229wrb.0 for ; Mon, 27 Sep 2021 05:33:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=s644ahboex6q+shoMOYNj6OgxWUg8OUuJkbKuV6OFsQ=; b=G1lB8Gg/8hcjY8nLI1dJLiELnZQ/kvLaryfinkAROrfZrLpvpacj41dxA8RoJt7Ej9 7D2X/W4pJHpHj9zaWjekOpmjHVo76HcV9r2vkVDGxelSPDwJ6V7PLJfEa2TKbN4xnzZv 637hkUvhxZ5+9KFai64AUweAxUjm6iFEKAHgINfFbici0Els+yR1DP90tB3f7AqTPoHf O8vWYTte52IuMa4X7Lt9rJjtV4E3nsdTjUAvty6wa6lyevAeTdnpidymV6OfOUK+tl+A qfUuXw+idZ8fU6btgoX3iuYdZQtlMhxUFTkD230HaPm5s8jA6Dxwnb9KEP1DdBUbM+2u A19g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=s644ahboex6q+shoMOYNj6OgxWUg8OUuJkbKuV6OFsQ=; b=ZhXQrhJjfacq+6p0a4W4FFnvTLfPMWRZPogWhiJHcDQNb2hFyOVFhGDyRTGCGLu3cY 2VtPVvjNJS+5aSpCNT5U58P8xK+ZltN6zVTimh72xfrAUDvnr8tFY3x+fSz0x+uVa5sR HqoqVeX4BGY+KmsVUVW2czE4YVvZ1+SlVC1kpFISvYXUDTP973BpQwcAr6SKytqP5i1Y Ip1ks3bvbYtiFl34FS3/S4Okoy4xFaE3HqEu7u8XZikTKWNfeWZ0tqpDwty7GC3SleKW Z397QlrtNhBh0B46DbCJLGZuor5zqkOL5iL+IO/cKXyRB4y9QCvTpPXE/liYu9UKiBdI djcw== X-Gm-Message-State: AOAM531KdFtFmrQkAakHRPJJPWfl5he0DC/MfKH62ajqq/TABOw/C16v WjnO/Pp78QoMoGk9PSRdslGlIdyZosxTU4gv X-Google-Smtp-Source: ABdhPJznuqsg+KO1+cVrfXfy8YQFStL/o+s4AXI9756wfwP6lr5GBCRe2YPatDKZkxW7SzaTHXM/Sw== X-Received: by 2002:a5d:4eca:: with SMTP id s10mr27953478wrv.255.1632746013310; Mon, 27 Sep 2021 05:33:33 -0700 (PDT) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id r9sm16285110wru.2.2021.09.27.05.33.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Sep 2021 05:33:32 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Mon, 27 Sep 2021 13:33:27 +0100 Message-Id: <20210927123327.14554-4-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210927123327.14554-1-david.plowman@raspberrypi.com> References: <20210927123327.14554-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 3/3] libcamera: pipeline: raspberrypi: Support colour spaces X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The Raspberry Pi pipeline handler now sets colour spaces correctly. In generateConfiguration() is sets them to reasonable default values based on the stream role. Note how video recording streams use the "VIDEO" YCbCr encoding, which will later be fixed up according to the requested resolution. validate() and configure() check the colour space is valid, and also force all (non-raw) output streams to share the same colour space (which is a hardware restriction). Finally, the "VIDEO" YCbCr encoding is corrected by configure() to be one of REC601, REC709 or REC2020. Signed-off-by: David Plowman --- .../pipeline/raspberrypi/raspberrypi.cpp | 84 +++++++++++++++++++ 1 file changed, 84 insertions(+) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 0bdfa727..33ab49d6 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -281,6 +281,24 @@ RPiCameraConfiguration::RPiCameraConfiguration(const RPiCameraData *data) { } +static bool validateColorSpace(ColorSpace &colorSpace) +{ + const std::vector validColorSpaces = { + ColorSpace::JFIF, + ColorSpace::SMPTE170M, + ColorSpace::REC709, + ColorSpace::REC2020, + ColorSpace::VIDEO, + }; + auto it = std::find_if(validColorSpaces.begin(), validColorSpaces.end(), + [&colorSpace](const ColorSpace &item) { return colorSpace == item; }); + if (it != validColorSpaces.end()) + return true; + + colorSpace = ColorSpace::JFIF; + return false; +} + CameraConfiguration::Status RPiCameraConfiguration::validate() { Status status = Valid; @@ -346,6 +364,7 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() unsigned int rawCount = 0, outCount = 0, count = 0, maxIndex = 0; std::pair outSize[2]; Size maxSize; + ColorSpace colorSpace; /* colour space for all non-raw output streams */ for (StreamConfiguration &cfg : config_) { if (isRaw(cfg.pixelFormat)) { /* @@ -354,6 +373,7 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() */ V4L2VideoDevice::Formats fmts = data_->unicam_[Unicam::Image].dev()->formats(); V4L2DeviceFormat sensorFormat = findBestMode(fmts, cfg.size); + sensorFormat.colorSpace = ColorSpace::RAW; int ret = data_->unicam_[Unicam::Image].dev()->tryFormat(&sensorFormat); if (ret) return Invalid; @@ -392,6 +412,7 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() if (maxSize < cfg.size) { maxSize = cfg.size; maxIndex = outCount; + colorSpace = cfg.colorSpace; } outCount++; } @@ -405,6 +426,10 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() } } + /* Ensure the output colour space (if present) is one we handle. */ + if (outCount) + validateColorSpace(colorSpace); + /* * Now do any fixups needed. For the two ISP outputs, one stream must be * equal or smaller than the other in all dimensions. @@ -446,10 +471,26 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() status = Adjusted; } + /* Output streams must share the same colour space. */ + if (cfg.colorSpace != colorSpace) { + LOG(RPI, Warning) << "Output stream " << (i == maxIndex ? 0 : 1) + << " colour space changed"; + cfg.colorSpace = colorSpace; + status = Adjusted; + } + V4L2DeviceFormat format; format.fourcc = V4L2PixelFormat::fromPixelFormat(cfg.pixelFormat); format.size = cfg.size; + /* + * Request a sensible colour space. Note that "VIDEO" isn't a real + * encoding, so substitute something else sensible. + */ + format.colorSpace = colorSpace; + if (format.colorSpace.encoding == ColorSpace::Encoding::VIDEO) + format.colorSpace.encoding = ColorSpace::Encoding::REC601; + int ret = dev->tryFormat(&format); if (ret) return Invalid; @@ -475,6 +516,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, V4L2DeviceFormat sensorFormat; unsigned int bufferCount; PixelFormat pixelFormat; + ColorSpace colorSpace; V4L2VideoDevice::Formats fmts; Size size; @@ -490,6 +532,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, fmts = data->unicam_[Unicam::Image].dev()->formats(); sensorFormat = findBestMode(fmts, size); pixelFormat = sensorFormat.fourcc.toPixelFormat(); + colorSpace = ColorSpace::RAW; ASSERT(pixelFormat.isValid()); bufferCount = 2; rawCount++; @@ -501,6 +544,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, /* Return the largest sensor resolution. */ size = data->sensor_->resolution(); bufferCount = 1; + colorSpace = ColorSpace::JFIF; outCount++; break; @@ -517,6 +561,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, pixelFormat = formats::YUV420; size = { 1920, 1080 }; bufferCount = 4; + colorSpace = ColorSpace::VIDEO; outCount++; break; @@ -525,6 +570,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, pixelFormat = formats::ARGB8888; size = { 800, 600 }; bufferCount = 4; + colorSpace = ColorSpace::JFIF; outCount++; break; @@ -554,6 +600,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, StreamConfiguration cfg(formats); cfg.size = size; cfg.pixelFormat = pixelFormat; + cfg.colorSpace = colorSpace; cfg.bufferCount = bufferCount; config->addConfiguration(cfg); } @@ -575,6 +622,7 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) Size maxSize, sensorSize; unsigned int maxIndex = 0; bool rawStream = false; + ColorSpace colorSpace; /* colour space for all non-raw output streams */ /* * Look for the RAW stream (if given) size as well as the largest @@ -593,14 +641,40 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) } else { if (cfg.size > maxSize) { maxSize = config->at(i).size; + colorSpace = config->at(i).colorSpace; maxIndex = i; } } } + if (maxSize.isNull()) { + /* + * No non-raw streams, so some will get made below. Doesn't matter + * what colour space we assign to them. + */ + colorSpace = ColorSpace::JFIF; + } else { + /* Make sure we can handle this colour space. */ + validateColorSpace(colorSpace); + + /* + * The "VIDEO" colour encoding means that we choose one of REC601, + * REC709 or REC2020 automatically according to the resolution. + */ + if (colorSpace.encoding == ColorSpace::Encoding::VIDEO) { + if (maxSize.width >= 3840) + colorSpace.encoding = ColorSpace::Encoding::REC2020; + else if (maxSize.width >= 1280) + colorSpace.encoding = ColorSpace::Encoding::REC709; + else + colorSpace.encoding = ColorSpace::Encoding::REC601; + } + } + /* First calculate the best sensor mode we can use based on the user request. */ V4L2VideoDevice::Formats fmts = data->unicam_[Unicam::Image].dev()->formats(); V4L2DeviceFormat sensorFormat = findBestMode(fmts, rawStream ? sensorSize : maxSize); + sensorFormat.colorSpace = ColorSpace::RAW; /* * Unicam image output format. The ISP input format gets set at start, @@ -638,11 +712,18 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) StreamConfiguration &cfg = config->at(i); if (isRaw(cfg.pixelFormat)) { + cfg.colorSpace = ColorSpace::RAW; cfg.setStream(&data->unicam_[Unicam::Image]); data->unicam_[Unicam::Image].setExternal(true); continue; } + /* All other streams share the same colour space. */ + if (cfg.colorSpace != colorSpace) { + LOG(RPI, Warning) << "Stream " << i << " colour space changed"; + cfg.colorSpace = colorSpace; + } + /* The largest resolution gets routed to the ISP Output 0 node. */ RPi::Stream *stream = i == maxIndex ? &data->isp_[Isp::Output0] : &data->isp_[Isp::Output1]; @@ -650,6 +731,7 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) V4L2PixelFormat fourcc = V4L2PixelFormat::fromPixelFormat(cfg.pixelFormat); format.size = cfg.size; format.fourcc = fourcc; + format.colorSpace = cfg.colorSpace; LOG(RPI, Debug) << "Setting " << stream->name() << " to " << format.toString(); @@ -689,6 +771,7 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) format = {}; format.size = maxSize; format.fourcc = V4L2PixelFormat::fromPixelFormat(formats::YUV420); + format.colorSpace = colorSpace; ret = data->isp_[Isp::Output0].dev()->setFormat(&format); if (ret) { LOG(RPI, Error) @@ -718,6 +801,7 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) const Size limit = maxDimensions.boundedToAspectRatio(format.size); output1Format.size = (format.size / 2).boundedTo(limit).alignedDownTo(2, 2); + output1Format.colorSpace = colorSpace; LOG(RPI, Debug) << "Setting ISP Output1 (internal) to " << output1Format.toString();