From patchwork Tue Jun 8 14:44:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 12519 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 13323BD22E for ; Tue, 8 Jun 2021 14:44:21 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B4122602A7; Tue, 8 Jun 2021 16:44:19 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="oklJMHz9"; dkim-atps=neutral Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D27E5602A7 for ; Tue, 8 Jun 2021 16:44:17 +0200 (CEST) Received: by mail-wr1-x42d.google.com with SMTP id a20so21936961wrc.0 for ; Tue, 08 Jun 2021 07:44:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=tU0Kgc4GIfvHcEcZhIZFYCoGFnSm56EXIbRc7F5nySU=; b=oklJMHz9WudEoA/SEDjRqhilKCdt2ARNfDY40tgi6+24jgMElRlxDtcfYrlQEeIgaO nXYh6nzkQJlLqONLI3+g2j03jYbfXXyotNa0f2+UV1FsbjoIS+dpOIDaq5XStF53Frlm 4LafXOe6xujysS6xBMYIuGvKDTs8QgtI6iTPKRD25/8Ci1wWo3fu9Fa1yWK7GPpTXdRF 9ZYJZoiyVK1b/bvWzA9oyUNjsjCTTAVnHQ0C6gqio/ivDwRRiIIzzHOfFYEUGg/tciN3 Lr0BsFGkyzPyNrK5Oi/R1UEvulPGjHAyrJXIJgzfl0NBmM/P+GT9/LmET48tBV6Rk4lK 0ULQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tU0Kgc4GIfvHcEcZhIZFYCoGFnSm56EXIbRc7F5nySU=; b=rh8CSxAlZrvZrzgRQacQBpYzi6XaL2B22/SOh6YqAKf/xYgOvW+W1YUChoOMX4vdC6 vhKYK6/3XR0hYFytlh3NpoLFtWuZVyOhmj6W0RxLX5xGrrbN66UzyjY/h/8JUuY9zy1H XgTg5FAan4xQP5tDWQKjrv4qtxl3dIhxKRE+ScdBsuT/KdPs5qOGmgEtPn+q6yz3Hvt9 mZ/rZmQPpJwjpprideKdtF+eoYN2QjgUQRmofG19v0g+WhKb0C+VNVxFOXrnwVRy2Ck8 fjT8Hn3EuHM20JTMcCOZFeMue63tOF0qz/lm5HsROaT9nTF2dszYX4gitUqQNswo0jsU sqow== X-Gm-Message-State: AOAM532MTIiz8GhbqQyrjJPiLzMLhkJfVKjBmKTLHKGIG5/a9qjbQURT K34epNjQ3uR93AamA6AbSoOlK0FA/vJ/xmK2 X-Google-Smtp-Source: ABdhPJzW44Lcju/Xck5Bm/h69OihMCNS0vxZ5WDW/l8vyIILhTJsjFvnHYYuwpSnDsxFJz887iwsYA== X-Received: by 2002:a5d:638b:: with SMTP id p11mr4363402wru.380.1623163457180; Tue, 08 Jun 2021 07:44:17 -0700 (PDT) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id i21sm3053173wmq.12.2021.06.08.07.44.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 07:44:16 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Tue, 8 Jun 2021 15:44:11 +0100 Message-Id: <20210608144413.1529-2-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210608144413.1529-1-david.plowman@raspberrypi.com> References: <20210608144413.1529-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 1/3] libcamera: Add ColorSpace class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This class represents a colour space by defining its YCbCr encoding, the transfer (gamma) function is uses, and whether the output is full or limited range. --- include/libcamera/color_space.h | 94 +++++++++++++++++++++++++++++++++ include/libcamera/meson.build | 1 + src/libcamera/color_space.cpp | 43 +++++++++++++++ src/libcamera/meson.build | 1 + 4 files changed, 139 insertions(+) create mode 100644 include/libcamera/color_space.h create mode 100644 src/libcamera/color_space.cpp diff --git a/include/libcamera/color_space.h b/include/libcamera/color_space.h new file mode 100644 index 00000000..3d990f99 --- /dev/null +++ b/include/libcamera/color_space.h @@ -0,0 +1,94 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Raspberry Pi (Trading) Limited + * + * color_space.h - color space definitions + */ + +#ifndef __LIBCAMERA_COLOR_SPACE_H__ +#define __LIBCAMERA_COLOR_SPACE_H__ + +#include + +namespace libcamera { + +class ColorSpace +{ +public: + enum class Encoding : int { + UNDEFINED, + RAW, + REC601, + REC709, + REC2020, + VIDEO, + }; + + enum class TransferFunction : int { + UNDEFINED, + IDENTITY, + SRGB, + REC709, + }; + + enum class Range : int { + UNDEFINED, + FULL, + LIMITED, + }; + + constexpr ColorSpace(Encoding e, TransferFunction t, Range r) + : encoding(e), transferFunction(t), range(r) + { + } + + constexpr ColorSpace() + : ColorSpace(Encoding::UNDEFINED, TransferFunction::UNDEFINED, Range::UNDEFINED) + { + } + + static const ColorSpace UNDEFINED; + static const ColorSpace RAW; + static const ColorSpace JFIF; + static const ColorSpace SMPTE170M; + static const ColorSpace REC709; + static const ColorSpace REC2020; + static const ColorSpace VIDEO; + + Encoding encoding; + TransferFunction transferFunction; + Range range; + + bool isFullyDefined() const + { + return encoding != Encoding::UNDEFINED && + transferFunction != TransferFunction::UNDEFINED && + range != Range::UNDEFINED; + } + + const std::string toString() const; +}; + +constexpr ColorSpace ColorSpace::UNDEFINED = { Encoding::UNDEFINED, TransferFunction::UNDEFINED, Range::UNDEFINED }; +constexpr ColorSpace ColorSpace::RAW = { Encoding::RAW, TransferFunction::IDENTITY, Range::FULL }; +constexpr ColorSpace ColorSpace::JFIF = { Encoding::REC601, TransferFunction::SRGB, Range::FULL }; +constexpr ColorSpace ColorSpace::SMPTE170M = { Encoding::REC601, TransferFunction::REC709, Range::LIMITED }; +constexpr ColorSpace ColorSpace::REC709 = { Encoding::REC709, TransferFunction::REC709, Range::LIMITED }; +constexpr ColorSpace ColorSpace::REC2020 = { Encoding::REC2020, TransferFunction::REC709, Range::LIMITED }; +constexpr ColorSpace ColorSpace::VIDEO = { Encoding::VIDEO, TransferFunction::REC709, Range::LIMITED }; + +static inline bool operator==(const ColorSpace &lhs, const ColorSpace &rhs) +{ + return lhs.encoding == rhs.encoding && + lhs.transferFunction == rhs.transferFunction && + lhs.range == rhs.range; +} + +static inline bool operator!=(const ColorSpace &lhs, const ColorSpace &rhs) +{ + return !(lhs == rhs); +} + +} /* namespace libcamera */ + +#endif /* __LIBCAMERA_COLOR_SPACE_H__ */ diff --git a/include/libcamera/meson.build b/include/libcamera/meson.build index 086c958b..7d61d285 100644 --- a/include/libcamera/meson.build +++ b/include/libcamera/meson.build @@ -6,6 +6,7 @@ libcamera_public_headers = files([ 'camera.h', 'camera_manager.h', 'class.h', + 'color_space.h', 'compiler.h', 'controls.h', 'file_descriptor.h', diff --git a/src/libcamera/color_space.cpp b/src/libcamera/color_space.cpp new file mode 100644 index 00000000..888762ae --- /dev/null +++ b/src/libcamera/color_space.cpp @@ -0,0 +1,43 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Raspberry Pi (Trading) Limited + * + * color_space.cpp - color spaces. + */ + +#include + +/** + * \file color_space.h + * \brief Class and enums to represent colour spaces. + */ + +namespace libcamera { + +const std::string ColorSpace::toString() const +{ + static const char *encodings[] = { + "UNDEFINED", + "RAW", + "REC601", + "REC709", + "REC2020", + }; + static const char *transferFunctions[] = { + "UNDEFINED", + "IDENTITY", + "SRGB", + "REC709", + }; + static const char *ranges[] = { + "UNDEFINED", + "FULL", + "LIMITED", + }; + + return std::string(encodings[static_cast(encoding)]) + "+" + + std::string(transferFunctions[static_cast(transferFunction)]) + "+" + + std::string(ranges[static_cast(range)]); +} + +} /* namespace libcamera */ diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build index 7e19a177..f7b48a32 100644 --- a/src/libcamera/meson.build +++ b/src/libcamera/meson.build @@ -11,6 +11,7 @@ libcamera_sources = files([ 'camera_sensor.cpp', 'camera_sensor_properties.cpp', 'class.cpp', + 'color_space.cpp', 'controls.cpp', 'control_serializer.cpp', 'control_validator.cpp', From patchwork Tue Jun 8 14:44:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 12520 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A3C25C320B for ; Tue, 8 Jun 2021 14:44:21 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1E8F068935; Tue, 8 Jun 2021 16:44:20 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="dgyP4oDG"; dkim-atps=neutral Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6E16D602A7 for ; Tue, 8 Jun 2021 16:44:18 +0200 (CEST) Received: by mail-wr1-x429.google.com with SMTP id o3so3490591wri.8 for ; Tue, 08 Jun 2021 07:44:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=D9/f6O7zN/BBzUimu5fdwYXvAs0jlUvjQorkBozOPyo=; b=dgyP4oDGqGNsErz1owPaE+XLUZoU89dt3l1JUpwsjiLUhwQT1rnPzQKBTVeG+4iQAO EfwQnJtr5f3geUimo1wHPal3/yz5yfir2HhQYhMY0d0GT1e6ZIyZaoJTaM8lMBpmW99X KEH5p13aQvjYmbeISRKhzdSsWEVCyh75upoF1SaQUQLzpcoetoL67wyDdCQ98OT7q9H4 yaKis7x2Onag9E22TtpYVmJbRHMpxrxug5yWNt6uivzmEEE1pnS87ms5z6K6yIAxba4z oaytpS9C4bYixQikrfihNY/8jes4NUPkTW+NHKECs3uVeQlrNANt20tu53229oYf8ka/ ri4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=D9/f6O7zN/BBzUimu5fdwYXvAs0jlUvjQorkBozOPyo=; b=Hm+3n+yd0FK2O22xiJYTzCuWwjx+0lpzzuBzh/x2MJOEQ6OkGtWOxuplKzDkUDbHHP SMf6bLwK1NEgMd/tTLl59LDILEEOaHQqOzAAE4joC0tghUz1MPHePytHQ8eb1Ws8mXoH MQ7LFzZUsKJWIAbn3hUS0ebyfhvEAaEyMTSAbIR/uQNwZrZOwgg5Z0NhmskTCf7j299v FyeO3cHJwdnApXx9yHhnvh8kfpk93cTbMNqK6o4/WPxwXvPku//9NI4+u3SAptUCTIh9 SNMNKRzPBE6Omk/ZedHsuc3MbQSkJAAO7KERMRJoeYI0dvS2UbWXyA5Q554fqmkAnRz5 Qv8A== X-Gm-Message-State: AOAM533TMMdizaMsWvkHZVrnXu1kHitnabm7dbcit+GtGLOmoVro3qCM Si/BYgadYl7voiATU5OXXqoLcknnaq7hZb70 X-Google-Smtp-Source: ABdhPJzcrnxppY9bLf8JEM1Sbgd2f6yGgZ6AYj9xEBNbpnP/hyZz0S4FCPRn7LUAR6cFwlQUi7Uzwg== X-Received: by 2002:adf:a195:: with SMTP id u21mr22724502wru.367.1623163457952; Tue, 08 Jun 2021 07:44:17 -0700 (PDT) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id i21sm3053173wmq.12.2021.06.08.07.44.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 07:44:17 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Tue, 8 Jun 2021 15:44:12 +0100 Message-Id: <20210608144413.1529-3-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210608144413.1529-1-david.plowman@raspberrypi.com> References: <20210608144413.1529-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 2/3] libcamera: Support passing ColorSpaces to V4L2 drivers X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The ColorSpace class is added to the StreamConfiguration, and is now passed to V4L2 devices where it is handled appropriately. Note how this means that the colour space is configured per-stream (though platforms may restrict this). --- include/libcamera/internal/v4l2_videodevice.h | 2 + include/libcamera/stream.h | 3 + src/libcamera/v4l2_videodevice.cpp | 112 ++++++++++++++++++ 3 files changed, 117 insertions(+) diff --git a/include/libcamera/internal/v4l2_videodevice.h b/include/libcamera/internal/v4l2_videodevice.h index 7938343b..5122d422 100644 --- a/include/libcamera/internal/v4l2_videodevice.h +++ b/include/libcamera/internal/v4l2_videodevice.h @@ -18,6 +18,7 @@ #include #include +#include #include #include #include @@ -162,6 +163,7 @@ public: V4L2PixelFormat fourcc; Size size; + ColorSpace colorSpace; std::array planes; unsigned int planesCount = 0; diff --git a/include/libcamera/stream.h b/include/libcamera/stream.h index bb47c390..f79c6987 100644 --- a/include/libcamera/stream.h +++ b/include/libcamera/stream.h @@ -13,6 +13,7 @@ #include #include +#include #include #include @@ -47,6 +48,8 @@ struct StreamConfiguration { unsigned int bufferCount; + ColorSpace colorSpace; + Stream *stream() const { return stream_; } void setStream(Stream *stream) { stream_ = stream; } const StreamFormats &formats() const { return formats_; } diff --git a/src/libcamera/v4l2_videodevice.cpp b/src/libcamera/v4l2_videodevice.cpp index 12c09dc7..d606f81d 100644 --- a/src/libcamera/v4l2_videodevice.cpp +++ b/src/libcamera/v4l2_videodevice.cpp @@ -730,6 +730,114 @@ int V4L2VideoDevice::getFormat(V4L2DeviceFormat *format) return getFormatSingleplane(format); } +static const std::vector> colorSpaceToV4l2 = { + { ColorSpace::RAW, V4L2_COLORSPACE_RAW }, + { ColorSpace::JFIF, V4L2_COLORSPACE_JPEG }, + { ColorSpace::SMPTE170M, V4L2_COLORSPACE_SMPTE170M }, + { ColorSpace::REC709, V4L2_COLORSPACE_REC709 }, + { ColorSpace::REC2020, V4L2_COLORSPACE_BT2020 }, +}; + +static const std::map encodingToV4l2 = { + { ColorSpace::Encoding::REC601, V4L2_YCBCR_ENC_601 }, + { ColorSpace::Encoding::REC709, V4L2_YCBCR_ENC_709 }, + { ColorSpace::Encoding::REC2020, V4L2_YCBCR_ENC_BT2020 }, +}; + +static const std::map transferFunctionToV4l2 = { + { ColorSpace::TransferFunction::IDENTITY, V4L2_XFER_FUNC_NONE }, + { ColorSpace::TransferFunction::SRGB, V4L2_XFER_FUNC_SRGB }, + { ColorSpace::TransferFunction::REC709, V4L2_XFER_FUNC_709 }, +}; + +static const std::map rangeToV4l2 = { + { ColorSpace::Range::FULL, V4L2_QUANTIZATION_FULL_RANGE }, + { ColorSpace::Range::LIMITED, V4L2_QUANTIZATION_LIM_RANGE }, +}; + +template +static void setColorSpace(const ColorSpace &colorSpace, T &v4l2PixFormat) +{ + if (!colorSpace.isFullyDefined()) + LOG(V4L2, Warning) << "Setting non-fully defined colour space" + << colorSpace.toString(); + + v4l2PixFormat.colorspace = V4L2_COLORSPACE_DEFAULT; + v4l2PixFormat.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; + v4l2PixFormat.xfer_func = V4L2_XFER_FUNC_DEFAULT; + v4l2PixFormat.quantization = V4L2_QUANTIZATION_DEFAULT; + + auto it_color = std::find_if(colorSpaceToV4l2.begin(), colorSpaceToV4l2.end(), + [&colorSpace](const std::pair &item) { return colorSpace == item.first; }); + if (it_color != colorSpaceToV4l2.end()) + v4l2PixFormat.colorspace = it_color->second; + + auto it_encoding = encodingToV4l2.find(colorSpace.encoding); + if (it_encoding != encodingToV4l2.end()) + v4l2PixFormat.ycbcr_enc = it_encoding->second; + + auto it_transfer = transferFunctionToV4l2.find(colorSpace.transferFunction); + if (it_transfer != transferFunctionToV4l2.end()) + v4l2PixFormat.xfer_func = it_transfer->second; + + auto it_range = rangeToV4l2.find(colorSpace.range); + if (it_range != rangeToV4l2.end()) + v4l2PixFormat.quantization = it_range->second; +} + +static const std::map v4l2ToColorSpace = { + { V4L2_COLORSPACE_RAW, ColorSpace::RAW }, + { V4L2_COLORSPACE_JPEG, ColorSpace::JFIF }, + { V4L2_COLORSPACE_SRGB, ColorSpace::JFIF }, + { V4L2_COLORSPACE_SMPTE170M, ColorSpace::SMPTE170M }, + { V4L2_COLORSPACE_REC709, ColorSpace::REC709 }, + { V4L2_COLORSPACE_BT2020, ColorSpace::REC2020 }, +}; + +static const std::map v4l2ToEncoding = { + { V4L2_YCBCR_ENC_601, ColorSpace::Encoding::REC601 }, + { V4L2_YCBCR_ENC_709, ColorSpace::Encoding::REC709 }, + { V4L2_YCBCR_ENC_BT2020, ColorSpace::Encoding::REC2020 }, +}; + +static const std::map v4l2ToTransferFunction = { + { V4L2_XFER_FUNC_NONE, ColorSpace::TransferFunction::IDENTITY }, + { V4L2_XFER_FUNC_SRGB, ColorSpace::TransferFunction::SRGB }, + { V4L2_XFER_FUNC_709, ColorSpace::TransferFunction::REC709 }, +}; + +static const std::map v4l2ToRange = { + { V4L2_QUANTIZATION_FULL_RANGE, ColorSpace::Range::FULL }, + { V4L2_QUANTIZATION_LIM_RANGE, ColorSpace::Range::LIMITED }, +}; + +template +ColorSpace getColorSpace(const T &v4l2PixFormat) +{ + ColorSpace colorSpace; + + auto it_color = v4l2ToColorSpace.find(v4l2PixFormat.colorspace); + if (it_color != v4l2ToColorSpace.end()) + colorSpace = it_color->second; + + auto it_encoding = v4l2ToEncoding.find(v4l2PixFormat.ycbcr_enc); + if (it_encoding != v4l2ToEncoding.end()) + colorSpace.encoding = it_encoding->second; + + auto it_transfer = v4l2ToTransferFunction.find(v4l2PixFormat.xfer_func); + if (it_transfer != v4l2ToTransferFunction.end()) + colorSpace.transferFunction = it_transfer->second; + + auto it_range = v4l2ToRange.find(v4l2PixFormat.quantization); + if (it_range != v4l2ToRange.end()) + colorSpace.range = it_range->second; + + if (!colorSpace.isFullyDefined()) + LOG(V4L2, Warning) << "Returning non-fully defined colour space" + << colorSpace.toString(); + return colorSpace; +} + /** * \brief Try an image format on the V4L2 video device * \param[inout] format The image format to test applicability to the video device @@ -839,6 +947,7 @@ int V4L2VideoDevice::getFormatMultiplane(V4L2DeviceFormat *format) format->size.width = pix->width; format->size.height = pix->height; format->fourcc = V4L2PixelFormat(pix->pixelformat); + format->colorSpace = getColorSpace(*pix); format->planesCount = pix->num_planes; for (unsigned int i = 0; i < format->planesCount; ++i) { @@ -861,6 +970,7 @@ int V4L2VideoDevice::trySetFormatMultiplane(V4L2DeviceFormat *format, bool set) pix->pixelformat = format->fourcc; pix->num_planes = format->planesCount; pix->field = V4L2_FIELD_NONE; + setColorSpace(format->colorSpace, *pix); ASSERT(pix->num_planes <= std::size(pix->plane_fmt)); @@ -909,6 +1019,7 @@ int V4L2VideoDevice::getFormatSingleplane(V4L2DeviceFormat *format) format->size.width = pix->width; format->size.height = pix->height; format->fourcc = V4L2PixelFormat(pix->pixelformat); + format->colorSpace = getColorSpace(*pix); format->planesCount = 1; format->planes[0].bpl = pix->bytesperline; format->planes[0].size = pix->sizeimage; @@ -928,6 +1039,7 @@ int V4L2VideoDevice::trySetFormatSingleplane(V4L2DeviceFormat *format, bool set) pix->pixelformat = format->fourcc; pix->bytesperline = format->planes[0].bpl; pix->field = V4L2_FIELD_NONE; + setColorSpace(format->colorSpace, *pix); ret = ioctl(set ? VIDIOC_S_FMT : VIDIOC_TRY_FMT, &v4l2Format); if (ret) { LOG(V4L2, Error) From patchwork Tue Jun 8 14:44:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 12521 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0E377BD22E for ; Tue, 8 Jun 2021 14:44:23 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C1D5C6892E; Tue, 8 Jun 2021 16:44:22 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="qwFgYoOb"; dkim-atps=neutral Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com [IPv6:2a00:1450:4864:20::32a]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C525768933 for ; Tue, 8 Jun 2021 16:44:19 +0200 (CEST) Received: by mail-wm1-x32a.google.com with SMTP id v206-20020a1cded70000b02901a586d3fa23so2043673wmg.4 for ; Tue, 08 Jun 2021 07:44:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=tTuxQ33qJmOJc/sX7N6lKV1NPaMROOvZ7PIlBB8W7Z8=; b=qwFgYoObGTUdxJVjZvAOFun++LOXIj0pA76LfGpOpJ6Oc21hmmjQVxmhZfRdWaMYk3 YA2Dx5xKtHtP69Ii9ReA+kVGdADXsgAgp9kELt31+fbOnIJDy7sPdx9m0H0hnmJ+kQTG n5L4NZeHcZXy3xWdtCrG2hd5XeZZ3KZCiJdp3a5QDlpi7z2lahkMx0ciA1syD8p/1v+j Rk3oOPLmNeQUfnnAj82xBMO/ay++SVZaQXbkSzWgsYCZ/Zryqoyzz8UkjI1L7tacluLY ebYy/1sbTJAHpDFTLszCiLhFFwteN1Khri++nZKweWu23TMyRCilx3dDfKjRP2RSPO8a BdWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tTuxQ33qJmOJc/sX7N6lKV1NPaMROOvZ7PIlBB8W7Z8=; b=aR4ji5smsXzQWWFjxAJlLenNjzseAwEwUtPgTaG4usIOEUsIj2SIDgAq0twMUb9W1V aXcypEVh+TfR4MWUnea46fx0Mto3vl9qWCx638AmRZ3sEzWV1bIQiFwxzJbaDLfBccXk OlrUk2pDqvWH/tsproxcXvmPMVtude7NXl1xfYwRpJyoh/m8hVfDk7Oa7st1/yu/RNJN 1YhVc4R95kPTdeYkEKP4GUSEjoMcrVp7374KQhFpdMo5R6aK2B327S+6pb62lj5APvZH 0/dg57uknGCOVmiBLEDKVNei4AenCnE0wXJ+cXmqQz2DDDIhSzdsg2kirvwCmT4IvRTr XJLQ== X-Gm-Message-State: AOAM533MU8XYM6+4D4In+IPVQ00Kf1uiPcqw0ueWBMr/+ACv7T66RILo hOglLT37y/i/ILNR+0i8RTNlMZoeEVx72JtD X-Google-Smtp-Source: ABdhPJzQLpWQ9q7p9OtNvly79X6Wjy1Yf/6GkijIIxbce9f2bJFqySS9rTn+JhU9C6Zt9/FHNvKSpw== X-Received: by 2002:a05:600c:410a:: with SMTP id j10mr4566629wmi.6.1623163459223; Tue, 08 Jun 2021 07:44:19 -0700 (PDT) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id i21sm3053173wmq.12.2021.06.08.07.44.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 07:44:18 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Tue, 8 Jun 2021 15:44:13 +0100 Message-Id: <20210608144413.1529-4-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210608144413.1529-1-david.plowman@raspberrypi.com> References: <20210608144413.1529-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 3/3] libcamera: pipeline: raspberrypi: Support colour spaces X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The Raspberry Pi pipeline handler now sets colour spaces correctly. In generateConfiguration() is sets them to reasonable default values based on the stream role. Note how video recording streams use the "VIDEO" YCbCr encoding, which will later be fixed up according to the requested resolution. validate() and configure() check the colour space is valid, and also force all (non-raw) output streams to share the same colour space (which is a hardware restriction). Finally, the "VIDEO" YCbCr encoding is corrected by configure() to be one of REC601, REC709 or REC2020. Reviewed-by: Naushir Patuck --- .../pipeline/raspberrypi/raspberrypi.cpp | 84 +++++++++++++++++++ 1 file changed, 84 insertions(+) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index a65b4568..acb4a1c3 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -279,6 +279,24 @@ RPiCameraConfiguration::RPiCameraConfiguration(const RPiCameraData *data) { } +static bool validateColorSpace(ColorSpace &colorSpace) +{ + const std::vector validColorSpaces = { + ColorSpace::JFIF, + ColorSpace::SMPTE170M, + ColorSpace::REC709, + ColorSpace::REC2020, + ColorSpace::VIDEO, + }; + auto it = std::find_if(validColorSpaces.begin(), validColorSpaces.end(), + [&colorSpace](const ColorSpace &item) { return colorSpace == item; }); + if (it != validColorSpaces.end()) + return true; + + colorSpace = ColorSpace::JFIF; + return false; +} + CameraConfiguration::Status RPiCameraConfiguration::validate() { Status status = Valid; @@ -344,6 +362,7 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() unsigned int rawCount = 0, outCount = 0, count = 0, maxIndex = 0; std::pair outSize[2]; Size maxSize; + ColorSpace colorSpace; /* colour space for all non-raw output streams */ for (StreamConfiguration &cfg : config_) { if (isRaw(cfg.pixelFormat)) { /* @@ -352,6 +371,7 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() */ V4L2VideoDevice::Formats fmts = data_->unicam_[Unicam::Image].dev()->formats(); V4L2DeviceFormat sensorFormat = findBestMode(fmts, cfg.size); + sensorFormat.colorSpace = ColorSpace::RAW; int ret = data_->unicam_[Unicam::Image].dev()->tryFormat(&sensorFormat); if (ret) return Invalid; @@ -390,6 +410,7 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() if (maxSize < cfg.size) { maxSize = cfg.size; maxIndex = outCount; + colorSpace = cfg.colorSpace; } outCount++; } @@ -403,6 +424,10 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() } } + /* Ensure the output colour space (if present) is one we handle. */ + if (outCount) + validateColorSpace(colorSpace); + /* * Now do any fixups needed. For the two ISP outputs, one stream must be * equal or smaller than the other in all dimensions. @@ -444,10 +469,26 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() status = Adjusted; } + /* Output streams must share the same colour space. */ + if (cfg.colorSpace != colorSpace) { + LOG(RPI, Warning) << "Output stream " << (i == maxIndex ? 0 : 1) + << " colour space changed"; + cfg.colorSpace = colorSpace; + status = Adjusted; + } + V4L2DeviceFormat format; format.fourcc = dev->toV4L2PixelFormat(cfg.pixelFormat); format.size = cfg.size; + /* + * Request a sensible colour space. Note that "VIDEO" isn't a real + * encoding, so substitute something else sensible. + */ + format.colorSpace = colorSpace; + if (format.colorSpace.encoding == ColorSpace::Encoding::VIDEO) + format.colorSpace.encoding = ColorSpace::Encoding::REC601; + int ret = dev->tryFormat(&format); if (ret) return Invalid; @@ -473,6 +514,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, V4L2DeviceFormat sensorFormat; unsigned int bufferCount; PixelFormat pixelFormat; + ColorSpace colorSpace; V4L2VideoDevice::Formats fmts; Size size; @@ -488,6 +530,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, fmts = data->unicam_[Unicam::Image].dev()->formats(); sensorFormat = findBestMode(fmts, size); pixelFormat = sensorFormat.fourcc.toPixelFormat(); + colorSpace = ColorSpace::RAW; ASSERT(pixelFormat.isValid()); bufferCount = 2; rawCount++; @@ -499,6 +542,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, /* Return the largest sensor resolution. */ size = data->sensor_->resolution(); bufferCount = 1; + colorSpace = ColorSpace::JFIF; outCount++; break; @@ -515,6 +559,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, pixelFormat = formats::YUV420; size = { 1920, 1080 }; bufferCount = 4; + colorSpace = ColorSpace::VIDEO; outCount++; break; @@ -523,6 +568,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, pixelFormat = formats::ARGB8888; size = { 800, 600 }; bufferCount = 4; + colorSpace = ColorSpace::JFIF; outCount++; break; @@ -552,6 +598,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, StreamConfiguration cfg(formats); cfg.size = size; cfg.pixelFormat = pixelFormat; + cfg.colorSpace = colorSpace; cfg.bufferCount = bufferCount; config->addConfiguration(cfg); } @@ -573,6 +620,7 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) Size maxSize, sensorSize; unsigned int maxIndex = 0; bool rawStream = false; + ColorSpace colorSpace; /* colour space for all non-raw output streams */ /* * Look for the RAW stream (if given) size as well as the largest @@ -591,14 +639,40 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) } else { if (cfg.size > maxSize) { maxSize = config->at(i).size; + colorSpace = config->at(i).colorSpace; maxIndex = i; } } } + if (maxSize.isNull()) { + /* + * No non-raw streams, so some will get made below. Doesn't matter + * what colour space we assign to them. + */ + colorSpace = ColorSpace::JFIF; + } else { + /* Make sure we can handle this colour space. */ + validateColorSpace(colorSpace); + + /* + * The "VIDEO" colour encoding means that we choose one of REC601, + * REC709 or REC2020 automatically according to the resolution. + */ + if (colorSpace.encoding == ColorSpace::Encoding::VIDEO) { + if (maxSize.width >= 3840) + colorSpace.encoding = ColorSpace::Encoding::REC2020; + else if (maxSize.width >= 1280) + colorSpace.encoding = ColorSpace::Encoding::REC709; + else + colorSpace.encoding = ColorSpace::Encoding::REC601; + } + } + /* First calculate the best sensor mode we can use based on the user request. */ V4L2VideoDevice::Formats fmts = data->unicam_[Unicam::Image].dev()->formats(); V4L2DeviceFormat sensorFormat = findBestMode(fmts, rawStream ? sensorSize : maxSize); + sensorFormat.colorSpace = ColorSpace::RAW; /* * Unicam image output format. The ISP input format gets set at start, @@ -636,11 +710,18 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) StreamConfiguration &cfg = config->at(i); if (isRaw(cfg.pixelFormat)) { + cfg.colorSpace = ColorSpace::RAW; cfg.setStream(&data->unicam_[Unicam::Image]); data->unicam_[Unicam::Image].setExternal(true); continue; } + /* All other streams share the same colour space. */ + if (cfg.colorSpace != colorSpace) { + LOG(RPI, Warning) << "Stream " << i << " colour space changed"; + cfg.colorSpace = colorSpace; + } + /* The largest resolution gets routed to the ISP Output 0 node. */ RPi::Stream *stream = i == maxIndex ? &data->isp_[Isp::Output0] : &data->isp_[Isp::Output1]; @@ -648,6 +729,7 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) V4L2PixelFormat fourcc = stream->dev()->toV4L2PixelFormat(cfg.pixelFormat); format.size = cfg.size; format.fourcc = fourcc; + format.colorSpace = cfg.colorSpace; LOG(RPI, Debug) << "Setting " << stream->name() << " to " << format.toString(); @@ -687,6 +769,7 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) format = {}; format.size = maxSize; format.fourcc = V4L2PixelFormat::fromPixelFormat(formats::YUV420, false); + format.colorSpace = colorSpace; ret = data->isp_[Isp::Output0].dev()->setFormat(&format); if (ret) { LOG(RPI, Error) @@ -716,6 +799,7 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) const Size limit = maxDimensions.boundedToAspectRatio(format.size); output1Format.size = (format.size / 2).boundedTo(limit).alignedDownTo(2, 2); + output1Format.colorSpace = colorSpace; LOG(RPI, Debug) << "Setting ISP Output1 (internal) to " << output1Format.toString();