From patchwork Thu Sep 21 16:55:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 19078 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 58DF7BE080 for ; Thu, 21 Sep 2023 20:51:55 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9FAA962944; Thu, 21 Sep 2023 22:51:54 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1695329514; bh=wQAOVVolZr+lzqz9rw0S0k1komJoFVjV9h6sK8wDq9M=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=Y+RjyVzZ6rKVG21ph9D4VpmUJejrbaHPN5XP6/SAdn4t23kwZUSotseQm8zvBrAnD H1Ri48LZm+5tNtSz2p57VDHoq8iqJY0/ttGHliXxpRsVImmTDCy+ssBkhxha/W3xDb lxdhMcgYU1PDT6JegKGtVDZcckIjUYr2QfLw49XeyK9LkEAOZqknoWe+SG028jOJt6 mSMy9IDN+CJSqE/qY86UMIwKhKaCXzVtzMFSWt5BWDNH/RnKEGaLeMqFQ79clLV3Ku mJbV0KnhmkUh9jea5ziscMHyHIy+ETqtHanOeq77ko4Xpke7aRWlBie05C1s9PhoID 3tMNInA7LFkVQ== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 424AF628D8 for ; Thu, 21 Sep 2023 18:56:02 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="GDCEV/fP"; dkim-atps=neutral Received: from uno.lan (93-46-82-201.ip106.fastwebnet.it [93.46.82.201]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 476941257; Thu, 21 Sep 2023 18:54:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1695315264; bh=wQAOVVolZr+lzqz9rw0S0k1komJoFVjV9h6sK8wDq9M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GDCEV/fPrAh+ENtQbEUqp1svIumUB1cewZqCJt791i7OUOdhiaP+JX5CUfnfNXCRS r8etNCNGPScAWzc2dv/DXCZPxwWi0vtNNMkMqLOYXl/Y1T2sht2H5NGjbWOCyhvyqz pJ4o6HXeWyy4HFTT7O/i1bqKajKGpkqqUEvY1v1o= To: libcamera-devel@lists.libcamera.org Date: Thu, 21 Sep 2023 18:55:38 +0200 Message-ID: <20230921165550.50956-2-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20230921165550.50956-1-jacopo.mondi@ideasonboard.com> References: <20230921165550.50956-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 X-Mailman-Approved-At: Thu, 21 Sep 2023 22:51:53 +0200 Subject: [libcamera-devel] [PATCH v5 01/13] documentation: Introduce Camera Sensor Model X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Introduce a documentation page about the 'camera sensor model' implemented by libcamera. The camera sensor model serves to provide to applications a reference description of the processing steps that take place in a camera sensor in order to precisely control the sensor configuration through the forthcoming SensorConfiguration class. Signed-off-by: Jacopo Mondi Reviewed-by: Naushir Patuck Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- .reuse/dep5 | 6 + Documentation/binning.svg | 5053 +++++++++++++++++++++++++ Documentation/camera-sensor-model.rst | 174 + Documentation/index.rst | 1 + Documentation/meson.build | 1 + Documentation/sensor_model.svg | 4870 ++++++++++++++++++++++++ Documentation/skipping.svg | 1720 +++++++++ 7 files changed, 11825 insertions(+) create mode 100644 Documentation/binning.svg create mode 100644 Documentation/camera-sensor-model.rst create mode 100644 Documentation/sensor_model.svg create mode 100644 Documentation/skipping.svg diff --git a/.reuse/dep5 b/.reuse/dep5 index d283a0c9e856..45e7eca3d714 100644 --- a/.reuse/dep5 +++ b/.reuse/dep5 @@ -23,3 +23,9 @@ Files: utils/ipc/mojo Copyright: Copyright 2013-2020 The Chromium Authors. All rights reserved. License: BSD-3-Clause Source: https://chromium.googlesource.com/chromium/src.git/ + +Files: Documentation/binning.svg + Documentation/camera-sensor-model.rst + Documentation/sensor_model.svg +Copyright: Copyright 2023 Ideas On Board Oy +License: CC-BY-SA-4.0 diff --git a/Documentation/binning.svg b/Documentation/binning.svg new file mode 100644 index 000000000000..c6a3b6394acd --- /dev/null +++ b/Documentation/binning.svg @@ -0,0 +1,5053 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/Documentation/camera-sensor-model.rst b/Documentation/camera-sensor-model.rst new file mode 100644 index 000000000000..58bf70e1dc5a --- /dev/null +++ b/Documentation/camera-sensor-model.rst @@ -0,0 +1,174 @@ +.. SPDX-License-Identifier: CC-BY-SA-4.0 + +.. _camera-sensor-model: + +.. todo: Move to Doxygen-generated documentation + +The libcamera camera sensor model +================================= + +libcamera defines an abstract camera sensor model in order to provide +a description of each of the processing steps that result in image data being +sent on the media bus and that form the image stream delivered to applications. + +Applications should use the abstract camera sensor model defined here to +precisely control the operations of the camera sensor. + +The libcamera camera sensor model targets image sensors producing frames in +RAW format, delivered through a MIPI CSI-2 compliant bus implementation. + +The abstract sensor model maps libcamera components to the characteristics and +operations of an image sensor, and serves as a reference to model the libcamera +CameraSensor class and SensorConfiguration classes and operations. + +In order to control the configuration of the camera sensor through the +SensorConfiguration class, applications should understand this model and map it +to the combination of image sensor and kernel driver in use. + +The camera sensor model defined here is based on the *MIPI CCS specification*, +particularly on *Section 8.2 - Image readout* of *Chapter 8 - Video Timings*. + + +Glossary +-------- + +.. glossary:: + + Pixel array + The full grid of pixels, active and inactive ones + + Pixel array active area + The portion(s) of the pixel array that contains valid and readable pixels; + corresponds to the libcamera properties::PixelArrayActiveAreas + + Analog crop rectangle + The portion of the *pixel array active area* which is read out and passed + to further processing stages + + Subsampling + Pixel processing techniques that reduce the image size by binning or by + skipping adjacent pixels + + Digital crop + Crop of the sub-sampled image data before scaling + + Frame output + The frame (image) as output on the media bus by the camera sensor + +Camera sensor model +------------------- + +The abstract sensor model is described in the following diagram. + +.. figure:: sensor_model.svg + + +1. The sensor reads pixels from the *pixel array*. The pixels being read out are + selected by the *analog crop rectangle*. + +2. The pixels can be subsampled to reduce the image size without affecting the + field of view. Two subsampling techniques can be used: + + - Binning: combines adjacent pixels of the same colour by averaging or + summing their values, in the analog domain and/or the digital domain. + + .. figure:: binning.svg + + + - Skipping: skips the read out of a number of adjacent pixels. + + .. figure:: skipping.svg + + +3. The output of the optional sub-sampling stage is then cropped after the + conversion of the analogue pixel values in the digital domain. + +4. The resulting output frame is sent on the media bus by the sensor. + +Camera Sensor configuration parameters +-------------------------------------- + +The libcamera camera sensor model defines parameters that allow users to +control: + +1. The image format bit depth + +2. The size and position of the *Analog crop rectangle* + +3. The subsampling factors used to downscale the pixel array readout data to a + smaller frame size without reducing the image *field of view*. Two + configuration parameters are made available to control the downscaling + factor: + + - binning + A vertical and horizontal binning factor can be specified, the image + will be downscaled in its vertical and horizontal sizes by the specified + factor. + + .. code-block:: + :caption: Definition: The horizontal and vertical binning factors + + horizontal_binning = xBin; + vertical_binning = yBin; + + + - skipping + Skipping reduces the image resolution by skipping the read-out of a number + of adjacent pixels. The skipping factor is specified by the 'increment' + number (number of pixels to 'skip') in the vertical and horizontal + directions and for even and odd rows and columns. + + .. code-block:: + :caption: Definition: The horizontal and vertical skipping factors + + horizontal_skipping = (xOddInc + xEvenInc) / 2 + vertical_skipping = (yOddInc + yEvenInc) / 2 + + Different sensors perform the binning and skipping stages in different + orders. For the sake of computing the final output image size the order of + execution is not relevant. The overall down-scaling factor is obtained by + combining the binning and skipping factors. + + .. code-block:: + :caption: Definition: The total scaling factor (binning + sub-sampling) + + total_horizontal_downscale = horizontal_binning + horizontal_skipping + total_vertical_downscale = vertical_binning + vertical_skipping + + +4. The output size is used to specify any additional cropping on the sub-sampled + frame. + +5. The total line length and frame height (*visibile* pixels + *blankings*) as + sent on the MIPI CSI-2 bus. + +6. The pixel transmission rate on the MIPI CSI-2 bus. + +The above parameters are combined to obtain the following high-level +configurations: + +- **frame output size** + + Obtained by applying a crop to the physical pixel array size in the analog + domain, followed by optional binning and sub-sampling (in any order), + followed by an optional crop step in the output digital domain. + +- **frame rate** + + The combination of the *total frame size*, the image format *bit depth* and + the *pixel rate* of the data sent on the MIPI CSI-2 bus allows to compute the + image stream frame rate. The equation is the well known: + + .. code-block:: + + frame_duration = total_frame_size / pixel_rate + frame_rate = 1 / frame_duration + + + where the *pixel_rate* parameter is the result of the sensor's configuration + of the MIPI CSI-2 bus *(the following formula applies to MIPI CSI-2 when + used on MIPI D-PHY physical protocol layer only)* + + .. code-block:: + + pixel_rate = CSI-2_link_freq * 2 * nr_of_lanes / bits_per_sample diff --git a/Documentation/index.rst b/Documentation/index.rst index 43d8b017d3b4..63fac72d11ed 100644 --- a/Documentation/index.rst +++ b/Documentation/index.rst @@ -23,3 +23,4 @@ Sensor driver requirements Lens driver requirements Python Bindings + Camera Sensor Model diff --git a/Documentation/meson.build b/Documentation/meson.build index b2a5bf15e6ea..7c1502592baa 100644 --- a/Documentation/meson.build +++ b/Documentation/meson.build @@ -63,6 +63,7 @@ endif if sphinx.found() docs_sources = [ + 'camera-sensor-model.rst', 'coding-style.rst', 'conf.py', 'contributing.rst', diff --git a/Documentation/sensor_model.svg b/Documentation/sensor_model.svg new file mode 100644 index 000000000000..02dc55a8018d --- /dev/null +++ b/Documentation/sensor_model.svg @@ -0,0 +1,4870 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Pixel array + Analog crop + Subsampling + Digital crop + Media Bus + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + 1 + + 2 + + 4 + + 3 + + diff --git a/Documentation/skipping.svg b/Documentation/skipping.svg new file mode 100644 index 000000000000..7bef37cfcc64 --- /dev/null +++ b/Documentation/skipping.svg @@ -0,0 +1,1720 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + x_even_inc = 1 + y_even_inc = 1 + 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 + x_odd_inc = 3 + y_odd_inc = 3 + 0 + 1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 + + + + From patchwork Thu Sep 21 16:55:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 19066 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 928E6BE080 for ; Thu, 21 Sep 2023 16:56:06 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 29D6562931; Thu, 21 Sep 2023 18:56:06 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1695315366; bh=SDH2KAbVTR5TNQvMXIpZykP2Hw6R4tHgoHOdK/Qwe8c=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=QTGmfs5biQq5TkiYMhnyXYihGDlmC+JhDbc4H6DugDJ+xxcpr9XZ+TL9bU3wAuq3a nFCbeAMwYPpdf5qeStvp1e5CRmf74HvWnH4P4CkseBqceiegz19dgOX/zNQG9LEC0T IByAIbc2by0slfrMKA/ZKGLLLcEDRPEEiwSL9etsOEJo3g//+qNyp/8WjIIBKvzRMN FwDFADZZ73awQ22/aKBhLBpprj4owtLRSYdMBNugxAXqp8GTYXdcCwbvqLgeoxSZ4B Fq6CfPqLhDaNVGpq3o/gVwaHxzcDbcFnie/FvcyppZhKYUFPGMw4J8+LIXCSSYb73+ /Ks6s0NikPebA== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9B7D3628D8 for ; Thu, 21 Sep 2023 18:56:02 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="s7jcjVMX"; dkim-atps=neutral Received: from uno.lan (93-46-82-201.ip106.fastwebnet.it [93.46.82.201]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 0AA491102; Thu, 21 Sep 2023 18:54:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1695315265; bh=SDH2KAbVTR5TNQvMXIpZykP2Hw6R4tHgoHOdK/Qwe8c=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=s7jcjVMXmgbLfskUF6KPRCKP2p2LC8a1SCbi0QiFWnQndey8iYyTLz8unE/RBZm5c e0nhyqOd7BFTA5dWTgpzu4RuD69X0dB0uoJIQJ/0x2n3l+JioKwXZwnh6Ep8DSI+7Y y0cNDTAOV1pfVbvTJpIQ/b8pfATUbxw8DMr3D/jU= To: libcamera-devel@lists.libcamera.org Date: Thu, 21 Sep 2023 18:55:39 +0200 Message-ID: <20230921165550.50956-3-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20230921165550.50956-1-jacopo.mondi@ideasonboard.com> References: <20230921165550.50956-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 02/13] libcamera: camera: Introduce SensorConfiguration X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Introduce SensorConfiguration in the libcamera API. The SensorConfiguration is part of the CameraConfiguration class and allows applications to control the sensor settings. Signed-off-by: Jacopo Mondi Reviewed-by: Naushir Patuck Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- include/libcamera/camera.h | 27 +++++++ src/libcamera/camera.cpp | 144 +++++++++++++++++++++++++++++++++++++ 2 files changed, 171 insertions(+) diff --git a/include/libcamera/camera.h b/include/libcamera/camera.h index 004bc89455f5..92a948e0f53b 100644 --- a/include/libcamera/camera.h +++ b/include/libcamera/camera.h @@ -9,6 +9,7 @@ #include #include +#include #include #include #include @@ -19,6 +20,7 @@ #include #include +#include #include #include #include @@ -30,6 +32,30 @@ class FrameBufferAllocator; class PipelineHandler; class Request; +class SensorConfiguration +{ +public: + unsigned int bitDepth = 0; + + Rectangle analogCrop; + + struct { + unsigned int binX = 1; + unsigned int binY = 1; + } binning; + + struct { + unsigned int xOddInc = 1; + unsigned int xEvenInc = 1; + unsigned int yOddInc = 1; + unsigned int yEvenInc = 1; + } skipping; + + Size outputSize; + + bool valid() const; +}; + class CameraConfiguration { public: @@ -66,6 +92,7 @@ public: bool empty() const; std::size_t size() const; + std::optional sensorConfig; Transform transform; protected: diff --git a/src/libcamera/camera.cpp b/src/libcamera/camera.cpp index 0eecee766f00..05a47444be89 100644 --- a/src/libcamera/camera.cpp +++ b/src/libcamera/camera.cpp @@ -97,6 +97,16 @@ * implemented in the above order at the hardware level. The libcamera pipeline * handlers translate the pipeline model to the real hardware configuration. * + * \subsection camera-sensor-model Camera Sensor Model + * + * By default, libcamera configures the camera sensor automatically based on the + * configuration of the streams. Applications may instead specify a manual + * configuration for the camera sensor. This allows precise control of the frame + * geometry and frame rate delivered by the sensor. + * + * More details about the camera sensor model implemented by libcamera are + * available in the libcamera camera-sensor-model documentation page. + * * \subsection digital-zoom Digital Zoom * * Digital zoom is implemented as a combination of the cropping and scaling @@ -111,6 +121,127 @@ namespace libcamera { LOG_DECLARE_CATEGORY(Camera) +/** + * \class SensorConfiguration + * \brief Camera sensor configuration + * + * The SensorConfiguration class collects parameters to control the operations + * of the camera sensor, accordingly to the abstract camera sensor model + * implemented by libcamera. + * + * \todo Applications shall fully populate all fields of the + * CameraConfiguration::sensorConfig class members before validating the + * CameraConfiguration. If the SensorConfiguration is not fully populated, or if + * any of its parameters cannot be applied to the sensor in use, the + * CameraConfiguration validation process will fail and return + * CameraConfiguration::Status::Invalid. + * + * Applications that populate the SensorConfiguration class members are + * expected to be highly-specialized applications that know what sensor + * they are operating with and what parameters are valid for the sensor in use. + * + * A detailed description of the abstract camera sensor model implemented by + * libcamera and the description of its configuration parameters is available + * in the libcamera documentation camera-sensor-model file. + */ + +/** + * \var SensorConfiguration::bitDepth + * \brief The sensor image format bit depth + * + * The number of bits (resolution) used to represent a pixel sample. + */ + +/** + * \var SensorConfiguration::analogCrop + * \brief The analog crop rectangle + * + * The selected portion of the active pixel array used to produce the image + * frame. + */ + +/** + * \var SensorConfiguration::binning + * \brief Sensor binning configuration + * + * Refer to the camera-sensor-model documentation for an accurate description + * of the binning operations. Disabled by default. + */ + +/** + * \var SensorConfiguration::binX + * \brief Horizontal binning factor + * + * The horizontal binning factor. Default to 1. + */ + +/** + * \var SensorConfiguration::binY + * \brief Vertical binning factor + * + * The vertical binning factor. Default to 1. + */ + +/** + * \var SensorConfiguration::skipping + * \brief The sensor skipping configuration + * + * Refer to the camera-sensor-model documentation for an accurate description + * of the skipping operations. + * + * If no skipping is performed, all the structure fields should be + * set to 1. Disabled by default. + */ + +/** + * \var SensorConfiguration::xOddInc + * \brief Horizontal increment for odd rows. Default to 1. + */ + +/** + * \var SensorConfiguration::xEvenInc + * \brief Horizontal increment for even rows. Default to 1. + */ + +/** + * \var SensorConfiguration::yOddInc + * \brief Vertical increment for odd columns. Default to 1. + */ + +/** + * \var SensorConfiguration::yEvenInc + * \brief Vertical increment for even columns. Default to 1. + */ + +/** + * \var SensorConfiguration::outputSize + * \brief The frame output (visible) size + * + * The size of the data frame as received by the host processor. + */ + +/** + * \brief Validate the sensor configuration + * + * A sensor configuration is valid if it's fully populated. + * + * \todo For now allow applications to populate the bitDepth and the outputSize + * only as skipping and binnings factors are initialized to 1 and the analog + * crop is ignored. + * + * \return True if the sensor configuration is valid, false otherwise. + */ +bool SensorConfiguration::valid() const +{ + if (bitDepth && binning.binX && binning.binY && + skipping.xOddInc && skipping.yOddInc && + skipping.xEvenInc && skipping.yEvenInc && + !outputSize.isNull()) + return true; + + return false; +} + /** * \class CameraConfiguration * \brief Hold configuration for streams of the camera @@ -391,6 +522,19 @@ CameraConfiguration::Status CameraConfiguration::validateColorSpaces(ColorSpaceF return status; } +/** + * \var CameraConfiguration::sensorConfig + * \brief The camera sensor configuration + * + * The sensorConfig member allows control of the configuration of the camera + * sensor. Refer to the camera-sensor-model documentation and to the + * SensorConfiguration class documentation for details about the sensor + * configuration process. + * + * The camera sensor configuration applies to all streams produced by a camera + * from the same image source. + */ + /** * \var CameraConfiguration::transform * \brief User-specified transform to be applied to the image From patchwork Thu Sep 21 16:55:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 19067 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4B1AEBE080 for ; Thu, 21 Sep 2023 16:56:08 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 21FDF62950; Thu, 21 Sep 2023 18:56:07 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1695315367; bh=NPHplOidD9/bicVixBB26kVaoSilg+Le877fl+rVAf4=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=kjCK6mADB+zUsrv2SC98IeEP4heDjoaX+14asCJ0o6IRtSG+drdWutk2tsvUEJ7dh qFEVx58XasoPPqUQWWugS/1jofiu+1gNVF/EuQb6ZlH4qJAbZ1w3O9x5q768VB0sHm 3OIJx0fUkFsB5Ima0NcCeOzKjbDJmYd8+NvTc5AQtWBDPThoiV57D/0Yxi3FSqicWH 32a+l1pPg8wXtMlv4nAEOL8/98tOLUnSAIUHTtvy+aHMZGc5cDN+bJ1QWiwtmgGVNm ZrxHFkGLHIgjYB09q5/d4DHsN08WODv5VkWmlO8z1Fq4f2dYjomBjEs9TLzu1Sf+gg PeEqdFOeGkYxw== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0A1EB628D8 for ; Thu, 21 Sep 2023 18:56:03 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Gw87q2qG"; dkim-atps=neutral Received: from uno.lan (93-46-82-201.ip106.fastwebnet.it [93.46.82.201]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 6967C2F6C; Thu, 21 Sep 2023 18:54:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1695315265; bh=NPHplOidD9/bicVixBB26kVaoSilg+Le877fl+rVAf4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Gw87q2qG4BeIA1t6JwuI7Mtiyi0keHv/UlrKAPoe7nEdXI2elRvJjd+gyxnb0nKwx wPCfVyfv0Swgzw6+JTZw+omt2YgNuhNJCNRdwNa0M6pY/4OIdt77CsphE3ccWX1Kxg le/Fn5vhB5g+9BUaBOHucfFiSLJvmAAYfdClFLdE= To: libcamera-devel@lists.libcamera.org Date: Thu, 21 Sep 2023 18:55:40 +0200 Message-ID: <20230921165550.50956-4-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20230921165550.50956-1-jacopo.mondi@ideasonboard.com> References: <20230921165550.50956-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 03/13] libcamera: camera_sensor: Support SensorConfiguration X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a class function to the CameraSensor class to apply a full configuration to the sensor. The configuration shall be fully populated and shall apply without modifications to the sensor. Signed-off-by: Jacopo Mondi Reviewed-by: Naushir Patuck Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- include/libcamera/internal/camera_sensor.h | 5 ++ src/libcamera/camera_sensor.cpp | 86 ++++++++++++++++++++++ 2 files changed, 91 insertions(+) diff --git a/include/libcamera/internal/camera_sensor.h b/include/libcamera/internal/camera_sensor.h index 02c77ab037da..06791c3c6425 100644 --- a/include/libcamera/internal/camera_sensor.h +++ b/include/libcamera/internal/camera_sensor.h @@ -29,6 +29,7 @@ namespace libcamera { class BayerFormat; class CameraLens; class MediaEntity; +class SensorConfiguration; struct CameraSensorProperties; @@ -58,6 +59,10 @@ public: Transform transform = Transform::Identity); int tryFormat(V4L2SubdeviceFormat *format) const; + int applyConfiguration(const SensorConfiguration &config, + Transform transform = Transform::Identity, + V4L2SubdeviceFormat *sensorFormat = nullptr); + const ControlInfoMap &controls() const; ControlList getControls(const std::vector &ids); int setControls(ControlList *ctrls); diff --git a/src/libcamera/camera_sensor.cpp b/src/libcamera/camera_sensor.cpp index f3a5aa37149f..323081d25ce7 100644 --- a/src/libcamera/camera_sensor.cpp +++ b/src/libcamera/camera_sensor.cpp @@ -15,6 +15,7 @@ #include #include +#include #include #include @@ -822,6 +823,91 @@ int CameraSensor::tryFormat(V4L2SubdeviceFormat *format) const V4L2Subdevice::Whence::TryFormat); } +/** + * \brief Apply a sensor configuration to the camera sensor + * \param[in] config The sensor configuration + * \param[in] transform The transform to be applied on the sensor. + * Defaults to Identity + * \param[out] sensorFormat Optional output parameter where the format + * actually applied to the sensor is returned to the caller + * + * Apply to the camera sensor the configuration \a config. + * + * \todo The configuration shall be fully populated and if any of the fields + * specified cannot be applied exactly, an error code is returned. + * + * \return 0 if \a config unpopulated or is applied correctly to the camera + * sensor, a negative error code otherwise + */ +int CameraSensor::applyConfiguration(const SensorConfiguration &config, + Transform transform, + V4L2SubdeviceFormat *sensorFormat) +{ + if (!config.valid()) { + LOG(CameraSensor, Error) << "Invalid sensor configuration"; + return -EINVAL; + } + + std::vector filteredCodes; + std::copy_if(mbusCodes_.begin(), mbusCodes_.end(), + std::back_inserter(filteredCodes), + [&config](unsigned int mbusCode) { + BayerFormat bayer = BayerFormat::fromMbusCode(mbusCode); + if (bayer.bitDepth == config.bitDepth) + return true; + return false; + }); + if (filteredCodes.empty()) { + LOG(CameraSensor, Error) + << "Cannot find any format with bit depth " + << config.bitDepth; + return -EINVAL; + } + + /* + * Compute the sensor's data frame size by applying the cropping + * rectangle, subsampling and output crop to the sensor's pixel array + * size. + * + * \todo The actual size computation is for now ignored and only the + * output size is considered. This implies that resolutions obtained + * with two different cropping/subsampling will look identical and + * only the first found one will be considered. + */ + V4L2SubdeviceFormat subdevFormat = {}; + for (unsigned int code : filteredCodes) { + for (const Size &size : sizes(code)) { + if (size.width != config.outputSize.width || + size.height != config.outputSize.height) + continue; + + subdevFormat.mbus_code = code; + subdevFormat.size = size; + break; + } + } + if (!subdevFormat.mbus_code) { + LOG(CameraSensor, Error) << "Invalid output size in sensor configuration"; + return -EINVAL; + } + + int ret = setFormat(&subdevFormat, transform); + if (ret) + return ret; + + /* + * Return to the caller the format actually applied to the sensor. + * This is relevant if transform has changed the bayer pattern order. + */ + if (sensorFormat) + *sensorFormat = subdevFormat; + + /* \todo Handle AnalogCrop. Most sensors do not support set_selection */ + /* \todo Handle scaling in the digital domain. */ + + return 0; +} + /** * \brief Retrieve the supported V4L2 controls and their information * From patchwork Thu Sep 21 16:55:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 19068 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A22FBC326C for ; Thu, 21 Sep 2023 16:56:09 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A887A62947; Thu, 21 Sep 2023 18:56:07 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1695315367; bh=qXGx6RyMquUh9GOu6AN6eszqs/7WzI1BqcrEed/s/+k=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=WWkYuKB+16ydkCkm19t9lz+VGlTczRF1sDSn8zpdK09CSCO+ZkBQmbbQLhyAB+yrJ 1/eR9sOtxPT2QaF5pAm6gfNp6uyya+AG27Jpwbqf8Yh05LpbjZv2D+Ks9G4QRnS+sy +Dgy///nyGwLpkxVEPAi6bjClAot2vjtab0CoeDZrXjuPpiqP6QMagEoCsuUxJbFdy TFTUUUWip/V4L7tlpJPmh8BwzYoo7GhDwwvVHsNtkVg+8MZP7XHyNR1jM+I6+2hbGj QiEn6b7QrmvXvt8Zjy0vYJ/2KHHgVnmYkkXd6ri8Z2XbORXQch4BxBRpRghbzhzoOt bkU6DXU0bPGWQ== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6665F628D8 for ; Thu, 21 Sep 2023 18:56:03 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="L7uHlzuV"; dkim-atps=neutral Received: from uno.lan (93-46-82-201.ip106.fastwebnet.it [93.46.82.201]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id CCDDD8201; Thu, 21 Sep 2023 18:54:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1695315266; bh=qXGx6RyMquUh9GOu6AN6eszqs/7WzI1BqcrEed/s/+k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=L7uHlzuVchT42s2S6oRuGCVlg1tAiizOoVBQ3uJcYGxEgGCUkfOrUxguyn3wgoz2t nf1shEEH6EgHQlbeN/EMDGPek4M1vRmalq8TRLv3+ZfVEoCc00A85VY7fovEI7JGIv 9d0LdZTqR28pBGwZPCSf/qzMGnImJ4J8NdFU31zk= To: libcamera-devel@lists.libcamera.org Date: Thu, 21 Sep 2023 18:55:41 +0200 Message-ID: <20230921165550.50956-5-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20230921165550.50956-1-jacopo.mondi@ideasonboard.com> References: <20230921165550.50956-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 04/13] libcamera: rpi: Handle SensorConfiguration X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Handle the SensorConfiguration provided by the application in the pipeline validate() and configure() call chains. During validation, first make sure SensorConfiguration is valid, then handle it to compute the sensor format. For the VC4 platform where the RAW stream follows the sensor's configuration adjust the RAW stream configuration to match the sensor configuration. Reviewed-by: Naushir Patuck Reviewed-by: Kieran Bingham Signed-off-by: Naushir Patuck Reviewed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi --- .../pipeline/rpi/common/pipeline_base.cpp | 66 ++++++++++++++++--- .../pipeline/rpi/common/pipeline_base.h | 4 +- src/libcamera/pipeline/rpi/vc4/vc4.cpp | 28 +++++++- 3 files changed, 84 insertions(+), 14 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 51fa1bbf9aa9..c02ceb65cae8 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -180,6 +180,15 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() if (config_.empty()) return Invalid; + /* + * Make sure that if a sensor configuration has been requested it + * is valid. + */ + if (sensorConfig && !sensorConfig->valid()) { + LOG(RPI, Error) << "Invalid sensor configuration request"; + return Invalid; + } + status = validateColorSpaces(ColorSpaceFlag::StreamsShareColorSpace); /* @@ -207,19 +216,43 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() std::sort(outStreams.begin(), outStreams.end(), [](auto &l, auto &r) { return l.cfg->size > r.cfg->size; }); - /* Compute the sensor configuration. */ - unsigned int bitDepth = defaultRawBitDepth; - if (!rawStreams.empty()) { + /* Compute the sensor's format then do any platform specific fixups. */ + unsigned int bitDepth; + Size sensorSize; + + if (sensorConfig) { + /* Use the application provided sensor configuration. */ + bitDepth = sensorConfig->bitDepth; + sensorSize = sensorConfig->outputSize; + } else if (!rawStreams.empty()) { + /* Use the RAW stream format and size. */ BayerFormat bayerFormat = BayerFormat::fromPixelFormat(rawStreams[0].cfg->pixelFormat); bitDepth = bayerFormat.bitDepth; + sensorSize = rawStreams[0].cfg->size; + } else { + bitDepth = defaultRawBitDepth; + sensorSize = outStreams[0].cfg->size; } - sensorFormat_ = data_->findBestFormat(rawStreams.empty() ? outStreams[0].cfg->size - : rawStreams[0].cfg->size, - bitDepth); + sensorFormat_ = data_->findBestFormat(sensorSize, bitDepth); + + /* + * If a sensor configuration has been requested, it should apply + * without modifications. + */ + if (sensorConfig) { + BayerFormat bayer = BayerFormat::fromMbusCode(sensorFormat_.mbus_code); + + if (bayer.bitDepth != sensorConfig->bitDepth || + sensorFormat_.size != sensorConfig->outputSize) { + LOG(RPI, Error) << "Invalid sensor configuration: " + << "bitDepth/size mismatch"; + return Invalid; + } + } /* Do any platform specific fixups. */ - status = data_->platformValidate(rawStreams, outStreams); + status = data_->platformValidate(this, rawStreams, outStreams); if (status == Invalid) return Invalid; @@ -467,12 +500,25 @@ int PipelineHandlerBase::configure(Camera *camera, CameraConfiguration *config) std::sort(ispStreams.begin(), ispStreams.end(), [](auto &l, auto &r) { return l.cfg->size > r.cfg->size; }); - /* Apply the format on the sensor with any cached transform. */ + /* + * Apply the format on the sensor with any cached transform. + * + * If the application has provided a sensor configuration apply it + * instead of just applying a format. + */ const RPiCameraConfiguration *rpiConfig = static_cast(config); - V4L2SubdeviceFormat sensorFormat = rpiConfig->sensorFormat_; + V4L2SubdeviceFormat sensorFormat; - ret = data->sensor_->setFormat(&sensorFormat, rpiConfig->combinedTransform_); + if (rpiConfig->sensorConfig) { + ret = data->sensor_->applyConfiguration(*rpiConfig->sensorConfig, + rpiConfig->combinedTransform_, + &sensorFormat); + } else { + sensorFormat = rpiConfig->sensorFormat_; + ret = data->sensor_->setFormat(&sensorFormat, + rpiConfig->combinedTransform_); + } if (ret) return ret; diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index dbabc61ea48c..81b2b7d2f4d1 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -42,6 +42,7 @@ namespace RPi { /* Map of mbus codes to supported sizes reported by the sensor. */ using SensorFormats = std::map>; +class RPiCameraConfiguration; class CameraData : public Camera::Private { public: @@ -72,7 +73,8 @@ public: V4L2VideoDevice *dev; }; - virtual CameraConfiguration::Status platformValidate(std::vector &rawStreams, + virtual CameraConfiguration::Status platformValidate(RPiCameraConfiguration *rpiConfig, + std::vector &rawStreams, std::vector &outStreams) const = 0; virtual int platformConfigure(const V4L2SubdeviceFormat &sensorFormat, std::optional packing, diff --git a/src/libcamera/pipeline/rpi/vc4/vc4.cpp b/src/libcamera/pipeline/rpi/vc4/vc4.cpp index 018cf4881d0e..2670eb8c4bbc 100644 --- a/src/libcamera/pipeline/rpi/vc4/vc4.cpp +++ b/src/libcamera/pipeline/rpi/vc4/vc4.cpp @@ -65,7 +65,8 @@ public: { } - CameraConfiguration::Status platformValidate(std::vector &rawStreams, + CameraConfiguration::Status platformValidate(RPi::RPiCameraConfiguration *rpiConfig, + std::vector &rawStreams, std::vector &outStreams) const override; int platformPipelineConfigure(const std::unique_ptr &root) override; @@ -394,7 +395,8 @@ int PipelineHandlerVc4::platformRegister(std::unique_ptr &camer return 0; } -CameraConfiguration::Status Vc4CameraData::platformValidate(std::vector &rawStreams, +CameraConfiguration::Status Vc4CameraData::platformValidate(RPi::RPiCameraConfiguration *rpiConfig, + std::vector &rawStreams, std::vector &outStreams) const { CameraConfiguration::Status status = CameraConfiguration::Status::Valid; @@ -405,9 +407,27 @@ CameraConfiguration::Status Vc4CameraData::platformValidate(std::vectorsensorFormat_.mbus_code); + + /* Handle flips to make sure to match the RAW stream format. */ + if (flipsAlterBayerOrder_) + rawBayer = rawBayer.transform(rpiConfig->combinedTransform_); + PixelFormat rawFormat = rawBayer.toPixelFormat(); + + if (rawStream->pixelFormat != rawFormat || + rawStream->size != rpiConfig->sensorFormat_.size) { + rawStream->pixelFormat = rawFormat; + rawStream->size = rpiConfig->sensorFormat_.size; + + status = CameraConfiguration::Adjusted; + } + } + /* * For the two ISP outputs, one stream must be equal or smaller than the * other in all dimensions. @@ -417,6 +437,8 @@ CameraConfiguration::Status Vc4CameraData::platformValidate(std::vectorsize.width, outStreams[0].cfg->size.width); size.height = std::min(outStreams[i].cfg->size.height, From patchwork Thu Sep 21 16:55:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 19069 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 83AA4BE080 for ; Thu, 21 Sep 2023 16:56:10 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A3BD26294B; Thu, 21 Sep 2023 18:56:08 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1695315368; bh=okpaM9IVcdE0ATLSllywNITjoe6yEZ6SVs+WCpQmepE=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=JqlOPQJvIeVNHc42iP62eIi5Dq3u1HwrzgcJQB8ZZPl2HUWh1/PBzBqQta33hhZZu MiqAQcbAL0QoZePERR1i/nDvHk8psI+nrBl6sBO4qmfQNo5zfn9npV6XNuKLJ4vPye uLCY6AqRbwchSXDdja53VJeQwVVfJfFAUfk3bJHCFwGapYsxbxJmt2erIqzhHi7TFX qN3H9HabkliwRXfsdCk1i1KwgoQo8zSWjyS3avpvi5HUpQGFDEuu7sMO6+AGEDeo9w pPQkMygSrytY/4nQ9sdM7mXq9mBBbGtyniUzcy/ajmJ1/x4P/kxEJMtaNDjqqp/wuy Y9k0s/lLD+axw== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C27FC62949 for ; Thu, 21 Sep 2023 18:56:03 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="uhBzdXUI"; dkim-atps=neutral Received: from uno.lan (93-46-82-201.ip106.fastwebnet.it [93.46.82.201]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 384FB1102; Thu, 21 Sep 2023 18:54:26 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1695315266; bh=okpaM9IVcdE0ATLSllywNITjoe6yEZ6SVs+WCpQmepE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=uhBzdXUIEi0N7f3QWFfIHqy1AnRE0bolP3fv4f+vG5cCgplq0gyG5EQmYsplv0n6v uOcur21SXd1dQvOcA02NaIA0+KvKjsKs21LfH93O8Vq4oju0uz6yG3r6FgCge4TR9C IZwyvk4KlHdtqWLUrc8fM59rWyixpBntTI4Gx9k4= To: libcamera-devel@lists.libcamera.org Date: Thu, 21 Sep 2023 18:55:42 +0200 Message-ID: <20230921165550.50956-6-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20230921165550.50956-1-jacopo.mondi@ideasonboard.com> References: <20230921165550.50956-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 05/13] libcamera: rpi: Allow platformValidate() to adjust format strides X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Naushir Patuck Propagate any changes to the format stride done by platformValidate(). The stride value may be adjusted for performace reasons. Signed-off-by: Naushir Patuck Reviewed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/rpi/common/pipeline_base.cpp | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index c02ceb65cae8..502cdff0051b 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -309,6 +309,17 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() V4L2DeviceFormat format; format.fourcc = out.dev->toV4L2PixelFormat(cfg.pixelFormat); format.size = cfg.size; + + /* + * platformValidate may have worked out the correct stride so we + * must pass it in. This also needs the planesCount to be set + * correctly or the stride will be ignored. + */ + const PixelFormat &pixFormat = format.fourcc.toPixelFormat(); + const PixelFormatInfo &info = PixelFormatInfo::info(pixFormat); + format.planesCount = info.numPlanes(); + format.planes[0].bpl = cfg.stride; + /* We want to send the associated YCbCr info through to the driver. */ format.colorSpace = yuvColorSpace_; From patchwork Thu Sep 21 16:55:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 19070 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6BBB6C326C for ; Thu, 21 Sep 2023 16:56:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4D5A062959; Thu, 21 Sep 2023 18:56:09 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1695315369; bh=w9AusMuCSjpYZIrUzlAUxsdXUujTwmIu5v1OblYSQOc=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=awWrl8MT0hzpbVMvcxpfrE7YxARI1GtqvmDEirszsNPkd6yWvh5S9X1SQPs8j/Cyn oGVUG4l/n02zBNFuixEhfG8wF434E1olMfpTz6eyKDTMQr91mcnvET1owO7C+J2PMS eRLIXpDpOquUeqLlx6Cfubs5rX+KOM34ZY2E6Tn01mFoWK/G6axycD+oVJ8u0JcDUK VG6+N2d1Ec3m8qpQAAl+MNq21CBDlB7scJ4bAhYfPb3cE+6g8kIqVUU/z0Rg1X0BcL M1bGt5BuRdh9kgKBHkP9CxMFzqXNBSO3po65XmU8w50GM5qTWg0KrYjqy1d/lTc1BT 2jfdzbwKYCOdA== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1956762944 for ; Thu, 21 Sep 2023 18:56:04 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="G8RxOdQX"; dkim-atps=neutral Received: from uno.lan (93-46-82-201.ip106.fastwebnet.it [93.46.82.201]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 8E5E71257; Thu, 21 Sep 2023 18:54:26 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1695315266; bh=w9AusMuCSjpYZIrUzlAUxsdXUujTwmIu5v1OblYSQOc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=G8RxOdQXyLHw5HNnanDJG3UGwwOLUD6BHtqVHNeemBWrYc4i4MRkCBVe50FX3/NfI e/wxVYo+kkn7774b0KzvzrwW6be0/HK8dgvo8quZYIj421P292Ugvx3eS/E3jC1M9x v5CXopYK2tKdtL+bH4XelmPAedqZkCZRgb0qwVDY= To: libcamera-devel@lists.libcamera.org Date: Thu, 21 Sep 2023 18:55:43 +0200 Message-ID: <20230921165550.50956-7-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20230921165550.50956-1-jacopo.mondi@ideasonboard.com> References: <20230921165550.50956-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 06/13] libcamera: rpi: Fix wrong comment indentation X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The closing line of a comment block was aligned with spaces and not tabs. Fix it. Signed-off-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- src/libcamera/pipeline/rpi/common/pipeline_base.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index 81b2b7d2f4d1..3e7c487fb0bf 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -273,7 +273,7 @@ private: * Store the colour spaces that all our streams will have. RGB format streams * will have the same colorspace as YUV streams, with YCbCr field cleared and * range set to full. - */ + */ std::optional yuvColorSpace_; std::optional rgbColorSpace_; }; From patchwork Thu Sep 21 16:55:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 19071 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 17077C32B0 for ; Thu, 21 Sep 2023 16:56:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C733862951; Thu, 21 Sep 2023 18:56:09 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1695315369; bh=KYw8rMgmbFYHbYUts551PNuej39xt6wtjn+gIchmE9w=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=MQ8LdeCg0nAKOv4qH0iQ1GVR7o0l7IwQMOtma1dcnmTlUAm/aYaR295vdKabPQ3VH +gc2Pqn6QCSzinzHenfMaQ02mgyi4sovzFZKLueJIrltJ5UiL/VgfOtdOGBUc+OUOb 04fjB4QSZPCO9hh/Yb7TWn6iqip1VC2ZWn/ryi/NMEfVGxAf0PzwfnOh4mfHuTo7nA RAXnvJgxkDUmiSWrc/M1iJVoPWFgVsZD1dAHK9Bm3uMb4F3sL9RcPUEh4C3kQCN7fl CJmDkx6jMy1jEq3gs5BBpk8S+cawABPz5qLAf5eveNQga1RdzqeENdj5tbFguh9lmK viOVUWbaH1PEQ== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7397B6294B for ; Thu, 21 Sep 2023 18:56:04 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Gi4Qv4lW"; dkim-atps=neutral Received: from uno.lan (93-46-82-201.ip106.fastwebnet.it [93.46.82.201]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E48668201; Thu, 21 Sep 2023 18:54:26 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1695315267; bh=KYw8rMgmbFYHbYUts551PNuej39xt6wtjn+gIchmE9w=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Gi4Qv4lWd3c/Bot7TWbLbPWVWfdb7+uABp5wigsu6/O5a9V6BpesHnAhrEVqo0ufu PLe0C/cpXI8/xilQgIFjOppbtgs03xy1S9KqJUmQR2pzXsBBP58WxrU+5JsW3P4R6H Sq7odYE3vmo6S+262B3oqjjoXZHyHLAFlONX2y0Y= To: libcamera-devel@lists.libcamera.org Date: Thu, 21 Sep 2023 18:55:44 +0200 Message-ID: <20230921165550.50956-8-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20230921165550.50956-1-jacopo.mondi@ideasonboard.com> References: <20230921165550.50956-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 07/13] libcamera: rpi: Make isRaw/isYuv/isRgb static functions X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Naushir Patuck Move the existing isRaw()/isYuv()/isRgb()into a static function of PipelineHandlerBase. This will allow then to be shared with the pipeline handler derived class. Signed-off-by: Naushir Patuck Signed-off-by: Jacopo Mondi Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- .../pipeline/rpi/common/pipeline_base.cpp | 52 +++++++++---------- .../pipeline/rpi/common/pipeline_base.h | 4 ++ 2 files changed, 30 insertions(+), 26 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 502cdff0051b..f6f6185cbfef 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -37,12 +37,6 @@ namespace { constexpr unsigned int defaultRawBitDepth = 12; -bool isRaw(const PixelFormat &pixFmt) -{ - /* This test works for both Bayer and raw mono formats. */ - return BayerFormat::fromPixelFormat(pixFmt).isValid(); -} - PixelFormat mbusCodeToPixelFormat(unsigned int mbus_code, BayerFormat::Packing packingReq) { @@ -91,22 +85,6 @@ std::optional findValidColorSpace(const ColorSpace &colourSpace) return std::nullopt; } -bool isRgb(const PixelFormat &pixFmt) -{ - const PixelFormatInfo &info = PixelFormatInfo::info(pixFmt); - return info.colourEncoding == PixelFormatInfo::ColourEncodingRGB; -} - -bool isYuv(const PixelFormat &pixFmt) -{ - /* The code below would return true for raw mono streams, so weed those out first. */ - if (isRaw(pixFmt)) - return false; - - const PixelFormatInfo &info = PixelFormatInfo::info(pixFmt); - return info.colourEncoding == PixelFormatInfo::ColourEncodingYUV; -} - } /* namespace */ /* @@ -129,7 +107,7 @@ CameraConfiguration::Status RPiCameraConfiguration::validateColorSpaces([[maybe_ for (auto cfg : config_) { /* First fix up raw streams to have the "raw" colour space. */ - if (isRaw(cfg.pixelFormat)) { + if (PipelineHandlerBase::isRaw(cfg.pixelFormat)) { /* If there was no value here, that doesn't count as "adjusted". */ if (cfg.colorSpace && cfg.colorSpace != ColorSpace::Raw) status = Adjusted; @@ -156,13 +134,13 @@ CameraConfiguration::Status RPiCameraConfiguration::validateColorSpaces([[maybe_ if (cfg.colorSpace == ColorSpace::Raw) continue; - if (isYuv(cfg.pixelFormat) && cfg.colorSpace != yuvColorSpace_) { + if (PipelineHandlerBase::isYuv(cfg.pixelFormat) && cfg.colorSpace != yuvColorSpace_) { /* Again, no value means "not adjusted". */ if (cfg.colorSpace) status = Adjusted; cfg.colorSpace = yuvColorSpace_; } - if (isRgb(cfg.pixelFormat) && cfg.colorSpace != rgbColorSpace_) { + if (PipelineHandlerBase::isRgb(cfg.pixelFormat) && cfg.colorSpace != rgbColorSpace_) { /* Be nice, and let the YUV version count as non-adjusted too. */ if (cfg.colorSpace && cfg.colorSpace != yuvColorSpace_) status = Adjusted; @@ -203,7 +181,7 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() std::vector rawStreams, outStreams; for (const auto &[index, cfg] : utils::enumerate(config_)) { - if (isRaw(cfg.pixelFormat)) + if (PipelineHandlerBase::isRaw(cfg.pixelFormat)) rawStreams.emplace_back(index, &cfg); else outStreams.emplace_back(index, &cfg); @@ -351,6 +329,28 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() return status; } +bool PipelineHandlerBase::isRgb(const PixelFormat &pixFmt) +{ + const PixelFormatInfo &info = PixelFormatInfo::info(pixFmt); + return info.colourEncoding == PixelFormatInfo::ColourEncodingRGB; +} + +bool PipelineHandlerBase::isYuv(const PixelFormat &pixFmt) +{ + /* The code below would return true for raw mono streams, so weed those out first. */ + if (PipelineHandlerBase::isRaw(pixFmt)) + return false; + + const PixelFormatInfo &info = PixelFormatInfo::info(pixFmt); + return info.colourEncoding == PixelFormatInfo::ColourEncodingYUV; +} + +bool PipelineHandlerBase::isRaw(const PixelFormat &pixFmt) +{ + /* This test works for both Bayer and raw mono formats. */ + return BayerFormat::fromPixelFormat(pixFmt).isValid(); +} + V4L2DeviceFormat PipelineHandlerBase::toV4L2DeviceFormat(const V4L2VideoDevice *dev, const V4L2SubdeviceFormat &format, BayerFormat::Packing packingReq) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index 3e7c487fb0bf..135b74392140 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -213,6 +213,10 @@ public: { } + static bool isRgb(const PixelFormat &pixFmt); + static bool isYuv(const PixelFormat &pixFmt); + static bool isRaw(const PixelFormat &pixFmt); + static V4L2DeviceFormat toV4L2DeviceFormat(const V4L2VideoDevice *dev, const V4L2SubdeviceFormat &format, BayerFormat::Packing packingReq); From patchwork Thu Sep 21 16:55:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 19072 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8F1C4C32B1 for ; Thu, 21 Sep 2023 16:56:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0E61D6295D; Thu, 21 Sep 2023 18:56:11 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1695315371; bh=B3JHUkJABLI1JS0ycBDxHndMePy5rJNFEtVnPjofGvg=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=NWgn/W7IvBpHIFG+9LBwk/kxT5PG1RokG1Iqu0a1opcbKBgLblO8se+FRe1EEWWiJ nP6JuaJXBvfmQh/5SF9aZ0CBPE98SnM5CVPzxNtG9Bv+3hVNhKM3OawPVvAabXDuTS VyJqOUoirTiyQkwTOxpEi40Xr0CgwSLIR/BYsgHIkBNcsa78qkXvTPnqlUFwUisS7q 4BayyH31R6ZskgdsRPsb11UIAgwCxDuJLgYFCTART/Dd7eZqo/9QdLGcKKfsFAo6Zr T2i5yDA2/zV+mOj1sjPaTo09bs3ejTvyCzOs9bRgsL/dNl4ugkJ1R15ODc1yvK1z28 7WN7i3iVXtatg== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D04BE628D8 for ; Thu, 21 Sep 2023 18:56:04 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="XwQYD3k8"; dkim-atps=neutral Received: from uno.lan (93-46-82-201.ip106.fastwebnet.it [93.46.82.201]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 479731257; Thu, 21 Sep 2023 18:54:27 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1695315267; bh=B3JHUkJABLI1JS0ycBDxHndMePy5rJNFEtVnPjofGvg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XwQYD3k8ooPlizOIDLpnznXbKOsmYXqq9wM0AJ26IvREPYZJy3U9DC+KkT3zLi/hW vNz267e5swKOd+dTIzyvPuEE9IMY26rR4uLyTlUvSHTe6p6iG/BHrINw3Avpjf31su XyR3k7rZvlM3YKhKQDVAMBN4mBLci06Y7/4r/Vrk= To: libcamera-devel@lists.libcamera.org Date: Thu, 21 Sep 2023 18:55:45 +0200 Message-ID: <20230921165550.50956-9-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20230921165550.50956-1-jacopo.mondi@ideasonboard.com> References: <20230921165550.50956-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 08/13] libcamera: rpi: Cache rawStreams and outStreams X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Naushir Patuck Currently, the stream configuration is stored in two vectors, rawStreams and outStreams for convenience. However, these vectors are constructed in both platformValidate() and platformConfigure(). This change caches these vectors in the RPiCameraConfiguration class to construct them only once in platformValidate(). Pass a pointer to the current configuration to platformValidate() and platformConfigure() so that they can access the streams vectors. Signed-off-by: Naushir Patuck Reviewed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi --- .../pipeline/rpi/common/pipeline_base.cpp | 50 +++++++------------ .../pipeline/rpi/common/pipeline_base.h | 42 ++++++++-------- src/libcamera/pipeline/rpi/vc4/vc4.cpp | 20 ++++---- 3 files changed, 48 insertions(+), 64 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index f6f6185cbfef..e36bdce229b6 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -179,19 +179,21 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() if (transform != requestedTransform) status = Adjusted; - std::vector rawStreams, outStreams; + rawStreams_.clear(); + outStreams_.clear(); + for (const auto &[index, cfg] : utils::enumerate(config_)) { if (PipelineHandlerBase::isRaw(cfg.pixelFormat)) - rawStreams.emplace_back(index, &cfg); + rawStreams_.emplace_back(index, &cfg); else - outStreams.emplace_back(index, &cfg); + outStreams_.emplace_back(index, &cfg); } /* Sort the streams so the highest resolution is first. */ - std::sort(rawStreams.begin(), rawStreams.end(), + std::sort(rawStreams_.begin(), rawStreams_.end(), [](auto &l, auto &r) { return l.cfg->size > r.cfg->size; }); - std::sort(outStreams.begin(), outStreams.end(), + std::sort(outStreams_.begin(), outStreams_.end(), [](auto &l, auto &r) { return l.cfg->size > r.cfg->size; }); /* Compute the sensor's format then do any platform specific fixups. */ @@ -202,14 +204,14 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() /* Use the application provided sensor configuration. */ bitDepth = sensorConfig->bitDepth; sensorSize = sensorConfig->outputSize; - } else if (!rawStreams.empty()) { + } else if (!rawStreams_.empty()) { /* Use the RAW stream format and size. */ - BayerFormat bayerFormat = BayerFormat::fromPixelFormat(rawStreams[0].cfg->pixelFormat); + BayerFormat bayerFormat = BayerFormat::fromPixelFormat(rawStreams_[0].cfg->pixelFormat); bitDepth = bayerFormat.bitDepth; - sensorSize = rawStreams[0].cfg->size; + sensorSize = rawStreams_[0].cfg->size; } else { bitDepth = defaultRawBitDepth; - sensorSize = outStreams[0].cfg->size; + sensorSize = outStreams_[0].cfg->size; } sensorFormat_ = data_->findBestFormat(sensorSize, bitDepth); @@ -230,12 +232,12 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() } /* Do any platform specific fixups. */ - status = data_->platformValidate(this, rawStreams, outStreams); + status = data_->platformValidate(this); if (status == Invalid) return Invalid; /* Further fixups on the RAW streams. */ - for (auto &raw : rawStreams) { + for (auto &raw : rawStreams_) { StreamConfiguration &cfg = config_.at(raw.index); V4L2DeviceFormat rawFormat; @@ -273,7 +275,7 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() } /* Further fixups on the ISP output streams. */ - for (auto &out : outStreams) { + for (auto &out : outStreams_) { StreamConfiguration &cfg = config_.at(out.index); PixelFormat &cfgPixFmt = cfg.pixelFormat; V4L2VideoDevice::Formats fmts = out.dev->formats(); @@ -493,24 +495,6 @@ int PipelineHandlerBase::configure(Camera *camera, CameraConfiguration *config) for (auto const stream : data->streams_) stream->clearFlags(StreamFlag::External); - std::vector rawStreams, ispStreams; - - for (unsigned i = 0; i < config->size(); i++) { - StreamConfiguration *cfg = &config->at(i); - - if (isRaw(cfg->pixelFormat)) - rawStreams.emplace_back(i, cfg); - else - ispStreams.emplace_back(i, cfg); - } - - /* Sort the streams so the highest resolution is first. */ - std::sort(rawStreams.begin(), rawStreams.end(), - [](auto &l, auto &r) { return l.cfg->size > r.cfg->size; }); - - std::sort(ispStreams.begin(), ispStreams.end(), - [](auto &l, auto &r) { return l.cfg->size > r.cfg->size; }); - /* * Apply the format on the sensor with any cached transform. * @@ -535,9 +519,9 @@ int PipelineHandlerBase::configure(Camera *camera, CameraConfiguration *config) /* Use the user requested packing/bit-depth. */ std::optional packing; - if (!rawStreams.empty()) { + if (!rpiConfig->rawStreams_.empty()) { BayerFormat bayerFormat = - BayerFormat::fromPixelFormat(rawStreams[0].cfg->pixelFormat); + BayerFormat::fromPixelFormat(rpiConfig->rawStreams_[0].cfg->pixelFormat); packing = bayerFormat.packing; } @@ -545,7 +529,7 @@ int PipelineHandlerBase::configure(Camera *camera, CameraConfiguration *config) * Platform specific internal stream configuration. This also assigns * external streams which get configured below. */ - ret = data->platformConfigure(sensorFormat, packing, rawStreams, ispStreams); + ret = data->platformConfigure(sensorFormat, packing, rpiConfig); if (ret) return ret; diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index 135b74392140..1a3a13dbb6cf 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -57,29 +57,10 @@ public: { } - struct StreamParams { - StreamParams() - : index(0), cfg(nullptr), dev(nullptr) - { - } - - StreamParams(unsigned int index_, StreamConfiguration *cfg_) - : index(index_), cfg(cfg_), dev(nullptr) - { - } - - unsigned int index; - StreamConfiguration *cfg; - V4L2VideoDevice *dev; - }; - - virtual CameraConfiguration::Status platformValidate(RPiCameraConfiguration *rpiConfig, - std::vector &rawStreams, - std::vector &outStreams) const = 0; + virtual CameraConfiguration::Status platformValidate(RPiCameraConfiguration *rpiConfig) const = 0; virtual int platformConfigure(const V4L2SubdeviceFormat &sensorFormat, std::optional packing, - std::vector &rawStreams, - std::vector &outStreams) = 0; + const RPiCameraConfiguration *rpiConfig) = 0; virtual void platformStart() = 0; virtual void platformStop() = 0; @@ -270,6 +251,25 @@ public: /* The sensor format computed in validate() */ V4L2SubdeviceFormat sensorFormat_; + struct StreamParams { + StreamParams() + : index(0), cfg(nullptr), dev(nullptr) + { + } + + StreamParams(unsigned int index_, StreamConfiguration *cfg_) + : index(index_), cfg(cfg_), dev(nullptr) + { + } + + unsigned int index; + StreamConfiguration *cfg; + V4L2VideoDevice *dev; + }; + + std::vector rawStreams_; + std::vector outStreams_; + private: const CameraData *data_; diff --git a/src/libcamera/pipeline/rpi/vc4/vc4.cpp b/src/libcamera/pipeline/rpi/vc4/vc4.cpp index 2670eb8c4bbc..2308577a613b 100644 --- a/src/libcamera/pipeline/rpi/vc4/vc4.cpp +++ b/src/libcamera/pipeline/rpi/vc4/vc4.cpp @@ -25,6 +25,7 @@ namespace libcamera { LOG_DECLARE_CATEGORY(RPI) using StreamFlag = RPi::Stream::StreamFlag; +using StreamParams = RPi::RPiCameraConfiguration::StreamParams; namespace { @@ -65,9 +66,7 @@ public: { } - CameraConfiguration::Status platformValidate(RPi::RPiCameraConfiguration *rpiConfig, - std::vector &rawStreams, - std::vector &outStreams) const override; + CameraConfiguration::Status platformValidate(RPi::RPiCameraConfiguration *rpiConfig) const override; int platformPipelineConfigure(const std::unique_ptr &root) override; @@ -118,8 +117,7 @@ private: int platformConfigure(const V4L2SubdeviceFormat &sensorFormat, std::optional packing, - std::vector &rawStreams, - std::vector &outStreams) override; + const RPi::RPiCameraConfiguration *rpiConfig) override; int platformConfigureIpa(ipa::RPi::ConfigParams ¶ms) override; int platformInitIpa([[maybe_unused]] ipa::RPi::InitParams ¶ms) override @@ -395,10 +393,11 @@ int PipelineHandlerVc4::platformRegister(std::unique_ptr &camer return 0; } -CameraConfiguration::Status Vc4CameraData::platformValidate(RPi::RPiCameraConfiguration *rpiConfig, - std::vector &rawStreams, - std::vector &outStreams) const +CameraConfiguration::Status Vc4CameraData::platformValidate(RPi::RPiCameraConfiguration *rpiConfig) const { + std::vector &rawStreams = rpiConfig->rawStreams_; + std::vector &outStreams = rpiConfig->outStreams_; + CameraConfiguration::Status status = CameraConfiguration::Status::Valid; /* Can only output 1 RAW stream, or 2 YUV/RGB streams. */ @@ -503,9 +502,10 @@ int Vc4CameraData::platformPipelineConfigure(const std::unique_ptr & int Vc4CameraData::platformConfigure(const V4L2SubdeviceFormat &sensorFormat, std::optional packing, - std::vector &rawStreams, - std::vector &outStreams) + const RPi::RPiCameraConfiguration *rpiConfig) { + const std::vector &rawStreams = rpiConfig->rawStreams_; + const std::vector &outStreams = rpiConfig->outStreams_; int ret; if (!packing) From patchwork Thu Sep 21 16:55:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 19073 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 277BEBE080 for ; Thu, 21 Sep 2023 16:56:13 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9A73462953; Thu, 21 Sep 2023 18:56:11 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1695315371; bh=feqIYvUqFwwlOexfM6mLCpiGUPOe/6b+Cl5NSJ5mV1c=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=NLH0uTLYpQPl9cO//UcbG46IQwJHHqoZb3X8HIyLX20SQ06M95nW/sWlQqYrb/YjE UlbUaYk1YnCbPNIjTZskEzYPIBmxPIH2pTZ86shqqOOzsZ0FWXiAM6eiBN+B1gQpwB HMcttO4LYuO3JAOHu63whGHbnU14/coTdX3Ce6d4DZHQXeTqwLPLLfx9WphoaBqAy6 He5JUrUO7P/Bs/XqfKudH9LUZ+rFpcVOW2h9i+40uqQomJadEESj7368bc9+J9ivGz rxWpxxXu6/2lvjO0v5eH5QDYSodHN3bXCyUY2KHbA8uVyIJPrUUnZ91Y4wVAl1ECT2 7S+bC0Ch2n0xA== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 290826294D for ; Thu, 21 Sep 2023 18:56:05 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Vs9YiXGI"; dkim-atps=neutral Received: from uno.lan (93-46-82-201.ip106.fastwebnet.it [93.46.82.201]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 9D3E42F6C; Thu, 21 Sep 2023 18:54:27 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1695315267; bh=feqIYvUqFwwlOexfM6mLCpiGUPOe/6b+Cl5NSJ5mV1c=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Vs9YiXGIyOkGnJFa71YMrMDVT5NclADtAiP7jtL9DxF6SjaIihH5l/tj/Zx7sEtss Qt/HESz2VRU3UfTz/07Nj3nC2vCMqjsOqJcnPWdRLcGb1P137wndVbvgND0tqvkzDI 9M3KblWll31xVP+9za+/+92NzuBxofOMraYOEeq8= To: libcamera-devel@lists.libcamera.org Date: Thu, 21 Sep 2023 18:55:46 +0200 Message-ID: <20230921165550.50956-10-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20230921165550.50956-1-jacopo.mondi@ideasonboard.com> References: <20230921165550.50956-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 09/13] libcamera: rpi: Add some helpers to PipelineHandlerBase X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Naushir Patuck Add a helper updateStreamConfig() that updates the format related fields in a StreamConfiguration from a given V4L2DeviceFormat structure. Add and override to the toV4L2DeviceFormat() helper that returns a V4L2DeviceFormat structure populated from the format related fields in a StreamConfiguration. Both these helper functions will be used in a future commit to simplify the Raspberry Pi pipeline handler configuration/validation code. Signed-off-by: Naushir Patuck Reviewed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi --- .../pipeline/rpi/common/pipeline_base.cpp | 47 +++++++++++++++++++ .../pipeline/rpi/common/pipeline_base.h | 4 ++ 2 files changed, 51 insertions(+) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index e36bdce229b6..f9b67563af17 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -353,6 +353,53 @@ bool PipelineHandlerBase::isRaw(const PixelFormat &pixFmt) return BayerFormat::fromPixelFormat(pixFmt).isValid(); } +/* + * Adjust a StreamConfiguration fields to match a video device format. + * Returns true if the StreamConfiguration has been adjusted. + */ +bool PipelineHandlerBase::updateStreamConfig(StreamConfiguration *stream, + const V4L2DeviceFormat &format) +{ + const PixelFormat &pixFormat = format.fourcc.toPixelFormat(); + bool adjusted = false; + + if (stream->pixelFormat != pixFormat || stream->size != format.size) { + stream->pixelFormat = pixFormat; + stream->size = format.size; + adjusted = true; + } + + if (stream->colorSpace != format.colorSpace) { + stream->colorSpace = format.colorSpace; + adjusted = true; + LOG(RPI, Debug) + << "Color space changed from " + << ColorSpace::toString(stream->colorSpace) << " to " + << ColorSpace::toString(format.colorSpace); + } + + stream->stride = format.planes[0].bpl; + stream->frameSize = format.planes[0].size; + + return adjusted; +} + +/* + * Populate and return a video device format using a StreamConfiguration. */ +V4L2DeviceFormat PipelineHandlerBase::toV4L2DeviceFormat(const V4L2VideoDevice *dev, + const StreamConfiguration *stream) +{ + V4L2DeviceFormat deviceFormat; + + const PixelFormatInfo &info = PixelFormatInfo::info(stream->pixelFormat); + deviceFormat.planesCount = info.numPlanes(); + deviceFormat.fourcc = dev->toV4L2PixelFormat(stream->pixelFormat); + deviceFormat.size = stream->size; + deviceFormat.colorSpace = stream->colorSpace; + + return deviceFormat; +} + V4L2DeviceFormat PipelineHandlerBase::toV4L2DeviceFormat(const V4L2VideoDevice *dev, const V4L2SubdeviceFormat &format, BayerFormat::Packing packingReq) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index 1a3a13dbb6cf..491c5e98c4a1 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -198,6 +198,10 @@ public: static bool isYuv(const PixelFormat &pixFmt); static bool isRaw(const PixelFormat &pixFmt); + static bool updateStreamConfig(StreamConfiguration *stream, + const V4L2DeviceFormat &format); + static V4L2DeviceFormat toV4L2DeviceFormat(const V4L2VideoDevice *dev, + const StreamConfiguration *stream); static V4L2DeviceFormat toV4L2DeviceFormat(const V4L2VideoDevice *dev, const V4L2SubdeviceFormat &format, BayerFormat::Packing packingReq); From patchwork Thu Sep 21 16:55:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 19074 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C9825C32B2 for ; Thu, 21 Sep 2023 16:56:13 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 648546295E; Thu, 21 Sep 2023 18:56:12 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1695315372; bh=b7QR/oq6w7h5sgTDU/EyZf2vV029gv5VibFRR0wa3dI=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=slS2EDIw/+iRDvS8bVPpBpnvJ9vBYGyNPREk4YBszA/jCOJ/SbLwN77RMqDkmDP0l pPoY2zfoI8tzfc+HJALcP21LZmblskUp/SYTC54mZVhEs9WdYHDBnPiaVKowNKak/q 2h/yqukZjtusa8+VRwgPEp1CC/NUihsyIHK+WtQx+jCH2ha2gNfVBpDBaB7pCqhkpU P4Tfq8lGKWXwb+69Zpw9fcf0g8cSjWpmcTV2ZhAkIJAHA9woPOKbnAvV54GMu7gdRV RW1tiwzOmE1AoIDawcjO8vONPZ7sQfDCu4kL0607lWO/zil11xj2kTCOAl7r/Dop/w M6Dk1NbDinUPg== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8803962949 for ; Thu, 21 Sep 2023 18:56:05 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="GmIm67AG"; dkim-atps=neutral Received: from uno.lan (93-46-82-201.ip106.fastwebnet.it [93.46.82.201]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id F19478201; Thu, 21 Sep 2023 18:54:27 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1695315268; bh=b7QR/oq6w7h5sgTDU/EyZf2vV029gv5VibFRR0wa3dI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GmIm67AGuozR57+dQFEmUY5VxZbYYC5ydI+lVo1JGC956Ma8bcijCUUrU3yGHo51y inAxsf3uzzDbpYlyKYZhNtjOo2pET6SRDBGZYFKyjZkSfLAo97N4BJEWanP5L0SBm0 lBaxZ+BeLFHwxb/AryCvtqnW3QgSSeniRzaY4RMQ= To: libcamera-devel@lists.libcamera.org Date: Thu, 21 Sep 2023 18:55:47 +0200 Message-ID: <20230921165550.50956-11-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20230921165550.50956-1-jacopo.mondi@ideasonboard.com> References: <20230921165550.50956-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 10/13] libcamera: rpi: Simplify validate() and configure() for RAW streams X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Naushir Patuck This commit simplifies the validate() and configure() calls in the pipeline handler in a number of ways: - Only pass the RPiCameraConfiguration structure into platformValidate() and platformConfigure(). - Determine the V4L2DeviceFormat structure fields for all streams in validate(), cache them and reuse in configure() instead of re-generating this structure multiple times. - Use the recently added updateStreamConfig() and toV4L2DeviceFormat() helpers to populate fields in the V4L2DeviceFormat and StreamConfiguration structures to reduce code duplication. Signed-off-by: Naushir Patuck Reviewed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi --- .../pipeline/rpi/common/pipeline_base.cpp | 48 +++++-------------- .../pipeline/rpi/common/pipeline_base.h | 5 +- src/libcamera/pipeline/rpi/vc4/vc4.cpp | 35 ++++++++------ 3 files changed, 34 insertions(+), 54 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index f9b67563af17..c560e48c12fb 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -238,16 +238,10 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() /* Further fixups on the RAW streams. */ for (auto &raw : rawStreams_) { - StreamConfiguration &cfg = config_.at(raw.index); - - V4L2DeviceFormat rawFormat; - rawFormat.fourcc = raw.dev->toV4L2PixelFormat(cfg.pixelFormat); - rawFormat.size = cfg.size; - rawFormat.colorSpace = cfg.colorSpace; - - int ret = raw.dev->tryFormat(&rawFormat); + int ret = raw.dev->tryFormat(&raw.format); if (ret) return Invalid; + /* * Some sensors change their Bayer order when they are h-flipped * or v-flipped, according to the transform. If this one does, we @@ -255,23 +249,15 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() * Note how we must fetch the "native" (i.e. untransformed) Bayer * order, because the sensor may currently be flipped! */ - V4L2PixelFormat fourcc = rawFormat.fourcc; + BayerFormat bayer = BayerFormat::fromPixelFormat(raw.cfg->pixelFormat); if (data_->flipsAlterBayerOrder_) { - BayerFormat bayer = BayerFormat::fromV4L2PixelFormat(fourcc); bayer.order = data_->nativeBayerOrder_; bayer = bayer.transform(combinedTransform_); - fourcc = bayer.toV4L2PixelFormat(); } + raw.cfg->pixelFormat = bayer.toPixelFormat(); - PixelFormat inputPixFormat = fourcc.toPixelFormat(); - if (raw.cfg->size != rawFormat.size || raw.cfg->pixelFormat != inputPixFormat) { - raw.cfg->size = rawFormat.size; - raw.cfg->pixelFormat = inputPixFormat; + if (RPi::PipelineHandlerBase::updateStreamConfig(raw.cfg, raw.format)) status = Adjusted; - } - - raw.cfg->stride = rawFormat.planes[0].bpl; - raw.cfg->frameSize = rawFormat.planes[0].size; } /* Further fixups on the ISP output streams. */ @@ -548,35 +534,25 @@ int PipelineHandlerBase::configure(Camera *camera, CameraConfiguration *config) * If the application has provided a sensor configuration apply it * instead of just applying a format. */ - const RPiCameraConfiguration *rpiConfig = - static_cast(config); - V4L2SubdeviceFormat sensorFormat; + RPiCameraConfiguration *rpiConfig = static_cast(config); + V4L2SubdeviceFormat *sensorFormat = &rpiConfig->sensorFormat_; if (rpiConfig->sensorConfig) { ret = data->sensor_->applyConfiguration(*rpiConfig->sensorConfig, rpiConfig->combinedTransform_, - &sensorFormat); + sensorFormat); } else { - sensorFormat = rpiConfig->sensorFormat_; - ret = data->sensor_->setFormat(&sensorFormat, + ret = data->sensor_->setFormat(sensorFormat, rpiConfig->combinedTransform_); } if (ret) return ret; - /* Use the user requested packing/bit-depth. */ - std::optional packing; - if (!rpiConfig->rawStreams_.empty()) { - BayerFormat bayerFormat = - BayerFormat::fromPixelFormat(rpiConfig->rawStreams_[0].cfg->pixelFormat); - packing = bayerFormat.packing; - } - /* * Platform specific internal stream configuration. This also assigns * external streams which get configured below. */ - ret = data->platformConfigure(sensorFormat, packing, rpiConfig); + ret = data->platformConfigure(rpiConfig); if (ret) return ret; @@ -640,11 +616,11 @@ int PipelineHandlerBase::configure(Camera *camera, CameraConfiguration *config) */ link->setEnabled(true); const MediaPad *sinkPad = link->sink(); - ret = device->setFormat(sinkPad->index(), &sensorFormat); + ret = device->setFormat(sinkPad->index(), sensorFormat); if (ret) { LOG(RPI, Error) << "Failed to set format on " << device->entity()->name() << " pad " << sinkPad->index() - << " with format " << sensorFormat + << " with format " << *sensorFormat << ": " << ret; return ret; } diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index 491c5e98c4a1..0c4a59b7f590 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -58,9 +58,7 @@ public: } virtual CameraConfiguration::Status platformValidate(RPiCameraConfiguration *rpiConfig) const = 0; - virtual int platformConfigure(const V4L2SubdeviceFormat &sensorFormat, - std::optional packing, - const RPiCameraConfiguration *rpiConfig) = 0; + virtual int platformConfigure(const RPiCameraConfiguration *rpiConfig) = 0; virtual void platformStart() = 0; virtual void platformStop() = 0; @@ -269,6 +267,7 @@ public: unsigned int index; StreamConfiguration *cfg; V4L2VideoDevice *dev; + V4L2DeviceFormat format; }; std::vector rawStreams_; diff --git a/src/libcamera/pipeline/rpi/vc4/vc4.cpp b/src/libcamera/pipeline/rpi/vc4/vc4.cpp index 2308577a613b..1a03c88d7ee3 100644 --- a/src/libcamera/pipeline/rpi/vc4/vc4.cpp +++ b/src/libcamera/pipeline/rpi/vc4/vc4.cpp @@ -115,9 +115,7 @@ private: isp_[Isp::Input].dev()->setSelection(V4L2_SEL_TGT_CROP, &ispCrop_); } - int platformConfigure(const V4L2SubdeviceFormat &sensorFormat, - std::optional packing, - const RPi::RPiCameraConfiguration *rpiConfig) override; + int platformConfigure(const RPi::RPiCameraConfiguration *rpiConfig) override; int platformConfigureIpa(ipa::RPi::ConfigParams ¶ms) override; int platformInitIpa([[maybe_unused]] ipa::RPi::InitParams ¶ms) override @@ -416,6 +414,9 @@ CameraConfiguration::Status Vc4CameraData::platformValidate(RPi::RPiCameraConfig /* Handle flips to make sure to match the RAW stream format. */ if (flipsAlterBayerOrder_) rawBayer = rawBayer.transform(rpiConfig->combinedTransform_); + + /* Apply the user requested packing. */ + rawBayer.packing = BayerFormat::fromPixelFormat(rawStream->pixelFormat).packing; PixelFormat rawFormat = rawBayer.toPixelFormat(); if (rawStream->pixelFormat != rawFormat || @@ -425,6 +426,9 @@ CameraConfiguration::Status Vc4CameraData::platformValidate(RPi::RPiCameraConfig status = CameraConfiguration::Adjusted; } + + rawStreams[0].format = + RPi::PipelineHandlerBase::toV4L2DeviceFormat(unicam_[Unicam::Image].dev(), rawStream); } /* @@ -500,23 +504,14 @@ int Vc4CameraData::platformPipelineConfigure(const std::unique_ptr & return 0; } -int Vc4CameraData::platformConfigure(const V4L2SubdeviceFormat &sensorFormat, - std::optional packing, - const RPi::RPiCameraConfiguration *rpiConfig) +int Vc4CameraData::platformConfigure(const RPi::RPiCameraConfiguration *rpiConfig) { const std::vector &rawStreams = rpiConfig->rawStreams_; const std::vector &outStreams = rpiConfig->outStreams_; int ret; - if (!packing) - packing = BayerFormat::Packing::CSI2; - V4L2VideoDevice *unicam = unicam_[Unicam::Image].dev(); - V4L2DeviceFormat unicamFormat = RPi::PipelineHandlerBase::toV4L2DeviceFormat(unicam, sensorFormat, *packing); - - ret = unicam->setFormat(&unicamFormat); - if (ret) - return ret; + V4L2DeviceFormat unicamFormat; /* * See which streams are requested, and route the user @@ -525,14 +520,24 @@ int Vc4CameraData::platformConfigure(const V4L2SubdeviceFormat &sensorFormat, if (!rawStreams.empty()) { rawStreams[0].cfg->setStream(&unicam_[Unicam::Image]); unicam_[Unicam::Image].setFlags(StreamFlag::External); + unicamFormat = rawStreams[0].format; + } else { + unicamFormat = + RPi::PipelineHandlerBase::toV4L2DeviceFormat(unicam, + rpiConfig->sensorFormat_, + BayerFormat::Packing::CSI2); } + ret = unicam->setFormat(&unicamFormat); + if (ret) + return ret; + ret = isp_[Isp::Input].dev()->setFormat(&unicamFormat); if (ret) return ret; LOG(RPI, Info) << "Sensor: " << sensor_->id() - << " - Selected sensor format: " << sensorFormat + << " - Selected sensor format: " << rpiConfig->sensorFormat_ << " - Selected unicam format: " << unicamFormat; /* Use a sensible small default size if no output streams are configured. */ From patchwork Thu Sep 21 16:55:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 19075 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 545FCC326C for ; Thu, 21 Sep 2023 16:56:14 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E96B262951; Thu, 21 Sep 2023 18:56:13 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1695315374; bh=+ORVoFtWp9TDl0RMo01WYqmvHdIpoxAqgffn9ObCinY=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=M3RONvqQutRhc8ZOdJ89DmsDgjewbcO4RowUh3vq66ba2EEovwQDfg6pOduEbUPbR DQaykHmsE6pWRFNIfb6oWk1UyRj8A7+vEcXob/GKFe/rl6Z/xBGQHQcxgGS1b3qDK/ og3q5JhSGzJGjYjSzTaNIuutnnYQa9bF1K8oJddZJAw3ahL4abCEZQzgzEThf1+TNi LuNV9OPnOL4CqU4RO7nXEX3hpzknlx6PU2pCoNTDHe6yRmtIpD2Zfvzt9UhOuzyGXC N/jkTO4JvRasRcxOlwRV5ou2syFoglKYrX3dBzU3FwOwZ+iQC5kKaxjPctcGAxugSF MU0+INVxpk2rw== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E048C62944 for ; Thu, 21 Sep 2023 18:56:05 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="rWLxlt/2"; dkim-atps=neutral Received: from uno.lan (93-46-82-201.ip106.fastwebnet.it [93.46.82.201]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 53E831257; Thu, 21 Sep 2023 18:54:28 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1695315268; bh=+ORVoFtWp9TDl0RMo01WYqmvHdIpoxAqgffn9ObCinY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rWLxlt/2Lwg++7GfsEYZ9DlF4P8vyWd+7t/x2w7Se5z8K7WWyk5A0qC0Ir97JBPQe EX/jRpZu9mnj2+BHpc7x5lNOGhVkya7nqGdc5pQH6uwZKATIkzhGBbrkFGnXl+rXGI +yCKArWATFv9299mAVGWbJ/XM0iRrQTlQo7HTxfk= To: libcamera-devel@lists.libcamera.org Date: Thu, 21 Sep 2023 18:55:48 +0200 Message-ID: <20230921165550.50956-12-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20230921165550.50956-1-jacopo.mondi@ideasonboard.com> References: <20230921165550.50956-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 11/13] libcamera: rpi: Change default stream formats X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Naushir Patuck Switch to XRGB8888 as a default Viewfinder role output format, and YUV420 as a default output format for everything else. Signed-off-by: Naushir Patuck Reviewed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/rpi/common/pipeline_base.cpp | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index c560e48c12fb..f8e8e13dc837 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -268,7 +268,7 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() if (fmts.find(out.dev->toV4L2PixelFormat(cfgPixFmt)) == fmts.end()) { /* If we cannot find a native format, use a default one. */ - cfgPixFmt = formats::NV12; + cfgPixFmt = formats::YUV420; status = Adjusted; } @@ -431,7 +431,7 @@ PipelineHandlerBase::generateConfiguration(Camera *camera, SpanispFormats(); - pixelFormat = formats::NV12; + pixelFormat = formats::YUV420; /* * Still image codecs usually expect the sYCC color space. * Even RGB codecs will be fine as the RGB we get with the @@ -465,7 +465,7 @@ PipelineHandlerBase::generateConfiguration(Camera *camera, SpanispFormats(); - pixelFormat = formats::ARGB8888; + pixelFormat = formats::XRGB8888; colorSpace = ColorSpace::Sycc; size = { 800, 600 }; bufferCount = 4; From patchwork Thu Sep 21 16:55:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 19076 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EE4F2C32B3 for ; Thu, 21 Sep 2023 16:56:14 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9F17E6295D; Thu, 21 Sep 2023 18:56:14 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1695315374; bh=0gh+wY8QM+2l9+M1e2z0LnVNAjVgWuednSnUxnRvaWE=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=dN91WR3lIfMfnQdBCKdZhKrdRm+iZ3LJw8uGK/LdjvjEwEPUjOG2ZjatU5T33kFEm pDq/8kyLLXvDpygUrflfDyMwXHkpyq1iqE9dTMm0REwAnOYi2fo58wiERBRTTUXPAs TQC62D4t5IMqa0BRmaWUP10mvTjnALVw/xIExeX2Ljal57u4UvfnNP/l+X+/HrJ4ry 2XFZHxko8sWqczDLcHk5/0Ds9RK9SQsSMSQkEvRQWqZxiCBaFWNQDh5xF9b0fU9Keg oWyM8It9u0+UJwKZ/q5kqo0CgsjpIIS4kF1k4DES6K06xrjM18zBI0u2cZr98oD+ha 45F79T+QporgQ== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4356662955 for ; Thu, 21 Sep 2023 18:56:06 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="vkt9Xm9U"; dkim-atps=neutral Received: from uno.lan (93-46-82-201.ip106.fastwebnet.it [93.46.82.201]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id A95FA2F6C; Thu, 21 Sep 2023 18:54:28 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1695315268; bh=0gh+wY8QM+2l9+M1e2z0LnVNAjVgWuednSnUxnRvaWE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=vkt9Xm9U2A16JGJ77fhXIGfY5eHDay1rDaFgx+xKDmviXkXayjOlNIDo6qySVSTmH FN+fkPp7bNhTUhc/2sKmod4m8m9P0p2SoAZh7iVqQTK31SesbVQ3+c86sN8025q2XW M/E4OxgBX2NzhN6e88GAW0xCXXkNvDuRrp9HDVvk= To: libcamera-devel@lists.libcamera.org Date: Thu, 21 Sep 2023 18:55:49 +0200 Message-ID: <20230921165550.50956-13-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20230921165550.50956-1-jacopo.mondi@ideasonboard.com> References: <20230921165550.50956-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 12/13] libcamera: rpi: Simplify validate() and configure() for YUV/RGB streams X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Naushir Patuck This commit simplifies the validate() and configure() calls in the pipeline handler in a number of ways: - Determine the V4L2DeviceFormat structure fields for all streams in validate(), cache them and reuse in configure() instead of re-generating this structure multiple times. - Remove setting a default pixel format in validate(), this code patch will not be used. - Use the recently added updateStreamConfig() and toV4L2DeviceFormat() helpers to populate fields in the V4L2DeviceFormat and StreamConfiguration structures to reduce code duplication. Signed-off-by: Naushir Patuck Reviewed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi --- .../pipeline/rpi/common/pipeline_base.cpp | 49 ++++--------------- src/libcamera/pipeline/rpi/vc4/vc4.cpp | 15 ++---- 2 files changed, 12 insertions(+), 52 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index f8e8e13dc837..b19dad457e59 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -262,56 +262,25 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() /* Further fixups on the ISP output streams. */ for (auto &out : outStreams_) { - StreamConfiguration &cfg = config_.at(out.index); - PixelFormat &cfgPixFmt = cfg.pixelFormat; - V4L2VideoDevice::Formats fmts = out.dev->formats(); - - if (fmts.find(out.dev->toV4L2PixelFormat(cfgPixFmt)) == fmts.end()) { - /* If we cannot find a native format, use a default one. */ - cfgPixFmt = formats::YUV420; - status = Adjusted; - } - - V4L2DeviceFormat format; - format.fourcc = out.dev->toV4L2PixelFormat(cfg.pixelFormat); - format.size = cfg.size; /* - * platformValidate may have worked out the correct stride so we - * must pass it in. This also needs the planesCount to be set - * correctly or the stride will be ignored. + * We want to send the associated YCbCr info through to the driver. + * + * But for RGB streams, the YCbCr info gets overwritten on the way back + * so we must check against what the stream cfg says, not what we actually + * requested (which carefully included the YCbCr info)! */ - const PixelFormat &pixFormat = format.fourcc.toPixelFormat(); - const PixelFormatInfo &info = PixelFormatInfo::info(pixFormat); - format.planesCount = info.numPlanes(); - format.planes[0].bpl = cfg.stride; - - /* We want to send the associated YCbCr info through to the driver. */ - format.colorSpace = yuvColorSpace_; + out.format.colorSpace = yuvColorSpace_; LOG(RPI, Debug) - << "Try color space " << ColorSpace::toString(cfg.colorSpace); + << "Try color space " << ColorSpace::toString(out.cfg->colorSpace); - int ret = out.dev->tryFormat(&format); + int ret = out.dev->tryFormat(&out.format); if (ret) return Invalid; - /* - * But for RGB streams, the YCbCr info gets overwritten on the way back - * so we must check against what the stream cfg says, not what we actually - * requested (which carefully included the YCbCr info)! - */ - if (cfg.colorSpace != format.colorSpace) { + if (RPi::PipelineHandlerBase::updateStreamConfig(out.cfg, out.format)) status = Adjusted; - LOG(RPI, Debug) - << "Color space changed from " - << ColorSpace::toString(cfg.colorSpace) << " to " - << ColorSpace::toString(format.colorSpace); - } - - cfg.colorSpace = format.colorSpace; - cfg.stride = format.planes[0].bpl; - cfg.frameSize = format.planes[0].size; } return status; diff --git a/src/libcamera/pipeline/rpi/vc4/vc4.cpp b/src/libcamera/pipeline/rpi/vc4/vc4.cpp index 1a03c88d7ee3..a1813ba65725 100644 --- a/src/libcamera/pipeline/rpi/vc4/vc4.cpp +++ b/src/libcamera/pipeline/rpi/vc4/vc4.cpp @@ -457,6 +457,8 @@ CameraConfiguration::Status Vc4CameraData::platformValidate(RPi::RPiCameraConfig * have that fixed up in the code above. */ outStreams[i].dev = isp_[i == 0 ? Isp::Output0 : Isp::Output1].dev(); + + outStreams[i].format = RPi::PipelineHandlerBase::toV4L2DeviceFormat(outStreams[i].dev, outStreams[i].cfg); } return status; @@ -549,11 +551,7 @@ int Vc4CameraData::platformConfigure(const RPi::RPiCameraConfiguration *rpiConfi /* The largest resolution gets routed to the ISP Output 0 node. */ RPi::Stream *stream = i == 0 ? &isp_[Isp::Output0] : &isp_[Isp::Output1]; - - V4L2PixelFormat fourcc = stream->dev()->toV4L2PixelFormat(cfg->pixelFormat); - format.size = cfg->size; - format.fourcc = fourcc; - format.colorSpace = cfg->colorSpace; + format = outStreams[i].format; LOG(RPI, Debug) << "Setting " << stream->name() << " to " << format; @@ -562,13 +560,6 @@ int Vc4CameraData::platformConfigure(const RPi::RPiCameraConfiguration *rpiConfi if (ret) return -EINVAL; - if (format.size != cfg->size || format.fourcc != fourcc) { - LOG(RPI, Error) - << "Failed to set requested format on " << stream->name() - << ", returned " << format; - return -EINVAL; - } - LOG(RPI, Debug) << "Stream " << stream->name() << " has color space " << ColorSpace::toString(cfg->colorSpace); From patchwork Thu Sep 21 16:55:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 19077 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C2F80C32B0 for ; Thu, 21 Sep 2023 16:56:15 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7DA6C62963; Thu, 21 Sep 2023 18:56:15 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1695315375; bh=LOO4pFx1IRYfHHjHm+nfzSRXK168AXtwhSnNX5UrCIY=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=FegbitA24tyk+46IhVmfreVWwQ7MGv5l+yAXT/862Ms5hT2KFj2I9VSMF7kyENWpK F5Xv74eC9aX0WEjgxdQseLUylbVcmcRqc9VVpa66uo99j5hQpUJ/aCWNQ7Wf1Ynort Z/QuPofC5eGpN7qqA7oE+mXVT6wvJ1AArkubpXeWWNLlvLGiuNNuQJFmeE8I8OqI3B jAhcXc07gvWrzc8KZFywUyxeuXhTQkPVV3JijL9deKVgsyOMLzTD5+wubQRw1i18SV TIBY0nOYcbi5UdGMenETFKWCv1ICnSUQ/ifMDIkowJXzqEFJtHGE5N4Xg4kLTv+TeE AvWD7DPGOZVCw== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B94D162948 for ; Thu, 21 Sep 2023 18:56:06 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="KS3DEZPr"; dkim-atps=neutral Received: from uno.lan (93-46-82-201.ip106.fastwebnet.it [93.46.82.201]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 0865E1102; Thu, 21 Sep 2023 18:54:28 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1695315269; bh=LOO4pFx1IRYfHHjHm+nfzSRXK168AXtwhSnNX5UrCIY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=KS3DEZPr/xmrMZueQE5QQIlBIO/nT+Hf12bM+QH1oUGMATNJbb6cZl1wvJdOdN6Mb b2j2bhGCCQjPnxsbwDdJqrwUNO8dKpWQ9P0c31v6gG9SPIkJ4VtSag/wgleKe562TS XQETTqXOP1x45guGZRV1rsFE9zw5pi6f5QPbSmco= To: libcamera-devel@lists.libcamera.org Date: Thu, 21 Sep 2023 18:55:50 +0200 Message-ID: <20230921165550.50956-14-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20230921165550.50956-1-jacopo.mondi@ideasonboard.com> References: <20230921165550.50956-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 13/13] apps: cam: Add option to configure sensor X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a '-f|--sensor_format' option to cam to allow forcing a sensor configuration from the command line. As an example: cam -c1 -C -S --stream pixelformat=YUYV -f width=3840,height=2160,bitDepth=10 Signed-off-by: Jacopo Mondi --- src/apps/cam/camera_session.cpp | 7 +++++ src/apps/cam/main.cpp | 4 +++ src/apps/cam/main.h | 1 + src/apps/common/stream_options.cpp | 42 ++++++++++++++++++++++++++++++ src/apps/common/stream_options.h | 8 ++++++ 5 files changed, 62 insertions(+) diff --git a/src/apps/cam/camera_session.cpp b/src/apps/cam/camera_session.cpp index 066e397b5f47..4e01ebdc3e90 100644 --- a/src/apps/cam/camera_session.cpp +++ b/src/apps/cam/camera_session.cpp @@ -72,6 +72,13 @@ CameraSession::CameraSession(CameraManager *cm, return; } + /* Apply a sensor configuration is requested. */ + if (SensorKeyValueParser::updateConfiguration(config.get(), + options_[OptSensorFmt])) { + std::cerr << "Failed to apply sensor configuration" << std::endl; + return; + } + bool strictFormats = options_.isSet(OptStrictFormats); #ifdef HAVE_KMS diff --git a/src/apps/cam/main.cpp b/src/apps/cam/main.cpp index 5d8a57bc14e5..d15ee875d6b8 100644 --- a/src/apps/cam/main.cpp +++ b/src/apps/cam/main.cpp @@ -110,6 +110,7 @@ void CamApp::quit() int CamApp::parseOptions(int argc, char *argv[]) { StreamKeyValueParser streamKeyValue; + SensorKeyValueParser sensorKeyValue; OptionsParser parser; parser.addOption(OptCamera, OptionString, @@ -171,6 +172,9 @@ int CamApp::parseOptions(int argc, char *argv[]) "Load a capture session configuration script from a file", "script", ArgumentRequired, "script", false, OptCamera); + parser.addOption(OptSensorFmt, &sensorKeyValue, + "Apply a format to the sensor", "sensor_format", true, + OptCamera); options_ = parser.parse(argc, argv); if (!options_.valid()) diff --git a/src/apps/cam/main.h b/src/apps/cam/main.h index 526aecece3f3..1432cfeb5052 100644 --- a/src/apps/cam/main.h +++ b/src/apps/cam/main.h @@ -19,6 +19,7 @@ enum { OptMonitor = 'm', OptSDL = 'S', OptStream = 's', + OptSensorFmt = 'f', OptListControls = 256, OptStrictFormats = 257, OptMetadata = 258, diff --git a/src/apps/common/stream_options.cpp b/src/apps/common/stream_options.cpp index 663b979aa763..a61317ab7fad 100644 --- a/src/apps/common/stream_options.cpp +++ b/src/apps/common/stream_options.cpp @@ -119,3 +119,45 @@ std::optional StreamKeyValueParser::parseRole(const KeyVa return {}; } + +SensorKeyValueParser::SensorKeyValueParser() +{ + addOption("bitDepth", OptionInteger, "Sensor format bit depth", + ArgumentRequired); + addOption("width", OptionInteger, "Sensor frame width in pixels", + ArgumentRequired); + addOption("height", OptionInteger, "Sensor frame height in pixels", + ArgumentRequired); +} + +int SensorKeyValueParser::updateConfiguration(CameraConfiguration *config, + const OptionValue &values) +{ + if (!config) { + std::cerr << "No configuration provided" << std::endl; + return -EINVAL; + } + + /* If no configuration values nothing to do. */ + if (values.empty()) + return 0; + + const std::vector &streamParameters = values.toArray(); + SensorConfiguration sensorConfig; + + for (auto const &value : streamParameters) { + KeyValueParser::Options opts = value.toKeyValues(); + + if (opts.isSet("width") && opts.isSet("height")) { + sensorConfig.outputSize.width = opts["width"]; + sensorConfig.outputSize.height = opts["height"]; + } + + if (opts.isSet("bitDepth")) + sensorConfig.bitDepth = opts["bitDepth"]; + } + + config->sensorConfig = sensorConfig; + + return 0; +} diff --git a/src/apps/common/stream_options.h b/src/apps/common/stream_options.h index a5f3bde08da8..a338206bd005 100644 --- a/src/apps/common/stream_options.h +++ b/src/apps/common/stream_options.h @@ -27,3 +27,11 @@ public: private: static std::optional parseRole(const KeyValueParser::Options &options); }; + +class SensorKeyValueParser : public KeyValueParser +{ +public: + SensorKeyValueParser(); + static int updateConfiguration(libcamera::CameraConfiguration *config, + const OptionValue &values); +}; From patchwork Mon Sep 25 17:09:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 19088 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 38CA6BD808 for ; Mon, 25 Sep 2023 17:09:34 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8F0EC62944; Mon, 25 Sep 2023 19:09:33 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1695661773; bh=mOAhWaYS94aJWI+MYiMnRTt8MBKEteFTUQSCqRi7Qx8=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=vqbEZOI20jlWPtwZJSGkAeOcwd3kfp6uaJb0FqiOsLNaP1xNO+XxRNrS9Vk50HTRI Pjda2fY7yc9JlcJikkfuNBSGOgaHL9MUavRX+BURL47NYuORTMcNT/QnCWXpTi3i0Z a2JhWFtXRdzz4Un18vNgk+/RkoDm+irQXfAxjTntWXVni6kUtRfdpEbl9VVi8/C65M t59lVI1fbYDmuuY3Ao9go0Q9kPv89F+KMSMY1Pg/+GtPhsdq10VUdE2EUyce8bIQau eUsPPSPfAPHoG4Q/tecdyWtx59hKo1uXWOALiwLmk1JuQnXNOIITWd/G6qYbSRjR+Y 0vbb3WDP6/KgA== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 255EA628FA for ; Mon, 25 Sep 2023 19:09:32 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="WOZyQP6U"; dkim-atps=neutral Received: from Monstersaurus.local (aztw-30-b2-v4wan-166917-cust845.vm26.cable.virginm.net [82.37.23.78]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id ADF9A13C5; Mon, 25 Sep 2023 19:07:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1695661671; bh=mOAhWaYS94aJWI+MYiMnRTt8MBKEteFTUQSCqRi7Qx8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WOZyQP6U5xGMibNAi9df1CmKLrdGwBdf4aMC9+6o+RUjdcO9g27HpyO06gAoOdYr1 JnmBQ/AOnbM4zZIKl6jl203/HWevQR45WSXa+z5j6SgT2AA979/WaJhLiHKhXDNE5q Mg10LXBt0o6zb7yWRXR7/S/czJ6aKT8CaCm+1Xwk= To: libcamera devel Date: Mon, 25 Sep 2023 18:09:19 +0100 Message-Id: <20230925170919.301540-1-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230921165550.50956-1-jacopo.mondi@ideasonboard.com> References: <20230921165550.50956-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 14/13] apps: qcam: Add option to configure sensor X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Kieran Bingham via libcamera-devel From: Kieran Bingham Reply-To: Kieran Bingham Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a '-f|--sensor_format' option to qcam to allow forcing a sensor configuration from the command line. As an example: qcam -f width=3840,height=2160,bitDepth=10 Signed-off-by: Kieran Bingham --- Extending Jacopo's implementation for cam to qcam Helpful for testing with visual display of the selected modes. src/apps/qcam/main.cpp | 3 +++ src/apps/qcam/main_window.cpp | 7 +++++++ src/apps/qcam/main_window.h | 1 + 3 files changed, 11 insertions(+) diff --git a/src/apps/qcam/main.cpp b/src/apps/qcam/main.cpp index 36cb93a53701..7e3f252e9f09 100644 --- a/src/apps/qcam/main.cpp +++ b/src/apps/qcam/main.cpp @@ -30,6 +30,7 @@ void signalHandler([[maybe_unused]] int signal) OptionsParser::Options parseOptions(int argc, char *argv[]) { StreamKeyValueParser streamKeyValue; + SensorKeyValueParser sensorKeyValue; OptionsParser parser; parser.addOption(OptCamera, OptionString, @@ -42,6 +43,8 @@ OptionsParser::Options parseOptions(int argc, char *argv[]) "renderer", ArgumentRequired, "renderer"); parser.addOption(OptStream, &streamKeyValue, "Set configuration of a camera stream", "stream", true); + parser.addOption(OptSensorFmt, &sensorKeyValue, + "Apply a format to the sensor", "sensor_format", true); parser.addOption(OptVerbose, OptionNone, "Print verbose log messages", "verbose"); diff --git a/src/apps/qcam/main_window.cpp b/src/apps/qcam/main_window.cpp index 0f16c038d516..e7511b4c7aa4 100644 --- a/src/apps/qcam/main_window.cpp +++ b/src/apps/qcam/main_window.cpp @@ -414,6 +414,13 @@ int MainWindow::startCapture() return -EINVAL; } + /* Apply a sensor configuration if requested. */ + if (SensorKeyValueParser::updateConfiguration(config_.get(), + options_[OptSensorFmt])) { + qWarning() << "Failed to apply sensor configuration"; + return -EINVAL; + } + CameraConfiguration::Status validation = config_->validate(); if (validation == CameraConfiguration::Invalid) { qWarning() << "Failed to create valid camera configuration"; diff --git a/src/apps/qcam/main_window.h b/src/apps/qcam/main_window.h index 2e3e1b5c83c2..44646a87cfbf 100644 --- a/src/apps/qcam/main_window.h +++ b/src/apps/qcam/main_window.h @@ -42,6 +42,7 @@ enum { OptHelp = 'h', OptRenderer = 'r', OptStream = 's', + OptSensorFmt = 'f', OptVerbose = 'v', };