From patchwork Sat Sep 16 12:19:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 19051 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D0496BE080 for ; Sat, 16 Sep 2023 13:01:28 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0D91162918; Sat, 16 Sep 2023 15:01:28 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1694869288; bh=5+z2tMp896rjc5PopxTdc83skhSb2ghYuCeTUPiouVI=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=iHaIs7Ht1SwQfqhoaKIF7qlsCg8deuPSLf2CxBWn31KLC3rgHQNI5rzP9bjeVh766 IZ/4WWSAyjz4ZgiGTFps4FsX60oQeRLw/7VRgLCNeorNQCLIwYIA/dq+La0J+xAFuh zT3bS/CKIvRnvFu2ho5lPvOxlFpZKzJtWKm6kB2DDB19WhMxohZeBAo+mdJuKC1Hyf d3ZgtZqV/p2vTuU85cn2C0zE/JuBcTzoZZQBoVsT0wnHqY+VbMm8IJU4Gb1AI6LpQR 3xWIKPBN+LAp8WjbchR2D400AWq48+9SkhhApX0xcKFZIiC2cbVLsEeVQNVtRQV+eQ YFr9cBs1NnHOw== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C26C5628E9 for ; Sat, 16 Sep 2023 14:19:37 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="pbp3L4HW"; dkim-atps=neutral Received: from uno.LocalDomain (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 1841C1536; Sat, 16 Sep 2023 14:18:03 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1694866683; bh=5+z2tMp896rjc5PopxTdc83skhSb2ghYuCeTUPiouVI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=pbp3L4HWLOpFM8c4WRRzXiq+kohjhyE7UvpxHlcLUjma14ORIHcVx/X+G83Jh/FKm NeUP9kBGhZ+kW9squVGvaoObbYYaVxdAaQf4M7ZssPuVKkfFvTlsC/V8+uPXY56ieu 0l6smKfHud5WjKcKCM5zSptRTLOK/FgRrb9dhW7k= To: libcamera-devel@lists.libcamera.org Date: Sat, 16 Sep 2023 14:19:19 +0200 Message-ID: <20230916121930.29490-2-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20230916121930.29490-1-jacopo.mondi@ideasonboard.com> References: <20230916121930.29490-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 X-Mailman-Approved-At: Sat, 16 Sep 2023 15:01:26 +0200 Subject: [libcamera-devel] [PATCH v4 01/12] documentation: Introduce Camera Sensor Model X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Introduce a documentation page about the 'camera sensor model' implemented by libcamera. The camera sensor model serves to provide to applications a reference description of the processing steps that take place in a camera sensor in order to precisely control the sensor configuration through the forthcoming SensorConfiguration class. Signed-off-by: Jacopo Mondi Reviewed-by: Naushir Patuck Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- Documentation/binning.svg | 5053 +++++++++++++++++++++++++ Documentation/camera-sensor-model.rst | 170 + Documentation/index.rst | 1 + Documentation/meson.build | 1 + Documentation/sensor_model.svg | 4866 ++++++++++++++++++++++++ Documentation/skipping.svg | 1720 +++++++++ 6 files changed, 11811 insertions(+) create mode 100644 Documentation/binning.svg create mode 100644 Documentation/camera-sensor-model.rst create mode 100644 Documentation/sensor_model.svg create mode 100644 Documentation/skipping.svg diff --git a/Documentation/binning.svg b/Documentation/binning.svg new file mode 100644 index 000000000000..c6a3b6394acd --- /dev/null +++ b/Documentation/binning.svg @@ -0,0 +1,5053 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/Documentation/camera-sensor-model.rst b/Documentation/camera-sensor-model.rst new file mode 100644 index 000000000000..7ff0bdc50564 --- /dev/null +++ b/Documentation/camera-sensor-model.rst @@ -0,0 +1,170 @@ +.. SPDX-License-Identifier: CC-BY-SA-4.0 + +.. _camera-sensor-model: + +The libcamera camera sensor model +================================= + +libcamera defines an abstract camera sensor model in order to provide +a description of each of the processing steps that result in image data being +sent on the media bus and that form the image stream delivered to applications. + +Applications should use the abstracted camera sensor model defined here to +precisely control the operations of the camera sensor. + +The libcamera camera sensor model targets image sensors producing frames in +RAW format, delivered through a MIPI CSI-2 compliant bus implementation. + +The abstract sensor model maps libcamera components to the characteristics and +operations of an image sensor, and serves as a reference to model the libcamera +CameraSensor class and SensorConfiguration classes and operations. + +In order to control the configuration of the camera sensor through the +SensorConfiguration class, applications should understand this model and map it +to the combination of image sensor and kernel driver in use. + +The camera sensor model defined here is based on the *MIPI CCS specification*, +particularly on *Section 8.2 - Image readout* of *Chapter 8 - Video Timings*. + +Glossary +--------- + +- *Pixel array*: The full grid of pixels, active and inactive ones + +- *Pixel array active area*: The portion(s) of the pixel array that + contains valid and readable pixels; corresponds to the libcamera + properties::PixelArrayActiveAreas + +- *Analog crop rectangle*: The portion of the *pixel array active area* which + is read out and passed to further processing stages + +- *Subsampling*: Pixel processing techniques that reduce the image size by + binning or by skipping adjacent pixels + +- *Digital crop*: Crop of the sub-sampled image data before scaling + +- *Frame output*: The frame (image) as output on the media bus by the + camera sensor + +Camera sensor model +------------------- + +The abstract sensor model is described in the following diagram + +.. figure:: sensor_model.svg + + +1. The sensor reads pixels from the *pixel array*. The pixels being read out are + selected by the *analog crop rectangle*. + +2. The pixels can be subsampled to reduce the image size without affecting the + field of view. Two subsampling techniques can be used: + + - Binning: combines adjacent pixels of the same colour by averaging or + summing their values, in the analog domain and/or the digital domain. + + .. figure:: binning.svg + + + - Skipping: skips the read out of a number of adjacent pixels. + + .. figure:: skipping.svg + + +3. The output of the optional sub-sampling stage is then cropped after the +conversion of the analogue pixel values in the digital domain. + +4. The resulting output frame is sent on the media bus by the sensor. + +Camera Sensor configuration parameters +-------------------------------------- + +The libcamera camera sensor model defines parameters that allow users to +control: + +1. The image format bit depth + +2. The size and position of the *Analog crop rectangle* + +3. The subsampling factors used to downscale the pixel array readout data to a + smaller frame size without reducing the image *field of view*. Two + configuration parameters are made available to control the downscaling factor + + - binning + - a vertical and horizontal binning factor can be specified, the image + will be downscaled in its vertical and horizontal sizes by the specified + factor + + .. code-block:: + :caption: Definition: The horizontal and vertical binning factors + + horizontal_binning = xBin; + vertical_binning = yBin; + + + - skipping + - reduces the image resolution by skipping the read-out of a + number of adjacent pixels + - the skipping factor is specified by the 'increment' number (number of + pixels to 'skip') in the vertical and horizontal directions and for + even and odd rows and columns + + .. code-block:: + :caption: Definition: The horizontal and vertical skipping factors + + horizontal_skipping = (xOddInc + xEvenInc) / 2 + vertical_skipping = (yOddInc + yEvenInc) / 2 + + + - different sensors perform the binning and skipping stages in different + orders + - for the sake of computing the final output image size the order of + execution is not relevant. + + - the overall down-scaling factor is obtained by combining the binning and + skipping factors + + .. code-block:: + :caption: Definition: The total scaling factor (binning + sub-sampling) + + total_horizontal_downscale = horizontal_binning + horizontal_skipping + total_vertical_downscale = vertical_binning + vertical_skipping + + +4. The output data frame size + - the output size is used to specify any additional cropping on the + sub-sampled frame + +5. The total line length and frame height (*visibile* pixels + *blankings*) as + sent on the MIPI CSI-2 bus + +6. The pixel transmission rate on the MIPI CSI-2 bus + +The above parameters are combined to obtain the following high-level +configurations: + +- **frame output size** + + Obtained by applying a crop to the physical pixel array size in the analog + domain, followed by optional binning and sub-sampling (in any order), + followed by an optional crop step in the output digital domain. + +- **frame rate** + + The combination of the *total frame size*, the image format *bit depth* and + the *pixel rate* of the data sent on the MIPI CSI-2 bus allows to compute the + image stream frame rate. The equation is the well known: + + .. code-block:: + + frame_duration = total_frame_size / pixel_rate + frame_rate = 1 / frame_duration + + + where the *pixel_rate* parameter is the result of the sensor's configuration + of the MIPI CSI-2 bus *(the following formula applies to MIPI CSI-2 when + used on MIPI D-PHY physical protocol layer only)* + + .. code-block:: + + pixel_rate = CSI-2_link_freq * 2 * nr_of_lanes / bits_per_sample diff --git a/Documentation/index.rst b/Documentation/index.rst index 43d8b017d3b4..63fac72d11ed 100644 --- a/Documentation/index.rst +++ b/Documentation/index.rst @@ -23,3 +23,4 @@ Sensor driver requirements Lens driver requirements Python Bindings + Camera Sensor Model diff --git a/Documentation/meson.build b/Documentation/meson.build index b2a5bf15e6ea..7c1502592baa 100644 --- a/Documentation/meson.build +++ b/Documentation/meson.build @@ -63,6 +63,7 @@ endif if sphinx.found() docs_sources = [ + 'camera-sensor-model.rst', 'coding-style.rst', 'conf.py', 'contributing.rst', diff --git a/Documentation/sensor_model.svg b/Documentation/sensor_model.svg new file mode 100644 index 000000000000..1f76d41cf6a5 --- /dev/null +++ b/Documentation/sensor_model.svg @@ -0,0 +1,4866 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Pixel array + Analog crop + Subsampling + Digital crop + Media Bus + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + 1 + + 2 + + 4 + + 3 + + diff --git a/Documentation/skipping.svg b/Documentation/skipping.svg new file mode 100644 index 000000000000..7bef37cfcc64 --- /dev/null +++ b/Documentation/skipping.svg @@ -0,0 +1,1720 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + x_even_inc = 1 + y_even_inc = 1 + 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 + x_odd_inc = 3 + y_odd_inc = 3 + 0 + 1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 + + + + From patchwork Sat Sep 16 12:19:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 19040 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 34CFBC3260 for ; Sat, 16 Sep 2023 12:19:41 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4003362922; Sat, 16 Sep 2023 14:19:40 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1694866780; bh=fCrwnYOzbDNE3v4d/Tv6LJHPfte+QYIu6jJXBrOwKTg=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=Hg3tZP7eY714sm6IGPX0Kfw5mU0YdojkMW3FqEG5po6sOxZmelwGXUTcqbPyi0QsJ UaHkl6PseBRMwfY2X1XWXr58bdezKB+flgULdAJvfVjTh3lGXDWiYtlnStEjgO0I9/ FYMuhLGa6AvNxbA3SDiU0A6xwz+lPdoeKmEzyKaSYjioMHLJ73n2g/kXmLtg+Kaby6 UJWIp+Irdc5m+4jcdj+t8MWyMQj9/M4K4i2gMwFRXTqjcD8bIlilUYzab3i84KDIuk 1u8E3rkyfERLG5VG3rdzw6dJebF5MRo5v+1JKiATJsXj53x3SdKlSgTmd4TqcxaD3R WYLDdXUYPdfhw== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 51E1A628E9 for ; Sat, 16 Sep 2023 14:19:38 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Z6bYot7w"; dkim-atps=neutral Received: from uno.LocalDomain (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 3069210FE; Sat, 16 Sep 2023 14:18:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1694866684; bh=fCrwnYOzbDNE3v4d/Tv6LJHPfte+QYIu6jJXBrOwKTg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Z6bYot7wD235f4NsnpxgP4arAIYA3D0vw5qBb05T1DNhDIx7J6tTMc6cZ8YbkVD9E siGVWu0xbSQwkcgMLvHZiePptVEth/b8gr/k6TUj3m4TjnckPZJ8NxKJrZu5uF51rz YEWi0w3PvhUtnhQ9LhbaPEWin8So7k6G0JRAqe9M= To: libcamera-devel@lists.libcamera.org Date: Sat, 16 Sep 2023 14:19:20 +0200 Message-ID: <20230916121930.29490-3-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20230916121930.29490-1-jacopo.mondi@ideasonboard.com> References: <20230916121930.29490-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 02/12] libcamera: camera: Introduce SensorConfiguration X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Introduce SensorConfiguration in the libcamera API. The SensorConfiguration is part of the CameraConfiguration class and allows applications to control the sensor settings. Signed-off-by: Jacopo Mondi Reviewed-by: Naushir Patuck Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- include/libcamera/camera.h | 43 +++++++++ src/libcamera/camera.cpp | 183 +++++++++++++++++++++++++++++++++++++ 2 files changed, 226 insertions(+) diff --git a/include/libcamera/camera.h b/include/libcamera/camera.h index 004bc89455f5..c4f9aa869489 100644 --- a/include/libcamera/camera.h +++ b/include/libcamera/camera.h @@ -19,6 +19,7 @@ #include #include +#include #include #include #include @@ -30,6 +31,47 @@ class FrameBufferAllocator; class PipelineHandler; class Request; +class SensorConfiguration +{ +public: + unsigned int bitDepth = 0; + + Rectangle analogCrop; + + struct { + unsigned int binX = 1; + unsigned int binY = 1; + } binning; + + struct { + unsigned int xOddInc = 1; + unsigned int xEvenInc = 1; + unsigned int yOddInc = 1; + unsigned int yEvenInc = 1; + } skipping; + + Size outputSize; + + bool valid() const + { + return validate() != Invalid; + } + + bool populated() const + { + return validate() == Populated; + } + +private: + enum Status { + Unpopulated, + Populated, + Invalid, + }; + + Status validate() const; +}; + class CameraConfiguration { public: @@ -66,6 +108,7 @@ public: bool empty() const; std::size_t size() const; + SensorConfiguration sensorConfig; Transform transform; protected: diff --git a/src/libcamera/camera.cpp b/src/libcamera/camera.cpp index 0eecee766f00..3d8098239a7f 100644 --- a/src/libcamera/camera.cpp +++ b/src/libcamera/camera.cpp @@ -97,6 +97,16 @@ * implemented in the above order at the hardware level. The libcamera pipeline * handlers translate the pipeline model to the real hardware configuration. * + * \subsection camera-sensor-model Camera Sensor Model + * + * By default, libcamera configures the camera sensor automatically based on the + * configuration of the streams. Applications may instead specify a manual + * configuration for the camera sensor. This allows precise control of the frame + * geometry and frame rate delivered by the sensor. + * + * More details about the camera sensor model implemented by libcamera are + * available in the libcamera camera-sensor-model documentation page. + * * \subsection digital-zoom Digital Zoom * * Digital zoom is implemented as a combination of the cropping and scaling @@ -111,6 +121,166 @@ namespace libcamera { LOG_DECLARE_CATEGORY(Camera) +/** + * \class SensorConfiguration + * \brief Camera sensor configuration + * + * The SensorConfiguration class collects parameters to control the operations + * of the camera sensor, accordingly to the abstract camera sensor model + * implemented by libcamera. + * + * \todo Applications shall fully populate all fields of the + * CameraConfiguration::sensorConfig class members before validating the + * CameraConfiguration. If the SensorConfiguration is not fully populated, or if + * any of its parameters cannot be applied to the sensor in use, the + * CameraConfiguration validation process will fail and return + * CameraConfiguration::Status::Invalid. + * + * Applications that populate the SensorConfiguration class members are + * expected to be highly-specialized applications that know what sensor + * they are operating with and what parameters are valid for the sensor in use. + * + * A detailed description of the abstract camera sensor model implemented by + * libcamera and the description of its configuration parameters is available + * in the libcamera documentation camera-sensor-model file. + */ + +/** + * \enum SensorConfiguration::Status + * \brief The sensor configuration validation status + */ + +/** + * \var SensorConfiguration::bitDepth + * \brief The sensor image format bit depth + * + * The number of bits (resolution) used to represent a pixel sample. + */ + +/** + * \var SensorConfiguration::analogCrop + * \brief The analog crop rectangle + * + * The selected portion of the active pixel array used to produce the image + * frame. + */ + +/** + * \var SensorConfiguration::binning + * \brief Sensor binning configuration + * + * Refer to the camera-sensor-model documentation for an accurate description + * of the binning operations. Disabled by default. + */ + +/** + * \var SensorConfiguration::binX + * \brief Horizontal binning factor + * + * The horizontal binning factor. Default to 1. + */ + +/** + * \var SensorConfiguration::binY + * \brief Vertical binning factor + * + * The vertical binning factor. Default to 1. + */ + +/** + * \var SensorConfiguration::skipping + * \brief The sensor skipping configuration + * + * Refer to the camera-sensor-model documentation for an accurate description + * of the skipping operations. + * + * If no skipping is performed, all the structure fields should be + * set to 1. Disabled by default. + */ + +/** + * \var SensorConfiguration::xOddInc + * \brief Horizontal increment for odd rows. Default to 1. + */ + +/** + * \var SensorConfiguration::xEvenInc + * \brief Horizontal increment for even rows. Default to 1. + */ + +/** + * \var SensorConfiguration::yOddInc + * \brief Vertical increment for odd columns. Default to 1. + */ + +/** + * \var SensorConfiguration::yEvenInc + * \brief Vertical increment for even columns. Default to 1. + */ + +/** + * \var SensorConfiguration::outputSize + * \brief The frame output (visible) size + * + * The size of the data frame as received by the host processor. + */ + +/** + * \fn SensorConfiguration::valid() const + * \brief Validate the SensorConfiguration + * + * Validate the sensor configuration. + * + * \todo A sensor configuration is valid (or well-formed) if it's either + * completely un-populated or fully populated. For now allow applications to + * populate the bitDepth and the outputSize only. + * + * \return True if the SensorConfiguration is either fully populated or + * un-populated, false otherwise + */ + +/** + * \fn SensorConfiguration::populated() const + * \brief Test if a SensorConfiguration is fully populated + * \return True if the SensorConfiguration is fully populated + */ + +/** + * \brief Validate the sensor configuration + * + * \todo A sensor configuration is valid (or well-formed) if it's either + * completely un-populated or fully populated. For now allow applications to + * populate the bitDepth and the outputSize only. + * + * \return The sensor configuration status + * \retval Unpopulated The sensor configuration is fully unpopulated + * \retval Populated The sensor configuration is fully populated + * \retval Invalid The sensor configuration is invalid (not fully populated + * and not fully unpopulated) + */ +SensorConfiguration::Status SensorConfiguration::validate() const +{ + if (bitDepth && binning.binX && binning.binY && + skipping.xOddInc && skipping.yOddInc && + skipping.xEvenInc && skipping.yEvenInc && + !outputSize.isNull()) + return Populated; + + /* + * By default the binning and skipping factors are initialized to 1, but + * a zero-initialized SensorConfiguration is considered unpopulated + * as well. + */ + if (!bitDepth && + binning.binX <= 1 && binning.binY <= 1 && + skipping.xOddInc <= 1 && skipping.yOddInc <= 1 && + skipping.xEvenInc <= 1 && skipping.yEvenInc <= 1 && + outputSize.isNull()) + return Unpopulated; + + return Invalid; +} + /** * \class CameraConfiguration * \brief Hold configuration for streams of the camera @@ -391,6 +561,19 @@ CameraConfiguration::Status CameraConfiguration::validateColorSpaces(ColorSpaceF return status; } +/** + * \var CameraConfiguration::sensorConfig + * \brief The camera sensor configuration + * + * The sensorConfig field allows control of the configuration of the camera + * sensor. Refer to the camera-sensor-model documentation and to the + * SensorConfiguration class documentation for details about the sensor + * configuration process. + * + * The camera sensor configuration applies to all streams produced by a camera + * from the same image source. + */ + /** * \var CameraConfiguration::transform * \brief User-specified transform to be applied to the image From patchwork Sat Sep 16 12:19:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 19041 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BCBDBBD160 for ; Sat, 16 Sep 2023 12:19:42 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 490126292A; Sat, 16 Sep 2023 14:19:42 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1694866782; bh=Te5nnrRTr32+HLX+QA/gnxn2lc1Wxq8owOHIdBsRk1c=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=vdyoLt7BCy9tH3+Gb6pe1Yc2YtL4eLoSv5nrK6WGLwz+6QYYs8ICnX4HZ8+tTCZ1B mt8QWfibTIDUXpZaqAG5LK7/InPklyIsQx6bmKRwngyMqyco9bd8Nh/JqV0kJV2bMD blOjqnjxwzx3sFs4Z1ekd3SLNYVnVZtNfDayZuAXQtwEptCIWVT4oA+bxISqLnt8AE xvrDYVG6SB9gXQ836qtjJbYbjf6sxyPl8LoDaXtiDzKCJcftOzlnFZ/64zId7bDanD s5VoS3EGvQEPnS8vmLKE9grgiwMBmoRrMAofvfpyGERNaw6OKNo/l0ZOErzBgJH0Uh mh1eRGI32KB3w== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E23866291C for ; Sat, 16 Sep 2023 14:19:38 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Eut56pVd"; dkim-atps=neutral Received: from uno.LocalDomain (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 9DCF88BE; Sat, 16 Sep 2023 14:18:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1694866684; bh=Te5nnrRTr32+HLX+QA/gnxn2lc1Wxq8owOHIdBsRk1c=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Eut56pVdEiWj0VcVrM5ZiBzu3yEVzaTHgs5cPveP1Im2Afhi6VF/FBm7haGEkCnlO nA2csjeNeGK9QHm7QuvwGD3LBXkEnAyyUPrV45uJY6oJhKPy7B55dRGE/FAuUWo2sB 4se5gkqUuc54+IMvQfUznL0ipI7SwyAyewncrarY= To: libcamera-devel@lists.libcamera.org Date: Sat, 16 Sep 2023 14:19:21 +0200 Message-ID: <20230916121930.29490-4-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20230916121930.29490-1-jacopo.mondi@ideasonboard.com> References: <20230916121930.29490-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 03/12] libcamera: camera_sensor: Support SensorConfiguration X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a class function to the CameraSensor class to apply a full configuration to the sensor. The configuration shall be fully populated and shall apply without modifications to the sensor. Signed-off-by: Jacopo Mondi Reviewed-by: Naushir Patuck Reviewed-by: Kieran Bingham --- include/libcamera/internal/camera_sensor.h | 5 ++ src/libcamera/camera_sensor.cpp | 91 ++++++++++++++++++++++ 2 files changed, 96 insertions(+) diff --git a/include/libcamera/internal/camera_sensor.h b/include/libcamera/internal/camera_sensor.h index 02c77ab037da..06791c3c6425 100644 --- a/include/libcamera/internal/camera_sensor.h +++ b/include/libcamera/internal/camera_sensor.h @@ -29,6 +29,7 @@ namespace libcamera { class BayerFormat; class CameraLens; class MediaEntity; +class SensorConfiguration; struct CameraSensorProperties; @@ -58,6 +59,10 @@ public: Transform transform = Transform::Identity); int tryFormat(V4L2SubdeviceFormat *format) const; + int applyConfiguration(const SensorConfiguration &config, + Transform transform = Transform::Identity, + V4L2SubdeviceFormat *sensorFormat = nullptr); + const ControlInfoMap &controls() const; ControlList getControls(const std::vector &ids); int setControls(ControlList *ctrls); diff --git a/src/libcamera/camera_sensor.cpp b/src/libcamera/camera_sensor.cpp index f3a5aa37149f..6c74d0736252 100644 --- a/src/libcamera/camera_sensor.cpp +++ b/src/libcamera/camera_sensor.cpp @@ -15,6 +15,7 @@ #include #include +#include #include #include @@ -822,6 +823,96 @@ int CameraSensor::tryFormat(V4L2SubdeviceFormat *format) const V4L2Subdevice::Whence::TryFormat); } +/** + * \brief Apply a sensor configuration to the camera sensor + * \param[in] config The sensor configuration + * \param[in] transform The transform to be applied on the sensor. + * Defaults to Identity + * \param[out] sensorFormat Optional output parameter where the format + * actually applied to the sensor is returned to the caller + * + * Apply to the camera sensor the configuration \a config. + * + * \todo The configuration shall be fully populated and if any of the fields + * specified cannot be applied exactly, an error code is returned. + * + * \return 0 if \a config unpopulated or is applied correctly to the camera + * sensor, a negative error code otherwise + */ +int CameraSensor::applyConfiguration(const SensorConfiguration &config, + Transform transform, + V4L2SubdeviceFormat *sensorFormat) +{ + if (!config.valid()) { + LOG(CameraSensor, Error) << "Invalid sensor configuration"; + return -EINVAL; + } + + if (!config.populated()) { + LOG(CameraSensor, Debug) << "Sensor configuration unpopulated"; + return 0; + } + + std::vector filteredCodes; + std::copy_if(mbusCodes_.begin(), mbusCodes_.end(), + std::back_inserter(filteredCodes), + [&config](unsigned int mbusCode) { + BayerFormat bayer = BayerFormat::fromMbusCode(mbusCode); + if (bayer.bitDepth == config.bitDepth) + return true; + return false; + }); + if (filteredCodes.empty()) { + LOG(CameraSensor, Error) + << "Cannot find any format with bit depth " + << config.bitDepth; + return -EINVAL; + } + + /* + * Compute the sensor's data frame size by applying the cropping + * rectangle, subsampling and output crop to the sensor's pixel array + * size. + * + * \todo The actual size computation is for now ignored and only the + * output size is considered. This implies that resolutions obtained + * with two different cropping/subsampling will look identical and + * only the first found one will be considered. + */ + V4L2SubdeviceFormat subdevFormat = {}; + for (unsigned int code : filteredCodes) { + for (const Size &size : sizes(code)) { + if (size.width != config.outputSize.width || + size.height != config.outputSize.height) + continue; + + subdevFormat.mbus_code = code; + subdevFormat.size = size; + break; + } + } + if (!subdevFormat.mbus_code) { + LOG(CameraSensor, Error) << "Invalid output size in sensor configuration"; + return -EINVAL; + } + + int ret = setFormat(&subdevFormat, transform); + if (ret) + return ret; + + /* + * Return to the caller the format actually applied to the sensor. + * This is relevant if transform has changed the bayer pattern order. + */ + if (sensorFormat) + *sensorFormat = subdevFormat; + + /* \todo Handle AnalogCrop. Most sensors do not support set_selection */ + /* \todo Handle scaling in the digital domain. */ + + return 0; +} + /** * \brief Retrieve the supported V4L2 controls and their information * From patchwork Sat Sep 16 12:19:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 19042 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3A332C3260 for ; Sat, 16 Sep 2023 12:19:44 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id DDB036292D; Sat, 16 Sep 2023 14:19:43 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1694866783; bh=1UFnEb08D+qUBGtRZdYu02la5CdlORvxcjJ9ikyvB6g=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=ZGCctMU9WvgJ/FKxGm1sDFmXgvCrOybVfusyzV45eM8hS4j6+e5dOmfTsRAzYLsCA VIwNGNe1emaQhMPmFKb/xn5mpKxaZOEMvlCl+WSam6c1zANaGgxSaHVn7LMCSwYD40 1NQ5nG+gY9fEbJtloJuKwkFR1v5i/Wvv2plaqTYLA5JibnklwvZmIGZOe/9ft8d+3D StQJMbyCpKv65vrof3+KSOMOhWibVPDSjuQG91DXLc/15Zg09Ie9WzuSXPkF7b0AgD 8cw+/3zqG1hOC21LXAAbHy5L+/rrhVgtYWkXfKQUUZEl/7MUfL0s4su9+LRbTnVb2y Pti1WOJJizxfQ== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3BA4462918 for ; Sat, 16 Sep 2023 14:19:39 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="loXh4ijx"; dkim-atps=neutral Received: from uno.LocalDomain (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 16CAD1812; Sat, 16 Sep 2023 14:18:05 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1694866685; bh=1UFnEb08D+qUBGtRZdYu02la5CdlORvxcjJ9ikyvB6g=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=loXh4ijxmT1muAvVN7WF3kUceeCui8kZW9zM3kU3FhwozC8iZTnKLjCf+EaJjv626 BNhxw4YdznqlrzE95MOsfpI2+xlmg88C5iVfL3hZEDAbOVcy/Jf35kKoqNxRvFIuz3 khdnUbwWUYOTkeg4JvBuxBPDulSlFtrkTUgQKM2I= To: libcamera-devel@lists.libcamera.org Date: Sat, 16 Sep 2023 14:19:22 +0200 Message-ID: <20230916121930.29490-5-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20230916121930.29490-1-jacopo.mondi@ideasonboard.com> References: <20230916121930.29490-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 04/12] libcamera: rpi: Handle SensorConfiguration X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Handle the SensorConfiguration provided by the application in the pipeline validate() and configure() call chains. During validation, first make sure SensorConfiguration is valid, then handle it to compute the sensor format. For the VC4 platform where the RAW stream follows the sensor's configuration adjust the RAW stream configuration to match the sensor configuration. Reviewed-by: Naushir Patuck Reviewed-by: Kieran Bingham Signed-off-by: Naushir Patuck Reviewed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi --- .../pipeline/rpi/common/pipeline_base.cpp | 62 ++++++++++++++++--- .../pipeline/rpi/common/pipeline_base.h | 4 +- src/libcamera/pipeline/rpi/vc4/vc4.cpp | 28 ++++++++- 3 files changed, 80 insertions(+), 14 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 51fa1bbf9aa9..4d0d150ccf0c 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -180,6 +180,11 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() if (config_.empty()) return Invalid; + if (!sensorConfig.valid()) { + LOG(RPI, Error) << "Invalid sensor configuration request"; + return Invalid; + } + status = validateColorSpaces(ColorSpaceFlag::StreamsShareColorSpace); /* @@ -207,19 +212,43 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() std::sort(outStreams.begin(), outStreams.end(), [](auto &l, auto &r) { return l.cfg->size > r.cfg->size; }); - /* Compute the sensor configuration. */ - unsigned int bitDepth = defaultRawBitDepth; - if (!rawStreams.empty()) { + /* Compute the sensor's format then do any platform specific fixups. */ + unsigned int bitDepth; + Size sensorSize; + + if (sensorConfig.populated()) { + /* Use the application provided sensor configuration. */ + bitDepth = sensorConfig.bitDepth; + sensorSize = sensorConfig.outputSize; + } else if (!rawStreams.empty()) { + /* Use the RAW stream format and size. */ BayerFormat bayerFormat = BayerFormat::fromPixelFormat(rawStreams[0].cfg->pixelFormat); bitDepth = bayerFormat.bitDepth; + sensorSize = rawStreams[0].cfg->size; + } else { + bitDepth = defaultRawBitDepth; + sensorSize = outStreams[0].cfg->size; } - sensorFormat_ = data_->findBestFormat(rawStreams.empty() ? outStreams[0].cfg->size - : rawStreams[0].cfg->size, - bitDepth); + sensorFormat_ = data_->findBestFormat(sensorSize, bitDepth); + + /* + * If a sensor configuration has been requested, it should apply + * without modifications. + */ + if (sensorConfig.populated()) { + BayerFormat bayer = BayerFormat::fromMbusCode(sensorFormat_.mbus_code); + + if (bayer.bitDepth != sensorConfig.bitDepth || + sensorFormat_.size != sensorConfig.outputSize) { + LOG(RPI, Error) << "Invalid sensor configuration: " + << "bitDepth/size mismatch"; + return Invalid; + } + } /* Do any platform specific fixups. */ - status = data_->platformValidate(rawStreams, outStreams); + status = data_->platformValidate(this, rawStreams, outStreams); if (status == Invalid) return Invalid; @@ -467,12 +496,25 @@ int PipelineHandlerBase::configure(Camera *camera, CameraConfiguration *config) std::sort(ispStreams.begin(), ispStreams.end(), [](auto &l, auto &r) { return l.cfg->size > r.cfg->size; }); - /* Apply the format on the sensor with any cached transform. */ + /* + * Apply the format on the sensor with any cached transform. + * + * If the application has provided a sensor configuration apply it + * instead of just applying a format. + */ const RPiCameraConfiguration *rpiConfig = static_cast(config); - V4L2SubdeviceFormat sensorFormat = rpiConfig->sensorFormat_; + V4L2SubdeviceFormat sensorFormat; - ret = data->sensor_->setFormat(&sensorFormat, rpiConfig->combinedTransform_); + if (rpiConfig->sensorConfig.populated()) { + ret = data->sensor_->applyConfiguration(rpiConfig->sensorConfig, + rpiConfig->combinedTransform_, + &sensorFormat); + } else { + sensorFormat = rpiConfig->sensorFormat_; + ret = data->sensor_->setFormat(&sensorFormat, + rpiConfig->combinedTransform_); + } if (ret) return ret; diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index dbabc61ea48c..81b2b7d2f4d1 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -42,6 +42,7 @@ namespace RPi { /* Map of mbus codes to supported sizes reported by the sensor. */ using SensorFormats = std::map>; +class RPiCameraConfiguration; class CameraData : public Camera::Private { public: @@ -72,7 +73,8 @@ public: V4L2VideoDevice *dev; }; - virtual CameraConfiguration::Status platformValidate(std::vector &rawStreams, + virtual CameraConfiguration::Status platformValidate(RPiCameraConfiguration *rpiConfig, + std::vector &rawStreams, std::vector &outStreams) const = 0; virtual int platformConfigure(const V4L2SubdeviceFormat &sensorFormat, std::optional packing, diff --git a/src/libcamera/pipeline/rpi/vc4/vc4.cpp b/src/libcamera/pipeline/rpi/vc4/vc4.cpp index 018cf4881d0e..2670eb8c4bbc 100644 --- a/src/libcamera/pipeline/rpi/vc4/vc4.cpp +++ b/src/libcamera/pipeline/rpi/vc4/vc4.cpp @@ -65,7 +65,8 @@ public: { } - CameraConfiguration::Status platformValidate(std::vector &rawStreams, + CameraConfiguration::Status platformValidate(RPi::RPiCameraConfiguration *rpiConfig, + std::vector &rawStreams, std::vector &outStreams) const override; int platformPipelineConfigure(const std::unique_ptr &root) override; @@ -394,7 +395,8 @@ int PipelineHandlerVc4::platformRegister(std::unique_ptr &camer return 0; } -CameraConfiguration::Status Vc4CameraData::platformValidate(std::vector &rawStreams, +CameraConfiguration::Status Vc4CameraData::platformValidate(RPi::RPiCameraConfiguration *rpiConfig, + std::vector &rawStreams, std::vector &outStreams) const { CameraConfiguration::Status status = CameraConfiguration::Status::Valid; @@ -405,9 +407,27 @@ CameraConfiguration::Status Vc4CameraData::platformValidate(std::vectorsensorFormat_.mbus_code); + + /* Handle flips to make sure to match the RAW stream format. */ + if (flipsAlterBayerOrder_) + rawBayer = rawBayer.transform(rpiConfig->combinedTransform_); + PixelFormat rawFormat = rawBayer.toPixelFormat(); + + if (rawStream->pixelFormat != rawFormat || + rawStream->size != rpiConfig->sensorFormat_.size) { + rawStream->pixelFormat = rawFormat; + rawStream->size = rpiConfig->sensorFormat_.size; + + status = CameraConfiguration::Adjusted; + } + } + /* * For the two ISP outputs, one stream must be equal or smaller than the * other in all dimensions. @@ -417,6 +437,8 @@ CameraConfiguration::Status Vc4CameraData::platformValidate(std::vectorsize.width, outStreams[0].cfg->size.width); size.height = std::min(outStreams[i].cfg->size.height, From patchwork Sat Sep 16 12:19:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 19043 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2635CBD160 for ; Sat, 16 Sep 2023 12:19:46 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B51CE6291F; Sat, 16 Sep 2023 14:19:45 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1694866785; bh=QfR+JclEnCYqYSFFstRnyrbg8ePcbMNcod+QMO9f+wQ=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=LfYQ2gXr14ts9qxW5AfIITiICEDC0zPEQcYzKfviTtk9XH5HAtLlut+keo7tfyxGr 53l8ndlvEXptiKa4nsI3GBrSxqvSsDCT0i2xw/V9Igo8Vr529AAS3/UF+NaPWnVobF 88kVk3O5w2CkemAl/nr9vRZhnQAS4SJ6ZcD30XBHvNnypfWmwTbn9+YU0pDtg5XLJ4 HFexbDY0uhs4j1n0DKiBBcCKYxkjpz6R5OLfGoQxzPFuenghOitLjKg2zJqbKd/KmD dmmODVz+3ySxUUiUa3c7SOD8BDinSDP/TosuellyRveZRiWC5uRUb/eYwa8YsJpjQW BpXCSi22LQSKA== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A278162911 for ; Sat, 16 Sep 2023 14:19:39 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="DY6sQi6W"; dkim-atps=neutral Received: from uno.LocalDomain (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 881C91536; Sat, 16 Sep 2023 14:18:05 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1694866685; bh=QfR+JclEnCYqYSFFstRnyrbg8ePcbMNcod+QMO9f+wQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DY6sQi6WC1srU0fNNaqgOr/Z92TjOCLSm4unVQyQZBS6tibGVA/QQYj8KlcHZa/sI mQsjQVfv8yaTx0N2VL1SzasdQ6NQBP2okzB+0ZaFgO4N2PqXi6el2wUoCwqrqIIUms TEhtusxSR5wTTqF91A6qeAA8FhdyJR9RA9+HA7FY= To: libcamera-devel@lists.libcamera.org Date: Sat, 16 Sep 2023 14:19:23 +0200 Message-ID: <20230916121930.29490-6-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20230916121930.29490-1-jacopo.mondi@ideasonboard.com> References: <20230916121930.29490-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 05/12] libcamera: rpi: Allow platformValidate() to adjust format strides X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Naushir Patuck Propagate any changes to the format stride done by platformValidate(). The stride value may be adjusted for performace reasons. Signed-off-by: Naushir Patuck Reviewed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/rpi/common/pipeline_base.cpp | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 4d0d150ccf0c..566aa8f3211e 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -305,6 +305,17 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() V4L2DeviceFormat format; format.fourcc = out.dev->toV4L2PixelFormat(cfg.pixelFormat); format.size = cfg.size; + + /* + * platformValidate may have worked out the correct stride so we + * must pass it in. This also needs the planesCount to be set + * correctly or the stride will be ignored. + */ + const PixelFormat &pixFormat = format.fourcc.toPixelFormat(); + const PixelFormatInfo &info = PixelFormatInfo::info(pixFormat); + format.planesCount = info.numPlanes(); + format.planes[0].bpl = cfg.stride; + /* We want to send the associated YCbCr info through to the driver. */ format.colorSpace = yuvColorSpace_; From patchwork Sat Sep 16 12:19:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 19044 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 32DEEC3260 for ; Sat, 16 Sep 2023 12:19:47 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 55E316291C; Sat, 16 Sep 2023 14:19:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1694866786; bh=TgvmlQebFId82xkKIXRM6yTqPPtRc2xTjkzERob04hU=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=xCZjUo+aMUo5j6vKZa8qNzZDoTmcY3SIgXutaL4o0CrqH8z5FcNn7uYfuvVPYIchY SMH2CnyFBdIui7/CCRXRTDMhM+9QmE6D0n0vFRYjHz59mOWmePSvYfup7r9ZLd1WBc WiSvvGDLmA+PMEHfUV1CM1SkO8A4DVNhQZl7603v+XNJbtUzVIoGymYnR3grJ267Vz 68PEV3tJYGrZRpXvLOoP0vzLCQr/NzUcJ4+bFUtbJ6l61BIiMQGmpxJmQHrwH5xNB1 LdWywx2j46fth93c9MCysaEzhV/Zk4zhQnAs8nmFf8JQZMEXj4ZkhzyDyASLUDcBb4 Nl207qjQdSOMA== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 06F966291F for ; Sat, 16 Sep 2023 14:19:40 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="USeZ3JOT"; dkim-atps=neutral Received: from uno.LocalDomain (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E8AE910FE; Sat, 16 Sep 2023 14:18:05 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1694866686; bh=TgvmlQebFId82xkKIXRM6yTqPPtRc2xTjkzERob04hU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=USeZ3JOTaFLQ6K5OSLrUwX33V6U7eIowIrQZnvq4U/hkYGvM/3eRzHkkUMpbpdL3a SnWFQrI5gcv47AKe8hsGLFnz0jiIEGkDhVptHNdIvmuY/jBLUm3tRqkF6qDCfmJayy yEeCJh9iME3VqdO5HONj3HKzQS6iZNh8n+3y9yA8= To: libcamera-devel@lists.libcamera.org Date: Sat, 16 Sep 2023 14:19:24 +0200 Message-ID: <20230916121930.29490-7-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20230916121930.29490-1-jacopo.mondi@ideasonboard.com> References: <20230916121930.29490-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 06/12] libcamera: rpi: Fix wrong comment indentation X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The closing line of a comment block was aligned with spaces and not tabs. Fix it. Signed-off-by: Jacopo Mondi Reviewed-by: Kieran Bingham --- src/libcamera/pipeline/rpi/common/pipeline_base.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- 2.42.0 diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index 81b2b7d2f4d1..3e7c487fb0bf 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -273,7 +273,7 @@ private: * Store the colour spaces that all our streams will have. RGB format streams * will have the same colorspace as YUV streams, with YCbCr field cleared and * range set to full. - */ + */ std::optional yuvColorSpace_; std::optional rgbColorSpace_; }; From patchwork Sat Sep 16 12:19:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 19045 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 16DC5BD160 for ; Sat, 16 Sep 2023 12:19:48 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9FAE662905; Sat, 16 Sep 2023 14:19:47 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1694866787; bh=Cz7IxIHRU7epEvaalMTcAR4pQRDJV+Kx1uhamDjjM/o=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=VI0hzNWZ97MkzlYlh2cTPyogiz/ZE72ZBCU6gtMKKVY1EHtEhLtVBjPREm1nemxNF uCpV5hsttvpaChBi9YAvS0vo+QG6JhPySS2xw1twFmJKdpWzp3PuNi5KToFAyhvN4s m7Tqwf8WBqyk3wB+kJ00MeXZ4ChMHDBIn9sTAGV5q74/pVbRLxGYw4rWFMyQkUpgeL 8JnL5O/U5OBZUPr4i5R7sY/C5m1W71TLBGLCzvHpF4H5ld1QR8HPORKHFScBNBm6LW U1TGqIoH0oJ48FchKBsNNR0lqW4siY8iplvYTzvbDyNc1eIj66PPIyI/r9uCkgimUd 8RUFi9py/w5vQ== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7227262925 for ; Sat, 16 Sep 2023 14:19:40 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Plj2UhxQ"; dkim-atps=neutral Received: from uno.LocalDomain (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 53FC08BE; Sat, 16 Sep 2023 14:18:06 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1694866686; bh=Cz7IxIHRU7epEvaalMTcAR4pQRDJV+Kx1uhamDjjM/o=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Plj2UhxQCcE10Ng7NRPuoATe9+eN3rRpMw7Fmn5UZXwzMCK/W0FvnFu9PcuLo5k5P c5RfCd2ufqf2AJjKEfAgW4aR0Z5Dzdwa0K78416MYffIQbgC6giPbMXPnlc92y+eY+ z3NNct0qleQMFHhyFKZay43LoLKZ50dyX2Jp19mM= To: libcamera-devel@lists.libcamera.org Date: Sat, 16 Sep 2023 14:19:25 +0200 Message-ID: <20230916121930.29490-8-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20230916121930.29490-1-jacopo.mondi@ideasonboard.com> References: <20230916121930.29490-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 07/12] libcamera: rpi: Make isRaw/isYuv/isRgb static functions X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Naushir Patuck Move the existing isRaw()/isYuv()/isRgb()into a static function of PipelineHandlerBase. This will allow then to be shared with the pipeline handler derived class. Signed-off-by: Naushir Patuck Signed-off-by: Jacopo Mondi Reviewed-by: Jacopo Mondi --- .../pipeline/rpi/common/pipeline_base.cpp | 52 +++++++++---------- .../pipeline/rpi/common/pipeline_base.h | 4 ++ 2 files changed, 30 insertions(+), 26 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 566aa8f3211e..ebb79337d092 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -37,12 +37,6 @@ namespace { constexpr unsigned int defaultRawBitDepth = 12; -bool isRaw(const PixelFormat &pixFmt) -{ - /* This test works for both Bayer and raw mono formats. */ - return BayerFormat::fromPixelFormat(pixFmt).isValid(); -} - PixelFormat mbusCodeToPixelFormat(unsigned int mbus_code, BayerFormat::Packing packingReq) { @@ -91,22 +85,6 @@ std::optional findValidColorSpace(const ColorSpace &colourSpace) return std::nullopt; } -bool isRgb(const PixelFormat &pixFmt) -{ - const PixelFormatInfo &info = PixelFormatInfo::info(pixFmt); - return info.colourEncoding == PixelFormatInfo::ColourEncodingRGB; -} - -bool isYuv(const PixelFormat &pixFmt) -{ - /* The code below would return true for raw mono streams, so weed those out first. */ - if (isRaw(pixFmt)) - return false; - - const PixelFormatInfo &info = PixelFormatInfo::info(pixFmt); - return info.colourEncoding == PixelFormatInfo::ColourEncodingYUV; -} - } /* namespace */ /* @@ -129,7 +107,7 @@ CameraConfiguration::Status RPiCameraConfiguration::validateColorSpaces([[maybe_ for (auto cfg : config_) { /* First fix up raw streams to have the "raw" colour space. */ - if (isRaw(cfg.pixelFormat)) { + if (PipelineHandlerBase::isRaw(cfg.pixelFormat)) { /* If there was no value here, that doesn't count as "adjusted". */ if (cfg.colorSpace && cfg.colorSpace != ColorSpace::Raw) status = Adjusted; @@ -156,13 +134,13 @@ CameraConfiguration::Status RPiCameraConfiguration::validateColorSpaces([[maybe_ if (cfg.colorSpace == ColorSpace::Raw) continue; - if (isYuv(cfg.pixelFormat) && cfg.colorSpace != yuvColorSpace_) { + if (PipelineHandlerBase::isYuv(cfg.pixelFormat) && cfg.colorSpace != yuvColorSpace_) { /* Again, no value means "not adjusted". */ if (cfg.colorSpace) status = Adjusted; cfg.colorSpace = yuvColorSpace_; } - if (isRgb(cfg.pixelFormat) && cfg.colorSpace != rgbColorSpace_) { + if (PipelineHandlerBase::isRgb(cfg.pixelFormat) && cfg.colorSpace != rgbColorSpace_) { /* Be nice, and let the YUV version count as non-adjusted too. */ if (cfg.colorSpace && cfg.colorSpace != yuvColorSpace_) status = Adjusted; @@ -199,7 +177,7 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() std::vector rawStreams, outStreams; for (const auto &[index, cfg] : utils::enumerate(config_)) { - if (isRaw(cfg.pixelFormat)) + if (PipelineHandlerBase::isRaw(cfg.pixelFormat)) rawStreams.emplace_back(index, &cfg); else outStreams.emplace_back(index, &cfg); @@ -347,6 +325,28 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() return status; } +bool PipelineHandlerBase::isRgb(const PixelFormat &pixFmt) +{ + const PixelFormatInfo &info = PixelFormatInfo::info(pixFmt); + return info.colourEncoding == PixelFormatInfo::ColourEncodingRGB; +} + +bool PipelineHandlerBase::isYuv(const PixelFormat &pixFmt) +{ + /* The code below would return true for raw mono streams, so weed those out first. */ + if (PipelineHandlerBase::isRaw(pixFmt)) + return false; + + const PixelFormatInfo &info = PixelFormatInfo::info(pixFmt); + return info.colourEncoding == PixelFormatInfo::ColourEncodingYUV; +} + +bool PipelineHandlerBase::isRaw(const PixelFormat &pixFmt) +{ + /* This test works for both Bayer and raw mono formats. */ + return BayerFormat::fromPixelFormat(pixFmt).isValid(); +} + V4L2DeviceFormat PipelineHandlerBase::toV4L2DeviceFormat(const V4L2VideoDevice *dev, const V4L2SubdeviceFormat &format, BayerFormat::Packing packingReq) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index 3e7c487fb0bf..135b74392140 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -213,6 +213,10 @@ public: { } + static bool isRgb(const PixelFormat &pixFmt); + static bool isYuv(const PixelFormat &pixFmt); + static bool isRaw(const PixelFormat &pixFmt); + static V4L2DeviceFormat toV4L2DeviceFormat(const V4L2VideoDevice *dev, const V4L2SubdeviceFormat &format, BayerFormat::Packing packingReq); From patchwork Sat Sep 16 12:19:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 19046 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1811CC32B0 for ; Sat, 16 Sep 2023 12:19:49 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B4DA862925; Sat, 16 Sep 2023 14:19:48 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1694866788; bh=Zc9J6NLdY30XhehyRaFBy+qampP6ydKK6Hqcd/w0kz4=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=Fk0mJs6y6b/Vi047q8ohx5KPgwtj1BaO682a3oI7++KKWKgjAbj1t6reF53sYHVil At9LbraDigUEQQQD8qlIqLGhIdpEIKVfYER041p8gzpo2KhrcCNcNxOPJ2qpg3Hk3m +GRw5Jj4WtWkGU6AMSsrG7Xl6FWAOznYKnLgFP9sbwB1N81pWY9cvCkFB6gzwrkSLX 2yObEGlMA+GWSB7nMIAqTGnVIDhNG58qZZtn3AWl046wefHTPLCeirR8A9xDwDRnjP b8eIY9SReT4hSFMXcltlporRg1Xjnluq8pSPz1g3cfIp8hEoKICMTYxnSmOcGsevxG tS4ULDT9Gmxpg== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D015262921 for ; Sat, 16 Sep 2023 14:19:40 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="q32Cetsn"; dkim-atps=neutral Received: from uno.LocalDomain (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id B96E71812; Sat, 16 Sep 2023 14:18:06 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1694866687; bh=Zc9J6NLdY30XhehyRaFBy+qampP6ydKK6Hqcd/w0kz4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=q32CetsnGmPl0srVW0QBt2geSjekkCQcH6YuFonifaGiMIJZkiTsbI+bDyimLqOHn nBsAdgehsCkdGnvETjdtxGLWTPKAHq95rgrfVgCabodHlexePpMM4Q1VTPk5n10K+V uDxjk0wmkjzo0rWliLSpALMpJT5nl1IQzau3PyTM= To: libcamera-devel@lists.libcamera.org Date: Sat, 16 Sep 2023 14:19:26 +0200 Message-ID: <20230916121930.29490-9-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20230916121930.29490-1-jacopo.mondi@ideasonboard.com> References: <20230916121930.29490-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 08/12] libcamera: rpi: Cache rawStreams and outStreams X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Naushir Patuck Currently, the stream configuration is stored in two vectors, rawStreams and outStreams for convenience. However, these vectors are constructed in both platformValidate() and platformConfigure(). This change caches these vectors in the RPiCameraConfiguration class to construct them only once in platformValidate(). Pass a pointer to the current configuration to platformValidate() and platformConfigure() so that they can access the streams vectors. Signed-off-by: Naushir Patuck Reviewed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi --- .../pipeline/rpi/common/pipeline_base.cpp | 50 +++++++------------ .../pipeline/rpi/common/pipeline_base.h | 42 ++++++++-------- src/libcamera/pipeline/rpi/vc4/vc4.cpp | 20 ++++---- 3 files changed, 48 insertions(+), 64 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index ebb79337d092..e7fedca7cc8d 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -175,19 +175,21 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() if (transform != requestedTransform) status = Adjusted; - std::vector rawStreams, outStreams; + rawStreams_.clear(); + outStreams_.clear(); + for (const auto &[index, cfg] : utils::enumerate(config_)) { if (PipelineHandlerBase::isRaw(cfg.pixelFormat)) - rawStreams.emplace_back(index, &cfg); + rawStreams_.emplace_back(index, &cfg); else - outStreams.emplace_back(index, &cfg); + outStreams_.emplace_back(index, &cfg); } /* Sort the streams so the highest resolution is first. */ - std::sort(rawStreams.begin(), rawStreams.end(), + std::sort(rawStreams_.begin(), rawStreams_.end(), [](auto &l, auto &r) { return l.cfg->size > r.cfg->size; }); - std::sort(outStreams.begin(), outStreams.end(), + std::sort(outStreams_.begin(), outStreams_.end(), [](auto &l, auto &r) { return l.cfg->size > r.cfg->size; }); /* Compute the sensor's format then do any platform specific fixups. */ @@ -198,14 +200,14 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() /* Use the application provided sensor configuration. */ bitDepth = sensorConfig.bitDepth; sensorSize = sensorConfig.outputSize; - } else if (!rawStreams.empty()) { + } else if (!rawStreams_.empty()) { /* Use the RAW stream format and size. */ - BayerFormat bayerFormat = BayerFormat::fromPixelFormat(rawStreams[0].cfg->pixelFormat); + BayerFormat bayerFormat = BayerFormat::fromPixelFormat(rawStreams_[0].cfg->pixelFormat); bitDepth = bayerFormat.bitDepth; - sensorSize = rawStreams[0].cfg->size; + sensorSize = rawStreams_[0].cfg->size; } else { bitDepth = defaultRawBitDepth; - sensorSize = outStreams[0].cfg->size; + sensorSize = outStreams_[0].cfg->size; } sensorFormat_ = data_->findBestFormat(sensorSize, bitDepth); @@ -226,12 +228,12 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() } /* Do any platform specific fixups. */ - status = data_->platformValidate(this, rawStreams, outStreams); + status = data_->platformValidate(this); if (status == Invalid) return Invalid; /* Further fixups on the RAW streams. */ - for (auto &raw : rawStreams) { + for (auto &raw : rawStreams_) { StreamConfiguration &cfg = config_.at(raw.index); V4L2DeviceFormat rawFormat; @@ -269,7 +271,7 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() } /* Further fixups on the ISP output streams. */ - for (auto &out : outStreams) { + for (auto &out : outStreams_) { StreamConfiguration &cfg = config_.at(out.index); PixelFormat &cfgPixFmt = cfg.pixelFormat; V4L2VideoDevice::Formats fmts = out.dev->formats(); @@ -489,24 +491,6 @@ int PipelineHandlerBase::configure(Camera *camera, CameraConfiguration *config) for (auto const stream : data->streams_) stream->clearFlags(StreamFlag::External); - std::vector rawStreams, ispStreams; - - for (unsigned i = 0; i < config->size(); i++) { - StreamConfiguration *cfg = &config->at(i); - - if (isRaw(cfg->pixelFormat)) - rawStreams.emplace_back(i, cfg); - else - ispStreams.emplace_back(i, cfg); - } - - /* Sort the streams so the highest resolution is first. */ - std::sort(rawStreams.begin(), rawStreams.end(), - [](auto &l, auto &r) { return l.cfg->size > r.cfg->size; }); - - std::sort(ispStreams.begin(), ispStreams.end(), - [](auto &l, auto &r) { return l.cfg->size > r.cfg->size; }); - /* * Apply the format on the sensor with any cached transform. * @@ -531,9 +515,9 @@ int PipelineHandlerBase::configure(Camera *camera, CameraConfiguration *config) /* Use the user requested packing/bit-depth. */ std::optional packing; - if (!rawStreams.empty()) { + if (!rpiConfig->rawStreams_.empty()) { BayerFormat bayerFormat = - BayerFormat::fromPixelFormat(rawStreams[0].cfg->pixelFormat); + BayerFormat::fromPixelFormat(rpiConfig->rawStreams_[0].cfg->pixelFormat); packing = bayerFormat.packing; } @@ -541,7 +525,7 @@ int PipelineHandlerBase::configure(Camera *camera, CameraConfiguration *config) * Platform specific internal stream configuration. This also assigns * external streams which get configured below. */ - ret = data->platformConfigure(sensorFormat, packing, rawStreams, ispStreams); + ret = data->platformConfigure(sensorFormat, packing, rpiConfig); if (ret) return ret; diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index 135b74392140..1a3a13dbb6cf 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -57,29 +57,10 @@ public: { } - struct StreamParams { - StreamParams() - : index(0), cfg(nullptr), dev(nullptr) - { - } - - StreamParams(unsigned int index_, StreamConfiguration *cfg_) - : index(index_), cfg(cfg_), dev(nullptr) - { - } - - unsigned int index; - StreamConfiguration *cfg; - V4L2VideoDevice *dev; - }; - - virtual CameraConfiguration::Status platformValidate(RPiCameraConfiguration *rpiConfig, - std::vector &rawStreams, - std::vector &outStreams) const = 0; + virtual CameraConfiguration::Status platformValidate(RPiCameraConfiguration *rpiConfig) const = 0; virtual int platformConfigure(const V4L2SubdeviceFormat &sensorFormat, std::optional packing, - std::vector &rawStreams, - std::vector &outStreams) = 0; + const RPiCameraConfiguration *rpiConfig) = 0; virtual void platformStart() = 0; virtual void platformStop() = 0; @@ -270,6 +251,25 @@ public: /* The sensor format computed in validate() */ V4L2SubdeviceFormat sensorFormat_; + struct StreamParams { + StreamParams() + : index(0), cfg(nullptr), dev(nullptr) + { + } + + StreamParams(unsigned int index_, StreamConfiguration *cfg_) + : index(index_), cfg(cfg_), dev(nullptr) + { + } + + unsigned int index; + StreamConfiguration *cfg; + V4L2VideoDevice *dev; + }; + + std::vector rawStreams_; + std::vector outStreams_; + private: const CameraData *data_; diff --git a/src/libcamera/pipeline/rpi/vc4/vc4.cpp b/src/libcamera/pipeline/rpi/vc4/vc4.cpp index 2670eb8c4bbc..2308577a613b 100644 --- a/src/libcamera/pipeline/rpi/vc4/vc4.cpp +++ b/src/libcamera/pipeline/rpi/vc4/vc4.cpp @@ -25,6 +25,7 @@ namespace libcamera { LOG_DECLARE_CATEGORY(RPI) using StreamFlag = RPi::Stream::StreamFlag; +using StreamParams = RPi::RPiCameraConfiguration::StreamParams; namespace { @@ -65,9 +66,7 @@ public: { } - CameraConfiguration::Status platformValidate(RPi::RPiCameraConfiguration *rpiConfig, - std::vector &rawStreams, - std::vector &outStreams) const override; + CameraConfiguration::Status platformValidate(RPi::RPiCameraConfiguration *rpiConfig) const override; int platformPipelineConfigure(const std::unique_ptr &root) override; @@ -118,8 +117,7 @@ private: int platformConfigure(const V4L2SubdeviceFormat &sensorFormat, std::optional packing, - std::vector &rawStreams, - std::vector &outStreams) override; + const RPi::RPiCameraConfiguration *rpiConfig) override; int platformConfigureIpa(ipa::RPi::ConfigParams ¶ms) override; int platformInitIpa([[maybe_unused]] ipa::RPi::InitParams ¶ms) override @@ -395,10 +393,11 @@ int PipelineHandlerVc4::platformRegister(std::unique_ptr &camer return 0; } -CameraConfiguration::Status Vc4CameraData::platformValidate(RPi::RPiCameraConfiguration *rpiConfig, - std::vector &rawStreams, - std::vector &outStreams) const +CameraConfiguration::Status Vc4CameraData::platformValidate(RPi::RPiCameraConfiguration *rpiConfig) const { + std::vector &rawStreams = rpiConfig->rawStreams_; + std::vector &outStreams = rpiConfig->outStreams_; + CameraConfiguration::Status status = CameraConfiguration::Status::Valid; /* Can only output 1 RAW stream, or 2 YUV/RGB streams. */ @@ -503,9 +502,10 @@ int Vc4CameraData::platformPipelineConfigure(const std::unique_ptr & int Vc4CameraData::platformConfigure(const V4L2SubdeviceFormat &sensorFormat, std::optional packing, - std::vector &rawStreams, - std::vector &outStreams) + const RPi::RPiCameraConfiguration *rpiConfig) { + const std::vector &rawStreams = rpiConfig->rawStreams_; + const std::vector &outStreams = rpiConfig->outStreams_; int ret; if (!packing) From patchwork Sat Sep 16 12:19:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 19047 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CEDFEC32B1 for ; Sat, 16 Sep 2023 12:19:49 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5C2E462924; Sat, 16 Sep 2023 14:19:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1694866789; bh=/ln7JqqBJ30qhFJf6xExSqV9yCyElhc+wER0kQIz3MI=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=gemnU+XvwySnPCDpomvVflJ71WX70mD92xDROfT5yDfXgVhp4JYqpWbI5I1pi0wxG m1ei7iJJh9NXmCUvR0/ax96KCO3gN1QTfAt4oU750U/hvmAJkC10tJ14jpYzYuAios C5mbH+YGnVmxaY9hO4p791tICBXslkCxkgaRO7VlTMK693lYD6Wgz+OeEjfwvpUTnQ eq+r+0GOcTFkrKJ5hz1YOx9O/6eI9Zz0/4hJwr8/dmQfrOFZyHLUC0ALXUFR2vk9Vd 1idXn42bru3HlS+Kbl74vTi5Jz6Yfu55eUAI8qbK1FO51+KOHpars8jgm55yUyGAZq sq58cHox0Xh2w== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3DE2D62929 for ; Sat, 16 Sep 2023 14:19:41 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="kveqQFe6"; dkim-atps=neutral Received: from uno.LocalDomain (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 277461536; Sat, 16 Sep 2023 14:18:07 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1694866687; bh=/ln7JqqBJ30qhFJf6xExSqV9yCyElhc+wER0kQIz3MI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kveqQFe6bZRWxbIFeLAd/aoP4LWCtpZyimpdSfVWsBAoFNvy2iXbguSDHB0siYNyq oS5I1yT3taOp8yNzsfISImepryKGPRDi13GaLqbvRN9DcywXVYUMCDVx+LELe9A90m GDGCCITtcg2q4xErXljgdq/zpyb1EpSu7GeyU3Dc= To: libcamera-devel@lists.libcamera.org Date: Sat, 16 Sep 2023 14:19:27 +0200 Message-ID: <20230916121930.29490-10-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20230916121930.29490-1-jacopo.mondi@ideasonboard.com> References: <20230916121930.29490-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 09/12] libcamera: rpi: Add some helpers to PipelineHandlerBase X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Naushir Patuck Add a helper updateStreamConfig() that updates the format related fields in a StreamConfiguration from a given V4L2DeviceFormat structure. Add and override to the toV4L2DeviceFormat() helper that returns a V4L2DeviceFormat structure populated from the format related fields in a StreamConfiguration. Both these helper functions will be used in a future commit to simplify the Raspberry Pi pipeline handler configuration/validation code. Signed-off-by: Naushir Patuck Reviewed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi --- .../pipeline/rpi/common/pipeline_base.cpp | 47 +++++++++++++++++++ .../pipeline/rpi/common/pipeline_base.h | 4 ++ 2 files changed, 51 insertions(+) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index e7fedca7cc8d..93779556fd81 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -349,6 +349,53 @@ bool PipelineHandlerBase::isRaw(const PixelFormat &pixFmt) return BayerFormat::fromPixelFormat(pixFmt).isValid(); } +/* + * Adjust a StreamConfiguration fields to match a video device format. + * Returns true if the StreamConfiguration has been adjusted. + */ +bool PipelineHandlerBase::updateStreamConfig(StreamConfiguration *stream, + const V4L2DeviceFormat &format) +{ + const PixelFormat &pixFormat = format.fourcc.toPixelFormat(); + bool adjusted = false; + + if (stream->pixelFormat != pixFormat || stream->size != format.size) { + stream->pixelFormat = pixFormat; + stream->size = format.size; + adjusted = true; + } + + if (stream->colorSpace != format.colorSpace) { + stream->colorSpace = format.colorSpace; + adjusted = true; + LOG(RPI, Debug) + << "Color space changed from " + << ColorSpace::toString(stream->colorSpace) << " to " + << ColorSpace::toString(format.colorSpace); + } + + stream->stride = format.planes[0].bpl; + stream->frameSize = format.planes[0].size; + + return adjusted; +} + +/* + * Populate and return a video device format using a StreamConfiguration. */ +V4L2DeviceFormat PipelineHandlerBase::toV4L2DeviceFormat(const V4L2VideoDevice *dev, + const StreamConfiguration *stream) +{ + V4L2DeviceFormat deviceFormat; + + const PixelFormatInfo &info = PixelFormatInfo::info(stream->pixelFormat); + deviceFormat.planesCount = info.numPlanes(); + deviceFormat.fourcc = dev->toV4L2PixelFormat(stream->pixelFormat); + deviceFormat.size = stream->size; + deviceFormat.colorSpace = stream->colorSpace; + + return deviceFormat; +} + V4L2DeviceFormat PipelineHandlerBase::toV4L2DeviceFormat(const V4L2VideoDevice *dev, const V4L2SubdeviceFormat &format, BayerFormat::Packing packingReq) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index 1a3a13dbb6cf..491c5e98c4a1 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -198,6 +198,10 @@ public: static bool isYuv(const PixelFormat &pixFmt); static bool isRaw(const PixelFormat &pixFmt); + static bool updateStreamConfig(StreamConfiguration *stream, + const V4L2DeviceFormat &format); + static V4L2DeviceFormat toV4L2DeviceFormat(const V4L2VideoDevice *dev, + const StreamConfiguration *stream); static V4L2DeviceFormat toV4L2DeviceFormat(const V4L2VideoDevice *dev, const V4L2SubdeviceFormat &format, BayerFormat::Packing packingReq); From patchwork Sat Sep 16 12:19:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 19048 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B4356C3260 for ; Sat, 16 Sep 2023 12:19:50 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 68E5162922; Sat, 16 Sep 2023 14:19:50 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1694866790; bh=joF5SwOXNjXn9yy67Ol4wUMncn/0xR5d/g+WVYRbqCc=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=4ZXo51oIKOqOEk8XS7pqVnzUM9y8W02xex4WKdki92RrNvyU2PgJqtcJWdPPFeClz Jg6Peqane45z15peNehU1T/kSERzXbDozgjBH++uNdywBKKEbQfgRHUITfFOm3m0oJ dp/ZvYvW3lt97/kvSZ0Pzt2NEVeU+xeWE+0yeMbqkHjWjCjgl3oo32hzkinaQQdUVi DBvvG71ay4aQ69ULPnO5wLYUySwJVp4VMYlPxd+cS8R86YnmjMLLLzfrYK/3EQMuy3 PU9JrOQyWUlme/e4X4/BW+yaximLRGZfeSPRPQ2S82t38wT+4q+bDkR4Tvxa9L6UOW spLZGqdhciLmQ== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9A633628E9 for ; Sat, 16 Sep 2023 14:19:41 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="gVHP36gj"; dkim-atps=neutral Received: from uno.LocalDomain (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 85D2F8BE; Sat, 16 Sep 2023 14:18:07 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1694866687; bh=joF5SwOXNjXn9yy67Ol4wUMncn/0xR5d/g+WVYRbqCc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=gVHP36gjQaRgdyqb93iQRQtCwkomkTd39/Q168g/65Wl8OAfuZ3skRZ6xyAnYGJcL 5HasBl371kAAt4ajnnOVSutegKJG9P84KT/R00dxEJ4llGU4nrbIsKur0mIcce1cGt 45VhTxGApBlaVJqiyPsbfKgVnr+NoxMQzlOUMLK0= To: libcamera-devel@lists.libcamera.org Date: Sat, 16 Sep 2023 14:19:28 +0200 Message-ID: <20230916121930.29490-11-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20230916121930.29490-1-jacopo.mondi@ideasonboard.com> References: <20230916121930.29490-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 10/12] libcamera: rpi: Simplify validate() and configure() for RAW streams X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Naushir Patuck This commit simplifies the validate() and configure() calls in the pipeline handler in a number of ways: - Only pass the RPiCameraConfiguration structure into platformValidate() and platformConfigure(). - Determine the V4L2DeviceFormat structure fields for all streams in validate(), cache them and reuse in configure() instead of re-generating this structure multiple times. - Use the recently added updateStreamConfig() and toV4L2DeviceFormat() helpers to populate fields in the V4L2DeviceFormat and StreamConfiguration structures to reduce code duplication. Signed-off-by: Naushir Patuck Reviewed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi --- .../pipeline/rpi/common/pipeline_base.cpp | 48 +++++-------------- .../pipeline/rpi/common/pipeline_base.h | 5 +- src/libcamera/pipeline/rpi/vc4/vc4.cpp | 35 ++++++++------ 3 files changed, 34 insertions(+), 54 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 93779556fd81..9e4411f22902 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -234,16 +234,10 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() /* Further fixups on the RAW streams. */ for (auto &raw : rawStreams_) { - StreamConfiguration &cfg = config_.at(raw.index); - - V4L2DeviceFormat rawFormat; - rawFormat.fourcc = raw.dev->toV4L2PixelFormat(cfg.pixelFormat); - rawFormat.size = cfg.size; - rawFormat.colorSpace = cfg.colorSpace; - - int ret = raw.dev->tryFormat(&rawFormat); + int ret = raw.dev->tryFormat(&raw.format); if (ret) return Invalid; + /* * Some sensors change their Bayer order when they are h-flipped * or v-flipped, according to the transform. If this one does, we @@ -251,23 +245,15 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() * Note how we must fetch the "native" (i.e. untransformed) Bayer * order, because the sensor may currently be flipped! */ - V4L2PixelFormat fourcc = rawFormat.fourcc; + BayerFormat bayer = BayerFormat::fromPixelFormat(raw.cfg->pixelFormat); if (data_->flipsAlterBayerOrder_) { - BayerFormat bayer = BayerFormat::fromV4L2PixelFormat(fourcc); bayer.order = data_->nativeBayerOrder_; bayer = bayer.transform(combinedTransform_); - fourcc = bayer.toV4L2PixelFormat(); } + raw.cfg->pixelFormat = bayer.toPixelFormat(); - PixelFormat inputPixFormat = fourcc.toPixelFormat(); - if (raw.cfg->size != rawFormat.size || raw.cfg->pixelFormat != inputPixFormat) { - raw.cfg->size = rawFormat.size; - raw.cfg->pixelFormat = inputPixFormat; + if (RPi::PipelineHandlerBase::updateStreamConfig(raw.cfg, raw.format)) status = Adjusted; - } - - raw.cfg->stride = rawFormat.planes[0].bpl; - raw.cfg->frameSize = rawFormat.planes[0].size; } /* Further fixups on the ISP output streams. */ @@ -544,35 +530,25 @@ int PipelineHandlerBase::configure(Camera *camera, CameraConfiguration *config) * If the application has provided a sensor configuration apply it * instead of just applying a format. */ - const RPiCameraConfiguration *rpiConfig = - static_cast(config); - V4L2SubdeviceFormat sensorFormat; + RPiCameraConfiguration *rpiConfig = static_cast(config); + V4L2SubdeviceFormat *sensorFormat = &rpiConfig->sensorFormat_; if (rpiConfig->sensorConfig.populated()) { ret = data->sensor_->applyConfiguration(rpiConfig->sensorConfig, rpiConfig->combinedTransform_, - &sensorFormat); + sensorFormat); } else { - sensorFormat = rpiConfig->sensorFormat_; - ret = data->sensor_->setFormat(&sensorFormat, + ret = data->sensor_->setFormat(sensorFormat, rpiConfig->combinedTransform_); } if (ret) return ret; - /* Use the user requested packing/bit-depth. */ - std::optional packing; - if (!rpiConfig->rawStreams_.empty()) { - BayerFormat bayerFormat = - BayerFormat::fromPixelFormat(rpiConfig->rawStreams_[0].cfg->pixelFormat); - packing = bayerFormat.packing; - } - /* * Platform specific internal stream configuration. This also assigns * external streams which get configured below. */ - ret = data->platformConfigure(sensorFormat, packing, rpiConfig); + ret = data->platformConfigure(rpiConfig); if (ret) return ret; @@ -636,11 +612,11 @@ int PipelineHandlerBase::configure(Camera *camera, CameraConfiguration *config) */ link->setEnabled(true); const MediaPad *sinkPad = link->sink(); - ret = device->setFormat(sinkPad->index(), &sensorFormat); + ret = device->setFormat(sinkPad->index(), sensorFormat); if (ret) { LOG(RPI, Error) << "Failed to set format on " << device->entity()->name() << " pad " << sinkPad->index() - << " with format " << sensorFormat + << " with format " << *sensorFormat << ": " << ret; return ret; } diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index 491c5e98c4a1..0c4a59b7f590 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -58,9 +58,7 @@ public: } virtual CameraConfiguration::Status platformValidate(RPiCameraConfiguration *rpiConfig) const = 0; - virtual int platformConfigure(const V4L2SubdeviceFormat &sensorFormat, - std::optional packing, - const RPiCameraConfiguration *rpiConfig) = 0; + virtual int platformConfigure(const RPiCameraConfiguration *rpiConfig) = 0; virtual void platformStart() = 0; virtual void platformStop() = 0; @@ -269,6 +267,7 @@ public: unsigned int index; StreamConfiguration *cfg; V4L2VideoDevice *dev; + V4L2DeviceFormat format; }; std::vector rawStreams_; diff --git a/src/libcamera/pipeline/rpi/vc4/vc4.cpp b/src/libcamera/pipeline/rpi/vc4/vc4.cpp index 2308577a613b..1a03c88d7ee3 100644 --- a/src/libcamera/pipeline/rpi/vc4/vc4.cpp +++ b/src/libcamera/pipeline/rpi/vc4/vc4.cpp @@ -115,9 +115,7 @@ private: isp_[Isp::Input].dev()->setSelection(V4L2_SEL_TGT_CROP, &ispCrop_); } - int platformConfigure(const V4L2SubdeviceFormat &sensorFormat, - std::optional packing, - const RPi::RPiCameraConfiguration *rpiConfig) override; + int platformConfigure(const RPi::RPiCameraConfiguration *rpiConfig) override; int platformConfigureIpa(ipa::RPi::ConfigParams ¶ms) override; int platformInitIpa([[maybe_unused]] ipa::RPi::InitParams ¶ms) override @@ -416,6 +414,9 @@ CameraConfiguration::Status Vc4CameraData::platformValidate(RPi::RPiCameraConfig /* Handle flips to make sure to match the RAW stream format. */ if (flipsAlterBayerOrder_) rawBayer = rawBayer.transform(rpiConfig->combinedTransform_); + + /* Apply the user requested packing. */ + rawBayer.packing = BayerFormat::fromPixelFormat(rawStream->pixelFormat).packing; PixelFormat rawFormat = rawBayer.toPixelFormat(); if (rawStream->pixelFormat != rawFormat || @@ -425,6 +426,9 @@ CameraConfiguration::Status Vc4CameraData::platformValidate(RPi::RPiCameraConfig status = CameraConfiguration::Adjusted; } + + rawStreams[0].format = + RPi::PipelineHandlerBase::toV4L2DeviceFormat(unicam_[Unicam::Image].dev(), rawStream); } /* @@ -500,23 +504,14 @@ int Vc4CameraData::platformPipelineConfigure(const std::unique_ptr & return 0; } -int Vc4CameraData::platformConfigure(const V4L2SubdeviceFormat &sensorFormat, - std::optional packing, - const RPi::RPiCameraConfiguration *rpiConfig) +int Vc4CameraData::platformConfigure(const RPi::RPiCameraConfiguration *rpiConfig) { const std::vector &rawStreams = rpiConfig->rawStreams_; const std::vector &outStreams = rpiConfig->outStreams_; int ret; - if (!packing) - packing = BayerFormat::Packing::CSI2; - V4L2VideoDevice *unicam = unicam_[Unicam::Image].dev(); - V4L2DeviceFormat unicamFormat = RPi::PipelineHandlerBase::toV4L2DeviceFormat(unicam, sensorFormat, *packing); - - ret = unicam->setFormat(&unicamFormat); - if (ret) - return ret; + V4L2DeviceFormat unicamFormat; /* * See which streams are requested, and route the user @@ -525,14 +520,24 @@ int Vc4CameraData::platformConfigure(const V4L2SubdeviceFormat &sensorFormat, if (!rawStreams.empty()) { rawStreams[0].cfg->setStream(&unicam_[Unicam::Image]); unicam_[Unicam::Image].setFlags(StreamFlag::External); + unicamFormat = rawStreams[0].format; + } else { + unicamFormat = + RPi::PipelineHandlerBase::toV4L2DeviceFormat(unicam, + rpiConfig->sensorFormat_, + BayerFormat::Packing::CSI2); } + ret = unicam->setFormat(&unicamFormat); + if (ret) + return ret; + ret = isp_[Isp::Input].dev()->setFormat(&unicamFormat); if (ret) return ret; LOG(RPI, Info) << "Sensor: " << sensor_->id() - << " - Selected sensor format: " << sensorFormat + << " - Selected sensor format: " << rpiConfig->sensorFormat_ << " - Selected unicam format: " << unicamFormat; /* Use a sensible small default size if no output streams are configured. */ From patchwork Sat Sep 16 12:19:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 19049 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 55E86C32B2 for ; Sat, 16 Sep 2023 12:19:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0196662921; Sat, 16 Sep 2023 14:19:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1694866791; bh=14pKWXs3hK4uBwqI3dUv64hTKoGiK05/80YGxK+toS8=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=q/8C1srEMMQs+8I0dIdSP9DwSZ/Bpo6WNqoTpSAL/0C/gdxkwWYlTYMHv4jIGURnf 9jPhPn5TmcFDKuK/R0y+D0+oKF8L8UwZahBmVgc4sMLiefY/t3tHy1JqPvJ1BUwhGR xTgQ+4D6P924maZJYuPEAzssUEIdMY+tINGxlsabExL006spx8UEZtUoO2qAy0BBm/ VnNzmPRifYEjj8eTZftVvIAN9jbg+rHPKjx9zXP94CP38etpukD+bIwHz9o2XTAHdj Mjg+WpKTvRXvwM0bSguAqCFoWobxgB/d10oM9PM61FJ3dG1TiP9NHsCIzAKDvQvK6X s+sFtqori+8xw== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 17EF36291C for ; Sat, 16 Sep 2023 14:19:42 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="vMl0cfgH"; dkim-atps=neutral Received: from uno.LocalDomain (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E6C1C1812; Sat, 16 Sep 2023 14:18:07 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1694866688; bh=14pKWXs3hK4uBwqI3dUv64hTKoGiK05/80YGxK+toS8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=vMl0cfgH7eSfYOA6l9jwgW230nz6nOdgaGW7l4e9IZfh4iPYNPRQDlk3g7QUFAWpS ZoAvdxhEDaFfP4e8jwqAQvCLlqb9/iLIiPvQrQ9JAh5EKlb+5d4sKAcEKDxt82/sWg 5hYNhxeXHCOSt0hL+kZG2ItCz0lDC/Q229nFQE9M= To: libcamera-devel@lists.libcamera.org Date: Sat, 16 Sep 2023 14:19:29 +0200 Message-ID: <20230916121930.29490-12-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20230916121930.29490-1-jacopo.mondi@ideasonboard.com> References: <20230916121930.29490-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 11/12] libcamera: rpi: Change default stream formats X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Naushir Patuck Switch to XRGB8888 as a default Viewfinder role output format, and YUV420 as a default output format for everything else. Signed-off-by: Naushir Patuck Reviewed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/rpi/common/pipeline_base.cpp | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 9e4411f22902..c69e076aac71 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -264,7 +264,7 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() if (fmts.find(out.dev->toV4L2PixelFormat(cfgPixFmt)) == fmts.end()) { /* If we cannot find a native format, use a default one. */ - cfgPixFmt = formats::NV12; + cfgPixFmt = formats::YUV420; status = Adjusted; } @@ -427,7 +427,7 @@ PipelineHandlerBase::generateConfiguration(Camera *camera, SpanispFormats(); - pixelFormat = formats::NV12; + pixelFormat = formats::YUV420; /* * Still image codecs usually expect the sYCC color space. * Even RGB codecs will be fine as the RGB we get with the @@ -461,7 +461,7 @@ PipelineHandlerBase::generateConfiguration(Camera *camera, SpanispFormats(); - pixelFormat = formats::ARGB8888; + pixelFormat = formats::XRGB8888; colorSpace = ColorSpace::Sycc; size = { 800, 600 }; bufferCount = 4; From patchwork Sat Sep 16 12:19:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 19050 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D610FBD160 for ; Sat, 16 Sep 2023 12:19:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7BB2F6292B; Sat, 16 Sep 2023 14:19:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1694866791; bh=c2GvTrMZTJMcatCGIVz4vOVZW5KTZwYWXeD5Zy0NktE=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=Ut/RqphhMnBwM8uj3SCSHy4YbqZPExcQuGrUsdojRr2OfF55Pi1U1Vws2Xxfufl6R GYIorF59rGe/WpRIgIl6B5n9P2fCkuPWtO8ZiTWYFINYDjW7tRXDVPIts6dnQzdFVq q3Ru2HDNjXwwnmX6DLZxwa/V9qn1bFPxiyIvlDUadOkVBW0+RvpWnLc3o7ftyI/4Hj BS01wsy52y8WvQz6WWj4FCUPeRzmjPkMd6yE5w+GfZE9KJK4d7Foj3ajW6M2jYm/IU 7GU1tCfmb6GZXKIzvX9pVDN0iGDAjFzGw1NhYRZwDqQZmJx8Y5gACJ+/QIxSPxd50m WAxw/E7UvBkbw== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 73DBF6292C for ; Sat, 16 Sep 2023 14:19:42 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="R36eqtMi"; dkim-atps=neutral Received: from uno.LocalDomain (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 5E5181536; Sat, 16 Sep 2023 14:18:08 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1694866688; bh=c2GvTrMZTJMcatCGIVz4vOVZW5KTZwYWXeD5Zy0NktE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=R36eqtMi4VHmG4vRfdvSLey2yTwIizrrJ6EIiPE1PPO9kWVSuTt1LtonbwBQj5kQI n6Yg1H8MxxtVUe+xWCmiuYKXBjsMnx93VN3Kz8FdnUR422O1N7b9qQO92iMJ3xM8QU CUI/ubpxSuaA610om+ZTnhQtn0fptz37KIrlZgcs= To: libcamera-devel@lists.libcamera.org Date: Sat, 16 Sep 2023 14:19:30 +0200 Message-ID: <20230916121930.29490-13-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20230916121930.29490-1-jacopo.mondi@ideasonboard.com> References: <20230916121930.29490-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 12/12] libcamera: rpi: Simplify validate() and configure() for YUV/RGB streams X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Naushir Patuck This commit simplifies the validate() and configure() calls in the pipeline handler in a number of ways: - Determine the V4L2DeviceFormat structure fields for all streams in validate(), cache them and reuse in configure() instead of re-generating this structure multiple times. - Remove setting a default pixel format in validate(), this code patch will not be used. - Use the recently added updateStreamConfig() and toV4L2DeviceFormat() helpers to populate fields in the V4L2DeviceFormat and StreamConfiguration structures to reduce code duplication. Signed-off-by: Naushir Patuck Reviewed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi --- .../pipeline/rpi/common/pipeline_base.cpp | 49 ++++--------------- src/libcamera/pipeline/rpi/vc4/vc4.cpp | 15 ++---- 2 files changed, 12 insertions(+), 52 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index c69e076aac71..653faff12353 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -258,56 +258,25 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() /* Further fixups on the ISP output streams. */ for (auto &out : outStreams_) { - StreamConfiguration &cfg = config_.at(out.index); - PixelFormat &cfgPixFmt = cfg.pixelFormat; - V4L2VideoDevice::Formats fmts = out.dev->formats(); - - if (fmts.find(out.dev->toV4L2PixelFormat(cfgPixFmt)) == fmts.end()) { - /* If we cannot find a native format, use a default one. */ - cfgPixFmt = formats::YUV420; - status = Adjusted; - } - - V4L2DeviceFormat format; - format.fourcc = out.dev->toV4L2PixelFormat(cfg.pixelFormat); - format.size = cfg.size; /* - * platformValidate may have worked out the correct stride so we - * must pass it in. This also needs the planesCount to be set - * correctly or the stride will be ignored. + * We want to send the associated YCbCr info through to the driver. + * + * But for RGB streams, the YCbCr info gets overwritten on the way back + * so we must check against what the stream cfg says, not what we actually + * requested (which carefully included the YCbCr info)! */ - const PixelFormat &pixFormat = format.fourcc.toPixelFormat(); - const PixelFormatInfo &info = PixelFormatInfo::info(pixFormat); - format.planesCount = info.numPlanes(); - format.planes[0].bpl = cfg.stride; - - /* We want to send the associated YCbCr info through to the driver. */ - format.colorSpace = yuvColorSpace_; + out.format.colorSpace = yuvColorSpace_; LOG(RPI, Debug) - << "Try color space " << ColorSpace::toString(cfg.colorSpace); + << "Try color space " << ColorSpace::toString(out.cfg->colorSpace); - int ret = out.dev->tryFormat(&format); + int ret = out.dev->tryFormat(&out.format); if (ret) return Invalid; - /* - * But for RGB streams, the YCbCr info gets overwritten on the way back - * so we must check against what the stream cfg says, not what we actually - * requested (which carefully included the YCbCr info)! - */ - if (cfg.colorSpace != format.colorSpace) { + if (RPi::PipelineHandlerBase::updateStreamConfig(out.cfg, out.format)) status = Adjusted; - LOG(RPI, Debug) - << "Color space changed from " - << ColorSpace::toString(cfg.colorSpace) << " to " - << ColorSpace::toString(format.colorSpace); - } - - cfg.colorSpace = format.colorSpace; - cfg.stride = format.planes[0].bpl; - cfg.frameSize = format.planes[0].size; } return status; diff --git a/src/libcamera/pipeline/rpi/vc4/vc4.cpp b/src/libcamera/pipeline/rpi/vc4/vc4.cpp index 1a03c88d7ee3..a1813ba65725 100644 --- a/src/libcamera/pipeline/rpi/vc4/vc4.cpp +++ b/src/libcamera/pipeline/rpi/vc4/vc4.cpp @@ -457,6 +457,8 @@ CameraConfiguration::Status Vc4CameraData::platformValidate(RPi::RPiCameraConfig * have that fixed up in the code above. */ outStreams[i].dev = isp_[i == 0 ? Isp::Output0 : Isp::Output1].dev(); + + outStreams[i].format = RPi::PipelineHandlerBase::toV4L2DeviceFormat(outStreams[i].dev, outStreams[i].cfg); } return status; @@ -549,11 +551,7 @@ int Vc4CameraData::platformConfigure(const RPi::RPiCameraConfiguration *rpiConfi /* The largest resolution gets routed to the ISP Output 0 node. */ RPi::Stream *stream = i == 0 ? &isp_[Isp::Output0] : &isp_[Isp::Output1]; - - V4L2PixelFormat fourcc = stream->dev()->toV4L2PixelFormat(cfg->pixelFormat); - format.size = cfg->size; - format.fourcc = fourcc; - format.colorSpace = cfg->colorSpace; + format = outStreams[i].format; LOG(RPI, Debug) << "Setting " << stream->name() << " to " << format; @@ -562,13 +560,6 @@ int Vc4CameraData::platformConfigure(const RPi::RPiCameraConfiguration *rpiConfi if (ret) return -EINVAL; - if (format.size != cfg->size || format.fourcc != fourcc) { - LOG(RPI, Error) - << "Failed to set requested format on " << stream->name() - << ", returned " << format; - return -EINVAL; - } - LOG(RPI, Debug) << "Stream " << stream->name() << " has color space " << ColorSpace::toString(cfg->colorSpace);