From patchwork Thu Nov 24 02:51:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17852 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4A63DBDE6B for ; Thu, 24 Nov 2022 02:51:54 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E80B96331E; Thu, 24 Nov 2022 03:51:53 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1669258313; bh=40cx5Kok1BbswVCPQnu1hRgCA5VMy9jgv48Cub9+y5M=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=Zx7cOUTBdqgHFpMU2hRK07AeLEPjssaRtpZlKu0sncH3L4fcLW1qXoZPIU94S2K5D EDAd0BdIdHzNpqJ0mWMRqGmEYTMMUC22eSGde5eqEJm/z0cjizRhAdYCNfB8XpRRf+ mBGynAQkeEu3QzPEphQje6AKiFsb17a/YKpRf5F1sQF5+IVapnRiWLg/Zu7sAzUrbT MvgRrkV3vwoDID8M25SgEh6dqw2Qxu88nd4t4zx2iQKLKI/SjPlB6CyuIaOj3JWpAi K645iMBcPW7OxnYX2kkQ9BP0cbsIkG76/7bbhox7el93N5+McJXCKVtz8F/9dxBg6f dFWBHvWoLcNzA== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B95FE63313 for ; Thu, 24 Nov 2022 03:51:51 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="O2uDbMt4"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 48647496 for ; Thu, 24 Nov 2022 03:51:51 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1669258311; bh=40cx5Kok1BbswVCPQnu1hRgCA5VMy9jgv48Cub9+y5M=; h=From:To:Subject:Date:In-Reply-To:References:From; b=O2uDbMt4ydJ+YIYqMA0aRl8Z4FAZK9i2+oWgT1x+wUnwrqounlQPdHUmK4+ZA/6EN EcASrU2r46KWzqC+JT8N/FNDO6EJMR1u/G70CEy0tKUpiWMKP99/1BsP4ZHvE0jjuo wMjk9cP20ACf9WezOvzxcRgiYEj9lAvBSHBXhc5E= To: libcamera-devel@lists.libcamera.org Date: Thu, 24 Nov 2022 04:51:25 +0200 Message-Id: <20221124025133.17875-2-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.37.4 In-Reply-To: <20221124025133.17875-1-laurent.pinchart@ideasonboard.com> References: <20221124025133.17875-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 1/9] libcamera: stream: Add operator<<() to print StreamRole as a string X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" libcamera prints stream role values in log messages. To be more user-friendly, add a specialization of operator<<() to print the role name as a string instead of a numerical value. Signed-off-by: Laurent Pinchart Reviewed-by: Paul Elder Reviewed-by: Jacopo Mondi --- include/libcamera/stream.h | 3 +++ src/libcamera/stream.cpp | 19 +++++++++++++++++++ 2 files changed, 22 insertions(+) diff --git a/include/libcamera/stream.h b/include/libcamera/stream.h index f0ae7e62e0a3..efec695ad317 100644 --- a/include/libcamera/stream.h +++ b/include/libcamera/stream.h @@ -9,6 +9,7 @@ #include #include +#include #include #include @@ -70,6 +71,8 @@ enum StreamRole { using StreamRoles = std::vector; +std::ostream &operator<<(std::ostream &out, StreamRole role); + class Stream { public: diff --git a/src/libcamera/stream.cpp b/src/libcamera/stream.cpp index 686e693bccba..67f308157fbf 100644 --- a/src/libcamera/stream.cpp +++ b/src/libcamera/stream.cpp @@ -417,6 +417,25 @@ std::string StreamConfiguration::toString() const * acceptable. */ +/** + * \brief Insert a text representation of a StreamRole into an output stream + * \param[in] out The output stream + * \param[in] role The StreamRole + * \return The output stream \a out + */ +std::ostream &operator<<(std::ostream &out, StreamRole role) +{ + static constexpr std::array names{ + "Raw", + "StillCapture", + "VideoRecording", + "Viewfinder", + }; + + out << names[static_cast>(role)]; + return out; +} + /** * \typedef StreamRoles * \brief A vector of StreamRole From patchwork Thu Nov 24 02:51:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17853 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A75E9BDE6B for ; Thu, 24 Nov 2022 02:51:56 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6547A63324; Thu, 24 Nov 2022 03:51:56 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1669258316; bh=vUHwcxJAScu2Sbdsm+MFseawJOtM3gYrdpvjCuCcBCw=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=Yn0SQPhjVooukbxkRKUCLRkTyt0hGbsVoF5qodcMM6EO9KU6NxQU5MvbQree5QtjC u6zuk9f8NQmA3lYqIh8RyrmD8SIh2snfn/My7Pgd+gDRouFeMPF3n74GbxoqhS1Qmn 9DuN8CfCoo1uMbz8E1D74y2yeLfP0MA3lovBvPrDR4C9+PzgZ7OELiqq6qC1YyckH7 JGSxXT0y1oQXSNJ/hNg1s8dKkD19gutfwvgaMX0NQLlFmnT1zRfByyD0slBaMKb2wg LzCwPNWLu/0EKLCbNOzApiPZGBVf1GORbCojrlnaSPnU+qTU1bKbimDHVilIFiuTZX bIHMdsfqXySZQ== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0C9AB63316 for ; Thu, 24 Nov 2022 03:51:53 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="heQkZ3z3"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 90D09496 for ; Thu, 24 Nov 2022 03:51:52 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1669258312; bh=vUHwcxJAScu2Sbdsm+MFseawJOtM3gYrdpvjCuCcBCw=; h=From:To:Subject:Date:In-Reply-To:References:From; b=heQkZ3z3V1RbKLJelwM9mH2P/Qbhhp9VJEd/IGjnouj7D3ugSdUJybxSOfRAoz5EA nq6TMXdEEzohqeRu6gO+61okhmZW1u4T3crecz1nSpRUyNZSypBl65aqmW65REasIw k6+U0OmLwE+lbY6/Fn6hEDbozBM/gYUSL56nmUCw= To: libcamera-devel@lists.libcamera.org Date: Thu, 24 Nov 2022 04:51:26 +0200 Message-Id: <20221124025133.17875-3-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.37.4 In-Reply-To: <20221124025133.17875-1-laurent.pinchart@ideasonboard.com> References: <20221124025133.17875-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 2/9] libcamera: stream: Turn StreamRole into scoped enumeration X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The StreamRole enum has enumerators such as 'Raw' that are too generic to be in the global libcamera namespace. Turn it into a scoped enum to avoid namespace clashes, and update users accordingly. Signed-off-by: Laurent Pinchart Reviewed-by: Paul Elder Reviewed-by: Jacopo Mondi --- Changes since v3: - Address imx8-isi pipeline handler --- include/libcamera/stream.h | 2 +- src/android/camera_capabilities.cpp | 2 +- src/apps/lc-compliance/capture_test.cpp | 17 ++++++++++++----- src/gstreamer/gstlibcamerapad.cpp | 20 +++++++++++++++----- src/libcamera/pipeline/imx8-isi/imx8-isi.cpp | 8 ++++---- 5 files changed, 33 insertions(+), 16 deletions(-) diff --git a/include/libcamera/stream.h b/include/libcamera/stream.h index efec695ad317..29235ddf0d8a 100644 --- a/include/libcamera/stream.h +++ b/include/libcamera/stream.h @@ -62,7 +62,7 @@ private: StreamFormats formats_; }; -enum StreamRole { +enum class StreamRole { Raw, StillCapture, VideoRecording, diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp index ef0d10d08fbd..1bfeaea4b121 100644 --- a/src/android/camera_capabilities.cpp +++ b/src/android/camera_capabilities.cpp @@ -488,7 +488,7 @@ int CameraCapabilities::initializeStreamConfigurations() * \todo Get this from the camera properties once defined */ std::unique_ptr cameraConfig = - camera_->generateConfiguration({ StillCapture }); + camera_->generateConfiguration({ StreamRole::StillCapture }); if (!cameraConfig) { LOG(HAL, Error) << "Failed to get maximum resolution"; return -EINVAL; diff --git a/src/apps/lc-compliance/capture_test.cpp b/src/apps/lc-compliance/capture_test.cpp index 52578207c11f..1dcfcf92fc8c 100644 --- a/src/apps/lc-compliance/capture_test.cpp +++ b/src/apps/lc-compliance/capture_test.cpp @@ -16,7 +16,12 @@ using namespace libcamera; const std::vector NUMREQUESTS = { 1, 2, 3, 5, 8, 13, 21, 34, 55, 89 }; -const std::vector ROLES = { Raw, StillCapture, VideoRecording, Viewfinder }; +const std::vector ROLES = { + StreamRole::Raw, + StreamRole::StillCapture, + StreamRole::VideoRecording, + StreamRole::Viewfinder +}; class SingleStream : public testing::TestWithParam> { @@ -54,10 +59,12 @@ void SingleStream::TearDown() std::string SingleStream::nameParameters(const testing::TestParamInfo &info) { - std::map rolesMap = { { Raw, "Raw" }, - { StillCapture, "StillCapture" }, - { VideoRecording, "VideoRecording" }, - { Viewfinder, "Viewfinder" } }; + std::map rolesMap = { + { StreamRole::Raw, "Raw" }, + { StreamRole::StillCapture, "StillCapture" }, + { StreamRole::VideoRecording, "VideoRecording" }, + { StreamRole::Viewfinder, "Viewfinder" } + }; std::string roleName = rolesMap[std::get<0>(info.param)]; std::string numRequestsName = std::to_string(std::get<1>(info.param)); diff --git a/src/gstreamer/gstlibcamerapad.cpp b/src/gstreamer/gstlibcamerapad.cpp index 87b4057ac101..9e710a479368 100644 --- a/src/gstreamer/gstlibcamerapad.cpp +++ b/src/gstreamer/gstlibcamerapad.cpp @@ -54,7 +54,7 @@ gst_libcamera_pad_get_property(GObject *object, guint prop_id, GValue *value, switch (prop_id) { case PROP_STREAM_ROLE: - g_value_set_enum(value, self->role); + g_value_set_enum(value, static_cast(self->role)); break; default: G_OBJECT_WARN_INVALID_PROPERTY_ID(object, prop_id, pspec); @@ -87,9 +87,19 @@ gst_libcamera_stream_role_get_type() { static GType type = 0; static const GEnumValue values[] = { - { StillCapture, "libcamera::StillCapture", "still-capture" }, - { VideoRecording, "libcamera::VideoRecording", "video-recording" }, - { Viewfinder, "libcamera::Viewfinder", "view-finder" }, + { + static_cast(StreamRole::StillCapture), + "libcamera::StillCapture", + "still-capture", + }, { + static_cast(StreamRole::VideoRecording), + "libcamera::VideoRecording", + "video-recording", + }, { + static_cast(StreamRole::Viewfinder), + "libcamera::Viewfinder", + "view-finder", + }, { 0, NULL, NULL } }; @@ -110,7 +120,7 @@ gst_libcamera_pad_class_init(GstLibcameraPadClass *klass) auto *spec = g_param_spec_enum("stream-role", "Stream Role", "The selected stream role", gst_libcamera_stream_role_get_type(), - VideoRecording, + static_cast(StreamRole::VideoRecording), (GParamFlags)(GST_PARAM_MUTABLE_READY | G_PARAM_CONSTRUCT | G_PARAM_READWRITE diff --git a/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp b/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp index e51457ebc345..0c67e35dde73 100644 --- a/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp +++ b/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp @@ -599,7 +599,7 @@ PipelineHandlerISI::generateConfiguration(Camera *camera, Size size; switch (role) { - case StillCapture: + case StreamRole::StillCapture: /* * \todo Make sure the sensor can produce non-RAW formats * compatible with the ones supported by the pipeline. @@ -608,8 +608,8 @@ PipelineHandlerISI::generateConfiguration(Camera *camera, pixelFormat = formats::YUYV; break; - case Viewfinder: - case VideoRecording: + case StreamRole::Viewfinder: + case StreamRole::VideoRecording: /* * \todo Make sure the sensor can produce non-RAW formats * compatible with the ones supported by the pipeline. @@ -618,7 +618,7 @@ PipelineHandlerISI::generateConfiguration(Camera *camera, pixelFormat = formats::YUYV; break; - case Raw: { + case StreamRole::Raw: { /* * Make sure the sensor can generate a RAW format and * prefer the ones with a larger bitdepth. From patchwork Thu Nov 24 02:51:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17854 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9528CBDE6B for ; Thu, 24 Nov 2022 02:51:57 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2E0046331A; Thu, 24 Nov 2022 03:51:57 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1669258317; bh=rLVvinFiJ0iEgC2rv4fnV9rCavXASkwU+sf8AON4LRw=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=D+atPvl1bcCUW/bQq3eWUDUk1EBAOeMQiViHTfU31ImaeqKsOa3cfezIa32ivwvnA SdwjO5KWjcE4Oqi3Lqql8rQjebMgvRktFVzjhL3Ac4IqQ41B+ksnyaPEs8Vbs6IdnJ ewN3TmL1H2+2x93r4oLuLAc8X268PAhMd0Zuo8s4A1/cUCGCQqwsVGlfzvNTht79p7 NkmnOkbX/nwqeiSuKpdA0vM+VgGdHyZtx27fLqioEtYYCoB+wDQaBGHxV31e/05Ggb dh+Zl3rpq7KWBtbWqi14mSUJLNHWWBLN6gAMUBabj/prtRlNyabUcdQ3fLAppQsVfB hpE9rU09wiesw== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 74B3863316 for ; Thu, 24 Nov 2022 03:51:54 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="vAtQi8MM"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E31837FA for ; Thu, 24 Nov 2022 03:51:53 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1669258314; bh=rLVvinFiJ0iEgC2rv4fnV9rCavXASkwU+sf8AON4LRw=; h=From:To:Subject:Date:In-Reply-To:References:From; b=vAtQi8MMzuu0tma3Pi954H7i/YkTd6jKIMnUIuEikLSHkDQ/Bhi2k1vh9RhZQqezl a0zhWt4tw00p/3gb2nxI8yDZY5SyIm4Z3nzK1wU3Gk2Mf5KyHX48QiOk/nBOTELj3v gnSPI5wPIOa8HG+7xtgp5OebPq0KcDz0mTkKoKV8= To: libcamera-devel@lists.libcamera.org Date: Thu, 24 Nov 2022 04:51:27 +0200 Message-Id: <20221124025133.17875-4-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.37.4 In-Reply-To: <20221124025133.17875-1-laurent.pinchart@ideasonboard.com> References: <20221124025133.17875-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 3/9] ipa: rkisp1: Support raw capture in IPA operations X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The RkISP1 can capture raw frames by bypassing the ISP. In that mode, the ISP will not produce statistics nor consume parameters. Most algorithms will thus be unable to run, with one exception: the AGC will still be able to configure the sensor exposure time and analog gain in manual mode. To prepare for this, add the ability to disable algorithms for the duration of the capture session based on the camera configuration. Individual algorithms report whether they support raw formats at construction time, and the IPA module disables algorithms in configure() based on the stream configurations. Disabled algorithms are skipped during the capture session in the processStatsBuffer() operation. As the ISP doesn't produce statistics, don't try to access the stats buffer. There is no need for similar logic in fillParamsBuffer() as that operation won't be called for raw capture. All algorithms report not supporting raw capture by default. Raw support in AGC will be added separately. The feature is implemented in the RkISP1 module without any support from libipa at this point to avoid designing a generic API based on a single user. This may be changed in the future. Signed-off-by: Laurent Pinchart Reviewed-by: Paul Elder Reviewed-by: Jacopo Mondi --- Changes since v3: - Initialize stats to nullptr by default --- src/ipa/rkisp1/algorithms/algorithm.h | 12 +++++++- src/ipa/rkisp1/ipa_context.cpp | 5 ++++ src/ipa/rkisp1/ipa_context.h | 2 ++ src/ipa/rkisp1/rkisp1.cpp | 40 +++++++++++++++++++++++---- 4 files changed, 52 insertions(+), 7 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/algorithm.h b/src/ipa/rkisp1/algorithms/algorithm.h index c3212cff76fe..9454c9a1fc06 100644 --- a/src/ipa/rkisp1/algorithms/algorithm.h +++ b/src/ipa/rkisp1/algorithms/algorithm.h @@ -15,7 +15,17 @@ namespace libcamera { namespace ipa::rkisp1 { -using Algorithm = libcamera::ipa::Algorithm; +class Algorithm : public libcamera::ipa::Algorithm +{ +public: + Algorithm() + : disabled_(false), supportsRaw_(false) + { + } + + bool disabled_; + bool supportsRaw_; +}; } /* namespace ipa::rkisp1 */ diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp index 7a987497bd03..9bbf368432fa 100644 --- a/src/ipa/rkisp1/ipa_context.cpp +++ b/src/ipa/rkisp1/ipa_context.cpp @@ -89,6 +89,11 @@ namespace libcamera::ipa::rkisp1 { * \brief Sensor output resolution */ +/** + * \var IPASessionConfiguration::raw + * \brief Indicates if the camera is configured to capture raw frames + */ + /** * \struct IPAActiveState * \brief Active state for algorithms diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index bb60ab9eab72..3e47ac663c58 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -48,6 +48,8 @@ struct IPASessionConfiguration { struct { rkisp1_cif_isp_version revision; } hw; + + bool raw; }; struct IPAActiveState { diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 9265d3c9f53c..6ac29df8ec8d 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -24,6 +24,7 @@ #include #include +#include "libcamera/internal/formats.h" #include "libcamera/internal/mapped_framebuffer.h" #include "libcamera/internal/yaml_parser.h" @@ -207,7 +208,7 @@ void IPARkISP1::stop() } int IPARkISP1::configure(const IPAConfigInfo &ipaConfig, - [[maybe_unused]] const std::map &streamConfig, + const std::map &streamConfig, ControlInfoMap *ipaControls) { sensorControls_ = ipaConfig.sensorControls; @@ -255,7 +256,21 @@ int IPARkISP1::configure(const IPAConfigInfo &ipaConfig, context_.configuration.sensor.minAnalogueGain = camHelper_->gain(minGain); context_.configuration.sensor.maxAnalogueGain = camHelper_->gain(maxGain); - for (auto const &algo : algorithms()) { + context_.configuration.raw = std::any_of(streamConfig.begin(), streamConfig.end(), + [](auto &cfg) -> bool { + PixelFormat pixelFormat{ cfg.second.pixelFormat }; + const PixelFormatInfo &format = PixelFormatInfo::info(pixelFormat); + return format.colourEncoding == PixelFormatInfo::ColourEncodingRAW; + }); + + for (auto const &a : algorithms()) { + Algorithm *algo = static_cast(a.get()); + + /* Disable algorithms that don't support raw formats. */ + algo->disabled_ = context_.configuration.raw && !algo->supportsRaw_; + if (algo->disabled_) + continue; + int ret = algo->configure(context_, info); if (ret) return ret; @@ -298,8 +313,12 @@ void IPARkISP1::queueRequest(const uint32_t frame, const ControlList &controls) { IPAFrameContext &frameContext = context_.frameContexts.alloc(frame); - for (auto const &algo : algorithms()) + for (auto const &a : algorithms()) { + Algorithm *algo = static_cast(a.get()); + if (algo->disabled_) + continue; algo->queueRequest(context_, frame, frameContext, controls); + } } void IPARkISP1::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) @@ -324,8 +343,13 @@ void IPARkISP1::processStatsBuffer(const uint32_t frame, const uint32_t bufferId { IPAFrameContext &frameContext = context_.frameContexts.get(frame); - const rkisp1_stat_buffer *stats = - reinterpret_cast( + /* + * In raw capture mode, the ISP is bypassed and no statistics buffer is + * provided. + */ + const rkisp1_stat_buffer *stats = nullptr; + if (!context_.configuration.raw) + stats = reinterpret_cast( mappedBuffers_.at(bufferId).planes()[0].data()); frameContext.sensor.exposure = @@ -335,8 +359,12 @@ void IPARkISP1::processStatsBuffer(const uint32_t frame, const uint32_t bufferId ControlList metadata(controls::controls); - for (auto const &algo : algorithms()) + for (auto const &a : algorithms()) { + Algorithm *algo = static_cast(a.get()); + if (algo->disabled_) + continue; algo->process(context_, frame, frameContext, stats, metadata); + } setControls(frame); From patchwork Thu Nov 24 02:51:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17855 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2C83BC3286 for ; Thu, 24 Nov 2022 02:51:58 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B34C063322; Thu, 24 Nov 2022 03:51:57 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1669258317; bh=AvK98lEHmZhXRJ/k3wkhLy85q2ewWwDf1lrRQ8IukCc=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=h34fI41jnx8D8vRSbeuIOJyd5HCcLS4bBGRUTjZM4aCiFw26Z0kcUpqMIiA2dgW+t TlTVKBbLjo579g+czVjBaDH+gV4zJdq9+lY7/nJTXjnddEVPCyn2iukVlLjgGxABR2 ApXQub6H2VgybvHy3qqHHpuZAieoQBEuwQ1jQ1W3L/aEZBXRjXn1P2KiKo/x1tLtMb 3F3xhbhO8uWVagcgrihSOAeHbuIUKS4/bix3+fOvNAxCcftq1UJa2jx6CroG9wsFjW NYIJmBZi9AO1dWGY5IS02vsuIbAcn4HHM00BT5nxskblkxSRzaFd+g3dx8vIQlaHb/ rmgaxkNvUeMnQ== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CCF1563313 for ; Thu, 24 Nov 2022 03:51:55 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="cuk0GJ20"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 619C4496; Thu, 24 Nov 2022 03:51:55 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1669258315; bh=AvK98lEHmZhXRJ/k3wkhLy85q2ewWwDf1lrRQ8IukCc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cuk0GJ20gVrqC170Ayl8jdyjBU61TiuylynHIP6Zv49hnu+Bvxl7FOi9now+3l5q/ S9Uqsdnc1qup4ewdgo/X/TjC/e+LrDQY0j/RkAhN/JvPHnwvv19K6qoAlgEC1khQwb JprZ506NEagRS9Tzx5laGHknuIgywGsN92chJoBY= To: libcamera-devel@lists.libcamera.org Date: Thu, 24 Nov 2022 04:51:28 +0200 Message-Id: <20221124025133.17875-5-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.37.4 In-Reply-To: <20221124025133.17875-1-laurent.pinchart@ideasonboard.com> References: <20221124025133.17875-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 4/9] ipa: rkisp1: Add support for manual gain and exposure X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Paul Elder Add support for manual gain and exposure in the rkisp1 IPA. Signed-off-by: Paul Elder Reviewed-by: Laurent Pinchart Signed-off-by: Laurent Pinchart Reviewed-by: Jacopo Mondi Reviewed-by: Paul Elder --- Changes since v3: - Convert exposure and gain V4L2 controls limits to libcamera values - Convert exposure control from duration to lines in AGC - Initialize lineDuration in init() - Small fixes in debug messages - Drop unnecessary curly braces - Use std::piecewise_construct --- src/ipa/rkisp1/algorithms/agc.cpp | 63 +++++++++++++++++++++++++++---- src/ipa/rkisp1/algorithms/agc.h | 4 ++ src/ipa/rkisp1/ipa_context.h | 13 ++++++- src/ipa/rkisp1/rkisp1.cpp | 25 ++++++++++++ 4 files changed, 96 insertions(+), 9 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index 169afe372803..ccbcd4a9583d 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -74,9 +74,14 @@ Agc::Agc() int Agc::configure(IPAContext &context, const IPACameraSensorInfo &configInfo) { /* Configure the default exposure and gain. */ - context.activeState.agc.gain = std::max(context.configuration.sensor.minAnalogueGain, - kMinAnalogueGain); - context.activeState.agc.exposure = 10ms / context.configuration.sensor.lineDuration; + context.activeState.agc.automatic.gain = + std::max(context.configuration.sensor.minAnalogueGain, + kMinAnalogueGain); + context.activeState.agc.automatic.exposure = + 10ms / context.configuration.sensor.lineDuration; + context.activeState.agc.manual.gain = context.activeState.agc.automatic.gain; + context.activeState.agc.manual.exposure = context.activeState.agc.automatic.exposure; + context.activeState.agc.autoEnabled = true; /* * According to the RkISP1 documentation: @@ -108,14 +113,58 @@ int Agc::configure(IPAContext &context, const IPACameraSensorInfo &configInfo) return 0; } +/** + * \copydoc libcamera::ipa::Algorithm::queueRequest + */ +void Agc::queueRequest(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) +{ + auto &agc = context.activeState.agc; + + const auto &agcEnable = controls.get(controls::AeEnable); + if (agcEnable && *agcEnable != agc.autoEnabled) { + agc.autoEnabled = *agcEnable; + + LOG(RkISP1Agc, Debug) + << (agc.autoEnabled ? "Enabling" : "Disabling") << " AGC"; + } + + const auto &exposure = controls.get(controls::ExposureTime); + if (exposure && !agc.autoEnabled) { + agc.manual.exposure = *exposure * 1.0us + / context.configuration.sensor.lineDuration; + + LOG(RkISP1Agc, Debug) + << "Set exposure to " << agc.manual.exposure; + } + + const auto &gain = controls.get(controls::AnalogueGain); + if (gain && !agc.autoEnabled) { + agc.manual.gain = *gain; + + LOG(RkISP1Agc, Debug) << "Set gain to " << agc.manual.gain; + } + + frameContext.agc.autoEnabled = agc.autoEnabled; + + if (!frameContext.agc.autoEnabled) { + frameContext.agc.exposure = agc.manual.exposure; + frameContext.agc.gain = agc.manual.gain; + } +} + /** * \copydoc libcamera::ipa::Algorithm::prepare */ void Agc::prepare(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, rkisp1_params_cfg *params) { - frameContext.agc.exposure = context.activeState.agc.exposure; - frameContext.agc.gain = context.activeState.agc.gain; + if (frameContext.agc.autoEnabled) { + frameContext.agc.exposure = context.activeState.agc.automatic.exposure; + frameContext.agc.gain = context.activeState.agc.automatic.gain; + } if (frame > 0) return; @@ -263,8 +312,8 @@ void Agc::computeExposure(IPAContext &context, IPAFrameContext &frameContext, << stepGain; /* Update the estimated exposure and gain. */ - activeState.agc.exposure = shutterTime / configuration.sensor.lineDuration; - activeState.agc.gain = stepGain; + activeState.agc.automatic.exposure = shutterTime / configuration.sensor.lineDuration; + activeState.agc.automatic.gain = stepGain; } /** diff --git a/src/ipa/rkisp1/algorithms/agc.h b/src/ipa/rkisp1/algorithms/agc.h index da4d2d4e8359..a228d0c37768 100644 --- a/src/ipa/rkisp1/algorithms/agc.h +++ b/src/ipa/rkisp1/algorithms/agc.h @@ -26,6 +26,10 @@ public: ~Agc() = default; int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; + void queueRequest(IPAContext &context, + const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) override; void prepare(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, rkisp1_params_cfg *params) override; diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index 3e47ac663c58..b9b2065328d6 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -54,8 +54,16 @@ struct IPASessionConfiguration { struct IPAActiveState { struct { - uint32_t exposure; - double gain; + struct { + uint32_t exposure; + double gain; + } manual; + struct { + uint32_t exposure; + double gain; + } automatic; + + bool autoEnabled; } agc; struct { @@ -96,6 +104,7 @@ struct IPAFrameContext : public FrameContext { struct { uint32_t exposure; double gain; + bool autoEnabled; } agc; struct { diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 6ac29df8ec8d..76e818d2d11e 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -158,6 +158,9 @@ int IPARkISP1::init(const IPASettings &settings, unsigned int hwRevision, return -ENODEV; } + context_.configuration.sensor.lineDuration = sensorInfo.minLineLength + * 1.0s / sensorInfo.pixelRate; + /* Load the tuning data file. */ File file(settings.configurationFile); if (!file.open(File::OpenModeFlag::ReadOnly)) { @@ -377,6 +380,28 @@ void IPARkISP1::updateControls(const IPACameraSensorInfo &sensorInfo, { ControlInfoMap::Map ctrlMap = rkisp1Controls; + /* + * Compute exposure time limits from the V4L2_CID_EXPOSURE control + * limits and the line duration. + */ + double lineDuration = context_.configuration.sensor.lineDuration.get(); + const ControlInfo &v4l2Exposure = sensorControls.find(V4L2_CID_EXPOSURE)->second; + int32_t minExposure = v4l2Exposure.min().get() * lineDuration; + int32_t maxExposure = v4l2Exposure.max().get() * lineDuration; + int32_t defExposure = v4l2Exposure.def().get() * lineDuration; + ctrlMap.emplace(std::piecewise_construct, + std::forward_as_tuple(&controls::ExposureTime), + std::forward_as_tuple(minExposure, maxExposure, defExposure)); + + /* Compute the analogue gain limits. */ + const ControlInfo &v4l2Gain = sensorControls.find(V4L2_CID_ANALOGUE_GAIN)->second; + float minGain = camHelper_->gain(v4l2Gain.min().get()); + float maxGain = camHelper_->gain(v4l2Gain.max().get()); + float defGain = camHelper_->gain(v4l2Gain.def().get()); + ctrlMap.emplace(std::piecewise_construct, + std::forward_as_tuple(&controls::AnalogueGain), + std::forward_as_tuple(minGain, maxGain, defGain)); + /* * Compute the frame duration limits. * From patchwork Thu Nov 24 02:51:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17856 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B31B7BDE6B for ; Thu, 24 Nov 2022 02:52:00 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5FBFF6332C; Thu, 24 Nov 2022 03:52:00 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1669258320; bh=hR1H2XvPxMz/AJJBPZvPmizY8AIaDb20Qff5SjK4wkM=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=nzdnS980dXPpf3/M/P529KG8nDem98AcgXJKRaunFx491HQd3DMbOi6vGxW0xRrza NG0K0WMx7ulMpweCWcVwIkst/6aLLMrQZiGEF5xNGs5Cd+ODyHW6lRCTRZ4TcLQ79+ HLa5O+5nxo8qjT0AhSfBbR0F8+zAT4evOSE6nnC7WbdmAaGKAzlXvy5eddDwrsFbeW TDl+VfXEU7Fak/9btjxpi+bBjjT2M21NRVfWgGo2p0T1JKA2rpQvVvD6d3MQl4MQ5D E2CRCwmAK4CDrqq+okqihTsPkBgn4ak7ecFJwMobKBkXFztoRRimsAT9rNUQ6nNKeS FF3g/823MZK1Q== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3762B63325 for ; Thu, 24 Nov 2022 03:51:57 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="O+M5r1Hx"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id AEA3A496 for ; Thu, 24 Nov 2022 03:51:56 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1669258316; bh=hR1H2XvPxMz/AJJBPZvPmizY8AIaDb20Qff5SjK4wkM=; h=From:To:Subject:Date:In-Reply-To:References:From; b=O+M5r1HxTPR/dG6umaMUgvun92sd6gYX0yPI7kXWn7fN+K7YMtT1yn93tSRRVGNfK Ec/ecEN5a34xs4f+DustC8IMZHUO12k8UQ6TXgvIHZ9CyKm5AYEG71vQlWwIr34yT4 zkx+AWaqHplO1nX7oWN+KmLgdWtFVIDChc+EVFxQ= To: libcamera-devel@lists.libcamera.org Date: Thu, 24 Nov 2022 04:51:29 +0200 Message-Id: <20221124025133.17875-6-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.37.4 In-Reply-To: <20221124025133.17875-1-laurent.pinchart@ideasonboard.com> References: <20221124025133.17875-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 5/9] ipa: rkisp1: agc: Support raw capture X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Support raw capture by allowing manual control of the exposure time and analogue gain. Signed-off-by: Laurent Pinchart Reviewed-by: Paul Elder Reviewed-by: Jacopo Mondi --- Changes since v3: - Drop unneeded params null check --- src/ipa/rkisp1/algorithms/agc.cpp | 49 ++++++++++++++++++++----------- src/ipa/rkisp1/algorithms/agc.h | 2 ++ 2 files changed, 34 insertions(+), 17 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index ccbcd4a9583d..6bf92743f4cf 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -62,6 +62,7 @@ static constexpr double kRelativeLuminanceTarget = 0.4; Agc::Agc() : frameCount_(0), numCells_(0), numHistBins_(0), filteredExposure_(0s) { + supportsRaw_ = true; } /** @@ -81,7 +82,7 @@ int Agc::configure(IPAContext &context, const IPACameraSensorInfo &configInfo) 10ms / context.configuration.sensor.lineDuration; context.activeState.agc.manual.gain = context.activeState.agc.automatic.gain; context.activeState.agc.manual.exposure = context.activeState.agc.automatic.exposure; - context.activeState.agc.autoEnabled = true; + context.activeState.agc.autoEnabled = !context.configuration.raw; /* * According to the RkISP1 documentation: @@ -123,12 +124,15 @@ void Agc::queueRequest(IPAContext &context, { auto &agc = context.activeState.agc; - const auto &agcEnable = controls.get(controls::AeEnable); - if (agcEnable && *agcEnable != agc.autoEnabled) { - agc.autoEnabled = *agcEnable; + if (!context.configuration.raw) { + const auto &agcEnable = controls.get(controls::AeEnable); + if (agcEnable && *agcEnable != agc.autoEnabled) { + agc.autoEnabled = *agcEnable; - LOG(RkISP1Agc, Debug) - << (agc.autoEnabled ? "Enabling" : "Disabling") << " AGC"; + LOG(RkISP1Agc, Debug) + << (agc.autoEnabled ? "Enabling" : "Disabling") + << " AGC"; + } } const auto &exposure = controls.get(controls::ExposureTime); @@ -368,6 +372,22 @@ double Agc::measureBrightness(const rkisp1_cif_isp_hist_stat *hist) const return histogram.interQuantileMean(0.98, 1.0); } +void Agc::fillMetadata(IPAContext &context, IPAFrameContext &frameContext, + ControlList &metadata) +{ + utils::Duration exposureTime = context.configuration.sensor.lineDuration + * frameContext.sensor.exposure; + metadata.set(controls::AnalogueGain, frameContext.sensor.gain); + metadata.set(controls::ExposureTime, exposureTime.get()); + + /* \todo Use VBlank value calculated from each frame exposure. */ + uint32_t vTotal = context.configuration.sensor.size.height + + context.configuration.sensor.defVBlank; + utils::Duration frameDuration = context.configuration.sensor.lineDuration + * vTotal; + metadata.set(controls::FrameDuration, frameDuration.get()); +} + /** * \brief Process RkISP1 statistics, and run AGC operations * \param[in] context The shared IPA context @@ -383,6 +403,11 @@ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, IPAFrameContext &frameContext, const rkisp1_stat_buffer *stats, ControlList &metadata) { + if (!stats) { + fillMetadata(context, frameContext, metadata); + return; + } + /* * \todo Verify that the exposure and gain applied by the sensor for * this frame match what has been requested. This isn't a hard @@ -425,17 +450,7 @@ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, computeExposure(context, frameContext, yGain, iqMeanGain); frameCount_++; - utils::Duration exposureTime = context.configuration.sensor.lineDuration - * frameContext.sensor.exposure; - metadata.set(controls::AnalogueGain, frameContext.sensor.gain); - metadata.set(controls::ExposureTime, exposureTime.get()); - - /* \todo Use VBlank value calculated from each frame exposure. */ - uint32_t vTotal = context.configuration.sensor.size.height - + context.configuration.sensor.defVBlank; - utils::Duration frameDuration = context.configuration.sensor.lineDuration - * vTotal; - metadata.set(controls::FrameDuration, frameDuration.get()); + fillMetadata(context, frameContext, metadata); } REGISTER_IPA_ALGORITHM(Agc, "Agc") diff --git a/src/ipa/rkisp1/algorithms/agc.h b/src/ipa/rkisp1/algorithms/agc.h index a228d0c37768..8a22263741b6 100644 --- a/src/ipa/rkisp1/algorithms/agc.h +++ b/src/ipa/rkisp1/algorithms/agc.h @@ -44,6 +44,8 @@ private: utils::Duration filterExposure(utils::Duration exposureValue); double estimateLuminance(const rkisp1_cif_isp_ae_stat *ae, double gain); double measureBrightness(const rkisp1_cif_isp_hist_stat *hist) const; + void fillMetadata(IPAContext &context, IPAFrameContext &frameContext, + ControlList &metadata); uint64_t frameCount_; From patchwork Thu Nov 24 02:51:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17857 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3A1B3C3286 for ; Thu, 24 Nov 2022 02:52:01 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id EE6A263328; Thu, 24 Nov 2022 03:52:00 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1669258321; bh=vbZrXKcEm2GXr2rsjpJpTqeKEP7JchVUjp/cZ0lgleg=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=FcAZRNzVq+AnKekbd/52JWXU6ey2vVlwY2rUYHimA9uIuD784QFKlyO0B3+iIQI+K OMFe6TTXTm/v83Ahb7kbOR1Vny1ARCOrwbYG5WgHxNdY9B2aZGxGwYbRThg98WDuzk +F/64d10uG+/lKv7inIWzicuBozkolK+bkEfTrYuMGZ+X/dJf5PUzgTG3L0VNd8DJU gwm78pGTfHjoiK7fjaUuoY8yVcfN10y1BdTAqNq0uIy0gRA5HyFShPorU4WNqrg+iJ 0i1MmluDchwEZ7NEMC27gSx2N4KxQ0u6SgdvqdsR3S2K0fxBTbB9CJg+cCJE7V1l6q InW5EKYuUXtXg== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7CEA96331D for ; Thu, 24 Nov 2022 03:51:58 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="UbuAcE8O"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 04DCC7FA; Thu, 24 Nov 2022 03:51:57 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1669258318; bh=vbZrXKcEm2GXr2rsjpJpTqeKEP7JchVUjp/cZ0lgleg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UbuAcE8OG6OU+RMYsn0X4AQQ4qJvxiquWurytQASFriAqo/zLG5sni/4lHu+j+QBR xtyuZQzfncnat79SrsOKYxyAd2d8EJYMIDYYgz52pl3xOPlHqrfQUqyKu5dV88C8AP 7HrleVYQBzTYel8Q39dLVWmOYbHc92gNUyniKoCI= To: libcamera-devel@lists.libcamera.org Date: Thu, 24 Nov 2022 04:51:30 +0200 Message-Id: <20221124025133.17875-7-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.37.4 In-Reply-To: <20221124025133.17875-1-laurent.pinchart@ideasonboard.com> References: <20221124025133.17875-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 6/9] pipeline: rkisp1: Query the driver for formats X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Paul Elder Query the driver for the output formats and sizes that it supports, instead of hardcoding them. This allows future-proofing for formats that are supported by some but not all versions of the driver. As the rkisp1 driver currently does not support VIDIOC_ENUM_FRAMESIZES, fallback to the hardcoded list of supported formats and framesizes. This feature will be added to the driver in parallel, though we cannot guarantee that users will have a new enough kernel for it. Signed-off-by: Paul Elder Signed-off-by: Laurent Pinchart Reviewed-by: Jacopo Mondi --- Changes since v3: - Demote Warning message to Info and rewrite it Changes since v2: - Don't pass V4L2VideoDevice to RkISP1Path::populateFormats() - Use structured binding in for range loop - Use std::set::count() to replace std::find_if() - Don't cache sizes, use std::set instead of std::map - Update min and max resolutions based on enumerated sizes - Drop comment about aspect ratio Changes since v1: - Enumerate and cache framesizes as well - Massage generateConfiguration accordingly - This lets us skip modifying V4L2VideoDevice::formats() to support lack of ENUM_FRAMESIZES - Also requires us to keep the list of hardcoded formats for backward compatibility --- src/libcamera/pipeline/rkisp1/rkisp1_path.cpp | 51 +++++++++++++++++-- src/libcamera/pipeline/rkisp1/rkisp1_path.h | 8 ++- 2 files changed, 54 insertions(+), 5 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp b/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp index 2d38f0fb37ab..7a2b0d172d84 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp @@ -41,6 +41,8 @@ bool RkISP1Path::init(MediaDevice *media) if (video_->open() < 0) return false; + populateFormats(); + link_ = media->link("rkisp1_isp", 2, resizer, 0); if (!link_) return false; @@ -48,6 +50,41 @@ bool RkISP1Path::init(MediaDevice *media) return true; } +void RkISP1Path::populateFormats() +{ + V4L2VideoDevice::Formats v4l2Formats = video_->formats(); + if (v4l2Formats.empty()) { + LOG(RkISP1, Info) + << "Failed to enumerate supported formats and sizes, using defaults"; + + for (const PixelFormat &format : formats_) + streamFormats_.insert(format); + return; + } + + minResolution_ = { 65535, 65535 }; + maxResolution_ = { 0, 0 }; + + std::vector formats; + for (const auto &[format, sizes] : v4l2Formats) { + const PixelFormat pixelFormat = format.toPixelFormat(); + const PixelFormatInfo &info = PixelFormatInfo::info(pixelFormat); + + /* \todo Add support for RAW formats. */ + if (info.colourEncoding == PixelFormatInfo::ColourEncodingRAW) + continue; + + streamFormats_.insert(pixelFormat); + + for (const auto &size : sizes) { + if (minResolution_ > size.min) + minResolution_ = size.min; + if (maxResolution_ < size.max) + maxResolution_ = size.max; + } + } +} + StreamConfiguration RkISP1Path::generateConfiguration(const Size &resolution) { Size maxResolution = maxResolution_.boundedToAspectRatio(resolution) @@ -55,7 +92,7 @@ StreamConfiguration RkISP1Path::generateConfiguration(const Size &resolution) Size minResolution = minResolution_.expandedToAspectRatio(resolution); std::map> streamFormats; - for (const PixelFormat &format : formats_) + for (const auto &format : streamFormats_) streamFormats[format] = { { minResolution, maxResolution } }; StreamFormats formats(streamFormats); @@ -72,8 +109,12 @@ CameraConfiguration::Status RkISP1Path::validate(StreamConfiguration *cfg) const StreamConfiguration reqCfg = *cfg; CameraConfiguration::Status status = CameraConfiguration::Valid; - if (std::find(formats_.begin(), formats_.end(), cfg->pixelFormat) == - formats_.end()) + /* + * Default to NV12 if the requested format is not supported. All + * versions of the ISP are guaranteed to support NV12 on both the main + * and self paths. + */ + if (!streamFormats_.count(cfg->pixelFormat)) cfg->pixelFormat = formats::NV12; cfg->size.boundTo(maxResolution_); @@ -204,6 +245,10 @@ void RkISP1Path::stop() running_ = false; } +/* + * \todo Remove the hardcoded resolutions and formats once all users will have + * migrated to a recent enough kernel. + */ namespace { constexpr Size RKISP1_RSZ_MP_SRC_MIN{ 32, 16 }; constexpr Size RKISP1_RSZ_MP_SRC_MAX{ 4416, 3312 }; diff --git a/src/libcamera/pipeline/rkisp1/rkisp1_path.h b/src/libcamera/pipeline/rkisp1/rkisp1_path.h index f3f1ae391d65..d88effbb6f56 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1_path.h +++ b/src/libcamera/pipeline/rkisp1/rkisp1_path.h @@ -8,6 +8,7 @@ #pragma once #include +#include #include #include @@ -57,14 +58,17 @@ public: Signal &bufferReady() { return video_->bufferReady; } private: + void populateFormats(); + static constexpr unsigned int RKISP1_BUFFER_COUNT = 4; const char *name_; bool running_; const Span formats_; - const Size minResolution_; - const Size maxResolution_; + std::set streamFormats_; + Size minResolution_; + Size maxResolution_; std::unique_ptr resizer_; std::unique_ptr video_; From patchwork Thu Nov 24 02:51:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17858 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id DD57FBDE6B for ; Thu, 24 Nov 2022 02:52:01 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7403A6331B; Thu, 24 Nov 2022 03:52:01 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1669258321; bh=WptlNbyQJXl6QIbjV1sxEkl7ZEFKPR4Ew90c3gsadbU=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=lluR/EMrhhO7LfM7uOyWZIC3CfyI44qYFSctl5E1M+bYqk1mPW/tQIEzHAILEGewi xC9lQkRs+dGRzqeMWk3t0gwvXzzG+om9tQEcGkhHzS7LOwzNYMhXvnj8lP+Oiw6LKn +YMlWj2U1Pvctosg3K4CuEXkUevEpJMFNADf9CD4LQTmgfMlgOH/XR0pQw2k2oKSmL AKrjv4jWe5P1wj0eaG4asXR0r1GgY6Jp+K1sFpC31r6KwbnnSt7KyzOiR8T19jJeCy kYSrLfuAcr13J20wRC2vsrw3ZImLIuHpXrvd3TjyqPTddF8EOzVrXpL6aWn0geggIg KZUIqRkwBQqFw== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E9ED263325 for ; Thu, 24 Nov 2022 03:51:59 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="YrabK7AE"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 7DA1F496 for ; Thu, 24 Nov 2022 03:51:59 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1669258319; bh=WptlNbyQJXl6QIbjV1sxEkl7ZEFKPR4Ew90c3gsadbU=; h=From:To:Subject:Date:In-Reply-To:References:From; b=YrabK7AEU1gSGJqN5uOOoPbeUe2dE6Ph1u6sXQo0O72XAzzTzuFBbCec7xeiPBL/O z/3IZB6nJukAxC/4ayArjEZlBt0ehOSNn3hP7GNmPUNud13C5UmZRU5OueCeW4JBaK WvZzIoRyW9mAV2aJ6C3TpGav8qKUGwXtB1CNPDTw= To: libcamera-devel@lists.libcamera.org Date: Thu, 24 Nov 2022 04:51:31 +0200 Message-Id: <20221124025133.17875-8-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.37.4 In-Reply-To: <20221124025133.17875-1-laurent.pinchart@ideasonboard.com> References: <20221124025133.17875-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 7/9] pipeline: rkisp1: Fix stream size validation X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Unlike RkISP1Path::generateConfiguration(), the validate() function doesn't take the camera sensor resolution into account but only considers the absolute minimum and maximum sizes supported by the ISP to validate the stream size. Fix it by using the same logic as when generating the configuration. Instead of passing the sensor resolution to the validate() function, pass the CameraSensor pointer to prepare for subsequent changes that will require access to more camera sensor data. While at it, update the generateConfiguration() function similarly for the same reason. Signed-off-by: Laurent Pinchart Reviewed-by: Paul Elder Reviewed-by: Jacopo Mondi --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 17 +++++++------- src/libcamera/pipeline/rkisp1/rkisp1_path.cpp | 22 +++++++++++++++---- src/libcamera/pipeline/rkisp1/rkisp1_path.h | 6 +++-- 3 files changed, 31 insertions(+), 14 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 231b16eca110..3eb31a49bd92 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -411,14 +411,15 @@ RkISP1CameraConfiguration::RkISP1CameraConfiguration(Camera *camera, bool RkISP1CameraConfiguration::fitsAllPaths(const StreamConfiguration &cfg) { + const CameraSensor *sensor = data_->sensor_.get(); StreamConfiguration config; config = cfg; - if (data_->mainPath_->validate(&config) != Valid) + if (data_->mainPath_->validate(sensor, &config) != Valid) return false; config = cfg; - if (data_->selfPath_ && data_->selfPath_->validate(&config) != Valid) + if (data_->selfPath_ && data_->selfPath_->validate(sensor, &config) != Valid) return false; return true; @@ -466,7 +467,7 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() /* Try to match stream without adjusting configuration. */ if (mainPathAvailable) { StreamConfiguration tryCfg = cfg; - if (data_->mainPath_->validate(&tryCfg) == Valid) { + if (data_->mainPath_->validate(sensor, &tryCfg) == Valid) { mainPathAvailable = false; cfg = tryCfg; cfg.setStream(const_cast(&data_->mainPathStream_)); @@ -476,7 +477,7 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() if (selfPathAvailable) { StreamConfiguration tryCfg = cfg; - if (data_->selfPath_->validate(&tryCfg) == Valid) { + if (data_->selfPath_->validate(sensor, &tryCfg) == Valid) { selfPathAvailable = false; cfg = tryCfg; cfg.setStream(const_cast(&data_->selfPathStream_)); @@ -487,7 +488,7 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() /* Try to match stream allowing adjusting configuration. */ if (mainPathAvailable) { StreamConfiguration tryCfg = cfg; - if (data_->mainPath_->validate(&tryCfg) == Adjusted) { + if (data_->mainPath_->validate(sensor, &tryCfg) == Adjusted) { mainPathAvailable = false; cfg = tryCfg; cfg.setStream(const_cast(&data_->mainPathStream_)); @@ -498,7 +499,7 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() if (selfPathAvailable) { StreamConfiguration tryCfg = cfg; - if (data_->selfPath_->validate(&tryCfg) == Adjusted) { + if (data_->selfPath_->validate(sensor, &tryCfg) == Adjusted) { selfPathAvailable = false; cfg = tryCfg; cfg.setStream(const_cast(&data_->selfPathStream_)); @@ -610,11 +611,11 @@ PipelineHandlerRkISP1::generateConfiguration(Camera *camera, StreamConfiguration cfg; if (useMainPath) { cfg = data->mainPath_->generateConfiguration( - data->sensor_->resolution()); + data->sensor_.get()); mainPathAvailable = false; } else { cfg = data->selfPath_->generateConfiguration( - data->sensor_->resolution()); + data->sensor_.get()); selfPathAvailable = false; } diff --git a/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp b/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp index 7a2b0d172d84..f60680556052 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp @@ -12,6 +12,7 @@ #include #include +#include "libcamera/internal/camera_sensor.h" #include "libcamera/internal/media_device.h" #include "libcamera/internal/v4l2_subdevice.h" #include "libcamera/internal/v4l2_videodevice.h" @@ -85,8 +86,10 @@ void RkISP1Path::populateFormats() } } -StreamConfiguration RkISP1Path::generateConfiguration(const Size &resolution) +StreamConfiguration RkISP1Path::generateConfiguration(const CameraSensor *sensor) { + const Size &resolution = sensor->resolution(); + Size maxResolution = maxResolution_.boundedToAspectRatio(resolution) .boundedTo(resolution); Size minResolution = minResolution_.expandedToAspectRatio(resolution); @@ -104,8 +107,11 @@ StreamConfiguration RkISP1Path::generateConfiguration(const Size &resolution) return cfg; } -CameraConfiguration::Status RkISP1Path::validate(StreamConfiguration *cfg) +CameraConfiguration::Status RkISP1Path::validate(const CameraSensor *sensor, + StreamConfiguration *cfg) { + const Size &resolution = sensor->resolution(); + const StreamConfiguration reqCfg = *cfg; CameraConfiguration::Status status = CameraConfiguration::Valid; @@ -117,8 +123,16 @@ CameraConfiguration::Status RkISP1Path::validate(StreamConfiguration *cfg) if (!streamFormats_.count(cfg->pixelFormat)) cfg->pixelFormat = formats::NV12; - cfg->size.boundTo(maxResolution_); - cfg->size.expandTo(minResolution_); + /* + * Adjust the size based on the sensor resolution and absolute limits + * of the ISP. + */ + Size maxResolution = maxResolution_.boundedToAspectRatio(resolution) + .boundedTo(resolution); + Size minResolution = minResolution_.expandedToAspectRatio(resolution); + + cfg->size.boundTo(maxResolution); + cfg->size.expandTo(minResolution); cfg->bufferCount = RKISP1_BUFFER_COUNT; V4L2DeviceFormat format; diff --git a/src/libcamera/pipeline/rkisp1/rkisp1_path.h b/src/libcamera/pipeline/rkisp1/rkisp1_path.h index d88effbb6f56..bf4ad18fbbf2 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1_path.h +++ b/src/libcamera/pipeline/rkisp1/rkisp1_path.h @@ -23,6 +23,7 @@ namespace libcamera { +class CameraSensor; class MediaDevice; class V4L2Subdevice; struct StreamConfiguration; @@ -39,8 +40,9 @@ public: int setEnabled(bool enable) { return link_->setEnabled(enable); } bool isEnabled() const { return link_->flags() & MEDIA_LNK_FL_ENABLED; } - StreamConfiguration generateConfiguration(const Size &resolution); - CameraConfiguration::Status validate(StreamConfiguration *cfg); + StreamConfiguration generateConfiguration(const CameraSensor *sensor); + CameraConfiguration::Status validate(const CameraSensor *sensor, + StreamConfiguration *cfg); int configure(const StreamConfiguration &config, const V4L2SubdeviceFormat &inputFormat); From patchwork Thu Nov 24 02:51:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17859 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CD02FBDE6B for ; Thu, 24 Nov 2022 02:52:03 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8706463326; Thu, 24 Nov 2022 03:52:03 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1669258323; bh=vBrMrU2YIZwERR5/f3yxMrEXrqpQ7RG72DN+vkmfEGI=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=U59bJ9SkwLFgt81qIFXUN/0yNg+v0lTAFP1NTMqpRa+hMo4yd/HWLeUTcw5Y/4uvy hKf0nk8TiffKeZE4iP2+nLFN5BztUiLp8T8/ZtgwF+7GXk0CPZjVlIuM7H0YZyFOQQ E3Yzh77V/cB7dPfSm5w/oTM8koUEenwPvoZQQqUrvbB3PJfOXeYIfHtogD5fvud6EH rM4Fh1tSDcOJSSMHmMwgP+85e150lwWSRd7dkjsm33G8aC76pVNMnyvNiPUQME32Zq Ogvk040dXuLxFU7d0xVABnrwzpQi2DKkJ/z1nw5ry6Cikq+PnDjR3eUc2qp/YM1Lxe ndZj5NE71rdyw== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 956706331D for ; Thu, 24 Nov 2022 03:52:01 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="bc02A7G6"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id C9524496; Thu, 24 Nov 2022 03:52:00 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1669258321; bh=vBrMrU2YIZwERR5/f3yxMrEXrqpQ7RG72DN+vkmfEGI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bc02A7G6qbg3l6DvaMhxOKhV7fjKxDewj2MPsalPQbJGhwzpC4sp0+u9ONU+iFh8p Qs5StEuH8Sg9Q+nuR2ihQ9IHbRdQXPzE1ZB3ZQRlwRFJHWQK98MJJSreISNXXu1iEO EoUOxvnwsPtioZnGRW53vvHnCuXBCfDY+n3pBFXQ= To: libcamera-devel@lists.libcamera.org Date: Thu, 24 Nov 2022 04:51:32 +0200 Message-Id: <20221124025133.17875-9-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.37.4 In-Reply-To: <20221124025133.17875-1-laurent.pinchart@ideasonboard.com> References: <20221124025133.17875-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 8/9] pipeline: rkisp1: Support raw Bayer capture at runtime X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Florian Sylvestre Implement support for raw Bayer capture at runtime, from start() to stop(). Support of raw formats in the camera configuration is split to a subsequent change to ease review. In raw mode, the ISP is bypassed. There is no need to provide parameter buffers, and the ISP will not generate statistics. This requires multiple changes in the buffer handling: - The params and stats buffers don't need to be allocated, and the corresponding video nodes don't need to be started or stopped. - The IPA module fillParamsBuffer() operation must not be called in queueRequestDevice(). As a result, the IPA module thus doesn't emit the paramsBufferReady signal. The main and self path video buffers must thus be queued directly in queueRequestDevice(). - The tryCompleteRequest() function must not to wait until the params buffer has been dequeued. - When the frame buffer has been captured, the IPA module processStatsBuffer() operation must be called directly to fill request metadata. Signed-off-by: Florian Sylvestre Signed-off-by: Paul Elder Signed-off-by: Laurent Pinchart Reviewed-by: Paul Elder Tested-by: Jacopo Mondi Reviewed-by: Jacopo Mondi --- Changes since v2: - Split from "pipeline: rkisp1: Support raw Bayer capture" - Drop new completeRaw() IPA operation - Don't queue params buffers in raw capture mode - Fix assertion failure when stopping capture --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 148 ++++++++++++++--------- 1 file changed, 93 insertions(+), 55 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 3eb31a49bd92..6073fee8ec20 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -67,7 +67,8 @@ class RkISP1Frames public: RkISP1Frames(PipelineHandler *pipe); - RkISP1FrameInfo *create(const RkISP1CameraData *data, Request *request); + RkISP1FrameInfo *create(const RkISP1CameraData *data, Request *request, + bool isRaw); int destroy(unsigned int frame); void clear(); @@ -184,6 +185,7 @@ private: std::unique_ptr csi_; bool hasSelfPath_; + bool isRaw_; RkISP1MainPath mainPath_; RkISP1SelfPath selfPath_; @@ -203,28 +205,35 @@ RkISP1Frames::RkISP1Frames(PipelineHandler *pipe) { } -RkISP1FrameInfo *RkISP1Frames::create(const RkISP1CameraData *data, Request *request) +RkISP1FrameInfo *RkISP1Frames::create(const RkISP1CameraData *data, Request *request, + bool isRaw) { unsigned int frame = data->frame_; - if (pipe_->availableParamBuffers_.empty()) { - LOG(RkISP1, Error) << "Parameters buffer underrun"; - return nullptr; - } - FrameBuffer *paramBuffer = pipe_->availableParamBuffers_.front(); + FrameBuffer *paramBuffer = nullptr; + FrameBuffer *statBuffer = nullptr; + + if (!isRaw) { + if (pipe_->availableParamBuffers_.empty()) { + LOG(RkISP1, Error) << "Parameters buffer underrun"; + return nullptr; + } + + if (pipe_->availableStatBuffers_.empty()) { + LOG(RkISP1, Error) << "Statisitc buffer underrun"; + return nullptr; + } + + paramBuffer = pipe_->availableParamBuffers_.front(); + pipe_->availableParamBuffers_.pop(); - if (pipe_->availableStatBuffers_.empty()) { - LOG(RkISP1, Error) << "Statisitc buffer underrun"; - return nullptr; + statBuffer = pipe_->availableStatBuffers_.front(); + pipe_->availableStatBuffers_.pop(); } - FrameBuffer *statBuffer = pipe_->availableStatBuffers_.front(); FrameBuffer *mainPathBuffer = request->findBuffer(&data->mainPathStream_); FrameBuffer *selfPathBuffer = request->findBuffer(&data->selfPathStream_); - pipe_->availableParamBuffers_.pop(); - pipe_->availableStatBuffers_.pop(); - RkISP1FrameInfo *info = new RkISP1FrameInfo; info->frame = frame; @@ -672,6 +681,8 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) << "ISP input pad configured with " << format << " crop " << rect; + isRaw_ = false; + /* YUYV8_2X8 is required on the ISP source path pad for YUV output. */ format.mbus_code = MEDIA_BUS_FMT_YUYV8_2X8; LOG(RkISP1, Debug) @@ -764,13 +775,15 @@ int PipelineHandlerRkISP1::allocateBuffers(Camera *camera) data->selfPathStream_.configuration().bufferCount, }); - ret = param_->allocateBuffers(maxCount, ¶mBuffers_); - if (ret < 0) - goto error; + if (!isRaw_) { + ret = param_->allocateBuffers(maxCount, ¶mBuffers_); + if (ret < 0) + goto error; - ret = stat_->allocateBuffers(maxCount, &statBuffers_); - if (ret < 0) - goto error; + ret = stat_->allocateBuffers(maxCount, &statBuffers_); + if (ret < 0) + goto error; + } for (std::unique_ptr &buffer : paramBuffers_) { buffer->setCookie(ipaBufferId++); @@ -846,23 +859,25 @@ int PipelineHandlerRkISP1::start(Camera *camera, [[maybe_unused]] const ControlL data->frame_ = 0; - ret = param_->streamOn(); - if (ret) { - data->ipa_->stop(); - freeBuffers(camera); - LOG(RkISP1, Error) - << "Failed to start parameters " << camera->id(); - return ret; - } + if (!isRaw_) { + ret = param_->streamOn(); + if (ret) { + data->ipa_->stop(); + freeBuffers(camera); + LOG(RkISP1, Error) + << "Failed to start parameters " << camera->id(); + return ret; + } - ret = stat_->streamOn(); - if (ret) { - param_->streamOff(); - data->ipa_->stop(); - freeBuffers(camera); - LOG(RkISP1, Error) - << "Failed to start statistics " << camera->id(); - return ret; + ret = stat_->streamOn(); + if (ret) { + param_->streamOff(); + data->ipa_->stop(); + freeBuffers(camera); + LOG(RkISP1, Error) + << "Failed to start statistics " << camera->id(); + return ret; + } } if (data->mainPath_->isEnabled()) { @@ -907,15 +922,17 @@ void PipelineHandlerRkISP1::stopDevice(Camera *camera) selfPath_.stop(); mainPath_.stop(); - ret = stat_->streamOff(); - if (ret) - LOG(RkISP1, Warning) - << "Failed to stop statistics for " << camera->id(); + if (!isRaw_) { + ret = stat_->streamOff(); + if (ret) + LOG(RkISP1, Warning) + << "Failed to stop statistics for " << camera->id(); - ret = param_->streamOff(); - if (ret) - LOG(RkISP1, Warning) - << "Failed to stop parameters for " << camera->id(); + ret = param_->streamOff(); + if (ret) + LOG(RkISP1, Warning) + << "Failed to stop parameters for " << camera->id(); + } ASSERT(data->queuedRequests_.empty()); data->frameInfo_.clear(); @@ -929,12 +946,21 @@ int PipelineHandlerRkISP1::queueRequestDevice(Camera *camera, Request *request) { RkISP1CameraData *data = cameraData(camera); - RkISP1FrameInfo *info = data->frameInfo_.create(data, request); + RkISP1FrameInfo *info = data->frameInfo_.create(data, request, isRaw_); if (!info) return -ENOENT; data->ipa_->queueRequest(data->frame_, request->controls()); - data->ipa_->fillParamsBuffer(data->frame_, info->paramBuffer->cookie()); + if (isRaw_) { + if (info->mainPathBuffer) + data->mainPath_->queueBuffer(info->mainPathBuffer); + + if (data->selfPath_ && info->selfPathBuffer) + data->selfPath_->queueBuffer(info->selfPathBuffer); + } else { + data->ipa_->fillParamsBuffer(data->frame_, + info->paramBuffer->cookie()); + } data->frame_++; @@ -1139,7 +1165,7 @@ void PipelineHandlerRkISP1::tryCompleteRequest(RkISP1FrameInfo *info) if (!info->metadataProcessed) return; - if (!info->paramDequeued) + if (!isRaw_ && !info->paramDequeued) return; data->frameInfo_.destroy(info->frame); @@ -1156,16 +1182,28 @@ void PipelineHandlerRkISP1::bufferReady(FrameBuffer *buffer) if (!info) return; + const FrameMetadata &metadata = buffer->metadata(); Request *request = buffer->request(); - /* - * Record the sensor's timestamp in the request metadata. - * - * \todo The sensor timestamp should be better estimated by connecting - * to the V4L2Device::frameStart signal. - */ - request->metadata().set(controls::SensorTimestamp, - buffer->metadata().timestamp); + if (metadata.status != FrameMetadata::FrameCancelled) { + /* + * Record the sensor's timestamp in the request metadata. + * + * \todo The sensor timestamp should be better estimated by connecting + * to the V4L2Device::frameStart signal. + */ + request->metadata().set(controls::SensorTimestamp, + metadata.timestamp); + + if (isRaw_) { + const ControlList &ctrls = + data->delayedCtrls_->get(metadata.sequence); + data->ipa_->processStatsBuffer(info->frame, 0, ctrls); + } + } else { + if (isRaw_) + info->metadataProcessed = true; + } completeBuffer(request, buffer); tryCompleteRequest(info); From patchwork Thu Nov 24 02:51:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17860 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 73D16BDE6B for ; Thu, 24 Nov 2022 02:52:04 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 34B166332F; Thu, 24 Nov 2022 03:52:04 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1669258324; bh=QaZCJhIf6DdDRXOF1mG0FfD8BNnU0QsgY1baN6/UzWM=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=ttBFxsZO36qs6/v4OeTHBrOeigB0uwmTj0jnd9Dpv5HwdrCAmBDloHI1p5usWD2Xg h+lxJ5acvgyM00clXlr7MUiHZI3TSgksRPaJQz2diBFJ/YtXXbDOgcMuVen+zWC8el 7dzjhIcLBE8vKVymHpJo2ApvPy9QT1h1qYV/0PFqCdABz5AjqBf9gp254KPyJ4FFpd JQt3kihNs540XbUkZJVWXvEgQ00pu/qrlg4ASKMeVFXo2uWLS2/Zza+LG+cVceUww5 WxF7wgDBtCFYrqqlb1brvWC28wgm1UJi0zj9JhuABZyJCcdWPl4MUdgMajgLEa8VLV FDZ2v8PNPQtMg== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9A3EF6332D for ; Thu, 24 Nov 2022 03:52:02 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="guZg0Aif"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 1D0C6496; Thu, 24 Nov 2022 03:52:02 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1669258322; bh=QaZCJhIf6DdDRXOF1mG0FfD8BNnU0QsgY1baN6/UzWM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=guZg0Aifklfvi/VToNlADwP5sct+rmV6obM0b2O7yVvPC09ua5GIxuC2SzuL9rLYB GeXq20vxg67dsnDGrLCvMDMYWOuzvawugSBAJ+gocM+tlfExkVreku2ntLjsnoBYEw rKo/BzbtKlxCR2rXQOzFAXt+TqyPuQCYaD5KiR3M= To: libcamera-devel@lists.libcamera.org Date: Thu, 24 Nov 2022 04:51:33 +0200 Message-Id: <20221124025133.17875-10-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.37.4 In-Reply-To: <20221124025133.17875-1-laurent.pinchart@ideasonboard.com> References: <20221124025133.17875-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 9/9] pipeline: rkisp1: Support raw Bayer capture configuration X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Florian Sylvestre Implement support for raw Bayer capture during configuration generation, validation and camera configuration. While at it, fix a typo in a comment. Signed-off-by: Florian Sylvestre Signed-off-by: Paul Elder Signed-off-by: Laurent Pinchart Reviewed-by: Paul Elder Reviewed-by: Jacopo Mondi --- Changes since v3: - Fix misplaced break - Reduce indentation level in loop - s/higher/highest/ Changes since v2: - Nearly complete rewrite --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 116 +++++++--- src/libcamera/pipeline/rkisp1/rkisp1_path.cpp | 210 +++++++++++++++--- src/libcamera/pipeline/rkisp1/rkisp1_path.h | 3 +- 3 files changed, 274 insertions(+), 55 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 6073fee8ec20..eb9ad65cb114 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -410,6 +410,30 @@ void RkISP1CameraData::metadataReady(unsigned int frame, const ControlList &meta pipe()->tryCompleteRequest(info); } +/* ----------------------------------------------------------------------------- + * Camera Configuration + */ + +namespace { + +/* Keep in sync with the supported raw formats in rkisp1_path.cpp. */ +const std::map rawFormats = { + { formats::SBGGR8, MEDIA_BUS_FMT_SBGGR8_1X8 }, + { formats::SGBRG8, MEDIA_BUS_FMT_SGBRG8_1X8 }, + { formats::SGRBG8, MEDIA_BUS_FMT_SGRBG8_1X8 }, + { formats::SRGGB8, MEDIA_BUS_FMT_SRGGB8_1X8 }, + { formats::SBGGR10, MEDIA_BUS_FMT_SBGGR10_1X10 }, + { formats::SGBRG10, MEDIA_BUS_FMT_SGBRG10_1X10 }, + { formats::SGRBG10, MEDIA_BUS_FMT_SGRBG10_1X10 }, + { formats::SRGGB10, MEDIA_BUS_FMT_SRGGB10_1X10 }, + { formats::SBGGR12, MEDIA_BUS_FMT_SBGGR12_1X12 }, + { formats::SGBRG12, MEDIA_BUS_FMT_SGBRG12_1X12 }, + { formats::SGRBG12, MEDIA_BUS_FMT_SGRBG12_1X12 }, + { formats::SRGGB12, MEDIA_BUS_FMT_SRGGB12_1X12 }, +}; + +} /* namespace */ + RkISP1CameraConfiguration::RkISP1CameraConfiguration(Camera *camera, RkISP1CameraData *data) : CameraConfiguration() @@ -456,6 +480,21 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() status = Adjusted; } + /* + * Simultaneous capture of raw and processed streams isn't possible. If + * there is any raw stream, cap the number of streams to one. + */ + if (config_.size() > 1) { + for (const auto &cfg : config_) { + if (PixelFormatInfo::info(cfg.pixelFormat).colourEncoding == + PixelFormatInfo::ColourEncodingRAW) { + config_.resize(1); + status = Adjusted; + break; + } + } + } + /* * If there are more than one stream in the configuration figure out the * order to evaluate the streams. The first stream has the highest @@ -517,45 +556,51 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() } } - /* All paths rejected configuraiton. */ + /* All paths rejected configuration. */ LOG(RkISP1, Debug) << "Camera configuration not supported " << cfg.toString(); return Invalid; } /* Select the sensor format. */ + PixelFormat rawFormat; Size maxSize; - for (const StreamConfiguration &cfg : config_) + + for (const StreamConfiguration &cfg : config_) { + const PixelFormatInfo &info = PixelFormatInfo::info(cfg.pixelFormat); + if (info.colourEncoding == PixelFormatInfo::ColourEncodingRAW) + rawFormat = cfg.pixelFormat; + maxSize = std::max(maxSize, cfg.size); + } + + std::vector mbusCodes; + + if (rawFormat.isValid()) { + mbusCodes = { rawFormats.at(rawFormat) }; + } else { + std::transform(rawFormats.begin(), rawFormats.end(), + std::back_inserter(mbusCodes), + [](const auto &value) { return value.second; }); + } + + sensorFormat_ = sensor->getFormat(mbusCodes, maxSize); - sensorFormat_ = sensor->getFormat({ MEDIA_BUS_FMT_SBGGR12_1X12, - MEDIA_BUS_FMT_SGBRG12_1X12, - MEDIA_BUS_FMT_SGRBG12_1X12, - MEDIA_BUS_FMT_SRGGB12_1X12, - MEDIA_BUS_FMT_SBGGR10_1X10, - MEDIA_BUS_FMT_SGBRG10_1X10, - MEDIA_BUS_FMT_SGRBG10_1X10, - MEDIA_BUS_FMT_SRGGB10_1X10, - MEDIA_BUS_FMT_SBGGR8_1X8, - MEDIA_BUS_FMT_SGBRG8_1X8, - MEDIA_BUS_FMT_SGRBG8_1X8, - MEDIA_BUS_FMT_SRGGB8_1X8 }, - maxSize); if (sensorFormat_.size.isNull()) sensorFormat_.size = sensor->resolution(); return status; } +/* ----------------------------------------------------------------------------- + * Pipeline Operations + */ + PipelineHandlerRkISP1::PipelineHandlerRkISP1(CameraManager *manager) : PipelineHandler(manager), hasSelfPath_(true) { } -/* ----------------------------------------------------------------------------- - * Pipeline Operations - */ - std::unique_ptr PipelineHandlerRkISP1::generateConfiguration(Camera *camera, const StreamRoles &roles) @@ -611,23 +656,38 @@ PipelineHandlerRkISP1::generateConfiguration(Camera *camera, colorSpace = ColorSpace::Rec709; break; + case StreamRole::Raw: + if (roles.size() > 1) { + LOG(RkISP1, Error) + << "Can't capture both raw and processed streams"; + return nullptr; + } + + useMainPath = true; + colorSpace = ColorSpace::Raw; + break; + default: LOG(RkISP1, Warning) << "Requested stream role not supported: " << role; return nullptr; } - StreamConfiguration cfg; + RkISP1Path *path; + if (useMainPath) { - cfg = data->mainPath_->generateConfiguration( - data->sensor_.get()); + path = data->mainPath_; mainPathAvailable = false; } else { - cfg = data->selfPath_->generateConfiguration( - data->sensor_.get()); + path = data->selfPath_; selfPathAvailable = false; } + StreamConfiguration cfg = + path->generateConfiguration(data->sensor_.get(), role); + if (!cfg.pixelFormat.isValid()) + return nullptr; + cfg.colorSpace = colorSpace; config->addConfiguration(cfg); } @@ -681,10 +741,14 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) << "ISP input pad configured with " << format << " crop " << rect; - isRaw_ = false; + const PixelFormat &streamFormat = config->at(0).pixelFormat; + const PixelFormatInfo &info = PixelFormatInfo::info(streamFormat); + isRaw_ = info.colourEncoding == PixelFormatInfo::ColourEncodingRAW; /* YUYV8_2X8 is required on the ISP source path pad for YUV output. */ - format.mbus_code = MEDIA_BUS_FMT_YUYV8_2X8; + if (!isRaw_) + format.mbus_code = MEDIA_BUS_FMT_YUYV8_2X8; + LOG(RkISP1, Debug) << "Configuring ISP output pad with " << format << " crop " << rect; diff --git a/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp b/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp index f60680556052..5079b268c464 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp @@ -21,6 +21,39 @@ namespace libcamera { LOG_DECLARE_CATEGORY(RkISP1) +namespace { + +/* Keep in sync with the supported raw formats in rkisp1.cpp. */ +const std::map formatToMediaBus = { + { formats::UYVY, MEDIA_BUS_FMT_YUYV8_2X8 }, + { formats::YUYV, MEDIA_BUS_FMT_YUYV8_2X8 }, + { formats::NV12, MEDIA_BUS_FMT_YUYV8_1_5X8 }, + { formats::NV21, MEDIA_BUS_FMT_YUYV8_1_5X8 }, + { formats::NV16, MEDIA_BUS_FMT_YUYV8_2X8 }, + { formats::NV61, MEDIA_BUS_FMT_YUYV8_2X8 }, + { formats::YUV420, MEDIA_BUS_FMT_YUYV8_1_5X8 }, + { formats::YVU420, MEDIA_BUS_FMT_YUYV8_1_5X8 }, + { formats::YUV422, MEDIA_BUS_FMT_YUYV8_2X8 }, + { formats::YVU422, MEDIA_BUS_FMT_YUYV8_2X8 }, + { formats::R8, MEDIA_BUS_FMT_YUYV8_2X8 }, + { formats::RGB565, MEDIA_BUS_FMT_YUYV8_2X8 }, + { formats::XRGB8888, MEDIA_BUS_FMT_YUYV8_2X8 }, + { formats::SBGGR8, MEDIA_BUS_FMT_SBGGR8_1X8 }, + { formats::SGBRG8, MEDIA_BUS_FMT_SGBRG8_1X8 }, + { formats::SGRBG8, MEDIA_BUS_FMT_SGRBG8_1X8 }, + { formats::SRGGB8, MEDIA_BUS_FMT_SRGGB8_1X8 }, + { formats::SBGGR10, MEDIA_BUS_FMT_SBGGR10_1X10 }, + { formats::SGBRG10, MEDIA_BUS_FMT_SGBRG10_1X10 }, + { formats::SGRBG10, MEDIA_BUS_FMT_SGRBG10_1X10 }, + { formats::SRGGB10, MEDIA_BUS_FMT_SRGGB10_1X10 }, + { formats::SBGGR12, MEDIA_BUS_FMT_SBGGR12_1X12 }, + { formats::SGBRG12, MEDIA_BUS_FMT_SGBRG12_1X12 }, + { formats::SGRBG12, MEDIA_BUS_FMT_SGRBG12_1X12 }, + { formats::SRGGB12, MEDIA_BUS_FMT_SRGGB12_1X12 }, +}; + +} /* namespace */ + RkISP1Path::RkISP1Path(const char *name, const Span &formats, const Size &minResolution, const Size &maxResolution) : name_(name), running_(false), formats_(formats), @@ -69,11 +102,18 @@ void RkISP1Path::populateFormats() std::vector formats; for (const auto &[format, sizes] : v4l2Formats) { const PixelFormat pixelFormat = format.toPixelFormat(); - const PixelFormatInfo &info = PixelFormatInfo::info(pixelFormat); - /* \todo Add support for RAW formats. */ - if (info.colourEncoding == PixelFormatInfo::ColourEncodingRAW) + /* + * As a defensive measure, skip any pixel format exposed by the + * driver that we don't know about. This ensures that looking up + * formats in formatToMediaBus using a key from streamFormats_ + * will never fail in any of the other functions. + */ + if (!formatToMediaBus.count(pixelFormat)) { + LOG(RkISP1, Warning) + << "Unsupported pixel format " << pixelFormat; continue; + } streamFormats_.insert(pixelFormat); @@ -86,21 +126,69 @@ void RkISP1Path::populateFormats() } } -StreamConfiguration RkISP1Path::generateConfiguration(const CameraSensor *sensor) +StreamConfiguration +RkISP1Path::generateConfiguration(const CameraSensor *sensor, StreamRole role) { + const std::vector &mbusCodes = sensor->mbusCodes(); const Size &resolution = sensor->resolution(); Size maxResolution = maxResolution_.boundedToAspectRatio(resolution) .boundedTo(resolution); Size minResolution = minResolution_.expandedToAspectRatio(resolution); + /* Create the list of supported stream formats. */ std::map> streamFormats; - for (const auto &format : streamFormats_) - streamFormats[format] = { { minResolution, maxResolution } }; + unsigned int rawBitsPerPixel = 0; + PixelFormat rawFormat; + + for (const auto &format : streamFormats_) { + const PixelFormatInfo &info = PixelFormatInfo::info(format); + + if (info.colourEncoding != PixelFormatInfo::ColourEncodingRAW) { + streamFormats[format] = { { minResolution, maxResolution } }; + continue; + } + + /* Skip raw formats not supported by the sensor. */ + uint32_t mbusCode = formatToMediaBus.at(format); + if (std::find(mbusCodes.begin(), mbusCodes.end(), mbusCode) == + mbusCodes.end()) + continue; + + streamFormats[format] = { { resolution, resolution } }; + + /* + * Store the raw format with the highest bits per pixel for + * later usage. + */ + if (info.bitsPerPixel > rawBitsPerPixel) { + rawBitsPerPixel = info.bitsPerPixel; + rawFormat = format; + } + } + + /* + * Pick a suitable pixel format for the role. Raw streams need to use a + * raw format, processed streams use NV12 by default. + */ + PixelFormat format; + + if (role == StreamRole::Raw) { + if (!rawFormat.isValid()) { + LOG(RkISP1, Error) + << "Sensor " << sensor->model() + << " doesn't support raw capture"; + return {}; + } + + format = rawFormat; + } else { + format = formats::NV12; + } StreamFormats formats(streamFormats); StreamConfiguration cfg(formats); - cfg.pixelFormat = formats::NV12; + cfg.pixelFormat = format; cfg.size = maxResolution; cfg.bufferCount = RKISP1_BUFFER_COUNT; @@ -110,26 +198,85 @@ StreamConfiguration RkISP1Path::generateConfiguration(const CameraSensor *sensor CameraConfiguration::Status RkISP1Path::validate(const CameraSensor *sensor, StreamConfiguration *cfg) { + const std::vector &mbusCodes = sensor->mbusCodes(); const Size &resolution = sensor->resolution(); const StreamConfiguration reqCfg = *cfg; CameraConfiguration::Status status = CameraConfiguration::Valid; /* - * Default to NV12 if the requested format is not supported. All - * versions of the ISP are guaranteed to support NV12 on both the main - * and self paths. + * Validate the pixel format. If the requested format isn't supported, + * default to either NV12 (all versions of the ISP are guaranteed to + * support NV12 on both the main and self paths) if the requested format + * is not a raw format, or to the supported raw format with the highest + * bits per pixel otherwise. */ - if (!streamFormats_.count(cfg->pixelFormat)) - cfg->pixelFormat = formats::NV12; + unsigned int rawBitsPerPixel = 0; + PixelFormat rawFormat; + bool found = false; + + for (const auto &format : streamFormats_) { + const PixelFormatInfo &info = PixelFormatInfo::info(format); + + if (info.colourEncoding == PixelFormatInfo::ColourEncodingRAW) { + /* Skip raw formats not supported by the sensor. */ + uint32_t mbusCode = formatToMediaBus.at(format); + if (std::find(mbusCodes.begin(), mbusCodes.end(), mbusCode) == + mbusCodes.end()) + continue; + + /* + * Store the raw format with the highest bits per pixel + * for later usage. + */ + if (info.bitsPerPixel > rawBitsPerPixel) { + rawBitsPerPixel = info.bitsPerPixel; + rawFormat = format; + } + } + + if (cfg->pixelFormat == format) { + found = true; + break; + } + } + + bool isRaw = PixelFormatInfo::info(cfg->pixelFormat).colourEncoding == + PixelFormatInfo::ColourEncodingRAW; /* - * Adjust the size based on the sensor resolution and absolute limits - * of the ISP. + * If no raw format supported by the sensor has been found, use a + * processed format. */ - Size maxResolution = maxResolution_.boundedToAspectRatio(resolution) - .boundedTo(resolution); - Size minResolution = minResolution_.expandedToAspectRatio(resolution); + if (!rawFormat.isValid()) + isRaw = false; + + if (!found) + cfg->pixelFormat = isRaw ? rawFormat : formats::NV12; + + Size minResolution; + Size maxResolution; + + if (isRaw) { + /* + * Use the sensor output size closest to the requested stream + * size. + */ + uint32_t mbusCode = formatToMediaBus.at(cfg->pixelFormat); + V4L2SubdeviceFormat sensorFormat = + sensor->getFormat({ mbusCode }, cfg->size); + + minResolution = sensorFormat.size; + maxResolution = sensorFormat.size; + } else { + /* + * Adjust the size based on the sensor resolution and absolute + * limits of the ISP. + */ + minResolution = minResolution_.expandedToAspectRatio(resolution); + maxResolution = maxResolution_.boundedToAspectRatio(resolution) + .boundedTo(resolution); + } cfg->size.boundTo(maxResolution); cfg->size.expandTo(minResolution); @@ -182,15 +329,11 @@ int RkISP1Path::configure(const StreamConfiguration &config, << "Configuring " << name_ << " resizer output pad with " << ispFormat; - switch (config.pixelFormat) { - case formats::NV12: - case formats::NV21: - ispFormat.mbus_code = MEDIA_BUS_FMT_YUYV8_1_5X8; - break; - default: - ispFormat.mbus_code = MEDIA_BUS_FMT_YUYV8_2X8; - break; - } + /* + * The configuration has been validated, the pixel format is guaranteed + * to be supported and thus found in formatToMediaBus. + */ + ispFormat.mbus_code = formatToMediaBus.at(config.pixelFormat); ret = resizer_->setFormat(1, &ispFormat); if (ret < 0) @@ -266,14 +409,25 @@ void RkISP1Path::stop() namespace { constexpr Size RKISP1_RSZ_MP_SRC_MIN{ 32, 16 }; constexpr Size RKISP1_RSZ_MP_SRC_MAX{ 4416, 3312 }; -constexpr std::array RKISP1_RSZ_MP_FORMATS{ +constexpr std::array RKISP1_RSZ_MP_FORMATS{ formats::YUYV, formats::NV16, formats::NV61, formats::NV21, formats::NV12, formats::R8, - /* \todo Add support for RAW formats. */ + formats::SBGGR8, + formats::SGBRG8, + formats::SGRBG8, + formats::SRGGB8, + formats::SBGGR10, + formats::SGBRG10, + formats::SGRBG10, + formats::SRGGB10, + formats::SBGGR12, + formats::SGBRG12, + formats::SGRBG12, + formats::SRGGB12, }; constexpr Size RKISP1_RSZ_SP_SRC_MIN{ 32, 16 }; diff --git a/src/libcamera/pipeline/rkisp1/rkisp1_path.h b/src/libcamera/pipeline/rkisp1/rkisp1_path.h index bf4ad18fbbf2..bdf3f95b95e1 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1_path.h +++ b/src/libcamera/pipeline/rkisp1/rkisp1_path.h @@ -40,7 +40,8 @@ public: int setEnabled(bool enable) { return link_->setEnabled(enable); } bool isEnabled() const { return link_->flags() & MEDIA_LNK_FL_ENABLED; } - StreamConfiguration generateConfiguration(const CameraSensor *sensor); + StreamConfiguration generateConfiguration(const CameraSensor *sensor, + StreamRole role); CameraConfiguration::Status validate(const CameraSensor *sensor, StreamConfiguration *cfg);