From patchwork Thu Nov 24 02:51:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17854 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9528CBDE6B for ; Thu, 24 Nov 2022 02:51:57 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2E0046331A; Thu, 24 Nov 2022 03:51:57 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1669258317; bh=rLVvinFiJ0iEgC2rv4fnV9rCavXASkwU+sf8AON4LRw=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=D+atPvl1bcCUW/bQq3eWUDUk1EBAOeMQiViHTfU31ImaeqKsOa3cfezIa32ivwvnA SdwjO5KWjcE4Oqi3Lqql8rQjebMgvRktFVzjhL3Ac4IqQ41B+ksnyaPEs8Vbs6IdnJ ewN3TmL1H2+2x93r4oLuLAc8X268PAhMd0Zuo8s4A1/cUCGCQqwsVGlfzvNTht79p7 NkmnOkbX/nwqeiSuKpdA0vM+VgGdHyZtx27fLqioEtYYCoB+wDQaBGHxV31e/05Ggb dh+Zl3rpq7KWBtbWqi14mSUJLNHWWBLN6gAMUBabj/prtRlNyabUcdQ3fLAppQsVfB hpE9rU09wiesw== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 74B3863316 for ; Thu, 24 Nov 2022 03:51:54 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="vAtQi8MM"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E31837FA for ; Thu, 24 Nov 2022 03:51:53 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1669258314; bh=rLVvinFiJ0iEgC2rv4fnV9rCavXASkwU+sf8AON4LRw=; h=From:To:Subject:Date:In-Reply-To:References:From; b=vAtQi8MMzuu0tma3Pi954H7i/YkTd6jKIMnUIuEikLSHkDQ/Bhi2k1vh9RhZQqezl a0zhWt4tw00p/3gb2nxI8yDZY5SyIm4Z3nzK1wU3Gk2Mf5KyHX48QiOk/nBOTELj3v gnSPI5wPIOa8HG+7xtgp5OebPq0KcDz0mTkKoKV8= To: libcamera-devel@lists.libcamera.org Date: Thu, 24 Nov 2022 04:51:27 +0200 Message-Id: <20221124025133.17875-4-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.37.4 In-Reply-To: <20221124025133.17875-1-laurent.pinchart@ideasonboard.com> References: <20221124025133.17875-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 3/9] ipa: rkisp1: Support raw capture in IPA operations X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The RkISP1 can capture raw frames by bypassing the ISP. In that mode, the ISP will not produce statistics nor consume parameters. Most algorithms will thus be unable to run, with one exception: the AGC will still be able to configure the sensor exposure time and analog gain in manual mode. To prepare for this, add the ability to disable algorithms for the duration of the capture session based on the camera configuration. Individual algorithms report whether they support raw formats at construction time, and the IPA module disables algorithms in configure() based on the stream configurations. Disabled algorithms are skipped during the capture session in the processStatsBuffer() operation. As the ISP doesn't produce statistics, don't try to access the stats buffer. There is no need for similar logic in fillParamsBuffer() as that operation won't be called for raw capture. All algorithms report not supporting raw capture by default. Raw support in AGC will be added separately. The feature is implemented in the RkISP1 module without any support from libipa at this point to avoid designing a generic API based on a single user. This may be changed in the future. Signed-off-by: Laurent Pinchart Reviewed-by: Paul Elder Reviewed-by: Jacopo Mondi --- Changes since v3: - Initialize stats to nullptr by default --- src/ipa/rkisp1/algorithms/algorithm.h | 12 +++++++- src/ipa/rkisp1/ipa_context.cpp | 5 ++++ src/ipa/rkisp1/ipa_context.h | 2 ++ src/ipa/rkisp1/rkisp1.cpp | 40 +++++++++++++++++++++++---- 4 files changed, 52 insertions(+), 7 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/algorithm.h b/src/ipa/rkisp1/algorithms/algorithm.h index c3212cff76fe..9454c9a1fc06 100644 --- a/src/ipa/rkisp1/algorithms/algorithm.h +++ b/src/ipa/rkisp1/algorithms/algorithm.h @@ -15,7 +15,17 @@ namespace libcamera { namespace ipa::rkisp1 { -using Algorithm = libcamera::ipa::Algorithm; +class Algorithm : public libcamera::ipa::Algorithm +{ +public: + Algorithm() + : disabled_(false), supportsRaw_(false) + { + } + + bool disabled_; + bool supportsRaw_; +}; } /* namespace ipa::rkisp1 */ diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp index 7a987497bd03..9bbf368432fa 100644 --- a/src/ipa/rkisp1/ipa_context.cpp +++ b/src/ipa/rkisp1/ipa_context.cpp @@ -89,6 +89,11 @@ namespace libcamera::ipa::rkisp1 { * \brief Sensor output resolution */ +/** + * \var IPASessionConfiguration::raw + * \brief Indicates if the camera is configured to capture raw frames + */ + /** * \struct IPAActiveState * \brief Active state for algorithms diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index bb60ab9eab72..3e47ac663c58 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -48,6 +48,8 @@ struct IPASessionConfiguration { struct { rkisp1_cif_isp_version revision; } hw; + + bool raw; }; struct IPAActiveState { diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 9265d3c9f53c..6ac29df8ec8d 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -24,6 +24,7 @@ #include #include +#include "libcamera/internal/formats.h" #include "libcamera/internal/mapped_framebuffer.h" #include "libcamera/internal/yaml_parser.h" @@ -207,7 +208,7 @@ void IPARkISP1::stop() } int IPARkISP1::configure(const IPAConfigInfo &ipaConfig, - [[maybe_unused]] const std::map &streamConfig, + const std::map &streamConfig, ControlInfoMap *ipaControls) { sensorControls_ = ipaConfig.sensorControls; @@ -255,7 +256,21 @@ int IPARkISP1::configure(const IPAConfigInfo &ipaConfig, context_.configuration.sensor.minAnalogueGain = camHelper_->gain(minGain); context_.configuration.sensor.maxAnalogueGain = camHelper_->gain(maxGain); - for (auto const &algo : algorithms()) { + context_.configuration.raw = std::any_of(streamConfig.begin(), streamConfig.end(), + [](auto &cfg) -> bool { + PixelFormat pixelFormat{ cfg.second.pixelFormat }; + const PixelFormatInfo &format = PixelFormatInfo::info(pixelFormat); + return format.colourEncoding == PixelFormatInfo::ColourEncodingRAW; + }); + + for (auto const &a : algorithms()) { + Algorithm *algo = static_cast(a.get()); + + /* Disable algorithms that don't support raw formats. */ + algo->disabled_ = context_.configuration.raw && !algo->supportsRaw_; + if (algo->disabled_) + continue; + int ret = algo->configure(context_, info); if (ret) return ret; @@ -298,8 +313,12 @@ void IPARkISP1::queueRequest(const uint32_t frame, const ControlList &controls) { IPAFrameContext &frameContext = context_.frameContexts.alloc(frame); - for (auto const &algo : algorithms()) + for (auto const &a : algorithms()) { + Algorithm *algo = static_cast(a.get()); + if (algo->disabled_) + continue; algo->queueRequest(context_, frame, frameContext, controls); + } } void IPARkISP1::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) @@ -324,8 +343,13 @@ void IPARkISP1::processStatsBuffer(const uint32_t frame, const uint32_t bufferId { IPAFrameContext &frameContext = context_.frameContexts.get(frame); - const rkisp1_stat_buffer *stats = - reinterpret_cast( + /* + * In raw capture mode, the ISP is bypassed and no statistics buffer is + * provided. + */ + const rkisp1_stat_buffer *stats = nullptr; + if (!context_.configuration.raw) + stats = reinterpret_cast( mappedBuffers_.at(bufferId).planes()[0].data()); frameContext.sensor.exposure = @@ -335,8 +359,12 @@ void IPARkISP1::processStatsBuffer(const uint32_t frame, const uint32_t bufferId ControlList metadata(controls::controls); - for (auto const &algo : algorithms()) + for (auto const &a : algorithms()) { + Algorithm *algo = static_cast(a.get()); + if (algo->disabled_) + continue; algo->process(context_, frame, frameContext, stats, metadata); + } setControls(frame);