{"id":17616,"url":"https://patchwork.libcamera.org/api/1.1/patches/17616/?format=json","web_url":"https://patchwork.libcamera.org/patch/17616/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20221018054913.2325021-1-paul.elder@ideasonboard.com>","date":"2022-10-18T05:49:13","name":"[libcamera-devel,v3] ipa: rkisp1: Add support for manual gain and exposure","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"8251f3d8010c31e726eb57b03bb5d3a36c2e1b7a","submitter":{"id":17,"url":"https://patchwork.libcamera.org/api/1.1/people/17/?format=json","name":"Paul Elder","email":"paul.elder@ideasonboard.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/17616/mbox/","series":[{"id":3560,"url":"https://patchwork.libcamera.org/api/1.1/series/3560/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=3560","date":"2022-10-18T05:49:13","name":"[libcamera-devel,v3] ipa: rkisp1: Add support for manual gain and exposure","version":3,"mbox":"https://patchwork.libcamera.org/series/3560/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/17616/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/17616/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id A3E98BD16B\n\tfor ;\n\tTue, 18 Oct 2022 05:49:28 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id DB7D962E0B;\n\tTue, 18 Oct 2022 07:49:27 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id C582761F55\n\tfor ;\n\tTue, 18 Oct 2022 07:49:25 +0200 (CEST)","from pyrite.rasen.tech (h175-177-042-159.catv02.itscom.jp\n\t[175.177.42.159])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 1B0E08CC;\n\tTue, 18 Oct 2022 07:49:23 +0200 (CEST)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1666072167;\n\tbh=s7SZpEZW6cZcmZlx/8VPfPvVh7YdGoyvSnqoj+t+mZo=;\n\th=To:Date:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post:\n\tList-Help:List-Subscribe:From:Reply-To:From;\n\tb=zftfA3A/hyhA4UK/x9sRumR0CUktAg9qgCaHet7G+H9XPPOd6zDtoAbY7A8l2gKn1\n\tkRafVmvu9xitc2Ty3rs3jhBdhkafA9+pWfiS7tbktYLqNmF1Zj13n1/m2Jn1G8V4n1\n\tq4Ogv072PqDQPXxnMApS4QHRAQrTy8YUPfPM7qAL4m2zrb6tfUESRY7PrMY+BfftLD\n\tX/IfC0tkaJsSRGDpTrzC70Wi4Z9Cs+mDzxf2Vqizahey7Z4S3MTndaBJSK4omvnnxj\n\t66feQL8AzaNTOEa3G2luyzqYGtL1TtKGHs2SMEkmludgyWAZWAReWNPQc4mCt+OAmE\n\tQJfNYxGKfHZjg==","v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1666072165;\n\tbh=s7SZpEZW6cZcmZlx/8VPfPvVh7YdGoyvSnqoj+t+mZo=;\n\th=From:To:Cc:Subject:Date:From;\n\tb=CngS3FtBpe4P91JW+s8GuJ2ihVAb3IuZCAeRVpyMR874VKDY1h+pVNbYpq4RiCYsR\n\t8hFZkjVeduPZryZw8kcKBDfZyaPsEmxI0OBEtL+bWAIaX5AR2N3sklwGzgBrm423qa\n\tEw/jcgSiZ/kJtmjnVfRsIKT2JT1ditQc/FzGW5yQ="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ideasonboard.com\n\theader.i=@ideasonboard.com\n\theader.b=\"CngS3FtB\"; dkim-atps=neutral","To":"libcamera-devel@lists.libcamera.org","Date":"Tue, 18 Oct 2022 14:49:13 +0900","Message-Id":"<20221018054913.2325021-1-paul.elder@ideasonboard.com>","X-Mailer":"git-send-email 2.30.2","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Subject":"[libcamera-devel] [PATCH v3] ipa: rkisp1: Add support for manual\n\tgain and exposure","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Paul Elder via libcamera-devel ","Reply-To":"Paul Elder ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Add support for manual gain and exposure in the rkisp1 IPA.\n\nSigned-off-by: Paul Elder \n\n---\nChanges in v3:\n- Report the exposure time and analogue gain in metadata\n\nChanges in v2:\n- set the min and max values of ExposureTime and AnalogueGain properly\n - to that end, plumb the sensorControls through the pipeline handler's\n loadIPA() and the IPA's init()\n---\n include/libcamera/ipa/rkisp1.mojom | 2 +-\n src/ipa/rkisp1/algorithms/agc.cpp | 59 +++++++++++++++++++++---\n src/ipa/rkisp1/algorithms/agc.h | 4 ++\n src/ipa/rkisp1/ipa_context.h | 13 +++++-\n src/ipa/rkisp1/rkisp1.cpp | 21 +++++++++\n src/libcamera/pipeline/rkisp1/rkisp1.cpp | 9 ++--\n 6 files changed, 95 insertions(+), 13 deletions(-)","diff":"diff --git a/include/libcamera/ipa/rkisp1.mojom b/include/libcamera/ipa/rkisp1.mojom\nindex eaf3955e..b526450d 100644\n--- a/include/libcamera/ipa/rkisp1.mojom\n+++ b/include/libcamera/ipa/rkisp1.mojom\n@@ -10,7 +10,7 @@ import \"include/libcamera/ipa/core.mojom\";\n \n interface IPARkISP1Interface {\n \tinit(libcamera.IPASettings settings,\n-\t uint32 hwRevision)\n+\t uint32 hwRevision, libcamera.ControlInfoMap sensorControls)\n \t\t=> (int32 ret, libcamera.ControlInfoMap ipaControls);\n \tstart() => (int32 ret);\n \tstop();\ndiff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp\nindex 04062a36..09ec6ac4 100644\n--- a/src/ipa/rkisp1/algorithms/agc.cpp\n+++ b/src/ipa/rkisp1/algorithms/agc.cpp\n@@ -14,6 +14,7 @@\n #include \n #include \n \n+#include \n #include \n \n #include \"libipa/histogram.h\"\n@@ -73,8 +74,11 @@ Agc::Agc()\n int Agc::configure(IPAContext &context, const IPACameraSensorInfo &configInfo)\n {\n \t/* Configure the default exposure and gain. */\n-\tcontext.activeState.agc.gain = std::max(context.configuration.agc.minAnalogueGain, kMinAnalogueGain);\n-\tcontext.activeState.agc.exposure = 10ms / context.configuration.sensor.lineDuration;\n+\tcontext.activeState.agc.automatic.gain = std::max(context.configuration.agc.minAnalogueGain, kMinAnalogueGain);\n+\tcontext.activeState.agc.automatic.exposure = 10ms / context.configuration.sensor.lineDuration;\n+\tcontext.activeState.agc.manual.gain = context.activeState.agc.automatic.gain;\n+\tcontext.activeState.agc.manual.exposure = context.activeState.agc.automatic.exposure;\n+\tcontext.activeState.agc.autoEnabled = true;\n \n \t/*\n \t * According to the RkISP1 documentation:\n@@ -221,8 +225,8 @@ void Agc::computeExposure(IPAContext &context, IPAFrameContext &frameContext,\n \t\t\t << stepGain;\n \n \t/* Update the estimated exposure and gain. */\n-\tactiveState.agc.exposure = shutterTime / configuration.sensor.lineDuration;\n-\tactiveState.agc.gain = stepGain;\n+\tactiveState.agc.automatic.exposure = shutterTime / configuration.sensor.lineDuration;\n+\tactiveState.agc.automatic.gain = stepGain;\n }\n \n /**\n@@ -339,8 +343,10 @@ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame,\n void Agc::prepare(IPAContext &context, const uint32_t frame,\n \t\t IPAFrameContext &frameContext, rkisp1_params_cfg *params)\n {\n-\tframeContext.agc.exposure = context.activeState.agc.exposure;\n-\tframeContext.agc.gain = context.activeState.agc.gain;\n+\tif (frameContext.agc.autoEnabled) {\n+\t\tframeContext.agc.exposure = context.activeState.agc.automatic.exposure;\n+\t\tframeContext.agc.gain = context.activeState.agc.automatic.gain;\n+\t}\n \n \tif (frame > 0)\n \t\treturn;\n@@ -373,6 +379,47 @@ void Agc::prepare(IPAContext &context, const uint32_t frame,\n \tparams->module_en_update |= RKISP1_CIF_ISP_MODULE_HST;\n }\n \n+/**\n+ * \\copydoc libcamera::ipa::Algorithm::queueRequest\n+ */\n+void Agc::queueRequest(IPAContext &context,\n+\t\t [[maybe_unused]] const uint32_t frame,\n+\t\t IPAFrameContext &frameContext,\n+\t\t const ControlList &controls)\n+{\n+\tauto &agc = context.activeState.agc;\n+\n+\tconst auto &agcEnable = controls.get(controls::AeEnable);\n+\tif (agcEnable && *agcEnable != agc.autoEnabled) {\n+\t\tagc.autoEnabled = *agcEnable;\n+\n+\t\tLOG(RkISP1Agc, Debug)\n+\t\t\t<< (*agcEnable ? \"Enabling\" : \"Disabling\") << \" AGC\";\n+\t}\n+\n+\tconst auto &exposure = controls.get(controls::ExposureTime);\n+\tif (exposure && !agc.autoEnabled) {\n+\t\tagc.manual.exposure = *exposure;\n+\n+\t\tLOG(RkISP1Agc, Debug)\n+\t\t\t<< \"Set exposure to: \" << agc.manual.exposure;\n+\t}\n+\n+\tconst auto &gain = controls.get(controls::AnalogueGain);\n+\tif (gain && !agc.autoEnabled) {\n+\t\tagc.manual.gain = *gain;\n+\n+\t\tLOG(RkISP1Agc, Debug) << \"Set gain to: \" << agc.manual.gain;\n+\t}\n+\n+\tframeContext.agc.autoEnabled = agc.autoEnabled;\n+\n+\tif (!frameContext.agc.autoEnabled) {\n+\t\tframeContext.agc.exposure = agc.manual.exposure;\n+\t\tframeContext.agc.gain = agc.manual.gain;\n+\t}\n+}\n+\n REGISTER_IPA_ALGORITHM(Agc, \"Agc\")\n \n } /* namespace ipa::rkisp1::algorithms */\ndiff --git a/src/ipa/rkisp1/algorithms/agc.h b/src/ipa/rkisp1/algorithms/agc.h\nindex 9ad5c32f..ebceeef4 100644\n--- a/src/ipa/rkisp1/algorithms/agc.h\n+++ b/src/ipa/rkisp1/algorithms/agc.h\n@@ -32,6 +32,10 @@ public:\n \tvoid process(IPAContext &context, const uint32_t frame,\n \t\t IPAFrameContext &frameContext,\n \t\t const rkisp1_stat_buffer *stats) override;\n+\tvoid queueRequest(IPAContext &context,\n+\t\t\t const uint32_t frame,\n+\t\t\t IPAFrameContext &frameContext,\n+\t\t\t const ControlList &controls) override;\n \n private:\n \tvoid computeExposure(IPAContext &Context, IPAFrameContext &frameContext,\ndiff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h\nindex c85d8abe..035d67e2 100644\n--- a/src/ipa/rkisp1/ipa_context.h\n+++ b/src/ipa/rkisp1/ipa_context.h\n@@ -50,8 +50,16 @@ struct IPASessionConfiguration {\n \n struct IPAActiveState {\n \tstruct {\n-\t\tuint32_t exposure;\n-\t\tdouble gain;\n+\t\tstruct {\n+\t\t\tuint32_t exposure;\n+\t\t\tdouble gain;\n+\t\t} manual;\n+\t\tstruct {\n+\t\t\tuint32_t exposure;\n+\t\t\tdouble gain;\n+\t\t} automatic;\n+\n+\t\tbool autoEnabled;\n \t} agc;\n \n \tstruct {\n@@ -92,6 +100,7 @@ struct IPAFrameContext : public FrameContext {\n \tstruct {\n \t\tuint32_t exposure;\n \t\tdouble gain;\n+\t\tbool autoEnabled;\n \t} agc;\n \n \tstruct {\ndiff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp\nindex 3f5c1a58..33692e5d 100644\n--- a/src/ipa/rkisp1/rkisp1.cpp\n+++ b/src/ipa/rkisp1/rkisp1.cpp\n@@ -49,6 +49,7 @@ public:\n \tIPARkISP1();\n \n \tint init(const IPASettings &settings, unsigned int hwRevision,\n+\t\t const ControlInfoMap &sensorControls,\n \t\t ControlInfoMap *ipaControls) override;\n \tint start() override;\n \tvoid stop() override;\n@@ -116,6 +117,7 @@ std::string IPARkISP1::logPrefix() const\n }\n \n int IPARkISP1::init(const IPASettings &settings, unsigned int hwRevision,\n+\t\t const ControlInfoMap &sensorControls,\n \t\t ControlInfoMap *ipaControls)\n {\n \t/* \\todo Add support for other revisions */\n@@ -183,6 +185,21 @@ int IPARkISP1::init(const IPASettings &settings, unsigned int hwRevision,\n \n \t/* Return the controls handled by the IPA. */\n \tControlInfoMap::Map ctrlMap = rkisp1Controls;\n+\n+\tauto exposureTimeInfo = sensorControls.find(V4L2_CID_EXPOSURE);\n+\tif (exposureTimeInfo != sensorControls.end()) {\n+\t\tctrlMap.emplace(&controls::ExposureTime,\n+\t\t\t\tControlInfo(exposureTimeInfo->second.min(),\n+\t\t\t\t\t exposureTimeInfo->second.max()));\n+\t}\n+\n+\tauto analogueGainInfo = sensorControls.find(V4L2_CID_ANALOGUE_GAIN);\n+\tif (analogueGainInfo != sensorControls.end()) {\n+\t\tctrlMap.emplace(&controls::AnalogueGain,\n+\t\t\t\tControlInfo(static_cast(analogueGainInfo->second.min().get()),\n+\t\t\t\t\t static_cast(analogueGainInfo->second.max().get())));\n+\t}\n+\n \t*ipaControls = ControlInfoMap(std::move(ctrlMap), controls::controls);\n \n \treturn 0;\n@@ -368,11 +385,15 @@ void IPARkISP1::setControls(unsigned int frame)\n \n void IPARkISP1::prepareMetadata(unsigned int frame, unsigned int aeState)\n {\n+\tIPAFrameContext &frameContext = context_.frameContexts.get(frame);\n \tControlList ctrls(controls::controls);\n \n \tif (aeState)\n \t\tctrls.set(controls::AeLocked, aeState == 2);\n \n+\tctrls.set(controls::ExposureTime, frameContext.agc.exposure);\n+\tctrls.set(controls::AnalogueGain, frameContext.agc.gain);\n+\n \tmetadataReady.emit(frame, ctrls);\n }\n \ndiff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp\nindex 455ee2a0..4f8e467a 100644\n--- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp\n+++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp\n@@ -91,7 +91,7 @@ public:\n \t}\n \n \tPipelineHandlerRkISP1 *pipe();\n-\tint loadIPA(unsigned int hwRevision);\n+\tint loadIPA(unsigned int hwRevision, const ControlInfoMap &sensorControls);\n \n \tStream mainPathStream_;\n \tStream selfPathStream_;\n@@ -319,7 +319,8 @@ PipelineHandlerRkISP1 *RkISP1CameraData::pipe()\n \treturn static_cast(Camera::Private::pipe());\n }\n \n-int RkISP1CameraData::loadIPA(unsigned int hwRevision)\n+int RkISP1CameraData::loadIPA(unsigned int hwRevision,\n+\t\t\t const ControlInfoMap &sensorControls)\n {\n \tipa_ = IPAManager::createIPA(pipe(), 1, 1);\n \tif (!ipa_)\n@@ -348,7 +349,7 @@ int RkISP1CameraData::loadIPA(unsigned int hwRevision)\n \t}\n \n \tint ret = ipa_->init({ ipaTuningFile, sensor_->model() }, hwRevision,\n-\t\t\t &controlInfo_);\n+\t\t\t sensorControls, &controlInfo_);\n \tif (ret < 0) {\n \t\tLOG(RkISP1, Error) << \"IPA initialization failure\";\n \t\treturn ret;\n@@ -1023,7 +1024,7 @@ int PipelineHandlerRkISP1::createCamera(MediaEntity *sensor)\n \tisp_->frameStart.connect(data->delayedCtrls_.get(),\n \t\t\t\t &DelayedControls::applyControls);\n \n-\tret = data->loadIPA(media_->hwRevision());\n+\tret = data->loadIPA(media_->hwRevision(), data->sensor_->controls());\n \tif (ret)\n \t\treturn ret;\n \n","prefixes":["libcamera-devel","v3"]}