{"id":22089,"url":"https://patchwork.libcamera.org/api/patches/22089/?format=json","web_url":"https://patchwork.libcamera.org/patch/22089/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20241125180310.1254519-9-mzamazal@redhat.com>","date":"2024-11-25T18:03:09","name":"[v2,8/9] libcamera: software_isp: Track whether CCM is enabled","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"a7faa9044f183c869fa84b74b3515122d70cdcc3","submitter":{"id":177,"url":"https://patchwork.libcamera.org/api/people/177/?format=json","name":"Milan Zamazal","email":"mzamazal@redhat.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/22089/mbox/","series":[{"id":4820,"url":"https://patchwork.libcamera.org/api/series/4820/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=4820","date":"2024-11-25T18:03:01","name":"Software ISP support for CCM","version":2,"mbox":"https://patchwork.libcamera.org/series/4820/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/22089/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/22089/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id D0E9EBD160\n\tfor ;\n\tMon, 25 Nov 2024 18:03:46 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 8982465F74;\n\tMon, 25 Nov 2024 19:03:46 +0100 (CET)","from us-smtp-delivery-124.mimecast.com\n\t(us-smtp-delivery-124.mimecast.com [170.10.129.124])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id E7B8E6604A\n\tfor ;\n\tMon, 25 Nov 2024 19:03:43 +0100 (CET)","from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com\n\t(ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63])\n\tby relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3,\n\tcipher=TLS_AES_256_GCM_SHA384) id us-mta-53-eZ65WHAWNrWsIxFFvxsFbw-1;\n\tMon, 25 Nov 2024 13:03:39 -0500","from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com\n\t(mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com\n\t[10.30.177.15])\n\t(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n\tkey-exchange X25519 server-signature RSA-PSS (2048 bits)\n\tserver-digest SHA256) (No client certificate requested)\n\tby mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix)\n\twith ESMTPS id 97F1A1956077; Mon, 25 Nov 2024 18:03:38 +0000 (UTC)","from nuthatch.redhat.com (unknown [10.45.224.109])\n\tby mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix)\n\twith ESMTP id 150531956086; Mon, 25 Nov 2024 18:03:36 +0000 (UTC)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=redhat.com header.i=@redhat.com\n\theader.b=\"amUSFl41\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;\n\ts=mimecast20190719; t=1732557823;\n\th=from:from:reply-to:subject:subject:date:date:message-id:message-id:\n\tto:to:cc:cc:mime-version:mime-version:content-type:content-type:\n\tcontent-transfer-encoding:content-transfer-encoding:\n\tin-reply-to:in-reply-to:references:references;\n\tbh=ZZX166gH/KMC6BrPwU8YOasZW6ngozEwlwhOYBHnPPw=;\n\tb=amUSFl41D7G1DYQLCg9naQti2NIgQnMyiifJH7jmdJW4w70tssjyqFyeubglLB7QMkDIQ5\n\td8WNKOIwSK6ghR5UCVq9kbtZrA+ytsVxUnoFoS6TNhqdBj1JyauuOq59V6ylwBu5F18lEL\n\tJFxSbCSpwR5Oi+uQKrt4o9TS1M/7KV8=","X-MC-Unique":"eZ65WHAWNrWsIxFFvxsFbw-1","X-Mimecast-MFC-AGG-ID":"eZ65WHAWNrWsIxFFvxsFbw","From":"Milan Zamazal ","To":"libcamera-devel@lists.libcamera.org","Cc":"Milan Zamazal ,\n\tRobert Mader ","Subject":"[PATCH v2 8/9] libcamera: software_isp: Track whether CCM is enabled","Date":"Mon, 25 Nov 2024 19:03:09 +0100","Message-ID":"<20241125180310.1254519-9-mzamazal@redhat.com>","In-Reply-To":"<20241125180310.1254519-1-mzamazal@redhat.com>","References":"<20241125180310.1254519-1-mzamazal@redhat.com>","MIME-Version":"1.0","X-Scanned-By":"MIMEDefang 3.0 on 10.30.177.15","X-Mimecast-Spam-Score":"0","X-Mimecast-MFC-PROC-ID":"U3-y4vN9IGhBUAh4DO82l2ecV2kR42zLZZB8GFkD-Mc_1732557818","X-Mimecast-Originator":"redhat.com","Content-Transfer-Encoding":"8bit","content-type":"text/plain; charset=\"US-ASCII\"; x-default=true","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Applying color correction matrix (CCM) in software ISP is optional due\nto performance reasons. CCM is applied if and only if `Ccm' algorithm\nis present in the tuning file and it defines some CCM.\n\nSoftware ISP debayering is a performance critical piece of code and we\ndo not want to use dynamic conditionals there. Therefore we pass\ninformation about CCM application to debayering configuration and let it\nselect the right versions of debayering functions using templates. This\nis a similar trick as the previously used one for adding or not adding\nan alpha channel to the output.\n\nDebayering gets this information but it ignores it in this patch.\nActual processing with CCM is added in the followup patch.\n\nSigned-off-by: Milan Zamazal \n---\n include/libcamera/ipa/soft.mojom | 2 +-\n src/ipa/simple/algorithms/ccm.cpp | 18 +++++------\n src/ipa/simple/algorithms/ccm.h | 1 +\n src/ipa/simple/soft_simple.cpp | 8 +++--\n src/libcamera/software_isp/debayer.h | 3 +-\n src/libcamera/software_isp/debayer_cpu.cpp | 35 ++++++++++++---------\n src/libcamera/software_isp/debayer_cpu.h | 24 +++++++-------\n src/libcamera/software_isp/software_isp.cpp | 5 +--\n 8 files changed, 54 insertions(+), 42 deletions(-)","diff":"diff --git a/include/libcamera/ipa/soft.mojom b/include/libcamera/ipa/soft.mojom\nindex a6c086f82..2ab67296d 100644\n--- a/include/libcamera/ipa/soft.mojom\n+++ b/include/libcamera/ipa/soft.mojom\n@@ -21,7 +21,7 @@ interface IPASoftInterface {\n \tstart() => (int32 ret);\n \tstop();\n \tconfigure(IPAConfigInfo configInfo)\n-\t\t=> (int32 ret);\n+\t\t=> (int32 ret, bool ccmEnabled);\n \n \t[async] queueRequest(uint32 frame, libcamera.ControlList sensorControls);\n \t[async] computeParams(uint32 frame);\ndiff --git a/src/ipa/simple/algorithms/ccm.cpp b/src/ipa/simple/algorithms/ccm.cpp\nindex bd2c03fe4..5d56af588 100644\n--- a/src/ipa/simple/algorithms/ccm.cpp\n+++ b/src/ipa/simple/algorithms/ccm.cpp\n@@ -25,26 +25,24 @@ unsigned int Ccm::kTemperatureThreshold = 100;\n int Ccm::init([[maybe_unused]] IPAContext &context, const YamlObject &tuningData)\n {\n \tint ret = ccm_.readYaml(tuningData[\"ccms\"], \"ct\", \"ccm\");\n-\tif (ret < 0) {\n+\tif (ret < 0)\n \t\tLOG(IPASoftCcm, Warning)\n \t\t\t<< \"Failed to parse 'ccm' \"\n \t\t\t<< \"parameter from tuning file; falling back to unit matrix\";\n-\t\tccmEnabled_ = false;\n-\t} else {\n-\t\tccmEnabled_ = true;\n-\t}\n \n \treturn 0;\n }\n \n+int Ccm::configure(IPAContext &context,\n+\t\t [[maybe_unused]] const IPAConfigInfo &configInfo)\n+{\n+\tcontext.activeState.ccm.enabled = true;\n+\treturn 0;\n+}\n+\n void Ccm::prepare(IPAContext &context, const uint32_t frame,\n \t\t IPAFrameContext &frameContext, [[maybe_unused]] DebayerParams *params)\n {\n-\tcontext.activeState.ccm.enabled = ccmEnabled_;\n-\n-\tif (!ccmEnabled_)\n-\t\treturn;\n-\n \tunsigned int ct = context.activeState.awb.temperatureK;\n \n \t/* Change CCM only on bigger temperature changes. */\ndiff --git a/src/ipa/simple/algorithms/ccm.h b/src/ipa/simple/algorithms/ccm.h\nindex 6b7efbe21..a7f6b6047 100644\n--- a/src/ipa/simple/algorithms/ccm.h\n+++ b/src/ipa/simple/algorithms/ccm.h\n@@ -23,6 +23,7 @@ public:\n \t~Ccm() = default;\n \n \tint init(IPAContext &context, const YamlObject &tuningData) override;\n+\tint configure(IPAContext &context, const IPAConfigInfo &configInfo) override;\n \tvoid prepare(IPAContext &context,\n \t\t const uint32_t frame,\n \t\t IPAFrameContext &frameContext,\ndiff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp\nindex ac2a94210..0b51e8b5d 100644\n--- a/src/ipa/simple/soft_simple.cpp\n+++ b/src/ipa/simple/soft_simple.cpp\n@@ -51,7 +51,7 @@ public:\n \t\t const SharedFD &fdStats,\n \t\t const SharedFD &fdParams,\n \t\t const ControlInfoMap &sensorInfoMap) override;\n-\tint configure(const IPAConfigInfo &configInfo) override;\n+\tint configure(const IPAConfigInfo &configInfo, bool *ccmEnabled) override;\n \n \tint start() override;\n \tvoid stop() override;\n@@ -177,7 +177,7 @@ int IPASoftSimple::init(const IPASettings &settings,\n \treturn 0;\n }\n \n-int IPASoftSimple::configure(const IPAConfigInfo &configInfo)\n+int IPASoftSimple::configure(const IPAConfigInfo &configInfo, bool *ccmEnabled)\n {\n \tsensorInfoMap_ = configInfo.sensorControls;\n \n@@ -238,12 +238,16 @@ int IPASoftSimple::configure(const IPAConfigInfo &configInfo)\n \t\tcontext_.configuration.agc.againMinStep = 1.0;\n \t}\n \n+\tcontext_.activeState.ccm.enabled = false;\n+\n \tfor (auto const &algo : algorithms()) {\n \t\tint ret = algo->configure(context_, configInfo);\n \t\tif (ret)\n \t\t\treturn ret;\n \t}\n \n+\t*ccmEnabled = context_.activeState.ccm.enabled;\n+\n \tLOG(IPASoft, Info)\n \t\t<< \"Exposure \" << context_.configuration.agc.exposureMin << \"-\"\n \t\t<< context_.configuration.agc.exposureMax\ndiff --git a/src/libcamera/software_isp/debayer.h b/src/libcamera/software_isp/debayer.h\nindex d7ca060da..ba033d440 100644\n--- a/src/libcamera/software_isp/debayer.h\n+++ b/src/libcamera/software_isp/debayer.h\n@@ -33,7 +33,8 @@ public:\n \tvirtual ~Debayer() = 0;\n \n \tvirtual int configure(const StreamConfiguration &inputCfg,\n-\t\t\t const std::vector> &outputCfgs) = 0;\n+\t\t\t const std::vector> &outputCfgs,\n+\t\t\t bool ccmEnabled) = 0;\n \n \tvirtual std::vector formats(PixelFormat inputFormat) = 0;\n \ndiff --git a/src/libcamera/software_isp/debayer_cpu.cpp b/src/libcamera/software_isp/debayer_cpu.cpp\nindex 7d080f00f..92f81baf2 100644\n--- a/src/libcamera/software_isp/debayer_cpu.cpp\n+++ b/src/libcamera/software_isp/debayer_cpu.cpp\n@@ -113,7 +113,7 @@ DebayerCpu::~DebayerCpu() = default;\n \t\t(prev[x] + curr[x - p] + curr[x + n] + next[x]) / (4 * (div)), \\\n \t\tcurr[x] / (div))\n \n-template\n+template\n void DebayerCpu::debayer8_BGBG_BGR888(uint8_t *dst, const uint8_t *src[])\n {\n \tDECLARE_SRC_POINTERS(uint8_t)\n@@ -124,7 +124,7 @@ void DebayerCpu::debayer8_BGBG_BGR888(uint8_t *dst, const uint8_t *src[])\n \t}\n }\n \n-template\n+template\n void DebayerCpu::debayer8_GRGR_BGR888(uint8_t *dst, const uint8_t *src[])\n {\n \tDECLARE_SRC_POINTERS(uint8_t)\n@@ -135,7 +135,7 @@ void DebayerCpu::debayer8_GRGR_BGR888(uint8_t *dst, const uint8_t *src[])\n \t}\n }\n \n-template\n+template\n void DebayerCpu::debayer10_BGBG_BGR888(uint8_t *dst, const uint8_t *src[])\n {\n \tDECLARE_SRC_POINTERS(uint16_t)\n@@ -147,7 +147,7 @@ void DebayerCpu::debayer10_BGBG_BGR888(uint8_t *dst, const uint8_t *src[])\n \t}\n }\n \n-template\n+template\n void DebayerCpu::debayer10_GRGR_BGR888(uint8_t *dst, const uint8_t *src[])\n {\n \tDECLARE_SRC_POINTERS(uint16_t)\n@@ -159,7 +159,7 @@ void DebayerCpu::debayer10_GRGR_BGR888(uint8_t *dst, const uint8_t *src[])\n \t}\n }\n \n-template\n+template\n void DebayerCpu::debayer12_BGBG_BGR888(uint8_t *dst, const uint8_t *src[])\n {\n \tDECLARE_SRC_POINTERS(uint16_t)\n@@ -171,7 +171,7 @@ void DebayerCpu::debayer12_BGBG_BGR888(uint8_t *dst, const uint8_t *src[])\n \t}\n }\n \n-template\n+template\n void DebayerCpu::debayer12_GRGR_BGR888(uint8_t *dst, const uint8_t *src[])\n {\n \tDECLARE_SRC_POINTERS(uint16_t)\n@@ -183,7 +183,7 @@ void DebayerCpu::debayer12_GRGR_BGR888(uint8_t *dst, const uint8_t *src[])\n \t}\n }\n \n-template\n+template\n void DebayerCpu::debayer10P_BGBG_BGR888(uint8_t *dst, const uint8_t *src[])\n {\n \tconst int widthInBytes = window_.width * 5 / 4;\n@@ -209,7 +209,7 @@ void DebayerCpu::debayer10P_BGBG_BGR888(uint8_t *dst, const uint8_t *src[])\n \t}\n }\n \n-template\n+template\n void DebayerCpu::debayer10P_GRGR_BGR888(uint8_t *dst, const uint8_t *src[])\n {\n \tconst int widthInBytes = window_.width * 5 / 4;\n@@ -230,7 +230,7 @@ void DebayerCpu::debayer10P_GRGR_BGR888(uint8_t *dst, const uint8_t *src[])\n \t}\n }\n \n-template\n+template\n void DebayerCpu::debayer10P_GBGB_BGR888(uint8_t *dst, const uint8_t *src[])\n {\n \tconst int widthInBytes = window_.width * 5 / 4;\n@@ -251,7 +251,7 @@ void DebayerCpu::debayer10P_GBGB_BGR888(uint8_t *dst, const uint8_t *src[])\n \t}\n }\n \n-template\n+template\n void DebayerCpu::debayer10P_RGRG_BGR888(uint8_t *dst, const uint8_t *src[])\n {\n \tconst int widthInBytes = window_.width * 5 / 4;\n@@ -367,10 +367,11 @@ int DebayerCpu::setupStandardBayerOrder(BayerFormat::Order order)\n \treturn 0;\n }\n \n-#define SET_DEBAYER_METHODS(method0, method1) \\\n-\tdebayer0_ = addAlphaByte ? &DebayerCpu::method0 : &DebayerCpu::method0; \\\n-\tdebayer1_ = addAlphaByte ? &DebayerCpu::method1 : &DebayerCpu::method1;\n+#define SET_DEBAYER_METHODS(method0, method1) \\\n+\tdebayer0_ = addAlphaByte ? &DebayerCpu::method0 : &DebayerCpu::method0; \\\n+\tdebayer1_ = addAlphaByte ? &DebayerCpu::method1 : &DebayerCpu::method1;\n \n+template\n int DebayerCpu::setDebayerFunctions(PixelFormat inputFormat, PixelFormat outputFormat)\n {\n \tBayerFormat bayerFormat =\n@@ -463,7 +464,8 @@ int DebayerCpu::setDebayerFunctions(PixelFormat inputFormat, PixelFormat outputF\n }\n \n int DebayerCpu::configure(const StreamConfiguration &inputCfg,\n-\t\t\t const std::vector> &outputCfgs)\n+\t\t\t const std::vector> &outputCfgs,\n+\t\t\t bool ccmEnabled)\n {\n \tif (getInputConfig(inputCfg.pixelFormat, inputConfig_) != 0)\n \t\treturn -EINVAL;\n@@ -502,7 +504,10 @@ int DebayerCpu::configure(const StreamConfiguration &inputCfg,\n \t\treturn -EINVAL;\n \t}\n \n-\tif (setDebayerFunctions(inputCfg.pixelFormat, outputCfg.pixelFormat) != 0)\n+\tint ret = ccmEnabled\n+\t\t\t ? setDebayerFunctions(inputCfg.pixelFormat, outputCfg.pixelFormat)\n+\t\t\t : setDebayerFunctions(inputCfg.pixelFormat, outputCfg.pixelFormat);\n+\tif (ret != 0)\n \t\treturn -EINVAL;\n \n \twindow_.x = ((inputCfg.size.width - outputCfg.size.width) / 2) &\ndiff --git a/src/libcamera/software_isp/debayer_cpu.h b/src/libcamera/software_isp/debayer_cpu.h\nindex 2c47e7c6e..b2ec8f1bd 100644\n--- a/src/libcamera/software_isp/debayer_cpu.h\n+++ b/src/libcamera/software_isp/debayer_cpu.h\n@@ -31,7 +31,8 @@ public:\n \t~DebayerCpu();\n \n \tint configure(const StreamConfiguration &inputCfg,\n-\t\t const std::vector> &outputCfgs);\n+\t\t const std::vector> &outputCfgs,\n+\t\t bool ccmEnabled);\n \tSize patternSize(PixelFormat inputFormat);\n \tstd::vector formats(PixelFormat input);\n \tstd::tuple\n@@ -85,28 +86,28 @@ private:\n \tusing debayerFn = void (DebayerCpu::*)(uint8_t *dst, const uint8_t *src[]);\n \n \t/* 8-bit raw bayer format */\n-\ttemplate\n+\ttemplate\n \tvoid debayer8_BGBG_BGR888(uint8_t *dst, const uint8_t *src[]);\n-\ttemplate\n+\ttemplate\n \tvoid debayer8_GRGR_BGR888(uint8_t *dst, const uint8_t *src[]);\n \t/* unpacked 10-bit raw bayer format */\n-\ttemplate\n+\ttemplate\n \tvoid debayer10_BGBG_BGR888(uint8_t *dst, const uint8_t *src[]);\n-\ttemplate\n+\ttemplate\n \tvoid debayer10_GRGR_BGR888(uint8_t *dst, const uint8_t *src[]);\n \t/* unpacked 12-bit raw bayer format */\n-\ttemplate\n+\ttemplate\n \tvoid debayer12_BGBG_BGR888(uint8_t *dst, const uint8_t *src[]);\n-\ttemplate\n+\ttemplate\n \tvoid debayer12_GRGR_BGR888(uint8_t *dst, const uint8_t *src[]);\n \t/* CSI-2 packed 10-bit raw bayer format (all the 4 orders) */\n-\ttemplate\n+\ttemplate\n \tvoid debayer10P_BGBG_BGR888(uint8_t *dst, const uint8_t *src[]);\n-\ttemplate\n+\ttemplate\n \tvoid debayer10P_GRGR_BGR888(uint8_t *dst, const uint8_t *src[]);\n-\ttemplate\n+\ttemplate\n \tvoid debayer10P_GBGB_BGR888(uint8_t *dst, const uint8_t *src[]);\n-\ttemplate\n+\ttemplate\n \tvoid debayer10P_RGRG_BGR888(uint8_t *dst, const uint8_t *src[]);\n \n \tstruct DebayerInputConfig {\n@@ -125,6 +126,7 @@ private:\n \tint getInputConfig(PixelFormat inputFormat, DebayerInputConfig &config);\n \tint getOutputConfig(PixelFormat outputFormat, DebayerOutputConfig &config);\n \tint setupStandardBayerOrder(BayerFormat::Order order);\n+\ttemplate\n \tint setDebayerFunctions(PixelFormat inputFormat, PixelFormat outputFormat);\n \tvoid setupInputMemcpy(const uint8_t *linePointers[]);\n \tvoid shiftLinePointers(const uint8_t *linePointers[], const uint8_t *src);\ndiff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp\nindex 2ccbeacc0..0e1d3a457 100644\n--- a/src/libcamera/software_isp/software_isp.cpp\n+++ b/src/libcamera/software_isp/software_isp.cpp\n@@ -232,11 +232,12 @@ int SoftwareIsp::configure(const StreamConfiguration &inputCfg,\n {\n \tASSERT(ipa_ && debayer_);\n \n-\tint ret = ipa_->configure(configInfo);\n+\tbool ccmEnabled;\n+\tint ret = ipa_->configure(configInfo, &ccmEnabled);\n \tif (ret < 0)\n \t\treturn ret;\n \n-\treturn debayer_->configure(inputCfg, outputCfgs);\n+\treturn debayer_->configure(inputCfg, outputCfgs, ccmEnabled);\n }\n \n /**\n","prefixes":["v2","8/9"]}