From patchwork Mon Jun 17 23:25:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 20350 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9040AC3293 for ; Mon, 17 Jun 2024 23:25:34 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2ACC365494; Tue, 18 Jun 2024 01:25:33 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="O5zBrsru"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4F1BA65489 for ; Tue, 18 Jun 2024 01:25:29 +0200 (CEST) Received: from Monstersaurus.local (cpc89244-aztw30-2-0-cust6594.18-1.cable.virginm.net [86.31.185.195]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 00A47564; Tue, 18 Jun 2024 01:25:11 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1718666712; bh=eKdiqXSezEgKBRUN4R9yfMCeps/4UvELQ9ohHGw9QgM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=O5zBrsruAPYnEde9mEZQxc4qq+4La0GkgzjOTSrKyyldjcNTvoHS7mjvp+rRxo6+1 4FCa6WDU+Ja03OTD13dnrA2XHC0o3qeSr51NxHOKWbak452/pPyJZSa5JsZczC40In 1mTMkmFViADQvh/imPtBCxJCugvn2tGbEQv3nXho= From: Kieran Bingham To: libcamera devel Cc: Kieran Bingham Subject: [PATCH 1/3] ipa: simple: softisp: Extend to pass metadata Date: Tue, 18 Jun 2024 00:25:23 +0100 Message-Id: <20240617232525.878530-2-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240617232525.878530-1-kieran.bingham@ideasonboard.com> References: <20240617232525.878530-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Extend the Simple IPA IPC to support returning a metadata controllist when the process call has completed. For efficiency, use the existing signal setIspParams() directly to avoid having an extra async callback to return the metadata. Merge the metadata reported by the ISP into any completing request to provide to the application. This should be further extended or improved to make use of the frame context structures so that the metadata is tied to a specific frame and request completion. Signed-off-by: Kieran Bingham --- include/libcamera/internal/software_isp/software_isp.h | 5 ++++- include/libcamera/ipa/soft.mojom | 2 +- src/ipa/simple/soft_simple.cpp | 4 +++- src/libcamera/pipeline/simple/simple.cpp | 7 +++++-- src/libcamera/software_isp/software_isp.cpp | 6 ++++-- 5 files changed, 17 insertions(+), 7 deletions(-) diff --git a/include/libcamera/internal/software_isp/software_isp.h b/include/libcamera/internal/software_isp/software_isp.h index c5338c05b99b..2631f40ef22a 100644 --- a/include/libcamera/internal/software_isp/software_isp.h +++ b/include/libcamera/internal/software_isp/software_isp.h @@ -80,8 +80,10 @@ public: Signal<> ispStatsReady; Signal setSensorControls; + const ControlList &metadata() { return metadata_; } + private: - void saveIspParams(); + void saveIspParams(const ControlList &metadata); void setSensorCtrls(const ControlList &sensorControls); void statsReady(); void inputReady(FrameBuffer *input); @@ -92,6 +94,7 @@ private: SharedMemObject sharedParams_; DebayerParams debayerParams_; DmaBufAllocator dmaHeap_; + ControlList metadata_; std::unique_ptr ipa_; }; diff --git a/include/libcamera/ipa/soft.mojom b/include/libcamera/ipa/soft.mojom index 3aa2066ebaa0..1a0221df7660 100644 --- a/include/libcamera/ipa/soft.mojom +++ b/include/libcamera/ipa/soft.mojom @@ -24,5 +24,5 @@ interface IPASoftInterface { interface IPASoftEventInterface { setSensorControls(libcamera.ControlList sensorControls); - setIspParams(); + setIspParams(libcamera.ControlList metadata); }; diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index b7746ce09206..09c7b575301e 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -250,6 +250,8 @@ void IPASoftSimple::processStats(const ControlList &sensorControls) SwIspStats::Histogram histogram = stats_->yHistogram; if (ignoreUpdates_ > 0) blackLevel_.update(histogram); + + ControlList metadata(controls::controls); const uint8_t blackLevel = blackLevel_.get(); /* @@ -303,7 +305,7 @@ void IPASoftSimple::processStats(const ControlList &sensorControls) params_->blue[i] = gammaTable_[idx]; } - setIspParams.emit(); + setIspParams.emit(metadata); /* \todo Switch to the libipa/algorithm.h API someday. */ diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index eb36578e320e..accdb744d61b 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -860,10 +860,13 @@ void SimpleCameraData::bufferReady(FrameBuffer *buffer) return; } - if (converter_) + if (converter_) { converter_->queueBuffers(buffer, conversionQueue_.front()); - else + } else { swIsp_->queueBuffers(buffer, conversionQueue_.front()); + if (request) + request->metadata().merge(swIsp_->metadata()); + } conversionQueue_.pop(); return; diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index 20fb6f48fdf9..efbbea2d2279 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -68,7 +68,8 @@ LOG_DEFINE_CATEGORY(SoftwareIsp) SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor) : dmaHeap_(DmaBufAllocator::DmaBufAllocatorFlag::CmaHeap | DmaBufAllocator::DmaBufAllocatorFlag::SystemHeap | - DmaBufAllocator::DmaBufAllocatorFlag::UDmaBuf) + DmaBufAllocator::DmaBufAllocatorFlag::UDmaBuf), + metadata_(controls::controls) { /* * debayerParams_ must be initialized because the initial value is used for @@ -349,9 +350,10 @@ void SoftwareIsp::process(FrameBuffer *input, FrameBuffer *output) ConnectionTypeQueued, input, output, debayerParams_); } -void SoftwareIsp::saveIspParams() +void SoftwareIsp::saveIspParams(const ControlList &metadata) { debayerParams_ = *sharedParams_; + metadata_.merge(metadata, ControlList::MergePolicy::OverwriteExisting); } void SoftwareIsp::setSensorCtrls(const ControlList &sensorControls) From patchwork Mon Jun 17 23:25:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 20351 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 35FB8BD87C for ; Mon, 17 Jun 2024 23:25:36 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 767BB65498; Tue, 18 Jun 2024 01:25:35 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="GDM6U9pB"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 879FA6548B for ; Tue, 18 Jun 2024 01:25:29 +0200 (CEST) Received: from Monstersaurus.local (cpc89244-aztw30-2-0-cust6594.18-1.cable.virginm.net [86.31.185.195]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 40555DD9; Tue, 18 Jun 2024 01:25:12 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1718666712; bh=R12+deRfTEHoHMFdRNStxNheQe4WfA4EzGBJahshAnc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GDM6U9pBGota1jXYNlnmU7Rz27cWu7fO/ffUBf/I4tUXs7ZXFRdwpbPoWCd522l7m p8vvbBK32bqEtxm9tO2afj8Lea8I///ltNoTQthKwNOSg6PNTJbPP/rMyHNObnBkK8 xoZYeh8o2BJEHBaCOY9mjpxhly95dEBjWMHKPYDc= From: Kieran Bingham To: libcamera devel Cc: Kieran Bingham Subject: [PATCH 2/3] ipa: simple: Report the ColourGains in metadata Date: Tue, 18 Jun 2024 00:25:24 +0100 Message-Id: <20240617232525.878530-3-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240617232525.878530-1-kieran.bingham@ideasonboard.com> References: <20240617232525.878530-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Provide the determined colour gains back into the metadata to add to completed requests. Signed-off-by: Kieran Bingham --- src/ipa/simple/soft_simple.cpp | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index 09c7b575301e..d35d28c26a3b 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -305,6 +305,10 @@ void IPASoftSimple::processStats(const ControlList &sensorControls) params_->blue[i] = gammaTable_[idx]; } + const float maxGain = 1024.0; + const float gains[] = { gainR / maxGain, gainB / maxGain }; + metadata.set(controls::ColourGains, gains); + setIspParams.emit(metadata); /* \todo Switch to the libipa/algorithm.h API someday. */ From patchwork Mon Jun 17 23:25:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 20352 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 671D1C3293 for ; Mon, 17 Jun 2024 23:25:38 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8DFBD65492; Tue, 18 Jun 2024 01:25:37 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="YwnUz4hZ"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id AF6476548D for ; Tue, 18 Jun 2024 01:25:29 +0200 (CEST) Received: from Monstersaurus.local (cpc89244-aztw30-2-0-cust6594.18-1.cable.virginm.net [86.31.185.195]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 87918289; Tue, 18 Jun 2024 01:25:12 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1718666712; bh=k3z1LjbhTNsU24ot7ttzPLAmYANO1cMflxdo1Clym20=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=YwnUz4hZ8Ga+HBywIWKicBCGjCkvD/ZFftrSA/Tlhr66qzQO5BOcTaFhxv9zmhSqL DjCbnSjOAA7ULMpjWvRsTPUp1i3MK2IhT72mchtPjLl/NksqF4MOVomvyGUstWeKpX dRq4AigAB7csfreIezQjUyGTZN8zcyhEGT067ngs= From: Kieran Bingham To: libcamera devel Cc: Kieran Bingham Subject: [PATCH 3/3] ipa: simple: Report black levels in metadata Date: Tue, 18 Jun 2024 00:25:25 +0100 Message-Id: <20240617232525.878530-4-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240617232525.878530-1-kieran.bingham@ideasonboard.com> References: <20240617232525.878530-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Provide the determined black level values in the metadata to add to the completed requests. Signed-off-by: Kieran Bingham --- src/ipa/simple/soft_simple.cpp | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index d35d28c26a3b..460fb216d49a 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -309,6 +309,10 @@ void IPASoftSimple::processStats(const ControlList &sensorControls) const float gains[] = { gainR / maxGain, gainB / maxGain }; metadata.set(controls::ColourGains, gains); + /* Assign each of the R G G B channels as the same black level. */ + const auto blackLevels = { blackLevel, blackLevel, blackLevel, blackLevel }; + metadata.set(controls::SensorBlackLevels, blackLevels); + setIspParams.emit(metadata); /* \todo Switch to the libipa/algorithm.h API someday. */