From patchwork Thu Mar 27 18:59:39 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23059 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 846A7C323E for ; Thu, 27 Mar 2025 19:00:02 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 33DF26897D; Thu, 27 Mar 2025 20:00:02 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="iVEwoiiy"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id F353C68947 for ; Thu, 27 Mar 2025 19:59:59 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1743101998; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TZRnrsWgi1XCphjvEgBR7IF0WE3NXR5ai2CXVj99ojo=; b=iVEwoiiyG/oZXGMnHxaA0ZwvBOYMe6WPj5iYnBzWrCxa9pu+HQlQ55DkcXUnIe181vnp3C KO9FmO6atydjKrcmnqXqA0iKrHwZ9lOPhrkeLRkM8Qs16jPIlDCL3720In095ZcMEiE3Y1 3fCDDn0E3vIklwNvzaleZ7GWZtvr1IE= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-619-PZ9Cuq8tMoqFfzT7b8gORQ-1; Thu, 27 Mar 2025 14:59:57 -0400 X-MC-Unique: PZ9Cuq8tMoqFfzT7b8gORQ-1 X-Mimecast-MFC-AGG-ID: PZ9Cuq8tMoqFfzT7b8gORQ_1743101996 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 5F95018EBE8F; Thu, 27 Mar 2025 18:59:56 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.45.224.17]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 95B7230001A1; Thu, 27 Mar 2025 18:59:54 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham Subject: [PATCH v6 1/6] libcamera: software_isp: Track frames and requests Date: Thu, 27 Mar 2025 19:59:39 +0100 Message-ID: <20250327185945.159481-2-mzamazal@redhat.com> In-Reply-To: <20250327185945.159481-1-mzamazal@redhat.com> References: <20250327185945.159481-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: yFfhjKwAeOWwaO-mt-beWDdpdWkR0ALE6AHawQtoY5A_1743101996 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Hardware pipelines track requests and other information related to particular frames. This hasn't been needed in software ISP so far. But in order to be able to track metadata corresponding to a given frame, frame-request tracking mechanism starts being useful. This patch introduces the basic tracking structure, actual metadata handling is added in the following patch. Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart Tested-by: Kieran Bingham # Lenovo X13s Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 72 ++++++++++++++++++++++-- 1 file changed, 68 insertions(+), 4 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 6e039bf3..004a8d53 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -181,6 +181,54 @@ LOG_DEFINE_CATEGORY(SimplePipeline) class SimplePipelineHandler; +struct SimpleFrameInfo { + SimpleFrameInfo(uint32_t f, Request *r) + : frame(f), request(r) + { + } + + uint32_t frame; + Request *request; +}; + +class SimpleFrames +{ +public: + void create(Request *request); + void destroy(uint32_t frame); + void clear(); + + SimpleFrameInfo *find(uint32_t frame); + +private: + std::map frameInfo_; +}; + +void SimpleFrames::create(Request *request) +{ + const uint32_t frame = request->sequence(); + auto [it, inserted] = frameInfo_.try_emplace(frame, frame, request); + ASSERT(inserted); +} + +void SimpleFrames::destroy(uint32_t frame) +{ + frameInfo_.erase(frame); +} + +void SimpleFrames::clear() +{ + frameInfo_.clear(); +} + +SimpleFrameInfo *SimpleFrames::find(uint32_t frame) +{ + auto info = frameInfo_.find(frame); + if (info == frameInfo_.end()) + return nullptr; + return &info->second; +} + struct SimplePipelineInfo { const char *driver; /* @@ -293,11 +341,13 @@ public: std::unique_ptr converter_; std::unique_ptr swIsp_; + SimpleFrames frameInfo_; private: void tryPipeline(unsigned int code, const Size &size); static std::vector routedSourcePads(MediaPad *sink); + void completeRequest(Request *request); void conversionInputDone(FrameBuffer *buffer); void conversionOutputDone(FrameBuffer *buffer); @@ -785,7 +835,7 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) /* No conversion, just complete the request. */ Request *request = buffer->request(); pipe->completeBuffer(request, buffer); - pipe->completeRequest(request); + completeRequest(request); return; } @@ -803,7 +853,7 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) const RequestOutputs &outputs = conversionQueue_.front(); for (auto &[stream, buf] : outputs.outputs) pipe->completeBuffer(outputs.request, buf); - pipe->completeRequest(outputs.request); + completeRequest(outputs.request); conversionQueue_.pop(); return; @@ -861,7 +911,7 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) /* Otherwise simply complete the request. */ pipe->completeBuffer(request, buffer); - pipe->completeRequest(request); + completeRequest(request); } void SimpleCameraData::clearIncompleteRequests() @@ -872,6 +922,18 @@ void SimpleCameraData::clearIncompleteRequests() } } +void SimpleCameraData::completeRequest(Request *request) +{ + SimpleFrameInfo *info = frameInfo_.find(request->sequence()); + if (!info) { + /* Something is really wrong, let's return. */ + return; + } + + frameInfo_.destroy(info->frame); + pipe()->completeRequest(request); +} + void SimpleCameraData::conversionInputDone(FrameBuffer *buffer) { /* Queue the input buffer back for capture. */ @@ -885,7 +947,7 @@ void SimpleCameraData::conversionOutputDone(FrameBuffer *buffer) /* Complete the buffer and the request. */ Request *request = buffer->request(); if (pipe->completeBuffer(request, buffer)) - pipe->completeRequest(request); + completeRequest(request); } void SimpleCameraData::ispStatsReady(uint32_t frame, uint32_t bufferId) @@ -1398,6 +1460,7 @@ void SimplePipelineHandler::stopDevice(Camera *camera) video->bufferReady.disconnect(data, &SimpleCameraData::imageBufferReady); + data->frameInfo_.clear(); data->clearIncompleteRequests(); data->conversionBuffers_.clear(); @@ -1426,6 +1489,7 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request) } } + data->frameInfo_.create(request); if (data->useConversion_) { data->conversionQueue_.push({ request, std::move(buffers) }); if (data->swIsp_) From patchwork Thu Mar 27 18:59:40 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23060 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0666FC323E for ; Thu, 27 Mar 2025 19:00:06 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6897C6897F; Thu, 27 Mar 2025 20:00:05 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="AWyeI70V"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2769A68971 for ; Thu, 27 Mar 2025 20:00:03 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1743102002; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FBL4j11uDGzu/i4OG1Qm8222bw75urCxPjWL1WsxBkk=; b=AWyeI70VDbvUbGvRfyTIDZVxlY3PZiUbBGrv/Rm+olJD3im8olU4xisdnZ+mNMUuvtq99O yjQ5FWsF1hZeD92gtvzUDsXFUv0BeUT2SjEfJ78FfutW5ALaEDwn07jiQj6B+lCafAzAZi VQiCPStq649PV2LxtgU6W3xcZac7oiw= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-65-W1Zqvj7pMEC13tgILEcWGQ-1; Thu, 27 Mar 2025 14:59:59 -0400 X-MC-Unique: W1Zqvj7pMEC13tgILEcWGQ-1 X-Mimecast-MFC-AGG-ID: W1Zqvj7pMEC13tgILEcWGQ_1743101998 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id D12FF180AB19; Thu, 27 Mar 2025 18:59:58 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.45.224.17]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 058D130001A1; Thu, 27 Mar 2025 18:59:56 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Kieran Bingham , Laurent Pinchart , Milan Zamazal Subject: [PATCH v6 2/6] ipa: simple: softisp: Extend to pass metadata Date: Thu, 27 Mar 2025 19:59:40 +0100 Message-ID: <20250327185945.159481-3-mzamazal@redhat.com> In-Reply-To: <20250327185945.159481-1-mzamazal@redhat.com> References: <20250327185945.159481-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: LOSSDaqZhgYEwO9JbqArGLdTlsRkXbOKdPkh6_29wyU_1743101998 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham Extend the Simple IPA IPC to support returning a metadata ControlList when the process call has completed. A new signal from the IPA is introduced to report the metadata, similarly to what the hardware pipelines do. Merge the metadata reported by the ISP into any completing request to provide to the application. Completion of a request is delayed until this is done; this doesn't apply to canceled requests. Reviewed-by: Kieran Bingham Signed-off-by: Kieran Bingham Signed-off-by: Milan Zamazal Reviewed-by: Laurent Pinchart --- .../internal/software_isp/software_isp.h | 2 + include/libcamera/ipa/soft.mojom | 1 + src/ipa/simple/soft_simple.cpp | 7 +-- src/libcamera/pipeline/simple/simple.cpp | 48 ++++++++++++++----- src/libcamera/software_isp/software_isp.cpp | 9 ++++ 5 files changed, 49 insertions(+), 18 deletions(-) diff --git a/include/libcamera/internal/software_isp/software_isp.h b/include/libcamera/internal/software_isp/software_isp.h index 0026cec2..1ee37dc7 100644 --- a/include/libcamera/internal/software_isp/software_isp.h +++ b/include/libcamera/internal/software_isp/software_isp.h @@ -85,12 +85,14 @@ public: Signal inputBufferReady; Signal outputBufferReady; Signal ispStatsReady; + Signal metadataReady; Signal setSensorControls; private: void saveIspParams(); void setSensorCtrls(const ControlList &sensorControls); void statsReady(uint32_t frame, uint32_t bufferId); + void saveMetadata(uint32_t frame, const ControlList &metadata); void inputReady(FrameBuffer *input); void outputReady(FrameBuffer *output); diff --git a/include/libcamera/ipa/soft.mojom b/include/libcamera/ipa/soft.mojom index ede74413..a8e6ecda 100644 --- a/include/libcamera/ipa/soft.mojom +++ b/include/libcamera/ipa/soft.mojom @@ -33,4 +33,5 @@ interface IPASoftInterface { interface IPASoftEventInterface { setSensorControls(libcamera.ControlList sensorControls); setIspParams(); + metadataReady(uint32 frame, libcamera.ControlList metadata); }; diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index a87c6cdd..4ef67c43 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -299,15 +299,10 @@ void IPASoftSimple::processStats(const uint32_t frame, int32_t again = sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get(); frameContext.sensor.gain = camHelper_ ? camHelper_->gain(again) : again; - /* - * Software ISP currently does not produce any metadata. Use an empty - * ControlList for now. - * - * \todo Implement proper metadata handling - */ ControlList metadata(controls::controls); for (auto const &algo : algorithms()) algo->process(context_, frame, frameContext, stats_, metadata); + metadataReady.emit(frame, metadata); /* Sanity check */ if (!sensorControls.contains(V4L2_CID_EXPOSURE) || diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 004a8d53..fd0ccdca 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -182,19 +182,21 @@ LOG_DEFINE_CATEGORY(SimplePipeline) class SimplePipelineHandler; struct SimpleFrameInfo { - SimpleFrameInfo(uint32_t f, Request *r) - : frame(f), request(r) + SimpleFrameInfo(uint32_t f, Request *r, bool m) + : frame(f), request(r), metadataRequired(m), metadataProcessed(false) { } uint32_t frame; Request *request; + bool metadataRequired; + bool metadataProcessed; }; class SimpleFrames { public: - void create(Request *request); + void create(Request *request, bool metadataRequested); void destroy(uint32_t frame); void clear(); @@ -204,10 +206,10 @@ private: std::map frameInfo_; }; -void SimpleFrames::create(Request *request) +void SimpleFrames::create(Request *request, bool metadataRequired) { const uint32_t frame = request->sequence(); - auto [it, inserted] = frameInfo_.try_emplace(frame, frame, request); + auto [it, inserted] = frameInfo_.try_emplace(frame, frame, request, metadataRequired); ASSERT(inserted); } @@ -347,11 +349,12 @@ private: void tryPipeline(unsigned int code, const Size &size); static std::vector routedSourcePads(MediaPad *sink); - void completeRequest(Request *request); + void tryCompleteRequest(Request *request); void conversionInputDone(FrameBuffer *buffer); void conversionOutputDone(FrameBuffer *buffer); void ispStatsReady(uint32_t frame, uint32_t bufferId); + void metadataReady(uint32_t frame, const ControlList &metadata); void setSensorControls(const ControlList &sensorControls); }; @@ -590,6 +593,7 @@ int SimpleCameraData::init() swIsp_->inputBufferReady.connect(this, &SimpleCameraData::conversionInputDone); swIsp_->outputBufferReady.connect(this, &SimpleCameraData::conversionOutputDone); swIsp_->ispStatsReady.connect(this, &SimpleCameraData::ispStatsReady); + swIsp_->metadataReady.connect(this, &SimpleCameraData::metadataReady); swIsp_->setSensorControls.connect(this, &SimpleCameraData::setSensorControls); } } @@ -835,7 +839,7 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) /* No conversion, just complete the request. */ Request *request = buffer->request(); pipe->completeBuffer(request, buffer); - completeRequest(request); + tryCompleteRequest(request); return; } @@ -853,7 +857,10 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) const RequestOutputs &outputs = conversionQueue_.front(); for (auto &[stream, buf] : outputs.outputs) pipe->completeBuffer(outputs.request, buf); - completeRequest(outputs.request); + SimpleFrameInfo *info = frameInfo_.find(outputs.request->sequence()); + if (info) + info->metadataRequired = false; + tryCompleteRequest(outputs.request); conversionQueue_.pop(); return; @@ -911,7 +918,7 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) /* Otherwise simply complete the request. */ pipe->completeBuffer(request, buffer); - completeRequest(request); + tryCompleteRequest(request); } void SimpleCameraData::clearIncompleteRequests() @@ -922,14 +929,20 @@ void SimpleCameraData::clearIncompleteRequests() } } -void SimpleCameraData::completeRequest(Request *request) +void SimpleCameraData::tryCompleteRequest(Request *request) { + if (request->hasPendingBuffers()) + return; + SimpleFrameInfo *info = frameInfo_.find(request->sequence()); if (!info) { /* Something is really wrong, let's return. */ return; } + if (info->metadataRequired && !info->metadataProcessed) + return; + frameInfo_.destroy(info->frame); pipe()->completeRequest(request); } @@ -947,7 +960,7 @@ void SimpleCameraData::conversionOutputDone(FrameBuffer *buffer) /* Complete the buffer and the request. */ Request *request = buffer->request(); if (pipe->completeBuffer(request, buffer)) - completeRequest(request); + tryCompleteRequest(request); } void SimpleCameraData::ispStatsReady(uint32_t frame, uint32_t bufferId) @@ -956,6 +969,17 @@ void SimpleCameraData::ispStatsReady(uint32_t frame, uint32_t bufferId) delayedCtrls_->get(frame)); } +void SimpleCameraData::metadataReady(uint32_t frame, const ControlList &metadata) +{ + SimpleFrameInfo *info = frameInfo_.find(frame); + if (!info) + return; + + info->request->metadata().merge(metadata); + info->metadataProcessed = true; + tryCompleteRequest(info->request); +} + void SimpleCameraData::setSensorControls(const ControlList &sensorControls) { delayedCtrls_->push(sensorControls); @@ -1489,7 +1513,7 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request) } } - data->frameInfo_.create(request); + data->frameInfo_.create(request, !!data->swIsp_); if (data->useConversion_) { data->conversionQueue_.push({ request, std::move(buffers) }); if (data->swIsp_) diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index 4a74dcb6..6baa3fec 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -55,6 +55,11 @@ LOG_DEFINE_CATEGORY(SoftwareIsp) * \brief A signal emitted when the statistics for IPA are ready */ +/** + * \var SoftwareIsp::metadataReady + * \brief A signal emitted when the metadata for IPA is ready + */ + /** * \var SoftwareIsp::setSensorControls * \brief A signal emitted when the values to write to the sensor controls are @@ -141,6 +146,10 @@ SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor, } ipa_->setIspParams.connect(this, &SoftwareIsp::saveIspParams); + ipa_->metadataReady.connect(this, + [this](uint32_t frame, const ControlList &metadata) { + metadataReady.emit(frame, metadata); + }); ipa_->setSensorControls.connect(this, &SoftwareIsp::setSensorCtrls); debayer_->moveToThread(&ispWorkerThread_); From patchwork Thu Mar 27 18:59:41 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23061 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4AFF1C323E for ; Thu, 27 Mar 2025 19:00:10 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C7A1D68986; Thu, 27 Mar 2025 20:00:09 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="aHrNA3qw"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E76F36897C for ; Thu, 27 Mar 2025 20:00:06 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1743102005; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1HwX+yC/EoaGzGDsNjwvjuR1ZRCrHrlU4Z/RHcxItwc=; b=aHrNA3qwVY0SABD9EApjMfuphtG/SFHtRIfSMXqh1CSncs1c6ZAC35Q8J4YkWTjLopPkmB UB+MxXr5FamkMdVaqkkCCvZM4hXUrLpLfPqyAZ7/UpRkwm3DZxbZP8keanE8+KSi/Wvawe eilSY5desE4k5oTJECS7xu0UU0FuWAo= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-210-pWnb6hbkMDeNxZE_JWiZ5Q-1; Thu, 27 Mar 2025 15:00:02 -0400 X-MC-Unique: pWnb6hbkMDeNxZE_JWiZ5Q-1 X-Mimecast-MFC-AGG-ID: pWnb6hbkMDeNxZE_JWiZ5Q_1743102001 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 33EC51800260; Thu, 27 Mar 2025 19:00:01 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.45.224.17]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 5743930001A1; Thu, 27 Mar 2025 18:59:59 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Kieran Bingham , Laurent Pinchart , Milan Zamazal Subject: [PATCH v6 3/6] ipa: simple: Report the ColourGains in metadata Date: Thu, 27 Mar 2025 19:59:41 +0100 Message-ID: <20250327185945.159481-4-mzamazal@redhat.com> In-Reply-To: <20250327185945.159481-1-mzamazal@redhat.com> References: <20250327185945.159481-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: QLGivWH9knNc5EjfbtX-ddQJ6BnefrhWXikahmbyIqg_1743102001 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham Provide the determined colour gains back into the metadata to add to completed requests. Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart Signed-off-by: Kieran Bingham Signed-off-by: Milan Zamazal --- src/ipa/simple/algorithms/awb.cpp | 21 ++++++++++++++++++++- src/ipa/simple/algorithms/awb.h | 6 +++++- src/ipa/simple/ipa_context.h | 4 ++++ 3 files changed, 29 insertions(+), 2 deletions(-) diff --git a/src/ipa/simple/algorithms/awb.cpp b/src/ipa/simple/algorithms/awb.cpp index ec77c6e5..55719059 100644 --- a/src/ipa/simple/algorithms/awb.cpp +++ b/src/ipa/simple/algorithms/awb.cpp @@ -17,6 +17,8 @@ #include "libipa/colours.h" #include "simple/ipa_context.h" +#include "control_ids.h" + namespace libcamera { LOG_DEFINE_CATEGORY(IPASoftAwb) @@ -32,15 +34,32 @@ int Awb::configure(IPAContext &context, return 0; } +void Awb::prepare(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + [[maybe_unused]] DebayerParams *params) +{ + auto &gains = context.activeState.awb.gains; + frameContext.gains.red = gains.r(); + frameContext.gains.blue = gains.b(); +} + void Awb::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, + IPAFrameContext &frameContext, const SwIspStats *stats, ControlList &metadata) { const SwIspStats::Histogram &histogram = stats->yHistogram; const uint8_t blackLevel = context.activeState.blc.level; + const float maxGain = 1024.0; + const float mdGains[] = { + static_cast(frameContext.gains.red / maxGain), + static_cast(frameContext.gains.blue / maxGain) + }; + metadata.set(controls::ColourGains, mdGains); + /* * Black level must be subtracted to get the correct AWB ratios, they * would be off if they were computed from the whole brightness range diff --git a/src/ipa/simple/algorithms/awb.h b/src/ipa/simple/algorithms/awb.h index db1496cd..ad993f39 100644 --- a/src/ipa/simple/algorithms/awb.h +++ b/src/ipa/simple/algorithms/awb.h @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: LGPL-2.1-or-later */ /* - * Copyright (C) 2024, Red Hat Inc. + * Copyright (C) 2024-2025 Red Hat Inc. * * Auto white balance */ @@ -20,6 +20,10 @@ public: ~Awb() = default; int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; + void prepare(IPAContext &context, + const uint32_t frame, + IPAFrameContext &frameContext, + DebayerParams *params) override; void process(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, diff --git a/src/ipa/simple/ipa_context.h b/src/ipa/simple/ipa_context.h index 17bcd4ca..bfac835b 100644 --- a/src/ipa/simple/ipa_context.h +++ b/src/ipa/simple/ipa_context.h @@ -70,6 +70,10 @@ struct IPAFrameContext : public FrameContext { int32_t exposure; double gain; } sensor; + struct { + double red; + double blue; + } gains; }; struct IPAContext { From patchwork Thu Mar 27 18:59:42 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23062 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 64313C323E for ; Thu, 27 Mar 2025 19:00:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C292E68988; Thu, 27 Mar 2025 20:00:11 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="OHzB1Mua"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 106506897D for ; Thu, 27 Mar 2025 20:00:06 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1743102005; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZBAUo+LO3+kzcudNM2LezLlYBcCCAP/Hq8Afwu2y0wI=; b=OHzB1MuaFwlLAbV0qWsCeFN/mleglYBmunubRN1+CTiPgPIctOZy0z0HntXSJp/5Kgday3 hGbEx5piHSwmX8aMS879SiBnOoEvymcLQ27JrXHuSBUJJ5TTkiHn75ubYJ7RBKqAdZ0J0N LDdzuEbVxZDI0LO+FHge2sltAMCSzjc= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-84-mPdN1Z7pNbqB6Hq5GW7lNw-1; Thu, 27 Mar 2025 15:00:04 -0400 X-MC-Unique: mPdN1Z7pNbqB6Hq5GW7lNw-1 X-Mimecast-MFC-AGG-ID: mPdN1Z7pNbqB6Hq5GW7lNw_1743102003 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 40A371933B53; Thu, 27 Mar 2025 19:00:03 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.45.224.17]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 98E3A30001A1; Thu, 27 Mar 2025 19:00:01 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Kieran Bingham , Laurent Pinchart , Milan Zamazal Subject: [PATCH v6 4/6] ipa: simple: Report black levels in metadata Date: Thu, 27 Mar 2025 19:59:42 +0100 Message-ID: <20250327185945.159481-5-mzamazal@redhat.com> In-Reply-To: <20250327185945.159481-1-mzamazal@redhat.com> References: <20250327185945.159481-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: 9B2hOxZLWgNknSJl1fUX8L_tNt4zdyVJMeiB6q5VtEM_1743102003 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham Provide the determined black level values in the metadata to add to the completed requests. Reviewed-by: Laurent Pinchart Signed-off-by: Kieran Bingham Signed-off-by: Milan Zamazal --- src/ipa/simple/algorithms/blc.cpp | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/src/ipa/simple/algorithms/blc.cpp b/src/ipa/simple/algorithms/blc.cpp index 1d7d370b..53c356c8 100644 --- a/src/ipa/simple/algorithms/blc.cpp +++ b/src/ipa/simple/algorithms/blc.cpp @@ -11,6 +11,8 @@ #include +#include "control_ids.h" + namespace libcamera { namespace ipa::soft::algorithms { @@ -49,8 +51,15 @@ void BlackLevel::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, IPAFrameContext &frameContext, const SwIspStats *stats, - [[maybe_unused]] ControlList &metadata) + ControlList &metadata) { + /* Assign each of the R G G B channels as the same black level. */ + const int32_t blackLevel = context.activeState.blc.level * 256; + const int32_t blackLevels[] = { + blackLevel, blackLevel, blackLevel, blackLevel + }; + metadata.set(controls::SensorBlackLevels, blackLevels); + if (context.configuration.black.level.has_value()) return; From patchwork Thu Mar 27 18:59:43 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23063 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 03651C323E for ; Thu, 27 Mar 2025 19:00:14 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0A19968983; Thu, 27 Mar 2025 20:00:13 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="Soxx1SAr"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 326EE68982 for ; Thu, 27 Mar 2025 20:00:09 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1743102008; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LFGtOZDi6AAxrfHyTW6fpfh0z6nzKONhGFxK3+o8AJA=; b=Soxx1SArY90lpcjxKagmXoi+bnP4T8i8Ix3lrvB7ZPU2BBmGFSklOiDrpPUtdIQsOgMXnS oLkdDMxhDtorf374P52IdEcxPcP9HoKxNZPNqDzDkHxmzIzum1Q8HY3zYDbsaflBlFYVHG xBTR5BeOK5lDh4a9w6Hrj8N4mj5g9/k= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-658-OKE-JFT5NGmWxHUGKqXjFA-1; Thu, 27 Mar 2025 15:00:06 -0400 X-MC-Unique: OKE-JFT5NGmWxHUGKqXjFA-1 X-Mimecast-MFC-AGG-ID: OKE-JFT5NGmWxHUGKqXjFA_1743102005 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id B004618EBE91; Thu, 27 Mar 2025 19:00:05 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.45.224.17]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id D839E3001D0F; Thu, 27 Mar 2025 19:00:03 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham Subject: [PATCH v6 5/6] ipa: simple: Report contrast in metadata Date: Thu, 27 Mar 2025 19:59:43 +0100 Message-ID: <20250327185945.159481-6-mzamazal@redhat.com> In-Reply-To: <20250327185945.159481-1-mzamazal@redhat.com> References: <20250327185945.159481-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: geddwMB-O_cF11uadVF97PxIqexEK8f-KDcKjp9Iszs_1743102005 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Provide the requested contrast value, if any, in the metadata to add to the completed requests. Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart Signed-off-by: Milan Zamazal --- src/ipa/simple/algorithms/lut.cpp | 15 ++++++++++++++- src/ipa/simple/algorithms/lut.h | 5 +++++ src/ipa/simple/ipa_context.h | 1 + 3 files changed, 20 insertions(+), 1 deletion(-) diff --git a/src/ipa/simple/algorithms/lut.cpp b/src/ipa/simple/algorithms/lut.cpp index a06cdeba..e8638f27 100644 --- a/src/ipa/simple/algorithms/lut.cpp +++ b/src/ipa/simple/algorithms/lut.cpp @@ -87,9 +87,11 @@ int16_t Lut::ccmValue(unsigned int i, float ccm) const void Lut::prepare(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, + IPAFrameContext &frameContext, DebayerParams *params) { + frameContext.contrast = context.activeState.knobs.contrast; + /* * Update the gamma table if needed. This means if black level changes * and since the black level gets updated only if a lower value is @@ -139,6 +141,17 @@ void Lut::prepare(IPAContext &context, } } +void Lut::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + [[maybe_unused]] const SwIspStats *stats, + ControlList &metadata) +{ + const auto &contrast = frameContext.contrast; + if (contrast) + metadata.set(controls::Contrast, contrast.value()); +} + REGISTER_IPA_ALGORITHM(Lut, "Lut") } /* namespace ipa::soft::algorithms */ diff --git a/src/ipa/simple/algorithms/lut.h b/src/ipa/simple/algorithms/lut.h index 77324800..ba8b9021 100644 --- a/src/ipa/simple/algorithms/lut.h +++ b/src/ipa/simple/algorithms/lut.h @@ -30,6 +30,11 @@ public: const uint32_t frame, IPAFrameContext &frameContext, DebayerParams *params) override; + void process(IPAContext &context, + const uint32_t frame, + IPAFrameContext &frameContext, + const SwIspStats *stats, + ControlList &metadata) override; private: void updateGammaTable(IPAContext &context); diff --git a/src/ipa/simple/ipa_context.h b/src/ipa/simple/ipa_context.h index bfac835b..10d539f5 100644 --- a/src/ipa/simple/ipa_context.h +++ b/src/ipa/simple/ipa_context.h @@ -74,6 +74,7 @@ struct IPAFrameContext : public FrameContext { double red; double blue; } gains; + std::optional contrast; }; struct IPAContext { From patchwork Thu Mar 27 18:59:44 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23064 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 02152C323E for ; Thu, 27 Mar 2025 19:00:17 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A8F9368989; Thu, 27 Mar 2025 20:00:16 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="P1WCT4Cz"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5E59268986 for ; Thu, 27 Mar 2025 20:00:13 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1743102012; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=epwQ+YMGEWl+tAab12DZxlFWIAFkGtrIJyPe8/konXc=; b=P1WCT4Cz5L6CCS+xXKIV2jjn0YfGHRYl2wZ5UOc1IGbaDZKEafQrsLD/jUnMvhJoDvaOAU qIAC3d184YRW9brkPeOz8ekVA+FLurnqN2AzoMx/qM9G055SpyvN5JOPd1MGladDeZ2rqi SV0vkOdTJwC+huJSF5P67XPG8zdisTA= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-271-4tjXZ-2YP7G4w4Mz6Ji0Vw-1; Thu, 27 Mar 2025 15:00:08 -0400 X-MC-Unique: 4tjXZ-2YP7G4w4Mz6Ji0Vw-1 X-Mimecast-MFC-AGG-ID: 4tjXZ-2YP7G4w4Mz6Ji0Vw_1743102008 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id F03CC18009FC; Thu, 27 Mar 2025 19:00:07 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.45.224.17]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 35F1C3001D11; Thu, 27 Mar 2025 19:00:06 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham Subject: [PATCH v6 6/6] ipa: simple: Report exposure in metadata Date: Thu, 27 Mar 2025 19:59:44 +0100 Message-ID: <20250327185945.159481-7-mzamazal@redhat.com> In-Reply-To: <20250327185945.159481-1-mzamazal@redhat.com> References: <20250327185945.159481-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: 9QXqshA2uKmAtoC_h87V_2W72Rq4Fh7-TCJke3kM3KQ_1743102008 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Report exposure and gain in metadata. This is more complicated than it could be expected because the exposure value should be in microseconds but it's handled using V4L2_CID_EXPOSURE control, which doesn't specify the unit, see https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/control.html. So the unit conversion is done in the way rkisp1 IPA uses. This requires getting and passing IPACameraSensorInfo around. To avoid naming confusion and to improve consistency with rkisp1 IPA, sensorCtrlInfoMap parameter is renamed to sensorControls. Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart Signed-off-by: Milan Zamazal --- include/libcamera/ipa/soft.mojom | 3 ++- src/ipa/simple/algorithms/agc.cpp | 11 +++++++++-- src/ipa/simple/ipa_context.h | 6 +++++- src/ipa/simple/soft_simple.cpp | 17 +++++++++++++---- src/libcamera/software_isp/software_isp.cpp | 20 ++++++++++++++------ 5 files changed, 43 insertions(+), 14 deletions(-) diff --git a/include/libcamera/ipa/soft.mojom b/include/libcamera/ipa/soft.mojom index a8e6ecda..77328c5f 100644 --- a/include/libcamera/ipa/soft.mojom +++ b/include/libcamera/ipa/soft.mojom @@ -16,7 +16,8 @@ interface IPASoftInterface { init(libcamera.IPASettings settings, libcamera.SharedFD fdStats, libcamera.SharedFD fdParams, - libcamera.ControlInfoMap sensorCtrlInfoMap) + libcamera.IPACameraSensorInfo sensorInfo, + libcamera.ControlInfoMap sensorControls) => (int32 ret, libcamera.ControlInfoMap ipaControls, bool ccmEnabled); start() => (int32 ret); stop(); diff --git a/src/ipa/simple/algorithms/agc.cpp b/src/ipa/simple/algorithms/agc.cpp index 72aade14..c46bb0eb 100644 --- a/src/ipa/simple/algorithms/agc.cpp +++ b/src/ipa/simple/algorithms/agc.cpp @@ -11,6 +11,8 @@ #include +#include "control_ids.h" + namespace libcamera { LOG_DEFINE_CATEGORY(IPASoftExposure) @@ -97,10 +99,15 @@ void Agc::updateExposure(IPAContext &context, IPAFrameContext &frameContext, dou void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, + IPAFrameContext &frameContext, const SwIspStats *stats, - [[maybe_unused]] ControlList &metadata) + ControlList &metadata) { + utils::Duration exposureTime = + context.configuration.agc.lineDuration * frameContext.sensor.exposure; + metadata.set(controls::ExposureTime, exposureTime.get()); + metadata.set(controls::AnalogueGain, frameContext.sensor.gain); + /* * Calculate Mean Sample Value (MSV) according to formula from: * https://www.araa.asn.au/acra/acra2007/papers/paper84final.pdf diff --git a/src/ipa/simple/ipa_context.h b/src/ipa/simple/ipa_context.h index 10d539f5..7dc2cd7a 100644 --- a/src/ipa/simple/ipa_context.h +++ b/src/ipa/simple/ipa_context.h @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: LGPL-2.1-or-later */ /* - * Copyright (C) 2024 Red Hat, Inc. + * Copyright (C) 2024-2025 Red Hat, Inc. * * Simple pipeline IPA Context */ @@ -18,6 +18,8 @@ #include +#include "core_ipa_interface.h" + namespace libcamera { namespace ipa::soft { @@ -27,6 +29,7 @@ struct IPASessionConfiguration { struct { int32_t exposureMin, exposureMax; double againMin, againMax, againMinStep; + utils::Duration lineDuration; } agc; struct { std::optional level; @@ -83,6 +86,7 @@ struct IPAContext { { } + IPACameraSensorInfo sensorInfo; IPASessionConfiguration configuration; IPAActiveState activeState; FCQueue frameContexts; diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index 4ef67c43..c94c4cd5 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -5,6 +5,7 @@ * Simple Software Image Processing Algorithm module */ +#include #include #include @@ -32,6 +33,8 @@ namespace libcamera { LOG_DEFINE_CATEGORY(IPASoft) +using namespace std::literals::chrono_literals; + namespace ipa::soft { /* Maximum number of frame contexts to be held */ @@ -50,7 +53,8 @@ public: int init(const IPASettings &settings, const SharedFD &fdStats, const SharedFD &fdParams, - const ControlInfoMap &sensorInfoMap, + const IPACameraSensorInfo &sensorInfo, + const ControlInfoMap &sensorControls, ControlInfoMap *ipaControls, bool *ccmEnabled) override; int configure(const IPAConfigInfo &configInfo) override; @@ -89,7 +93,8 @@ IPASoftSimple::~IPASoftSimple() int IPASoftSimple::init(const IPASettings &settings, const SharedFD &fdStats, const SharedFD &fdParams, - const ControlInfoMap &sensorInfoMap, + const IPACameraSensorInfo &sensorInfo, + const ControlInfoMap &sensorControls, ControlInfoMap *ipaControls, bool *ccmEnabled) { @@ -100,6 +105,8 @@ int IPASoftSimple::init(const IPASettings &settings, << settings.sensorModel; } + context_.sensorInfo = sensorInfo; + /* Load the tuning data file */ File file(settings.configurationFile); if (!file.open(File::OpenModeFlag::ReadOnly)) { @@ -173,12 +180,12 @@ int IPASoftSimple::init(const IPASettings &settings, * Don't save the min and max control values yet, as e.g. the limits * for V4L2_CID_EXPOSURE depend on the configured sensor resolution. */ - if (sensorInfoMap.find(V4L2_CID_EXPOSURE) == sensorInfoMap.end()) { + if (sensorControls.find(V4L2_CID_EXPOSURE) == sensorControls.end()) { LOG(IPASoft, Error) << "Don't have exposure control"; return -EINVAL; } - if (sensorInfoMap.find(V4L2_CID_ANALOGUE_GAIN) == sensorInfoMap.end()) { + if (sensorControls.find(V4L2_CID_ANALOGUE_GAIN) == sensorControls.end()) { LOG(IPASoft, Error) << "Don't have gain control"; return -EINVAL; } @@ -198,6 +205,8 @@ int IPASoftSimple::configure(const IPAConfigInfo &configInfo) context_.activeState = {}; context_.frameContexts.clear(); + context_.configuration.agc.lineDuration = + context_.sensorInfo.minLineLength * 1.0s / context_.sensorInfo.pixelRate; context_.configuration.agc.exposureMin = exposureInfo.min().get(); context_.configuration.agc.exposureMax = exposureInfo.max().get(); if (!context_.configuration.agc.exposureMin) { diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index 6baa3fec..28e2a360 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -133,12 +133,20 @@ SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor, std::string ipaTuningFile = ipa_->configurationFile(sensor->model() + ".yaml", "uncalibrated.yaml"); - int ret = ipa_->init(IPASettings{ ipaTuningFile, sensor->model() }, - debayer_->getStatsFD(), - sharedParams_.fd(), - sensor->controls(), - ipaControls, - &ccmEnabled_); + IPACameraSensorInfo sensorInfo{}; + int ret = sensor->sensorInfo(&sensorInfo); + if (ret) { + LOG(SoftwareIsp, Error) << "Camera sensor information not available"; + return; + } + + ret = ipa_->init(IPASettings{ ipaTuningFile, sensor->model() }, + debayer_->getStatsFD(), + sharedParams_.fd(), + sensorInfo, + sensor->controls(), + ipaControls, + &ccmEnabled_); if (ret) { LOG(SoftwareIsp, Error) << "IPA init failed"; debayer_.reset();