From patchwork Wed Mar 26 12:48:50 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23048 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0D890C3213 for ; Wed, 26 Mar 2025 12:49:10 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CC9836896E; Wed, 26 Mar 2025 13:49:09 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="KodGztlp"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 59AB668967 for ; Wed, 26 Mar 2025 13:49:08 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1742993347; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+Oag22kDtLWrTGKy0Bs3Zr2+XflUp24JhZrLHJnM3QU=; b=KodGztlpZsqNKtZAJoVJJz3fHK1CuFSxii3SelMX1f80XviRR1IUSMrgZ2Rl8XNK9ZnZRd 9omhVh/Xo5SPS+xOQNVjK0q2L0WHgUfcFuVCo2kCzQL9+cTOGAHdc9FwpEABUK5IFWkZXZ aFYiHZSOQWkKqgYZLmZJBC2VVn5oMoo= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-128-8jPBahDPMYu96Ot0sc9lFg-1; Wed, 26 Mar 2025 08:49:04 -0400 X-MC-Unique: 8jPBahDPMYu96Ot0sc9lFg-1 X-Mimecast-MFC-AGG-ID: 8jPBahDPMYu96Ot0sc9lFg_1742993343 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 50F68190308B; Wed, 26 Mar 2025 12:49:03 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.34.15]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 9121530001A1; Wed, 26 Mar 2025 12:49:01 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham Subject: [PATCH v5 1/6] libcamera: software_isp: Track frames and requests Date: Wed, 26 Mar 2025 13:48:50 +0100 Message-ID: <20250326124856.75709-2-mzamazal@redhat.com> In-Reply-To: <20250326124856.75709-1-mzamazal@redhat.com> References: <20250326124856.75709-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: 82rfOvGW_icrXQjQY0MOQmj3VuJovdSSbadqs8Ra5UM_1742993343 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Hardware pipelines track requests and other information related to particular frames. This hasn't been needed in software ISP so far. But in order to be able to track metadata corresponding to a given frame, frame-request tracking mechanism starts being useful. This patch introduces the basic tracking structure, actual metadata handling is added in the following patch. Signed-off-by: Milan Zamazal Reviewed-by: Kieran Bingham Tested-by: Kieran Bingham # Lenovo X13s Reviewed-by: Laurent Pinchart --- src/libcamera/pipeline/simple/simple.cpp | 85 ++++++++++++++++++++++-- 1 file changed, 80 insertions(+), 5 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 6e039bf3..2281c0a6 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -181,6 +181,64 @@ LOG_DEFINE_CATEGORY(SimplePipeline) class SimplePipelineHandler; +struct SimpleFrameInfo { + uint32_t frame; + Request *request; +}; + +class SimpleFrames +{ +public: + void create(Request *request); + void destroy(uint32_t frame); + void clear(); + + SimpleFrameInfo *find(uint32_t frame); + SimpleFrameInfo *find(Request *request); + +private: + std::map frameInfo_; +}; + +void SimpleFrames::create(Request *request) +{ + uint32_t frame = request->sequence(); + auto [it, inserted] = frameInfo_.try_emplace(request->sequence()); + ASSERT(inserted); + it->second.frame = frame; + it->second.request = request; +} + +void SimpleFrames::destroy(uint32_t frame) +{ + frameInfo_.erase(frame); +} + +void SimpleFrames::clear() +{ + frameInfo_.clear(); +} + +SimpleFrameInfo *SimpleFrames::find(uint32_t frame) +{ + auto info = frameInfo_.find(frame); + if (info == frameInfo_.end()) + return nullptr; + return &info->second; +} + +SimpleFrameInfo *SimpleFrames::find(Request *request) +{ + for (auto &itInfo : frameInfo_) { + SimpleFrameInfo *info = &itInfo.second; + + if (info->request == request) + return info; + } + + return nullptr; +} + struct SimplePipelineInfo { const char *driver; /* @@ -293,11 +351,13 @@ public: std::unique_ptr converter_; std::unique_ptr swIsp_; + SimpleFrames frameInfo_; private: void tryPipeline(unsigned int code, const Size &size); static std::vector routedSourcePads(MediaPad *sink); + void completeRequest(Request *request); void conversionInputDone(FrameBuffer *buffer); void conversionOutputDone(FrameBuffer *buffer); @@ -785,7 +845,7 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) /* No conversion, just complete the request. */ Request *request = buffer->request(); pipe->completeBuffer(request, buffer); - pipe->completeRequest(request); + completeRequest(request); return; } @@ -803,7 +863,7 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) const RequestOutputs &outputs = conversionQueue_.front(); for (auto &[stream, buf] : outputs.outputs) pipe->completeBuffer(outputs.request, buf); - pipe->completeRequest(outputs.request); + completeRequest(outputs.request); conversionQueue_.pop(); return; @@ -861,7 +921,7 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) /* Otherwise simply complete the request. */ pipe->completeBuffer(request, buffer); - pipe->completeRequest(request); + completeRequest(request); } void SimpleCameraData::clearIncompleteRequests() @@ -872,6 +932,18 @@ void SimpleCameraData::clearIncompleteRequests() } } +void SimpleCameraData::completeRequest(Request *request) +{ + SimpleFrameInfo *info = frameInfo_.find(request); + if (!info) { + /* Something is really wrong, let's return. */ + return; + } + + frameInfo_.destroy(info->frame); + pipe()->completeRequest(request); +} + void SimpleCameraData::conversionInputDone(FrameBuffer *buffer) { /* Queue the input buffer back for capture. */ @@ -885,7 +957,7 @@ void SimpleCameraData::conversionOutputDone(FrameBuffer *buffer) /* Complete the buffer and the request. */ Request *request = buffer->request(); if (pipe->completeBuffer(request, buffer)) - pipe->completeRequest(request); + completeRequest(request); } void SimpleCameraData::ispStatsReady(uint32_t frame, uint32_t bufferId) @@ -1398,6 +1470,7 @@ void SimplePipelineHandler::stopDevice(Camera *camera) video->bufferReady.disconnect(data, &SimpleCameraData::imageBufferReady); + data->frameInfo_.clear(); data->clearIncompleteRequests(); data->conversionBuffers_.clear(); @@ -1428,8 +1501,10 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request) if (data->useConversion_) { data->conversionQueue_.push({ request, std::move(buffers) }); - if (data->swIsp_) + if (data->swIsp_) { + data->frameInfo_.create(request); data->swIsp_->queueRequest(request->sequence(), request->controls()); + } } return 0; From patchwork Wed Mar 26 12:48:51 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23049 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E4074C3213 for ; Wed, 26 Mar 2025 12:49:13 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9601E68973; Wed, 26 Mar 2025 13:49:13 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="PmqlpqVb"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3AF7468950 for ; Wed, 26 Mar 2025 13:49:11 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1742993350; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kCQ8WKo50d9iz+K9QyKS/uB/T/B/IEndfTAIyagfGBE=; b=PmqlpqVbVtBPtBnYBBpvFgGtmxuym2q3nadHxvvniXtPSP3s7bIB8eLzY3ulGGT+Xv9VF1 Sx/AJm2J0TERct6RI54MBCzSqhXpExw9JZOkDAifpvFkeV4mPSApf2jlAuGVAlmCeXLxiV UaarCXHyUsjQkSuF3nPunJB2bsMT6ts= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-115-y1ol76bXOFuq3dToocBjhg-1; Wed, 26 Mar 2025 08:49:06 -0400 X-MC-Unique: y1ol76bXOFuq3dToocBjhg-1 X-Mimecast-MFC-AGG-ID: y1ol76bXOFuq3dToocBjhg_1742993345 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 70102180AF56; Wed, 26 Mar 2025 12:49:05 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.34.15]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id E79C630001A1; Wed, 26 Mar 2025 12:49:03 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Kieran Bingham , Laurent Pinchart , Milan Zamazal Subject: [PATCH v5 2/6] ipa: simple: softisp: Extend to pass metadata Date: Wed, 26 Mar 2025 13:48:51 +0100 Message-ID: <20250326124856.75709-3-mzamazal@redhat.com> In-Reply-To: <20250326124856.75709-1-mzamazal@redhat.com> References: <20250326124856.75709-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: W52Tajnbs1KqZ9o1Q8_xpg6fUq38FH63iNl5MViXgU4_1742993345 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham Extend the Simple IPA IPC to support returning a metadata ControlList when the process call has completed. A new signal from the IPA is introduced to report the metadata, similarly to what the hardware pipelines do. Merge the metadata reported by the ISP into any completing request to provide to the application. Completion of a request is delayed until this is done; this doesn't apply to canceled requests. Signed-off-by: Kieran Bingham Signed-off-by: Milan Zamazal Reviewed-by: Kieran Bingham --- .../internal/software_isp/software_isp.h | 2 ++ include/libcamera/ipa/soft.mojom | 1 + src/ipa/simple/soft_simple.cpp | 7 +--- src/libcamera/pipeline/simple/simple.cpp | 33 ++++++++++++++++++- src/libcamera/software_isp/software_isp.cpp | 9 +++++ 5 files changed, 45 insertions(+), 7 deletions(-) diff --git a/include/libcamera/internal/software_isp/software_isp.h b/include/libcamera/internal/software_isp/software_isp.h index 0026cec2..1ee37dc7 100644 --- a/include/libcamera/internal/software_isp/software_isp.h +++ b/include/libcamera/internal/software_isp/software_isp.h @@ -85,12 +85,14 @@ public: Signal inputBufferReady; Signal outputBufferReady; Signal ispStatsReady; + Signal metadataReady; Signal setSensorControls; private: void saveIspParams(); void setSensorCtrls(const ControlList &sensorControls); void statsReady(uint32_t frame, uint32_t bufferId); + void saveMetadata(uint32_t frame, const ControlList &metadata); void inputReady(FrameBuffer *input); void outputReady(FrameBuffer *output); diff --git a/include/libcamera/ipa/soft.mojom b/include/libcamera/ipa/soft.mojom index ede74413..a8e6ecda 100644 --- a/include/libcamera/ipa/soft.mojom +++ b/include/libcamera/ipa/soft.mojom @@ -33,4 +33,5 @@ interface IPASoftInterface { interface IPASoftEventInterface { setSensorControls(libcamera.ControlList sensorControls); setIspParams(); + metadataReady(uint32 frame, libcamera.ControlList metadata); }; diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index a87c6cdd..4ef67c43 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -299,15 +299,10 @@ void IPASoftSimple::processStats(const uint32_t frame, int32_t again = sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get(); frameContext.sensor.gain = camHelper_ ? camHelper_->gain(again) : again; - /* - * Software ISP currently does not produce any metadata. Use an empty - * ControlList for now. - * - * \todo Implement proper metadata handling - */ ControlList metadata(controls::controls); for (auto const &algo : algorithms()) algo->process(context_, frame, frameContext, stats_, metadata); + metadataReady.emit(frame, metadata); /* Sanity check */ if (!sensorControls.contains(V4L2_CID_EXPOSURE) || diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 2281c0a6..f34df926 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -184,6 +184,7 @@ class SimplePipelineHandler; struct SimpleFrameInfo { uint32_t frame; Request *request; + bool metadataProcessed; }; class SimpleFrames @@ -207,6 +208,7 @@ void SimpleFrames::create(Request *request) ASSERT(inserted); it->second.frame = frame; it->second.request = request; + it->second.metadataProcessed = false; } void SimpleFrames::destroy(uint32_t frame) @@ -357,11 +359,13 @@ private: void tryPipeline(unsigned int code, const Size &size); static std::vector routedSourcePads(MediaPad *sink); + void tryCompleteRequest(Request *request); void completeRequest(Request *request); void conversionInputDone(FrameBuffer *buffer); void conversionOutputDone(FrameBuffer *buffer); void ispStatsReady(uint32_t frame, uint32_t bufferId); + void metadataReady(uint32_t frame, const ControlList &metadata); void setSensorControls(const ControlList &sensorControls); }; @@ -600,6 +604,7 @@ int SimpleCameraData::init() swIsp_->inputBufferReady.connect(this, &SimpleCameraData::conversionInputDone); swIsp_->outputBufferReady.connect(this, &SimpleCameraData::conversionOutputDone); swIsp_->ispStatsReady.connect(this, &SimpleCameraData::ispStatsReady); + swIsp_->metadataReady.connect(this, &SimpleCameraData::metadataReady); swIsp_->setSensorControls.connect(this, &SimpleCameraData::setSensorControls); } } @@ -932,6 +937,21 @@ void SimpleCameraData::clearIncompleteRequests() } } +void SimpleCameraData::tryCompleteRequest(Request *request) +{ + if (request->hasPendingBuffers()) + return; + + SimpleFrameInfo *info = frameInfo_.find(request); + if (!info) + return; + + if (!info->metadataProcessed) + return; + + completeRequest(request); +} + void SimpleCameraData::completeRequest(Request *request) { SimpleFrameInfo *info = frameInfo_.find(request); @@ -957,7 +977,7 @@ void SimpleCameraData::conversionOutputDone(FrameBuffer *buffer) /* Complete the buffer and the request. */ Request *request = buffer->request(); if (pipe->completeBuffer(request, buffer)) - completeRequest(request); + tryCompleteRequest(request); } void SimpleCameraData::ispStatsReady(uint32_t frame, uint32_t bufferId) @@ -966,6 +986,17 @@ void SimpleCameraData::ispStatsReady(uint32_t frame, uint32_t bufferId) delayedCtrls_->get(frame)); } +void SimpleCameraData::metadataReady(uint32_t frame, const ControlList &metadata) +{ + SimpleFrameInfo *info = frameInfo_.find(frame); + if (!info) + return; + + info->request->metadata().merge(metadata); + info->metadataProcessed = true; + tryCompleteRequest(info->request); +} + void SimpleCameraData::setSensorControls(const ControlList &sensorControls) { delayedCtrls_->push(sensorControls); diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index 4a74dcb6..6baa3fec 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -55,6 +55,11 @@ LOG_DEFINE_CATEGORY(SoftwareIsp) * \brief A signal emitted when the statistics for IPA are ready */ +/** + * \var SoftwareIsp::metadataReady + * \brief A signal emitted when the metadata for IPA is ready + */ + /** * \var SoftwareIsp::setSensorControls * \brief A signal emitted when the values to write to the sensor controls are @@ -141,6 +146,10 @@ SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor, } ipa_->setIspParams.connect(this, &SoftwareIsp::saveIspParams); + ipa_->metadataReady.connect(this, + [this](uint32_t frame, const ControlList &metadata) { + metadataReady.emit(frame, metadata); + }); ipa_->setSensorControls.connect(this, &SoftwareIsp::setSensorCtrls); debayer_->moveToThread(&ispWorkerThread_); From patchwork Wed Mar 26 12:48:52 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23050 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CE9BCC3213 for ; Wed, 26 Mar 2025 12:49:16 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 85C5F68950; Wed, 26 Mar 2025 13:49:16 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="Xq3QOKIS"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DD0B768970 for ; Wed, 26 Mar 2025 13:49:12 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1742993351; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=txkaJYD0Svfhm+by4VOUnRTXlH6ZDr/2prGh2HQ2q1A=; b=Xq3QOKISUn/kTQPxouV9T2+kd3MGx8YA1dsRG4S0CwkoZN++jp9iSx9/ppzdTLjNkbdaGa 7AmU4e1WsMSLBpzosMBBuEovYTgOGbKOm5mnsXtltxvhWrnmMURry12425czouhA7A9X41 k1Kkp3+BHMIgo3+gMoD8kgpodIiKs7E= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-612-2Y6exn_MN0yRJ_m7F3u_Zw-1; Wed, 26 Mar 2025 08:49:08 -0400 X-MC-Unique: 2Y6exn_MN0yRJ_m7F3u_Zw-1 X-Mimecast-MFC-AGG-ID: 2Y6exn_MN0yRJ_m7F3u_Zw_1742993347 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 96E47180AF75; Wed, 26 Mar 2025 12:49:07 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.34.15]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 0F44030001A1; Wed, 26 Mar 2025 12:49:05 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Kieran Bingham , Laurent Pinchart , Milan Zamazal Subject: [PATCH v5 3/6] ipa: simple: Report the ColourGains in metadata Date: Wed, 26 Mar 2025 13:48:52 +0100 Message-ID: <20250326124856.75709-4-mzamazal@redhat.com> In-Reply-To: <20250326124856.75709-1-mzamazal@redhat.com> References: <20250326124856.75709-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: cUuKhj9B0TDfV98NfYu0RX6oX3yaKZjfrWZmW79buZM_1742993347 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham Provide the determined colour gains back into the metadata to add to completed requests. Signed-off-by: Kieran Bingham Signed-off-by: Milan Zamazal Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- src/ipa/simple/algorithms/awb.cpp | 21 ++++++++++++++++++++- src/ipa/simple/algorithms/awb.h | 6 +++++- src/ipa/simple/ipa_context.h | 4 ++++ 3 files changed, 29 insertions(+), 2 deletions(-) diff --git a/src/ipa/simple/algorithms/awb.cpp b/src/ipa/simple/algorithms/awb.cpp index ec77c6e5..55719059 100644 --- a/src/ipa/simple/algorithms/awb.cpp +++ b/src/ipa/simple/algorithms/awb.cpp @@ -17,6 +17,8 @@ #include "libipa/colours.h" #include "simple/ipa_context.h" +#include "control_ids.h" + namespace libcamera { LOG_DEFINE_CATEGORY(IPASoftAwb) @@ -32,15 +34,32 @@ int Awb::configure(IPAContext &context, return 0; } +void Awb::prepare(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + [[maybe_unused]] DebayerParams *params) +{ + auto &gains = context.activeState.awb.gains; + frameContext.gains.red = gains.r(); + frameContext.gains.blue = gains.b(); +} + void Awb::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, + IPAFrameContext &frameContext, const SwIspStats *stats, ControlList &metadata) { const SwIspStats::Histogram &histogram = stats->yHistogram; const uint8_t blackLevel = context.activeState.blc.level; + const float maxGain = 1024.0; + const float mdGains[] = { + static_cast(frameContext.gains.red / maxGain), + static_cast(frameContext.gains.blue / maxGain) + }; + metadata.set(controls::ColourGains, mdGains); + /* * Black level must be subtracted to get the correct AWB ratios, they * would be off if they were computed from the whole brightness range diff --git a/src/ipa/simple/algorithms/awb.h b/src/ipa/simple/algorithms/awb.h index db1496cd..ad993f39 100644 --- a/src/ipa/simple/algorithms/awb.h +++ b/src/ipa/simple/algorithms/awb.h @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: LGPL-2.1-or-later */ /* - * Copyright (C) 2024, Red Hat Inc. + * Copyright (C) 2024-2025 Red Hat Inc. * * Auto white balance */ @@ -20,6 +20,10 @@ public: ~Awb() = default; int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; + void prepare(IPAContext &context, + const uint32_t frame, + IPAFrameContext &frameContext, + DebayerParams *params) override; void process(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, diff --git a/src/ipa/simple/ipa_context.h b/src/ipa/simple/ipa_context.h index 17bcd4ca..bfac835b 100644 --- a/src/ipa/simple/ipa_context.h +++ b/src/ipa/simple/ipa_context.h @@ -70,6 +70,10 @@ struct IPAFrameContext : public FrameContext { int32_t exposure; double gain; } sensor; + struct { + double red; + double blue; + } gains; }; struct IPAContext { From patchwork Wed Mar 26 12:48:53 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23051 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B3602C3213 for ; Wed, 26 Mar 2025 12:49:17 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3B9F768978; Wed, 26 Mar 2025 13:49:17 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="H6SS6N3X"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2D34668950 for ; Wed, 26 Mar 2025 13:49:13 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1742993352; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=r5h0yEOrHyUVK/p/2Lmy0vgyxjmWPIdOBVmgz9dQYx4=; b=H6SS6N3XNR0laAHx3+CZLLd7kpHu3WTnJK52GsDz6sr7aaWCTj9dorqDiUGqUiOmeygVgv M38JhU6Q4Zlyjq7Eu58vqx8w5AVlXhHmCf9nJ0DQQH0FlFNtUReq9IEYHnbjaTsOqQ6Rpm ADkfknkKckUndRjdHg4qle96jmBrJsg= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-393-u-MePDNGMzu2NHCudEvz5Q-1; Wed, 26 Mar 2025 08:49:10 -0400 X-MC-Unique: u-MePDNGMzu2NHCudEvz5Q-1 X-Mimecast-MFC-AGG-ID: u-MePDNGMzu2NHCudEvz5Q_1742993349 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id D548E1800263; Wed, 26 Mar 2025 12:49:09 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.34.15]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 1B26E3001D0F; Wed, 26 Mar 2025 12:49:07 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Kieran Bingham , Laurent Pinchart , Milan Zamazal Subject: [PATCH v5 4/6] ipa: simple: Report black levels in metadata Date: Wed, 26 Mar 2025 13:48:53 +0100 Message-ID: <20250326124856.75709-5-mzamazal@redhat.com> In-Reply-To: <20250326124856.75709-1-mzamazal@redhat.com> References: <20250326124856.75709-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: -xo6tCtNhDnhzurX3Eq7XF6JWtlByQvvPnofSFxG84U_1742993349 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham Provide the determined black level values in the metadata to add to the completed requests. Signed-off-by: Kieran Bingham Signed-off-by: Milan Zamazal Reviewed-by: Laurent Pinchart --- src/ipa/simple/algorithms/blc.cpp | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/src/ipa/simple/algorithms/blc.cpp b/src/ipa/simple/algorithms/blc.cpp index 1d7d370b..53c356c8 100644 --- a/src/ipa/simple/algorithms/blc.cpp +++ b/src/ipa/simple/algorithms/blc.cpp @@ -11,6 +11,8 @@ #include +#include "control_ids.h" + namespace libcamera { namespace ipa::soft::algorithms { @@ -49,8 +51,15 @@ void BlackLevel::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, IPAFrameContext &frameContext, const SwIspStats *stats, - [[maybe_unused]] ControlList &metadata) + ControlList &metadata) { + /* Assign each of the R G G B channels as the same black level. */ + const int32_t blackLevel = context.activeState.blc.level * 256; + const int32_t blackLevels[] = { + blackLevel, blackLevel, blackLevel, blackLevel + }; + metadata.set(controls::SensorBlackLevels, blackLevels); + if (context.configuration.black.level.has_value()) return; From patchwork Wed Mar 26 12:48:54 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23052 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 99C9EC3213 for ; Wed, 26 Mar 2025 12:49:20 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2937F6897E; Wed, 26 Mar 2025 13:49:20 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="SCcBmE+g"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 292756896F for ; Wed, 26 Mar 2025 13:49:17 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1742993356; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=07//BQUffZrBR3eUxozOFm9taj4vtZ3e2VEa1P0y3z0=; b=SCcBmE+gDzLzVEzNYMzodBQfs1kUq5SA5WxRIB8DWmt9XGUBFOBEbWVsOIPUzPlJMwp1VA K8K0czTIuV3coAA4n7kK3m8i7wenaGfppNS7ct7+VZHI/u0A86TxpBeBU4/yIZzlQp7SaU jKwFOjV/3C9amwL7+FJlSgnOSLAv+74= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-106-JjS8ka2cMH6NZS0mm3cFyQ-1; Wed, 26 Mar 2025 08:49:13 -0400 X-MC-Unique: JjS8ka2cMH6NZS0mm3cFyQ-1 X-Mimecast-MFC-AGG-ID: JjS8ka2cMH6NZS0mm3cFyQ_1742993352 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 21CF21800262; Wed, 26 Mar 2025 12:49:12 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.34.15]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 71DF630001A1; Wed, 26 Mar 2025 12:49:10 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham Subject: [PATCH v5 5/6] ipa: simple: Report contrast in metadata Date: Wed, 26 Mar 2025 13:48:54 +0100 Message-ID: <20250326124856.75709-6-mzamazal@redhat.com> In-Reply-To: <20250326124856.75709-1-mzamazal@redhat.com> References: <20250326124856.75709-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: qbbxwXsTGdArcWD3ynCZLIni3b-ZoPUdHSkFMQbzJu4_1742993352 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Provide the requested contrast value, if any, in the metadata to add to the completed requests. Signed-off-by: Milan Zamazal Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- src/ipa/simple/algorithms/lut.cpp | 15 ++++++++++++++- src/ipa/simple/algorithms/lut.h | 5 +++++ src/ipa/simple/ipa_context.h | 1 + 3 files changed, 20 insertions(+), 1 deletion(-) diff --git a/src/ipa/simple/algorithms/lut.cpp b/src/ipa/simple/algorithms/lut.cpp index a06cdeba..e8638f27 100644 --- a/src/ipa/simple/algorithms/lut.cpp +++ b/src/ipa/simple/algorithms/lut.cpp @@ -87,9 +87,11 @@ int16_t Lut::ccmValue(unsigned int i, float ccm) const void Lut::prepare(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, + IPAFrameContext &frameContext, DebayerParams *params) { + frameContext.contrast = context.activeState.knobs.contrast; + /* * Update the gamma table if needed. This means if black level changes * and since the black level gets updated only if a lower value is @@ -139,6 +141,17 @@ void Lut::prepare(IPAContext &context, } } +void Lut::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + [[maybe_unused]] const SwIspStats *stats, + ControlList &metadata) +{ + const auto &contrast = frameContext.contrast; + if (contrast) + metadata.set(controls::Contrast, contrast.value()); +} + REGISTER_IPA_ALGORITHM(Lut, "Lut") } /* namespace ipa::soft::algorithms */ diff --git a/src/ipa/simple/algorithms/lut.h b/src/ipa/simple/algorithms/lut.h index 77324800..ba8b9021 100644 --- a/src/ipa/simple/algorithms/lut.h +++ b/src/ipa/simple/algorithms/lut.h @@ -30,6 +30,11 @@ public: const uint32_t frame, IPAFrameContext &frameContext, DebayerParams *params) override; + void process(IPAContext &context, + const uint32_t frame, + IPAFrameContext &frameContext, + const SwIspStats *stats, + ControlList &metadata) override; private: void updateGammaTable(IPAContext &context); diff --git a/src/ipa/simple/ipa_context.h b/src/ipa/simple/ipa_context.h index bfac835b..10d539f5 100644 --- a/src/ipa/simple/ipa_context.h +++ b/src/ipa/simple/ipa_context.h @@ -74,6 +74,7 @@ struct IPAFrameContext : public FrameContext { double red; double blue; } gains; + std::optional contrast; }; struct IPAContext { From patchwork Wed Mar 26 12:48:55 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23053 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4CAC5C3213 for ; Wed, 26 Mar 2025 12:49:22 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id ED75D6897C; Wed, 26 Mar 2025 13:49:21 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="NcvksSNt"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0558B68975 for ; Wed, 26 Mar 2025 13:49:17 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1742993357; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=d9Fm2hojPhcDYQvIamge0t9vFxP3zk4HeS1kz1Jkhic=; b=NcvksSNthOkrMftGqRg45eKQuaTTvXuSFohiY/zfDAIuCTSn59kZPwOLiTqr8DzuXgs8H+ zX5lWagzVPu5g1Xm+ejhTCrvMQsGesbbJ36aadNzdY6IH8aSayc+9HiliCHeXbol1pJiTG bNyQDONcG36Hd6fUexiq7QFHuoBL/40= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-156-xIJMofn0MhOe4LSsl6lvjQ-1; Wed, 26 Mar 2025 08:49:15 -0400 X-MC-Unique: xIJMofn0MhOe4LSsl6lvjQ-1 X-Mimecast-MFC-AGG-ID: xIJMofn0MhOe4LSsl6lvjQ_1742993354 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 48870196D2CC; Wed, 26 Mar 2025 12:49:14 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.34.15]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 9FA6D3001D0F; Wed, 26 Mar 2025 12:49:12 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham Subject: [PATCH v5 6/6] ipa: simple: Report exposure in metadata Date: Wed, 26 Mar 2025 13:48:55 +0100 Message-ID: <20250326124856.75709-7-mzamazal@redhat.com> In-Reply-To: <20250326124856.75709-1-mzamazal@redhat.com> References: <20250326124856.75709-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: x51ZkBIDaCSw6qj8F39UrL5tW-xdaUFlrm0CX46aOvE_1742993354 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Report exposure and gain in metadata. This is more complicated than it could be expected because the exposure value should be in microseconds but it's handled using V4L2_CID_EXPOSURE control, which doesn't specify the unit, see https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/control.html. So the unit conversion is done in the way rkisp1 IPA uses. This requires getting and passing IPACameraSensorInfo around. To avoid naming confusion and to improve consistency with rkisp1 IPA, sensorCtrlInfoMap parameter is renamed to sensorControls. Signed-off-by: Milan Zamazal Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- include/libcamera/ipa/soft.mojom | 3 ++- src/ipa/simple/algorithms/agc.cpp | 11 +++++++++-- src/ipa/simple/ipa_context.h | 6 +++++- src/ipa/simple/soft_simple.cpp | 17 +++++++++++++---- src/libcamera/software_isp/software_isp.cpp | 20 ++++++++++++++------ 5 files changed, 43 insertions(+), 14 deletions(-) diff --git a/include/libcamera/ipa/soft.mojom b/include/libcamera/ipa/soft.mojom index a8e6ecda..77328c5f 100644 --- a/include/libcamera/ipa/soft.mojom +++ b/include/libcamera/ipa/soft.mojom @@ -16,7 +16,8 @@ interface IPASoftInterface { init(libcamera.IPASettings settings, libcamera.SharedFD fdStats, libcamera.SharedFD fdParams, - libcamera.ControlInfoMap sensorCtrlInfoMap) + libcamera.IPACameraSensorInfo sensorInfo, + libcamera.ControlInfoMap sensorControls) => (int32 ret, libcamera.ControlInfoMap ipaControls, bool ccmEnabled); start() => (int32 ret); stop(); diff --git a/src/ipa/simple/algorithms/agc.cpp b/src/ipa/simple/algorithms/agc.cpp index 72aade14..c46bb0eb 100644 --- a/src/ipa/simple/algorithms/agc.cpp +++ b/src/ipa/simple/algorithms/agc.cpp @@ -11,6 +11,8 @@ #include +#include "control_ids.h" + namespace libcamera { LOG_DEFINE_CATEGORY(IPASoftExposure) @@ -97,10 +99,15 @@ void Agc::updateExposure(IPAContext &context, IPAFrameContext &frameContext, dou void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, + IPAFrameContext &frameContext, const SwIspStats *stats, - [[maybe_unused]] ControlList &metadata) + ControlList &metadata) { + utils::Duration exposureTime = + context.configuration.agc.lineDuration * frameContext.sensor.exposure; + metadata.set(controls::ExposureTime, exposureTime.get()); + metadata.set(controls::AnalogueGain, frameContext.sensor.gain); + /* * Calculate Mean Sample Value (MSV) according to formula from: * https://www.araa.asn.au/acra/acra2007/papers/paper84final.pdf diff --git a/src/ipa/simple/ipa_context.h b/src/ipa/simple/ipa_context.h index 10d539f5..7dc2cd7a 100644 --- a/src/ipa/simple/ipa_context.h +++ b/src/ipa/simple/ipa_context.h @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: LGPL-2.1-or-later */ /* - * Copyright (C) 2024 Red Hat, Inc. + * Copyright (C) 2024-2025 Red Hat, Inc. * * Simple pipeline IPA Context */ @@ -18,6 +18,8 @@ #include +#include "core_ipa_interface.h" + namespace libcamera { namespace ipa::soft { @@ -27,6 +29,7 @@ struct IPASessionConfiguration { struct { int32_t exposureMin, exposureMax; double againMin, againMax, againMinStep; + utils::Duration lineDuration; } agc; struct { std::optional level; @@ -83,6 +86,7 @@ struct IPAContext { { } + IPACameraSensorInfo sensorInfo; IPASessionConfiguration configuration; IPAActiveState activeState; FCQueue frameContexts; diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index 4ef67c43..c94c4cd5 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -5,6 +5,7 @@ * Simple Software Image Processing Algorithm module */ +#include #include #include @@ -32,6 +33,8 @@ namespace libcamera { LOG_DEFINE_CATEGORY(IPASoft) +using namespace std::literals::chrono_literals; + namespace ipa::soft { /* Maximum number of frame contexts to be held */ @@ -50,7 +53,8 @@ public: int init(const IPASettings &settings, const SharedFD &fdStats, const SharedFD &fdParams, - const ControlInfoMap &sensorInfoMap, + const IPACameraSensorInfo &sensorInfo, + const ControlInfoMap &sensorControls, ControlInfoMap *ipaControls, bool *ccmEnabled) override; int configure(const IPAConfigInfo &configInfo) override; @@ -89,7 +93,8 @@ IPASoftSimple::~IPASoftSimple() int IPASoftSimple::init(const IPASettings &settings, const SharedFD &fdStats, const SharedFD &fdParams, - const ControlInfoMap &sensorInfoMap, + const IPACameraSensorInfo &sensorInfo, + const ControlInfoMap &sensorControls, ControlInfoMap *ipaControls, bool *ccmEnabled) { @@ -100,6 +105,8 @@ int IPASoftSimple::init(const IPASettings &settings, << settings.sensorModel; } + context_.sensorInfo = sensorInfo; + /* Load the tuning data file */ File file(settings.configurationFile); if (!file.open(File::OpenModeFlag::ReadOnly)) { @@ -173,12 +180,12 @@ int IPASoftSimple::init(const IPASettings &settings, * Don't save the min and max control values yet, as e.g. the limits * for V4L2_CID_EXPOSURE depend on the configured sensor resolution. */ - if (sensorInfoMap.find(V4L2_CID_EXPOSURE) == sensorInfoMap.end()) { + if (sensorControls.find(V4L2_CID_EXPOSURE) == sensorControls.end()) { LOG(IPASoft, Error) << "Don't have exposure control"; return -EINVAL; } - if (sensorInfoMap.find(V4L2_CID_ANALOGUE_GAIN) == sensorInfoMap.end()) { + if (sensorControls.find(V4L2_CID_ANALOGUE_GAIN) == sensorControls.end()) { LOG(IPASoft, Error) << "Don't have gain control"; return -EINVAL; } @@ -198,6 +205,8 @@ int IPASoftSimple::configure(const IPAConfigInfo &configInfo) context_.activeState = {}; context_.frameContexts.clear(); + context_.configuration.agc.lineDuration = + context_.sensorInfo.minLineLength * 1.0s / context_.sensorInfo.pixelRate; context_.configuration.agc.exposureMin = exposureInfo.min().get(); context_.configuration.agc.exposureMax = exposureInfo.max().get(); if (!context_.configuration.agc.exposureMin) { diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index 6baa3fec..28e2a360 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -133,12 +133,20 @@ SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor, std::string ipaTuningFile = ipa_->configurationFile(sensor->model() + ".yaml", "uncalibrated.yaml"); - int ret = ipa_->init(IPASettings{ ipaTuningFile, sensor->model() }, - debayer_->getStatsFD(), - sharedParams_.fd(), - sensor->controls(), - ipaControls, - &ccmEnabled_); + IPACameraSensorInfo sensorInfo{}; + int ret = sensor->sensorInfo(&sensorInfo); + if (ret) { + LOG(SoftwareIsp, Error) << "Camera sensor information not available"; + return; + } + + ret = ipa_->init(IPASettings{ ipaTuningFile, sensor->model() }, + debayer_->getStatsFD(), + sharedParams_.fd(), + sensorInfo, + sensor->controls(), + ipaControls, + &ccmEnabled_); if (ret) { LOG(SoftwareIsp, Error) << "IPA init failed"; debayer_.reset();