From patchwork Wed Mar 26 12:48:51 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23049 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E4074C3213 for ; Wed, 26 Mar 2025 12:49:13 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9601E68973; Wed, 26 Mar 2025 13:49:13 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="PmqlpqVb"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3AF7468950 for ; Wed, 26 Mar 2025 13:49:11 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1742993350; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kCQ8WKo50d9iz+K9QyKS/uB/T/B/IEndfTAIyagfGBE=; b=PmqlpqVbVtBPtBnYBBpvFgGtmxuym2q3nadHxvvniXtPSP3s7bIB8eLzY3ulGGT+Xv9VF1 Sx/AJm2J0TERct6RI54MBCzSqhXpExw9JZOkDAifpvFkeV4mPSApf2jlAuGVAlmCeXLxiV UaarCXHyUsjQkSuF3nPunJB2bsMT6ts= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-115-y1ol76bXOFuq3dToocBjhg-1; Wed, 26 Mar 2025 08:49:06 -0400 X-MC-Unique: y1ol76bXOFuq3dToocBjhg-1 X-Mimecast-MFC-AGG-ID: y1ol76bXOFuq3dToocBjhg_1742993345 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 70102180AF56; Wed, 26 Mar 2025 12:49:05 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.34.15]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id E79C630001A1; Wed, 26 Mar 2025 12:49:03 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Kieran Bingham , Laurent Pinchart , Milan Zamazal Subject: [PATCH v5 2/6] ipa: simple: softisp: Extend to pass metadata Date: Wed, 26 Mar 2025 13:48:51 +0100 Message-ID: <20250326124856.75709-3-mzamazal@redhat.com> In-Reply-To: <20250326124856.75709-1-mzamazal@redhat.com> References: <20250326124856.75709-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: W52Tajnbs1KqZ9o1Q8_xpg6fUq38FH63iNl5MViXgU4_1742993345 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham Extend the Simple IPA IPC to support returning a metadata ControlList when the process call has completed. A new signal from the IPA is introduced to report the metadata, similarly to what the hardware pipelines do. Merge the metadata reported by the ISP into any completing request to provide to the application. Completion of a request is delayed until this is done; this doesn't apply to canceled requests. Signed-off-by: Kieran Bingham Signed-off-by: Milan Zamazal Reviewed-by: Kieran Bingham --- .../internal/software_isp/software_isp.h | 2 ++ include/libcamera/ipa/soft.mojom | 1 + src/ipa/simple/soft_simple.cpp | 7 +--- src/libcamera/pipeline/simple/simple.cpp | 33 ++++++++++++++++++- src/libcamera/software_isp/software_isp.cpp | 9 +++++ 5 files changed, 45 insertions(+), 7 deletions(-) diff --git a/include/libcamera/internal/software_isp/software_isp.h b/include/libcamera/internal/software_isp/software_isp.h index 0026cec2..1ee37dc7 100644 --- a/include/libcamera/internal/software_isp/software_isp.h +++ b/include/libcamera/internal/software_isp/software_isp.h @@ -85,12 +85,14 @@ public: Signal inputBufferReady; Signal outputBufferReady; Signal ispStatsReady; + Signal metadataReady; Signal setSensorControls; private: void saveIspParams(); void setSensorCtrls(const ControlList &sensorControls); void statsReady(uint32_t frame, uint32_t bufferId); + void saveMetadata(uint32_t frame, const ControlList &metadata); void inputReady(FrameBuffer *input); void outputReady(FrameBuffer *output); diff --git a/include/libcamera/ipa/soft.mojom b/include/libcamera/ipa/soft.mojom index ede74413..a8e6ecda 100644 --- a/include/libcamera/ipa/soft.mojom +++ b/include/libcamera/ipa/soft.mojom @@ -33,4 +33,5 @@ interface IPASoftInterface { interface IPASoftEventInterface { setSensorControls(libcamera.ControlList sensorControls); setIspParams(); + metadataReady(uint32 frame, libcamera.ControlList metadata); }; diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index a87c6cdd..4ef67c43 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -299,15 +299,10 @@ void IPASoftSimple::processStats(const uint32_t frame, int32_t again = sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get(); frameContext.sensor.gain = camHelper_ ? camHelper_->gain(again) : again; - /* - * Software ISP currently does not produce any metadata. Use an empty - * ControlList for now. - * - * \todo Implement proper metadata handling - */ ControlList metadata(controls::controls); for (auto const &algo : algorithms()) algo->process(context_, frame, frameContext, stats_, metadata); + metadataReady.emit(frame, metadata); /* Sanity check */ if (!sensorControls.contains(V4L2_CID_EXPOSURE) || diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 2281c0a6..f34df926 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -184,6 +184,7 @@ class SimplePipelineHandler; struct SimpleFrameInfo { uint32_t frame; Request *request; + bool metadataProcessed; }; class SimpleFrames @@ -207,6 +208,7 @@ void SimpleFrames::create(Request *request) ASSERT(inserted); it->second.frame = frame; it->second.request = request; + it->second.metadataProcessed = false; } void SimpleFrames::destroy(uint32_t frame) @@ -357,11 +359,13 @@ private: void tryPipeline(unsigned int code, const Size &size); static std::vector routedSourcePads(MediaPad *sink); + void tryCompleteRequest(Request *request); void completeRequest(Request *request); void conversionInputDone(FrameBuffer *buffer); void conversionOutputDone(FrameBuffer *buffer); void ispStatsReady(uint32_t frame, uint32_t bufferId); + void metadataReady(uint32_t frame, const ControlList &metadata); void setSensorControls(const ControlList &sensorControls); }; @@ -600,6 +604,7 @@ int SimpleCameraData::init() swIsp_->inputBufferReady.connect(this, &SimpleCameraData::conversionInputDone); swIsp_->outputBufferReady.connect(this, &SimpleCameraData::conversionOutputDone); swIsp_->ispStatsReady.connect(this, &SimpleCameraData::ispStatsReady); + swIsp_->metadataReady.connect(this, &SimpleCameraData::metadataReady); swIsp_->setSensorControls.connect(this, &SimpleCameraData::setSensorControls); } } @@ -932,6 +937,21 @@ void SimpleCameraData::clearIncompleteRequests() } } +void SimpleCameraData::tryCompleteRequest(Request *request) +{ + if (request->hasPendingBuffers()) + return; + + SimpleFrameInfo *info = frameInfo_.find(request); + if (!info) + return; + + if (!info->metadataProcessed) + return; + + completeRequest(request); +} + void SimpleCameraData::completeRequest(Request *request) { SimpleFrameInfo *info = frameInfo_.find(request); @@ -957,7 +977,7 @@ void SimpleCameraData::conversionOutputDone(FrameBuffer *buffer) /* Complete the buffer and the request. */ Request *request = buffer->request(); if (pipe->completeBuffer(request, buffer)) - completeRequest(request); + tryCompleteRequest(request); } void SimpleCameraData::ispStatsReady(uint32_t frame, uint32_t bufferId) @@ -966,6 +986,17 @@ void SimpleCameraData::ispStatsReady(uint32_t frame, uint32_t bufferId) delayedCtrls_->get(frame)); } +void SimpleCameraData::metadataReady(uint32_t frame, const ControlList &metadata) +{ + SimpleFrameInfo *info = frameInfo_.find(frame); + if (!info) + return; + + info->request->metadata().merge(metadata); + info->metadataProcessed = true; + tryCompleteRequest(info->request); +} + void SimpleCameraData::setSensorControls(const ControlList &sensorControls) { delayedCtrls_->push(sensorControls); diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index 4a74dcb6..6baa3fec 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -55,6 +55,11 @@ LOG_DEFINE_CATEGORY(SoftwareIsp) * \brief A signal emitted when the statistics for IPA are ready */ +/** + * \var SoftwareIsp::metadataReady + * \brief A signal emitted when the metadata for IPA is ready + */ + /** * \var SoftwareIsp::setSensorControls * \brief A signal emitted when the values to write to the sensor controls are @@ -141,6 +146,10 @@ SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor, } ipa_->setIspParams.connect(this, &SoftwareIsp::saveIspParams); + ipa_->metadataReady.connect(this, + [this](uint32_t frame, const ControlList &metadata) { + metadataReady.emit(frame, metadata); + }); ipa_->setSensorControls.connect(this, &SoftwareIsp::setSensorCtrls); debayer_->moveToThread(&ispWorkerThread_);