From patchwork Thu Dec 19 21:10:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 22419 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C49C4BD7D8 for ; Thu, 19 Dec 2024 21:10:31 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 412D868483; Thu, 19 Dec 2024 22:10:31 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="iOfRrU84"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6E16368471 for ; Thu, 19 Dec 2024 22:10:28 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734642627; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EMGQn+N/13YiKmyFMD99W6zD+mBu5cFtDRapzLfd4oc=; b=iOfRrU84ViiH5MPfhXlbNgyy0OhfmfvQDLzP7PrU6K/3ZcLd4Ngs52teSBUu9xwc0Q96pP /JuLdYMdwIaiIl1kLv7H15JfNlaqFZTNC38CAzziHdmNwA89E8Rr/bBdfDn4Wqs8h7j6dQ ApEqimKmT2pmHNLUKeR7YZ3yUL2w+fU= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-78-xm_P_HR-OiWtVuhrJBPyGg-1; Thu, 19 Dec 2024 16:10:25 -0500 X-MC-Unique: xm_P_HR-OiWtVuhrJBPyGg-1 X-Mimecast-MFC-AGG-ID: xm_P_HR-OiWtVuhrJBPyGg Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id C95A91956046; Thu, 19 Dec 2024 21:10:24 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.224.37]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id E8B0730044C1; Thu, 19 Dec 2024 21:10:22 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Kieran Bingham , Laurent Pinchart Subject: [PATCH v2 1/5] libcamera: software_isp: Track frames and requests Date: Thu, 19 Dec 2024 22:10:06 +0100 Message-ID: <20241219211010.103310-2-mzamazal@redhat.com> In-Reply-To: <20241219211010.103310-1-mzamazal@redhat.com> References: <20241219211010.103310-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: J9kojeCltSwfSnkiaVqSzI_Xbq5EiazZRpdzLoL7YLE_1734642624 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Hardware pipelines track requests and other information related to particular frames. This hasn't been needed in software ISP so far. But in order to be able to track metadata corresponding to a given frame, frame-request tracking mechanism starts being useful. This patch introduces the basic tracking structure, actual metadata handling is added in the following patch. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 87 ++++++++++++++++++++++-- 1 file changed, 82 insertions(+), 5 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 8ac24e6e..07c1cf1f 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -181,6 +181,70 @@ LOG_DEFINE_CATEGORY(SimplePipeline) class SimplePipelineHandler; +struct SimpleFrameInfo { + uint32_t frame; + Request *request; +}; + +class SimpleFrames +{ +public: + void create(Request *request); + void destroy(uint32_t frame); + void clear(); + + SimpleFrameInfo *find(uint32_t frame); + SimpleFrameInfo *find(Request *request); + +private: + std::map frameInfo_; +}; + +void SimpleFrames::create(Request *request) +{ + uint32_t frame = request->sequence(); + SimpleFrameInfo *info = new SimpleFrameInfo; + info->frame = frame; + info->request = request; + frameInfo_[frame] = info; +} + +void SimpleFrames::destroy(uint32_t frame) +{ + SimpleFrameInfo *info = find(frame); + if (!info) + return; + delete info; + frameInfo_.erase(frame); +} + +void SimpleFrames::clear() +{ + for (const auto &info : frameInfo_) + delete info.second; + frameInfo_.clear(); +} + +SimpleFrameInfo *SimpleFrames::find(uint32_t frame) +{ + auto info = frameInfo_.find(frame); + if (info == frameInfo_.end()) + return nullptr; + return info->second; +} + +SimpleFrameInfo *SimpleFrames::find(Request *request) +{ + for (auto &itInfo : frameInfo_) { + SimpleFrameInfo *info = itInfo.second; + + if (info->request == request) + return info; + } + + return nullptr; +} + struct SimplePipelineInfo { const char *driver; /* @@ -293,11 +357,13 @@ public: std::unique_ptr converter_; std::unique_ptr swIsp_; + SimpleFrames frameInfo_; private: void tryPipeline(unsigned int code, const Size &size); static std::vector routedSourcePads(MediaPad *sink); + void completeRequest(Request *request); void conversionInputDone(FrameBuffer *buffer); void conversionOutputDone(FrameBuffer *buffer); @@ -799,7 +865,7 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) /* No conversion, just complete the request. */ Request *request = buffer->request(); pipe->completeBuffer(request, buffer); - pipe->completeRequest(request); + completeRequest(request); return; } @@ -817,7 +883,7 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) const RequestOutputs &outputs = conversionQueue_.front(); for (auto &[stream, buf] : outputs.outputs) pipe->completeBuffer(outputs.request, buf); - pipe->completeRequest(outputs.request); + completeRequest(outputs.request); conversionQueue_.pop(); return; @@ -875,7 +941,7 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) /* Otherwise simply complete the request. */ pipe->completeBuffer(request, buffer); - pipe->completeRequest(request); + completeRequest(request); } void SimpleCameraData::clearIncompleteRequests() @@ -886,6 +952,14 @@ void SimpleCameraData::clearIncompleteRequests() } } +void SimpleCameraData::completeRequest(Request *request) +{ + SimpleFrameInfo *info = frameInfo_.find(request); + if (info) + frameInfo_.destroy(info->frame); + pipe()->completeRequest(request); +} + void SimpleCameraData::conversionInputDone(FrameBuffer *buffer) { /* Queue the input buffer back for capture. */ @@ -899,7 +973,7 @@ void SimpleCameraData::conversionOutputDone(FrameBuffer *buffer) /* Complete the buffer and the request. */ Request *request = buffer->request(); if (pipe->completeBuffer(request, buffer)) - pipe->completeRequest(request); + completeRequest(request); } void SimpleCameraData::ispStatsReady(uint32_t frame, uint32_t bufferId) @@ -1412,6 +1486,7 @@ void SimplePipelineHandler::stopDevice(Camera *camera) video->bufferReady.disconnect(data, &SimpleCameraData::imageBufferReady); + data->frameInfo_.clear(); data->clearIncompleteRequests(); data->conversionBuffers_.clear(); @@ -1442,8 +1517,10 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request) if (data->useConversion_) { data->conversionQueue_.push({ request, std::move(buffers) }); - if (data->swIsp_) + if (data->swIsp_) { + data->frameInfo_.create(request); data->swIsp_->queueRequest(request->sequence(), request->controls()); + } } return 0; From patchwork Thu Dec 19 21:10:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 22420 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EC7CEC3309 for ; Thu, 19 Dec 2024 21:10:33 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8204F68482; Thu, 19 Dec 2024 22:10:33 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="MIIZsQ2s"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 273CD68482 for ; Thu, 19 Dec 2024 22:10:31 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734642629; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=c5j7ZxE7aLGIhbwvU5fl9+xHHhhD2qdrdl+yM6OIXsA=; b=MIIZsQ2sNWUIBGJTDnoA1Gvf+irtslkdALNdZuuehZEUmjQXX7tH5WafU0ExBG00LLv1uw 5tle8iyuDCFd97aPSeQwv+RY0UiaNBM92HpxJNIKnc2a9qir1Mr0aETld6iDE4GColgn5p cHn5wd58HaGMd0XkTCbMNN4hhUSZSm0= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-456-idY7D-KvO4-8ZlPrhl-Zew-1; Thu, 19 Dec 2024 16:10:28 -0500 X-MC-Unique: idY7D-KvO4-8ZlPrhl-Zew-1 X-Mimecast-MFC-AGG-ID: idY7D-KvO4-8ZlPrhl-Zew Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 1260219560AF; Thu, 19 Dec 2024 21:10:27 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.224.37]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 54F4C30044C1; Thu, 19 Dec 2024 21:10:25 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Kieran Bingham , Laurent Pinchart , Milan Zamazal Subject: [PATCH v2 2/5] ipa: simple: softisp: Extend to pass metadata Date: Thu, 19 Dec 2024 22:10:07 +0100 Message-ID: <20241219211010.103310-3-mzamazal@redhat.com> In-Reply-To: <20241219211010.103310-1-mzamazal@redhat.com> References: <20241219211010.103310-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: BQcPcT_CACZOn92IHe3GAhRpRt5nwRWTFVYuxqXznHU_1734642627 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham Extend the Simple IPA IPC to support returning a metadata ControlList when the process call has completed. A new signal from the IPA is introduced to report the metadata, similarly to what the hardware pipelines do. Merge the metadata reported by the ISP into any completing request to provide to the application. Completion of a request is delayed until this is done; this doesn't apply to canceled requests. Signed-off-by: Kieran Bingham Signed-off-by: Milan Zamazal --- .../internal/software_isp/software_isp.h | 2 ++ include/libcamera/ipa/soft.mojom | 1 + src/ipa/simple/soft_simple.cpp | 7 +--- src/libcamera/pipeline/simple/simple.cpp | 35 +++++++++++++++---- src/libcamera/software_isp/software_isp.cpp | 11 ++++++ 5 files changed, 43 insertions(+), 13 deletions(-) diff --git a/include/libcamera/internal/software_isp/software_isp.h b/include/libcamera/internal/software_isp/software_isp.h index d51b03fd..3d248aba 100644 --- a/include/libcamera/internal/software_isp/software_isp.h +++ b/include/libcamera/internal/software_isp/software_isp.h @@ -83,12 +83,14 @@ public: Signal inputBufferReady; Signal outputBufferReady; Signal ispStatsReady; + Signal metadataReady; Signal setSensorControls; private: void saveIspParams(); void setSensorCtrls(const ControlList &sensorControls); void statsReady(uint32_t frame, uint32_t bufferId); + void saveMetadata(uint32_t frame, const ControlList &metadata); void inputReady(FrameBuffer *input); void outputReady(FrameBuffer *output); diff --git a/include/libcamera/ipa/soft.mojom b/include/libcamera/ipa/soft.mojom index d52e6f1a..934a6fd1 100644 --- a/include/libcamera/ipa/soft.mojom +++ b/include/libcamera/ipa/soft.mojom @@ -33,4 +33,5 @@ interface IPASoftInterface { interface IPASoftEventInterface { setSensorControls(libcamera.ControlList sensorControls); setIspParams(); + metadataReady(uint32 frame, libcamera.ControlList metadata); }; diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index b26e4e15..944c644e 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -295,15 +295,10 @@ void IPASoftSimple::processStats(const uint32_t frame, int32_t again = sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get(); frameContext.sensor.gain = camHelper_ ? camHelper_->gain(again) : again; - /* - * Software ISP currently does not produce any metadata. Use an empty - * ControlList for now. - * - * \todo Implement proper metadata handling - */ ControlList metadata(controls::controls); for (auto const &algo : algorithms()) algo->process(context_, frame, frameContext, stats_, metadata); + metadataReady.emit(frame, metadata); /* Sanity check */ if (!sensorControls.contains(V4L2_CID_EXPOSURE) || diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 07c1cf1f..1438bb22 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -184,6 +184,7 @@ class SimplePipelineHandler; struct SimpleFrameInfo { uint32_t frame; Request *request; + bool metadataProcessed; }; class SimpleFrames @@ -206,6 +207,7 @@ void SimpleFrames::create(Request *request) SimpleFrameInfo *info = new SimpleFrameInfo; info->frame = frame; info->request = request; + info->metadataProcessed = false; frameInfo_[frame] = info; } @@ -363,11 +365,12 @@ private: void tryPipeline(unsigned int code, const Size &size); static std::vector routedSourcePads(MediaPad *sink); - void completeRequest(Request *request); + void completeRequest(Request *request, bool checkCompleted); void conversionInputDone(FrameBuffer *buffer); void conversionOutputDone(FrameBuffer *buffer); void ispStatsReady(uint32_t frame, uint32_t bufferId); + void metadataReady(uint32_t frame, const ControlList &metadata); void setSensorControls(const ControlList &sensorControls); }; @@ -620,6 +623,7 @@ int SimpleCameraData::init() }); swIsp_->outputBufferReady.connect(this, &SimpleCameraData::conversionOutputDone); swIsp_->ispStatsReady.connect(this, &SimpleCameraData::ispStatsReady); + swIsp_->metadataReady.connect(this, &SimpleCameraData::metadataReady); swIsp_->setSensorControls.connect(this, &SimpleCameraData::setSensorControls); } } @@ -865,7 +869,7 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) /* No conversion, just complete the request. */ Request *request = buffer->request(); pipe->completeBuffer(request, buffer); - completeRequest(request); + completeRequest(request, false); return; } @@ -883,7 +887,7 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) const RequestOutputs &outputs = conversionQueue_.front(); for (auto &[stream, buf] : outputs.outputs) pipe->completeBuffer(outputs.request, buf); - completeRequest(outputs.request); + completeRequest(outputs.request, false); conversionQueue_.pop(); return; @@ -941,7 +945,7 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) /* Otherwise simply complete the request. */ pipe->completeBuffer(request, buffer); - completeRequest(request); + completeRequest(request, false); } void SimpleCameraData::clearIncompleteRequests() @@ -952,11 +956,17 @@ void SimpleCameraData::clearIncompleteRequests() } } -void SimpleCameraData::completeRequest(Request *request) +void SimpleCameraData::completeRequest(Request *request, bool checkCompleted) { + if (checkCompleted && request->hasPendingBuffers()) + return; + SimpleFrameInfo *info = frameInfo_.find(request); - if (info) + if (info) { + if (checkCompleted && !info->metadataProcessed) + return; frameInfo_.destroy(info->frame); + } pipe()->completeRequest(request); } @@ -973,7 +983,7 @@ void SimpleCameraData::conversionOutputDone(FrameBuffer *buffer) /* Complete the buffer and the request. */ Request *request = buffer->request(); if (pipe->completeBuffer(request, buffer)) - completeRequest(request); + completeRequest(request, true); } void SimpleCameraData::ispStatsReady(uint32_t frame, uint32_t bufferId) @@ -982,6 +992,17 @@ void SimpleCameraData::ispStatsReady(uint32_t frame, uint32_t bufferId) delayedCtrls_->get(frame)); } +void SimpleCameraData::metadataReady(uint32_t frame, const ControlList &metadata) +{ + SimpleFrameInfo *info = frameInfo_.find(frame); + if (!info) + return; + + info->request->metadata().merge(metadata); + info->metadataProcessed = true; + completeRequest(info->request, true); +} + void SimpleCameraData::setSensorControls(const ControlList &sensorControls) { delayedCtrls_->push(sensorControls); diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index 2bea64d9..bd4c25cc 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -51,6 +51,11 @@ LOG_DEFINE_CATEGORY(SoftwareIsp) * \brief A signal emitted when the statistics for IPA are ready */ +/** + * \var SoftwareIsp::metadataReady + * \brief A signal emitted when the metadata for IPA are ready + */ + /** * \var SoftwareIsp::setSensorControls * \brief A signal emitted when the values to write to the sensor controls are @@ -136,6 +141,7 @@ SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor, } ipa_->setIspParams.connect(this, &SoftwareIsp::saveIspParams); + ipa_->metadataReady.connect(this, &SoftwareIsp::saveMetadata); ipa_->setSensorControls.connect(this, &SoftwareIsp::setSensorCtrls); debayer_->moveToThread(&ispWorkerThread_); @@ -357,6 +363,11 @@ void SoftwareIsp::statsReady(uint32_t frame, uint32_t bufferId) ispStatsReady.emit(frame, bufferId); } +void SoftwareIsp::saveMetadata(uint32_t frame, const ControlList &metadata) +{ + metadataReady.emit(frame, metadata); +} + void SoftwareIsp::inputReady(FrameBuffer *input) { inputBufferReady.emit(input); From patchwork Thu Dec 19 21:10:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 22421 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7C464BD7D8 for ; Thu, 19 Dec 2024 21:10:38 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1621F68482; Thu, 19 Dec 2024 22:10:38 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="S8nKrmQJ"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 73C7068483 for ; Thu, 19 Dec 2024 22:10:34 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734642633; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Q88QYC47SQ9p6s7ro2hqAMhJcn/ZImIqRmqY0+UKbfU=; b=S8nKrmQJFdLqkTMhgzkENy4HmGA3tT6T0Yt2OPSGh6vpqaeAKiRD4TpWz3xC6RmSlf8v64 pglzlJfz2FRSkJtRk6FX8vEa45FKxO9ROHNHf7lVnABCC9vURElQD/OSYzWy3x+1i10FUx JnrjJm1Qu/kHCfDqgWTYrdtmKftlTag= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-621-Sf7AasC2MxKcMdqrigOChA-1; Thu, 19 Dec 2024 16:10:30 -0500 X-MC-Unique: Sf7AasC2MxKcMdqrigOChA-1 X-Mimecast-MFC-AGG-ID: Sf7AasC2MxKcMdqrigOChA Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 0F0BE19560A1; Thu, 19 Dec 2024 21:10:29 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.224.37]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id A229E30044C1; Thu, 19 Dec 2024 21:10:27 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Kieran Bingham , Laurent Pinchart Subject: [PATCH v2 3/5] ipa: simple: Report the ColourGains in metadata Date: Thu, 19 Dec 2024 22:10:08 +0100 Message-ID: <20241219211010.103310-4-mzamazal@redhat.com> In-Reply-To: <20241219211010.103310-1-mzamazal@redhat.com> References: <20241219211010.103310-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: N7-ru4cm-XJC-Oi0slSkttkxrofu5B0-IJt_eWqjTpQ_1734642629 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham Provide the determined colour gains back into the metadata to add to completed requests. Metadata must be set before computing the new gain values in order to report the metadata used to process the image rather than the metadata determined from the image (and to be used for processing the next image). Signed-off-by: Kieran Bingham --- src/ipa/simple/algorithms/awb.cpp | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/src/ipa/simple/algorithms/awb.cpp b/src/ipa/simple/algorithms/awb.cpp index 195de41d..95ff4434 100644 --- a/src/ipa/simple/algorithms/awb.cpp +++ b/src/ipa/simple/algorithms/awb.cpp @@ -14,6 +14,8 @@ #include "simple/ipa_context.h" +#include "control_ids.h" + namespace libcamera { LOG_DEFINE_CATEGORY(IPASoftAwb) @@ -33,10 +35,16 @@ void Awb::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, [[maybe_unused]] IPAFrameContext &frameContext, const SwIspStats *stats, - [[maybe_unused]] ControlList &metadata) + ControlList &metadata) { const SwIspStats::Histogram &histogram = stats->yHistogram; const uint8_t blackLevel = context.activeState.blc.level; + auto &gains = context.activeState.gains; + + const float maxGain = 1024.0; + const float mdGains[] = { static_cast(gains.red / maxGain), + static_cast(gains.blue / maxGain) }; + metadata.set(controls::ColourGains, mdGains); /* * Black level must be subtracted to get the correct AWB ratios, they @@ -54,7 +62,6 @@ void Awb::process(IPAContext &context, * Calculate red and blue gains for AWB. * Clamp max gain at 4.0, this also avoids 0 division. */ - auto &gains = context.activeState.gains; gains.red = sumR <= sumG / 4 ? 4.0 : static_cast(sumG) / sumR; gains.blue = sumB <= sumG / 4 ? 4.0 : static_cast(sumG) / sumB; /* Green gain is fixed to 1.0 */ From patchwork Thu Dec 19 21:10:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 22422 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 238D2C330A for ; Thu, 19 Dec 2024 21:10:41 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A1D4B6848B; Thu, 19 Dec 2024 22:10:40 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="ZFO9tsMF"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7277268480 for ; Thu, 19 Dec 2024 22:10:36 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734642635; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tNwTlbdunWD81spcI8ViHFWeu26EmXM8el1o5/75Now=; b=ZFO9tsMFhP4PpmX1xy876e0HwStgipXJydateit7hCQ8t20BR8EZ3BksraCnrjJxd3nTZ5 +hU5Ijfjowp5YZ3qSs9qynlEpwOFtVNjASXqD/1o0n/gRzIsd72iofXVCdboyya+0N8J+k XtddwifMp3RJ6fA6ZJUGAMkKHI+qJu0= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-34-Mncjq_R9Ngq4p7cPtU7ovA-1; Thu, 19 Dec 2024 16:10:32 -0500 X-MC-Unique: Mncjq_R9Ngq4p7cPtU7ovA-1 X-Mimecast-MFC-AGG-ID: Mncjq_R9Ngq4p7cPtU7ovA Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 5CDCB19560BC; Thu, 19 Dec 2024 21:10:31 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.224.37]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 9D1E5300F9B5; Thu, 19 Dec 2024 21:10:29 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Kieran Bingham , Laurent Pinchart , Milan Zamazal Subject: [PATCH v2 4/5] ipa: simple: Report black levels in metadata Date: Thu, 19 Dec 2024 22:10:09 +0100 Message-ID: <20241219211010.103310-5-mzamazal@redhat.com> In-Reply-To: <20241219211010.103310-1-mzamazal@redhat.com> References: <20241219211010.103310-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: P7iTrLL_8GtAkzpfdPhYblnQSyoPvagDtII3AxirQN4_1734642631 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham Provide the determined black level values in the metadata to add to the completed requests. Signed-off-by: Kieran Bingham Signed-off-by: Milan Zamazal --- src/ipa/simple/algorithms/blc.cpp | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/src/ipa/simple/algorithms/blc.cpp b/src/ipa/simple/algorithms/blc.cpp index 1d7d370b..d5759f92 100644 --- a/src/ipa/simple/algorithms/blc.cpp +++ b/src/ipa/simple/algorithms/blc.cpp @@ -11,6 +11,8 @@ #include +#include "control_ids.h" + namespace libcamera { namespace ipa::soft::algorithms { @@ -49,8 +51,13 @@ void BlackLevel::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, IPAFrameContext &frameContext, const SwIspStats *stats, - [[maybe_unused]] ControlList &metadata) + ControlList &metadata) { + /* Assign each of the R G G B channels as the same black level. */ + const int32_t blackLevel = context.activeState.blc.level * 256; + const int32_t blackLevels[] = { blackLevel, blackLevel, blackLevel, blackLevel }; + metadata.set(controls::SensorBlackLevels, blackLevels); + if (context.configuration.black.level.has_value()) return; From patchwork Thu Dec 19 21:10:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 22423 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 82791C3312 for ; Thu, 19 Dec 2024 21:10:42 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8612F6848D; Thu, 19 Dec 2024 22:10:41 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="e2JfuUZj"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E5AAB68480 for ; Thu, 19 Dec 2024 22:10:37 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734642637; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tQM+GajCGK8QL/RdzTuLWiiBSm8L6ggAWVvPgB+kSeg=; b=e2JfuUZjhVZKJnMUAiKKCTqqsS9+IRfn3wTjZDsFl9PpmdvA3EgEz7G4Rbzm3RPT1qYLRt t7TqxwpnOVVlZyzwndHWEG8FNESHk9KnvIaz5Y9y7gwTrgFjVJABhkdX3+iLRjpYzA4I1h VE3yqxHQr6H2nSBqs/t8ckabyZgFp40= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-670-ksjzk513PEKy3GDluAmUhg-1; Thu, 19 Dec 2024 16:10:34 -0500 X-MC-Unique: ksjzk513PEKy3GDluAmUhg-1 X-Mimecast-MFC-AGG-ID: ksjzk513PEKy3GDluAmUhg Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 8357E1956077; Thu, 19 Dec 2024 21:10:33 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.224.37]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id D138030044C1; Thu, 19 Dec 2024 21:10:31 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Kieran Bingham , Laurent Pinchart Subject: [PATCH v2 5/5] ipa: simple: Report contrast in metadata Date: Thu, 19 Dec 2024 22:10:10 +0100 Message-ID: <20241219211010.103310-6-mzamazal@redhat.com> In-Reply-To: <20241219211010.103310-1-mzamazal@redhat.com> References: <20241219211010.103310-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: Z9r3kUqiWdm3m2v2j95ox1hcPWf5PyvJoJplfYgu8wY_1734642633 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Provide the requested contrast value, if any, in the metadata to add to the completed requests. Signed-off-by: Milan Zamazal --- src/ipa/simple/algorithms/lut.cpp | 11 +++++++++++ src/ipa/simple/algorithms/lut.h | 5 +++++ 2 files changed, 16 insertions(+) diff --git a/src/ipa/simple/algorithms/lut.cpp b/src/ipa/simple/algorithms/lut.cpp index 0ba2391f..31208bfe 100644 --- a/src/ipa/simple/algorithms/lut.cpp +++ b/src/ipa/simple/algorithms/lut.cpp @@ -116,6 +116,17 @@ void Lut::prepare(IPAContext &context, } } +void Lut::process(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + [[maybe_unused]] const SwIspStats *stats, + ControlList &metadata) +{ + const auto contrast = context.activeState.knobs.contrast; + if (contrast) + metadata.set(controls::Contrast, contrast.value()); +} + REGISTER_IPA_ALGORITHM(Lut, "Lut") } /* namespace ipa::soft::algorithms */ diff --git a/src/ipa/simple/algorithms/lut.h b/src/ipa/simple/algorithms/lut.h index 889f864b..ab4e1094 100644 --- a/src/ipa/simple/algorithms/lut.h +++ b/src/ipa/simple/algorithms/lut.h @@ -30,6 +30,11 @@ public: const uint32_t frame, IPAFrameContext &frameContext, DebayerParams *params) override; + void process(IPAContext &context, + const uint32_t frame, + IPAFrameContext &frameContext, + const SwIspStats *stats, + ControlList &metadata) override; private: void updateGammaTable(IPAContext &context);