From patchwork Thu Jan 30 17:32:48 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 22692 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BB6D3C324C for ; Thu, 30 Jan 2025 17:33:13 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B73106856E; Thu, 30 Jan 2025 18:33:12 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="b1dT6FV5"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7F97168559 for ; Thu, 30 Jan 2025 18:33:09 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1738258388; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JsJK5TzQ2MRpZTNymsDFwD1Wt8wjT3TJfSgbZhPdRdk=; b=b1dT6FV5GSYn0lJ2RmbB+aNPy5OjC3RCFv3v1ZezTyNxFuMgf1jTsKJU7wKduKjsKs9jFH UnvOu+46miOHlCO/TPFL7cSzgKQfx2GvPy7rHhnVLjf3T+dvHqCSX4SKh9ZW3wlr2CIB6P vQMOS4niRzmuRCJgxrBUaYQMmlqZhrY= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-581-5BqbX7IlMlqP1NyD2hPwIg-1; Thu, 30 Jan 2025 12:33:04 -0500 X-MC-Unique: 5BqbX7IlMlqP1NyD2hPwIg-1 X-Mimecast-MFC-AGG-ID: 5BqbX7IlMlqP1NyD2hPwIg Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id EB12D19560AF; Thu, 30 Jan 2025 17:33:02 +0000 (UTC) Received: from mzamazal-thinkpadp1gen3.tpbc.com (unknown [10.45.224.122]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id EB26E1956094; Thu, 30 Jan 2025 17:33:00 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= Subject: [PATCH v4 1/6] libcamera: software_isp: Track frames and requests Date: Thu, 30 Jan 2025 18:32:48 +0100 Message-ID: <20250130173254.112895-2-mzamazal@redhat.com> In-Reply-To: <20250130173254.112895-1-mzamazal@redhat.com> References: <20250130173254.112895-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: DEJPoibluMg4b3b8Bcwy_wOy0eVbKI8sARCTf43RPIM_1738258383 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Hardware pipelines track requests and other information related to particular frames. This hasn't been needed in software ISP so far. But in order to be able to track metadata corresponding to a given frame, frame-request tracking mechanism starts being useful. This patch introduces the basic tracking structure, actual metadata handling is added in the following patch. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 85 ++++++++++++++++++++++-- 1 file changed, 80 insertions(+), 5 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 8ac24e6e..2cfa4e42 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -181,6 +181,64 @@ LOG_DEFINE_CATEGORY(SimplePipeline) class SimplePipelineHandler; +struct SimpleFrameInfo { + uint32_t frame; + Request *request; +}; + +class SimpleFrames +{ +public: + void create(Request *request); + void destroy(uint32_t frame); + void clear(); + + SimpleFrameInfo *find(uint32_t frame); + SimpleFrameInfo *find(Request *request); + +private: + std::map frameInfo_; +}; + +void SimpleFrames::create(Request *request) +{ + uint32_t frame = request->sequence(); + auto [it, inserted] = frameInfo_.try_emplace(request->sequence()); + ASSERT(inserted); + it->second.frame = frame; + it->second.request = request; +} + +void SimpleFrames::destroy(uint32_t frame) +{ + frameInfo_.erase(frame); +} + +void SimpleFrames::clear() +{ + frameInfo_.clear(); +} + +SimpleFrameInfo *SimpleFrames::find(uint32_t frame) +{ + auto info = frameInfo_.find(frame); + if (info == frameInfo_.end()) + return nullptr; + return &info->second; +} + +SimpleFrameInfo *SimpleFrames::find(Request *request) +{ + for (auto &itInfo : frameInfo_) { + SimpleFrameInfo *info = &itInfo.second; + + if (info->request == request) + return info; + } + + return nullptr; +} + struct SimplePipelineInfo { const char *driver; /* @@ -293,11 +351,13 @@ public: std::unique_ptr converter_; std::unique_ptr swIsp_; + SimpleFrames frameInfo_; private: void tryPipeline(unsigned int code, const Size &size); static std::vector routedSourcePads(MediaPad *sink); + void completeRequest(Request *request); void conversionInputDone(FrameBuffer *buffer); void conversionOutputDone(FrameBuffer *buffer); @@ -799,7 +859,7 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) /* No conversion, just complete the request. */ Request *request = buffer->request(); pipe->completeBuffer(request, buffer); - pipe->completeRequest(request); + completeRequest(request); return; } @@ -817,7 +877,7 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) const RequestOutputs &outputs = conversionQueue_.front(); for (auto &[stream, buf] : outputs.outputs) pipe->completeBuffer(outputs.request, buf); - pipe->completeRequest(outputs.request); + completeRequest(outputs.request); conversionQueue_.pop(); return; @@ -875,7 +935,7 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) /* Otherwise simply complete the request. */ pipe->completeBuffer(request, buffer); - pipe->completeRequest(request); + completeRequest(request); } void SimpleCameraData::clearIncompleteRequests() @@ -886,6 +946,18 @@ void SimpleCameraData::clearIncompleteRequests() } } +void SimpleCameraData::completeRequest(Request *request) +{ + SimpleFrameInfo *info = frameInfo_.find(request); + if (!info) { + /* Something is really wrong, let's return. */ + return; + } + + frameInfo_.destroy(info->frame); + pipe()->completeRequest(request); +} + void SimpleCameraData::conversionInputDone(FrameBuffer *buffer) { /* Queue the input buffer back for capture. */ @@ -899,7 +971,7 @@ void SimpleCameraData::conversionOutputDone(FrameBuffer *buffer) /* Complete the buffer and the request. */ Request *request = buffer->request(); if (pipe->completeBuffer(request, buffer)) - pipe->completeRequest(request); + completeRequest(request); } void SimpleCameraData::ispStatsReady(uint32_t frame, uint32_t bufferId) @@ -1412,6 +1484,7 @@ void SimplePipelineHandler::stopDevice(Camera *camera) video->bufferReady.disconnect(data, &SimpleCameraData::imageBufferReady); + data->frameInfo_.clear(); data->clearIncompleteRequests(); data->conversionBuffers_.clear(); @@ -1442,8 +1515,10 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request) if (data->useConversion_) { data->conversionQueue_.push({ request, std::move(buffers) }); - if (data->swIsp_) + if (data->swIsp_) { + data->frameInfo_.create(request); data->swIsp_->queueRequest(request->sequence(), request->controls()); + } } return 0; From patchwork Thu Jan 30 17:32:49 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 22691 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D46C9C324C for ; Thu, 30 Jan 2025 17:33:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9AEA86856D; Thu, 30 Jan 2025 18:33:11 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="I1EVhSOL"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 19AFD6851B for ; Thu, 30 Jan 2025 18:33:08 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1738258388; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2cTa3usip+dwFGx34k7MZC74ToWhGA/1Bqt03BkAitY=; b=I1EVhSOL6XgxGXpajpMX5BcHaGz2CGp/Vahk3vk0fOV/ljlH1hGcFBdG5x2nQ1KIGOZfuS wcuIXIfh9OZVMvNT/6tc1NYeFXWzl7nKYXzlCJncXF3LaRCt0E2IbXluJmtrjf1oxxcJuQ IVNVoK7LGPbSt9vkFXCoWMwVZZszNf8= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-605-fErskd6xNV6PDtWZN30LEw-1; Thu, 30 Jan 2025 12:33:06 -0500 X-MC-Unique: fErskd6xNV6PDtWZN30LEw-1 X-Mimecast-MFC-AGG-ID: fErskd6xNV6PDtWZN30LEw Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 8CC691955D80; Thu, 30 Jan 2025 17:33:05 +0000 (UTC) Received: from mzamazal-thinkpadp1gen3.tpbc.com (unknown [10.45.224.122]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 7B1991956094; Thu, 30 Jan 2025 17:33:03 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Kieran Bingham , Laurent Pinchart , =?utf-8?b?QmFybmFiw6FzIFDFkWN6?= =?utf-8?q?e?= , Milan Zamazal Subject: [PATCH v4 2/6] ipa: simple: softisp: Extend to pass metadata Date: Thu, 30 Jan 2025 18:32:49 +0100 Message-ID: <20250130173254.112895-3-mzamazal@redhat.com> In-Reply-To: <20250130173254.112895-1-mzamazal@redhat.com> References: <20250130173254.112895-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: 3cyw63fitNl8_ZlKDSf9rxzNnMsGQ_H0BwYJxzkgayo_1738258385 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham Extend the Simple IPA IPC to support returning a metadata ControlList when the process call has completed. A new signal from the IPA is introduced to report the metadata, similarly to what the hardware pipelines do. Merge the metadata reported by the ISP into any completing request to provide to the application. Completion of a request is delayed until this is done; this doesn't apply to canceled requests. Signed-off-by: Kieran Bingham Signed-off-by: Milan Zamazal --- .../internal/software_isp/software_isp.h | 2 ++ include/libcamera/ipa/soft.mojom | 1 + src/ipa/simple/soft_simple.cpp | 7 +--- src/libcamera/pipeline/simple/simple.cpp | 33 ++++++++++++++++++- src/libcamera/software_isp/software_isp.cpp | 9 +++++ 5 files changed, 45 insertions(+), 7 deletions(-) diff --git a/include/libcamera/internal/software_isp/software_isp.h b/include/libcamera/internal/software_isp/software_isp.h index d51b03fd..3d248aba 100644 --- a/include/libcamera/internal/software_isp/software_isp.h +++ b/include/libcamera/internal/software_isp/software_isp.h @@ -83,12 +83,14 @@ public: Signal inputBufferReady; Signal outputBufferReady; Signal ispStatsReady; + Signal metadataReady; Signal setSensorControls; private: void saveIspParams(); void setSensorCtrls(const ControlList &sensorControls); void statsReady(uint32_t frame, uint32_t bufferId); + void saveMetadata(uint32_t frame, const ControlList &metadata); void inputReady(FrameBuffer *input); void outputReady(FrameBuffer *output); diff --git a/include/libcamera/ipa/soft.mojom b/include/libcamera/ipa/soft.mojom index d52e6f1a..934a6fd1 100644 --- a/include/libcamera/ipa/soft.mojom +++ b/include/libcamera/ipa/soft.mojom @@ -33,4 +33,5 @@ interface IPASoftInterface { interface IPASoftEventInterface { setSensorControls(libcamera.ControlList sensorControls); setIspParams(); + metadataReady(uint32 frame, libcamera.ControlList metadata); }; diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index b26e4e15..944c644e 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -295,15 +295,10 @@ void IPASoftSimple::processStats(const uint32_t frame, int32_t again = sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get(); frameContext.sensor.gain = camHelper_ ? camHelper_->gain(again) : again; - /* - * Software ISP currently does not produce any metadata. Use an empty - * ControlList for now. - * - * \todo Implement proper metadata handling - */ ControlList metadata(controls::controls); for (auto const &algo : algorithms()) algo->process(context_, frame, frameContext, stats_, metadata); + metadataReady.emit(frame, metadata); /* Sanity check */ if (!sensorControls.contains(V4L2_CID_EXPOSURE) || diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 2cfa4e42..126db216 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -184,6 +184,7 @@ class SimplePipelineHandler; struct SimpleFrameInfo { uint32_t frame; Request *request; + bool metadataProcessed; }; class SimpleFrames @@ -207,6 +208,7 @@ void SimpleFrames::create(Request *request) ASSERT(inserted); it->second.frame = frame; it->second.request = request; + it->second.metadataProcessed = false; } void SimpleFrames::destroy(uint32_t frame) @@ -357,11 +359,13 @@ private: void tryPipeline(unsigned int code, const Size &size); static std::vector routedSourcePads(MediaPad *sink); + void tryCompleteRequest(Request *request); void completeRequest(Request *request); void conversionInputDone(FrameBuffer *buffer); void conversionOutputDone(FrameBuffer *buffer); void ispStatsReady(uint32_t frame, uint32_t bufferId); + void metadataReady(uint32_t frame, const ControlList &metadata); void setSensorControls(const ControlList &sensorControls); }; @@ -614,6 +618,7 @@ int SimpleCameraData::init() }); swIsp_->outputBufferReady.connect(this, &SimpleCameraData::conversionOutputDone); swIsp_->ispStatsReady.connect(this, &SimpleCameraData::ispStatsReady); + swIsp_->metadataReady.connect(this, &SimpleCameraData::metadataReady); swIsp_->setSensorControls.connect(this, &SimpleCameraData::setSensorControls); } } @@ -946,6 +951,21 @@ void SimpleCameraData::clearIncompleteRequests() } } +void SimpleCameraData::tryCompleteRequest(Request *request) +{ + if (request->hasPendingBuffers()) + return; + + SimpleFrameInfo *info = frameInfo_.find(request); + if (!info) + return; + + if (!info->metadataProcessed) + return; + + completeRequest(request); +} + void SimpleCameraData::completeRequest(Request *request) { SimpleFrameInfo *info = frameInfo_.find(request); @@ -971,7 +991,7 @@ void SimpleCameraData::conversionOutputDone(FrameBuffer *buffer) /* Complete the buffer and the request. */ Request *request = buffer->request(); if (pipe->completeBuffer(request, buffer)) - completeRequest(request); + tryCompleteRequest(request); } void SimpleCameraData::ispStatsReady(uint32_t frame, uint32_t bufferId) @@ -980,6 +1000,17 @@ void SimpleCameraData::ispStatsReady(uint32_t frame, uint32_t bufferId) delayedCtrls_->get(frame)); } +void SimpleCameraData::metadataReady(uint32_t frame, const ControlList &metadata) +{ + SimpleFrameInfo *info = frameInfo_.find(frame); + if (!info) + return; + + info->request->metadata().merge(metadata); + info->metadataProcessed = true; + tryCompleteRequest(info->request); +} + void SimpleCameraData::setSensorControls(const ControlList &sensorControls) { delayedCtrls_->push(sensorControls); diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index 44baf200..92f87acc 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -51,6 +51,11 @@ LOG_DEFINE_CATEGORY(SoftwareIsp) * \brief A signal emitted when the statistics for IPA are ready */ +/** + * \var SoftwareIsp::metadataReady + * \brief A signal emitted when the metadata for IPA is ready + */ + /** * \var SoftwareIsp::setSensorControls * \brief A signal emitted when the values to write to the sensor controls are @@ -136,6 +141,10 @@ SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor, } ipa_->setIspParams.connect(this, &SoftwareIsp::saveIspParams); + ipa_->metadataReady.connect(this, + [this](uint32_t frame, const ControlList &metadata) { + metadataReady.emit(frame, metadata); + }); ipa_->setSensorControls.connect(this, &SoftwareIsp::setSensorCtrls); debayer_->moveToThread(&ispWorkerThread_); From patchwork Thu Jan 30 17:32:50 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 22693 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B71EBC324C for ; Thu, 30 Jan 2025 17:33:16 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4A5B36856B; Thu, 30 Jan 2025 18:33:16 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="QhWoSaH7"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 96D5B6851B for ; Thu, 30 Jan 2025 18:33:11 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1738258390; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=H65FlS9UoNSiJcLrYf2GTy7oyK2WGvU+zKYtaQMlPgw=; b=QhWoSaH7enfzi00mrNs6PxnthTvEf+L4QTp8rm5j12CbYhNLD75/I26wJoTVOlHrZ5XvDa M1boiHKPTAeQSxyOsBq1P4gPw2Gt3/5tuXz+H4KUDKim6rRaFXNre+o/dR9Dqg1hmgVD2X 55YVFR7esnVQk8VIIiTGlqbgwzpQ7JQ= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-158-CqTav2mpMn-_LpOa1djgDA-1; Thu, 30 Jan 2025 12:33:09 -0500 X-MC-Unique: CqTav2mpMn-_LpOa1djgDA-1 X-Mimecast-MFC-AGG-ID: CqTav2mpMn-_LpOa1djgDA Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id E45AA1801F1F; Thu, 30 Jan 2025 17:33:07 +0000 (UTC) Received: from mzamazal-thinkpadp1gen3.tpbc.com (unknown [10.45.224.122]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 192001956094; Thu, 30 Jan 2025 17:33:05 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Kieran Bingham , Laurent Pinchart , =?utf-8?b?QmFybmFiw6FzIFDFkWN6?= =?utf-8?q?e?= , Milan Zamazal Subject: [PATCH v4 3/6] ipa: simple: Report the ColourGains in metadata Date: Thu, 30 Jan 2025 18:32:50 +0100 Message-ID: <20250130173254.112895-4-mzamazal@redhat.com> In-Reply-To: <20250130173254.112895-1-mzamazal@redhat.com> References: <20250130173254.112895-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: 9te4f1KAtozlbw63YdLwWoHNDchdqMDRieYgf0LV2t4_1738258388 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham Provide the determined colour gains back into the metadata to add to completed requests. Signed-off-by: Kieran Bingham Signed-off-by: Milan Zamazal --- src/ipa/simple/algorithms/awb.cpp | 23 +++++++++++++++++++++-- src/ipa/simple/algorithms/awb.h | 6 +++++- src/ipa/simple/ipa_context.h | 4 ++++ 3 files changed, 30 insertions(+), 3 deletions(-) diff --git a/src/ipa/simple/algorithms/awb.cpp b/src/ipa/simple/algorithms/awb.cpp index 195de41d..3b4146f1 100644 --- a/src/ipa/simple/algorithms/awb.cpp +++ b/src/ipa/simple/algorithms/awb.cpp @@ -14,6 +14,8 @@ #include "simple/ipa_context.h" +#include "control_ids.h" + namespace libcamera { LOG_DEFINE_CATEGORY(IPASoftAwb) @@ -29,15 +31,32 @@ int Awb::configure(IPAContext &context, return 0; } +void Awb::prepare(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + [[maybe_unused]] DebayerParams *params) +{ + auto &gains = context.activeState.gains; + frameContext.gains.red = gains.red; + frameContext.gains.blue = gains.blue; +} + void Awb::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, + IPAFrameContext &frameContext, const SwIspStats *stats, - [[maybe_unused]] ControlList &metadata) + ControlList &metadata) { const SwIspStats::Histogram &histogram = stats->yHistogram; const uint8_t blackLevel = context.activeState.blc.level; + const float maxGain = 1024.0; + const float mdGains[] = { + static_cast(frameContext.gains.red / maxGain), + static_cast(frameContext.gains.blue / maxGain) + }; + metadata.set(controls::ColourGains, mdGains); + /* * Black level must be subtracted to get the correct AWB ratios, they * would be off if they were computed from the whole brightness range diff --git a/src/ipa/simple/algorithms/awb.h b/src/ipa/simple/algorithms/awb.h index db1496cd..ad993f39 100644 --- a/src/ipa/simple/algorithms/awb.h +++ b/src/ipa/simple/algorithms/awb.h @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: LGPL-2.1-or-later */ /* - * Copyright (C) 2024, Red Hat Inc. + * Copyright (C) 2024-2025 Red Hat Inc. * * Auto white balance */ @@ -20,6 +20,10 @@ public: ~Awb() = default; int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; + void prepare(IPAContext &context, + const uint32_t frame, + IPAFrameContext &frameContext, + DebayerParams *params) override; void process(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, diff --git a/src/ipa/simple/ipa_context.h b/src/ipa/simple/ipa_context.h index 4af51306..99cb6284 100644 --- a/src/ipa/simple/ipa_context.h +++ b/src/ipa/simple/ipa_context.h @@ -58,6 +58,10 @@ struct IPAFrameContext : public FrameContext { int32_t exposure; double gain; } sensor; + struct { + double red; + double blue; + } gains; }; struct IPAContext { From patchwork Thu Jan 30 17:32:51 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 22694 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D0CD8C324C for ; Thu, 30 Jan 2025 17:33:19 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6DA806856E; Thu, 30 Jan 2025 18:33:19 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="COKD5WQe"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7691D68563 for ; Thu, 30 Jan 2025 18:33:14 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1738258393; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GK4OYP263HK0zpqaizaAGtJcLiUKdaz1IBRqRbsoPmo=; b=COKD5WQefDIGa7xO6txinInUTNx2aEyUUSwDhU1I8h77IPI9xU/sJgW+D3FbDCoPFJk27Y yxWTlxxIsH6Kfo+uqECZLqY0J49JfNpcwMiJxGHrL+HrZqldxjxpLgm5Se4L9FwFmOncog auEqpxWqVd0b0sSjBCMmGMYyeD94ByE= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-646-CI87SxSkOzmUGR7J1D2k9w-1; Thu, 30 Jan 2025 12:33:11 -0500 X-MC-Unique: CI87SxSkOzmUGR7J1D2k9w-1 X-Mimecast-MFC-AGG-ID: CI87SxSkOzmUGR7J1D2k9w Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 922A219560BB; Thu, 30 Jan 2025 17:33:10 +0000 (UTC) Received: from mzamazal-thinkpadp1gen3.tpbc.com (unknown [10.45.224.122]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 7F8901956094; Thu, 30 Jan 2025 17:33:08 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Kieran Bingham , Laurent Pinchart , =?utf-8?b?QmFybmFiw6FzIFDFkWN6?= =?utf-8?q?e?= , Milan Zamazal Subject: [PATCH v4 4/6] ipa: simple: Report black levels in metadata Date: Thu, 30 Jan 2025 18:32:51 +0100 Message-ID: <20250130173254.112895-5-mzamazal@redhat.com> In-Reply-To: <20250130173254.112895-1-mzamazal@redhat.com> References: <20250130173254.112895-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: Z5J6H7IOHj_xln8H-pQkEo88ASpd-I6NfNfrc_WOdX0_1738258390 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham Provide the determined black level values in the metadata to add to the completed requests. Signed-off-by: Kieran Bingham Signed-off-by: Milan Zamazal Reviewed-by: Laurent Pinchart --- src/ipa/simple/algorithms/blc.cpp | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/src/ipa/simple/algorithms/blc.cpp b/src/ipa/simple/algorithms/blc.cpp index 1d7d370b..53c356c8 100644 --- a/src/ipa/simple/algorithms/blc.cpp +++ b/src/ipa/simple/algorithms/blc.cpp @@ -11,6 +11,8 @@ #include +#include "control_ids.h" + namespace libcamera { namespace ipa::soft::algorithms { @@ -49,8 +51,15 @@ void BlackLevel::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, IPAFrameContext &frameContext, const SwIspStats *stats, - [[maybe_unused]] ControlList &metadata) + ControlList &metadata) { + /* Assign each of the R G G B channels as the same black level. */ + const int32_t blackLevel = context.activeState.blc.level * 256; + const int32_t blackLevels[] = { + blackLevel, blackLevel, blackLevel, blackLevel + }; + metadata.set(controls::SensorBlackLevels, blackLevels); + if (context.configuration.black.level.has_value()) return; From patchwork Thu Jan 30 17:32:52 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 22695 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EC146C324C for ; Thu, 30 Jan 2025 17:33:21 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 92D0268574; Thu, 30 Jan 2025 18:33:21 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="EtYtGTHt"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DA2F56856B for ; Thu, 30 Jan 2025 18:33:18 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1738258397; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TJHT5fRyj/79eLk8f6RKIl+xdWx6KHugvTFF9EnoIgo=; b=EtYtGTHtp2ZqX5CZjflb6dlAHiLiC4dgBI2TC4oypHuV6Kj10D4XknbWGAFTtsrJDBfq/2 iEvqHxmCFBLTRzQTrg0dLpXMT/6JyUXMd2Yl5BmmiaQUnlUTLWxlpleDTpkQGjGKZw5TwG jcxWkRgV5ShOZOiD7ZeKZzexY7WgUps= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-48-q8NCYwhZO3uijFRx0FzGGg-1; Thu, 30 Jan 2025 12:33:14 -0500 X-MC-Unique: q8NCYwhZO3uijFRx0FzGGg-1 X-Mimecast-MFC-AGG-ID: q8NCYwhZO3uijFRx0FzGGg Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 0240119560BA; Thu, 30 Jan 2025 17:33:13 +0000 (UTC) Received: from mzamazal-thinkpadp1gen3.tpbc.com (unknown [10.45.224.122]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 22CE11956094; Thu, 30 Jan 2025 17:33:10 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= Subject: [PATCH v4 5/6] ipa: simple: Report contrast in metadata Date: Thu, 30 Jan 2025 18:32:52 +0100 Message-ID: <20250130173254.112895-6-mzamazal@redhat.com> In-Reply-To: <20250130173254.112895-1-mzamazal@redhat.com> References: <20250130173254.112895-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: 7Bfo1zMldiLiQ-wo1ZciTIt-vKCqhU5zqgeEbZG2WE8_1738258393 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Provide the requested contrast value, if any, in the metadata to add to the completed requests. Signed-off-by: Milan Zamazal --- src/ipa/simple/algorithms/lut.cpp | 17 +++++++++++++++-- src/ipa/simple/algorithms/lut.h | 5 +++++ src/ipa/simple/ipa_context.h | 1 + 3 files changed, 21 insertions(+), 2 deletions(-) diff --git a/src/ipa/simple/algorithms/lut.cpp b/src/ipa/simple/algorithms/lut.cpp index 0ba2391f..a6a77b7c 100644 --- a/src/ipa/simple/algorithms/lut.cpp +++ b/src/ipa/simple/algorithms/lut.cpp @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: LGPL-2.1-or-later */ /* - * Copyright (C) 2024, Red Hat Inc. + * Copyright (C) 2024-2025 Red Hat Inc. * * Color lookup tables construction */ @@ -82,9 +82,11 @@ void Lut::updateGammaTable(IPAContext &context) void Lut::prepare(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, + IPAFrameContext &frameContext, [[maybe_unused]] DebayerParams *params) { + frameContext.contrast = context.activeState.knobs.contrast; + /* * Update the gamma table if needed. This means if black level changes * and since the black level gets updated only if a lower value is @@ -116,6 +118,17 @@ void Lut::prepare(IPAContext &context, } } +void Lut::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + [[maybe_unused]] const SwIspStats *stats, + ControlList &metadata) +{ + const auto &contrast = frameContext.contrast; + if (contrast) + metadata.set(controls::Contrast, contrast.value()); +} + REGISTER_IPA_ALGORITHM(Lut, "Lut") } /* namespace ipa::soft::algorithms */ diff --git a/src/ipa/simple/algorithms/lut.h b/src/ipa/simple/algorithms/lut.h index 889f864b..ab4e1094 100644 --- a/src/ipa/simple/algorithms/lut.h +++ b/src/ipa/simple/algorithms/lut.h @@ -30,6 +30,11 @@ public: const uint32_t frame, IPAFrameContext &frameContext, DebayerParams *params) override; + void process(IPAContext &context, + const uint32_t frame, + IPAFrameContext &frameContext, + const SwIspStats *stats, + ControlList &metadata) override; private: void updateGammaTable(IPAContext &context); diff --git a/src/ipa/simple/ipa_context.h b/src/ipa/simple/ipa_context.h index 99cb6284..07fd641a 100644 --- a/src/ipa/simple/ipa_context.h +++ b/src/ipa/simple/ipa_context.h @@ -62,6 +62,7 @@ struct IPAFrameContext : public FrameContext { double red; double blue; } gains; + std::optional contrast; }; struct IPAContext { From patchwork Thu Jan 30 17:32:53 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 22696 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CA463C324C for ; Thu, 30 Jan 2025 17:33:23 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 824B368576; Thu, 30 Jan 2025 18:33:23 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="QeBZHDoX"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DD4EF68573 for ; Thu, 30 Jan 2025 18:33:20 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1738258400; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CgmeRhJchBQ5+4nWoUKvpVKQA9dA1++jOuG4yjWjJz4=; b=QeBZHDoX8iy6oP1q8xUCcXTSu/4YX3TomPm/1K7ch5YCBhXI18Bv3bvpIky/zI8APhzcUb 2ahJD9tEM4kwmZ6wANbBYwSU5OjyalASFoGMPaR/yDd+kdSw5cBrVmoD6UCSRe9aivlmcV 87o782gh8+Q6t8SFwsh/0HY+3IAqVJU= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-393-Df8Hga56OOGHewkhB3QSvQ-1; Thu, 30 Jan 2025 12:33:16 -0500 X-MC-Unique: Df8Hga56OOGHewkhB3QSvQ-1 X-Mimecast-MFC-AGG-ID: Df8Hga56OOGHewkhB3QSvQ Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 49B11195608B; Thu, 30 Jan 2025 17:33:15 +0000 (UTC) Received: from mzamazal-thinkpadp1gen3.tpbc.com (unknown [10.45.224.122]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 7EA651956094; Thu, 30 Jan 2025 17:33:13 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= Subject: [PATCH v4 6/6] ipa: simple: Report exposure in metadata Date: Thu, 30 Jan 2025 18:32:53 +0100 Message-ID: <20250130173254.112895-7-mzamazal@redhat.com> In-Reply-To: <20250130173254.112895-1-mzamazal@redhat.com> References: <20250130173254.112895-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: EAqC_h0myaDwxVyYQ0_RDgPpT4FF68-0lrNhO9wuMck_1738258395 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Report exposure and gain in metadata. This is more complicated than it could be expected because the exposure value should be in microseconds but it's handled using V4L2_CID_EXPOSURE control, which doesn't specify the unit, see https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/control.html. So the unit conversion is done in the way rkisp1 IPA uses. This requires getting and passing IPACameraSensorInfo around. To avoid naming confusion and to improve consistency with rkisp1 IPA, sensorCtrlInfoMap parameter is renamed to sensorControls. Signed-off-by: Milan Zamazal --- include/libcamera/ipa/soft.mojom | 3 ++- src/ipa/simple/algorithms/agc.cpp | 11 +++++++++-- src/ipa/simple/ipa_context.h | 6 +++++- src/ipa/simple/soft_simple.cpp | 17 +++++++++++++---- src/libcamera/software_isp/software_isp.cpp | 18 +++++++++++++----- 5 files changed, 42 insertions(+), 13 deletions(-) diff --git a/include/libcamera/ipa/soft.mojom b/include/libcamera/ipa/soft.mojom index 934a6fd1..16870a4d 100644 --- a/include/libcamera/ipa/soft.mojom +++ b/include/libcamera/ipa/soft.mojom @@ -16,7 +16,8 @@ interface IPASoftInterface { init(libcamera.IPASettings settings, libcamera.SharedFD fdStats, libcamera.SharedFD fdParams, - libcamera.ControlInfoMap sensorCtrlInfoMap) + libcamera.IPACameraSensorInfo sensorInfo, + libcamera.ControlInfoMap sensorControls) => (int32 ret, libcamera.ControlInfoMap ipaControls); start() => (int32 ret); stop(); diff --git a/src/ipa/simple/algorithms/agc.cpp b/src/ipa/simple/algorithms/agc.cpp index 72aade14..c46bb0eb 100644 --- a/src/ipa/simple/algorithms/agc.cpp +++ b/src/ipa/simple/algorithms/agc.cpp @@ -11,6 +11,8 @@ #include +#include "control_ids.h" + namespace libcamera { LOG_DEFINE_CATEGORY(IPASoftExposure) @@ -97,10 +99,15 @@ void Agc::updateExposure(IPAContext &context, IPAFrameContext &frameContext, dou void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, + IPAFrameContext &frameContext, const SwIspStats *stats, - [[maybe_unused]] ControlList &metadata) + ControlList &metadata) { + utils::Duration exposureTime = + context.configuration.agc.lineDuration * frameContext.sensor.exposure; + metadata.set(controls::ExposureTime, exposureTime.get()); + metadata.set(controls::AnalogueGain, frameContext.sensor.gain); + /* * Calculate Mean Sample Value (MSV) according to formula from: * https://www.araa.asn.au/acra/acra2007/papers/paper84final.pdf diff --git a/src/ipa/simple/ipa_context.h b/src/ipa/simple/ipa_context.h index 07fd641a..22b80281 100644 --- a/src/ipa/simple/ipa_context.h +++ b/src/ipa/simple/ipa_context.h @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: LGPL-2.1-or-later */ /* - * Copyright (C) 2024 Red Hat, Inc. + * Copyright (C) 2024-2025 Red Hat, Inc. * * Simple pipeline IPA Context */ @@ -15,6 +15,8 @@ #include +#include "core_ipa_interface.h" + namespace libcamera { namespace ipa::soft { @@ -24,6 +26,7 @@ struct IPASessionConfiguration { struct { int32_t exposureMin, exposureMax; double againMin, againMax, againMinStep; + utils::Duration lineDuration; } agc; struct { std::optional level; @@ -71,6 +74,7 @@ struct IPAContext { { } + IPACameraSensorInfo sensorInfo; IPASessionConfiguration configuration; IPAActiveState activeState; FCQueue frameContexts; diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index 944c644e..c6e82e12 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -5,6 +5,7 @@ * Simple Software Image Processing Algorithm module */ +#include #include #include @@ -32,6 +33,8 @@ namespace libcamera { LOG_DEFINE_CATEGORY(IPASoft) +using namespace std::literals::chrono_literals; + namespace ipa::soft { /* Maximum number of frame contexts to be held */ @@ -50,7 +53,8 @@ public: int init(const IPASettings &settings, const SharedFD &fdStats, const SharedFD &fdParams, - const ControlInfoMap &sensorInfoMap, + const IPACameraSensorInfo &sensorInfo, + const ControlInfoMap &sensorControls, ControlInfoMap *ipaControls) override; int configure(const IPAConfigInfo &configInfo) override; @@ -88,7 +92,8 @@ IPASoftSimple::~IPASoftSimple() int IPASoftSimple::init(const IPASettings &settings, const SharedFD &fdStats, const SharedFD &fdParams, - const ControlInfoMap &sensorInfoMap, + const IPACameraSensorInfo &sensorInfo, + const ControlInfoMap &sensorControls, ControlInfoMap *ipaControls) { camHelper_ = CameraSensorHelperFactoryBase::create(settings.sensorModel); @@ -98,6 +103,8 @@ int IPASoftSimple::init(const IPASettings &settings, << settings.sensorModel; } + context_.sensorInfo = sensorInfo; + /* Load the tuning data file */ File file(settings.configurationFile); if (!file.open(File::OpenModeFlag::ReadOnly)) { @@ -169,12 +176,12 @@ int IPASoftSimple::init(const IPASettings &settings, * Don't save the min and max control values yet, as e.g. the limits * for V4L2_CID_EXPOSURE depend on the configured sensor resolution. */ - if (sensorInfoMap.find(V4L2_CID_EXPOSURE) == sensorInfoMap.end()) { + if (sensorControls.find(V4L2_CID_EXPOSURE) == sensorControls.end()) { LOG(IPASoft, Error) << "Don't have exposure control"; return -EINVAL; } - if (sensorInfoMap.find(V4L2_CID_ANALOGUE_GAIN) == sensorInfoMap.end()) { + if (sensorControls.find(V4L2_CID_ANALOGUE_GAIN) == sensorControls.end()) { LOG(IPASoft, Error) << "Don't have gain control"; return -EINVAL; } @@ -194,6 +201,8 @@ int IPASoftSimple::configure(const IPAConfigInfo &configInfo) context_.activeState = {}; context_.frameContexts.clear(); + context_.configuration.agc.lineDuration = + context_.sensorInfo.minLineLength * 1.0s / context_.sensorInfo.pixelRate; context_.configuration.agc.exposureMin = exposureInfo.min().get(); context_.configuration.agc.exposureMax = exposureInfo.max().get(); if (!context_.configuration.agc.exposureMin) { diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index 92f87acc..92fd97e3 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -129,11 +129,19 @@ SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor, std::string ipaTuningFile = ipa_->configurationFile(sensor->model() + ".yaml", "uncalibrated.yaml"); - int ret = ipa_->init(IPASettings{ ipaTuningFile, sensor->model() }, - debayer_->getStatsFD(), - sharedParams_.fd(), - sensor->controls(), - ipaControls); + IPACameraSensorInfo sensorInfo{}; + int ret = sensor->sensorInfo(&sensorInfo); + if (ret) { + LOG(SoftwareIsp, Error) << "Camera sensor information not available"; + return; + } + + ret = ipa_->init(IPASettings{ ipaTuningFile, sensor->model() }, + debayer_->getStatsFD(), + sharedParams_.fd(), + sensorInfo, + sensor->controls(), + ipaControls); if (ret) { LOG(SoftwareIsp, Error) << "IPA init failed"; debayer_.reset();