From patchwork Wed Mar 26 12:48:55 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23053 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4CAC5C3213 for ; Wed, 26 Mar 2025 12:49:22 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id ED75D6897C; Wed, 26 Mar 2025 13:49:21 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="NcvksSNt"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0558B68975 for ; Wed, 26 Mar 2025 13:49:17 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1742993357; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=d9Fm2hojPhcDYQvIamge0t9vFxP3zk4HeS1kz1Jkhic=; b=NcvksSNthOkrMftGqRg45eKQuaTTvXuSFohiY/zfDAIuCTSn59kZPwOLiTqr8DzuXgs8H+ zX5lWagzVPu5g1Xm+ejhTCrvMQsGesbbJ36aadNzdY6IH8aSayc+9HiliCHeXbol1pJiTG bNyQDONcG36Hd6fUexiq7QFHuoBL/40= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-156-xIJMofn0MhOe4LSsl6lvjQ-1; Wed, 26 Mar 2025 08:49:15 -0400 X-MC-Unique: xIJMofn0MhOe4LSsl6lvjQ-1 X-Mimecast-MFC-AGG-ID: xIJMofn0MhOe4LSsl6lvjQ_1742993354 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 48870196D2CC; Wed, 26 Mar 2025 12:49:14 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.34.15]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 9FA6D3001D0F; Wed, 26 Mar 2025 12:49:12 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham Subject: [PATCH v5 6/6] ipa: simple: Report exposure in metadata Date: Wed, 26 Mar 2025 13:48:55 +0100 Message-ID: <20250326124856.75709-7-mzamazal@redhat.com> In-Reply-To: <20250326124856.75709-1-mzamazal@redhat.com> References: <20250326124856.75709-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: x51ZkBIDaCSw6qj8F39UrL5tW-xdaUFlrm0CX46aOvE_1742993354 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Report exposure and gain in metadata. This is more complicated than it could be expected because the exposure value should be in microseconds but it's handled using V4L2_CID_EXPOSURE control, which doesn't specify the unit, see https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/control.html. So the unit conversion is done in the way rkisp1 IPA uses. This requires getting and passing IPACameraSensorInfo around. To avoid naming confusion and to improve consistency with rkisp1 IPA, sensorCtrlInfoMap parameter is renamed to sensorControls. Signed-off-by: Milan Zamazal Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- include/libcamera/ipa/soft.mojom | 3 ++- src/ipa/simple/algorithms/agc.cpp | 11 +++++++++-- src/ipa/simple/ipa_context.h | 6 +++++- src/ipa/simple/soft_simple.cpp | 17 +++++++++++++---- src/libcamera/software_isp/software_isp.cpp | 20 ++++++++++++++------ 5 files changed, 43 insertions(+), 14 deletions(-) diff --git a/include/libcamera/ipa/soft.mojom b/include/libcamera/ipa/soft.mojom index a8e6ecda..77328c5f 100644 --- a/include/libcamera/ipa/soft.mojom +++ b/include/libcamera/ipa/soft.mojom @@ -16,7 +16,8 @@ interface IPASoftInterface { init(libcamera.IPASettings settings, libcamera.SharedFD fdStats, libcamera.SharedFD fdParams, - libcamera.ControlInfoMap sensorCtrlInfoMap) + libcamera.IPACameraSensorInfo sensorInfo, + libcamera.ControlInfoMap sensorControls) => (int32 ret, libcamera.ControlInfoMap ipaControls, bool ccmEnabled); start() => (int32 ret); stop(); diff --git a/src/ipa/simple/algorithms/agc.cpp b/src/ipa/simple/algorithms/agc.cpp index 72aade14..c46bb0eb 100644 --- a/src/ipa/simple/algorithms/agc.cpp +++ b/src/ipa/simple/algorithms/agc.cpp @@ -11,6 +11,8 @@ #include +#include "control_ids.h" + namespace libcamera { LOG_DEFINE_CATEGORY(IPASoftExposure) @@ -97,10 +99,15 @@ void Agc::updateExposure(IPAContext &context, IPAFrameContext &frameContext, dou void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, + IPAFrameContext &frameContext, const SwIspStats *stats, - [[maybe_unused]] ControlList &metadata) + ControlList &metadata) { + utils::Duration exposureTime = + context.configuration.agc.lineDuration * frameContext.sensor.exposure; + metadata.set(controls::ExposureTime, exposureTime.get()); + metadata.set(controls::AnalogueGain, frameContext.sensor.gain); + /* * Calculate Mean Sample Value (MSV) according to formula from: * https://www.araa.asn.au/acra/acra2007/papers/paper84final.pdf diff --git a/src/ipa/simple/ipa_context.h b/src/ipa/simple/ipa_context.h index 10d539f5..7dc2cd7a 100644 --- a/src/ipa/simple/ipa_context.h +++ b/src/ipa/simple/ipa_context.h @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: LGPL-2.1-or-later */ /* - * Copyright (C) 2024 Red Hat, Inc. + * Copyright (C) 2024-2025 Red Hat, Inc. * * Simple pipeline IPA Context */ @@ -18,6 +18,8 @@ #include +#include "core_ipa_interface.h" + namespace libcamera { namespace ipa::soft { @@ -27,6 +29,7 @@ struct IPASessionConfiguration { struct { int32_t exposureMin, exposureMax; double againMin, againMax, againMinStep; + utils::Duration lineDuration; } agc; struct { std::optional level; @@ -83,6 +86,7 @@ struct IPAContext { { } + IPACameraSensorInfo sensorInfo; IPASessionConfiguration configuration; IPAActiveState activeState; FCQueue frameContexts; diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index 4ef67c43..c94c4cd5 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -5,6 +5,7 @@ * Simple Software Image Processing Algorithm module */ +#include #include #include @@ -32,6 +33,8 @@ namespace libcamera { LOG_DEFINE_CATEGORY(IPASoft) +using namespace std::literals::chrono_literals; + namespace ipa::soft { /* Maximum number of frame contexts to be held */ @@ -50,7 +53,8 @@ public: int init(const IPASettings &settings, const SharedFD &fdStats, const SharedFD &fdParams, - const ControlInfoMap &sensorInfoMap, + const IPACameraSensorInfo &sensorInfo, + const ControlInfoMap &sensorControls, ControlInfoMap *ipaControls, bool *ccmEnabled) override; int configure(const IPAConfigInfo &configInfo) override; @@ -89,7 +93,8 @@ IPASoftSimple::~IPASoftSimple() int IPASoftSimple::init(const IPASettings &settings, const SharedFD &fdStats, const SharedFD &fdParams, - const ControlInfoMap &sensorInfoMap, + const IPACameraSensorInfo &sensorInfo, + const ControlInfoMap &sensorControls, ControlInfoMap *ipaControls, bool *ccmEnabled) { @@ -100,6 +105,8 @@ int IPASoftSimple::init(const IPASettings &settings, << settings.sensorModel; } + context_.sensorInfo = sensorInfo; + /* Load the tuning data file */ File file(settings.configurationFile); if (!file.open(File::OpenModeFlag::ReadOnly)) { @@ -173,12 +180,12 @@ int IPASoftSimple::init(const IPASettings &settings, * Don't save the min and max control values yet, as e.g. the limits * for V4L2_CID_EXPOSURE depend on the configured sensor resolution. */ - if (sensorInfoMap.find(V4L2_CID_EXPOSURE) == sensorInfoMap.end()) { + if (sensorControls.find(V4L2_CID_EXPOSURE) == sensorControls.end()) { LOG(IPASoft, Error) << "Don't have exposure control"; return -EINVAL; } - if (sensorInfoMap.find(V4L2_CID_ANALOGUE_GAIN) == sensorInfoMap.end()) { + if (sensorControls.find(V4L2_CID_ANALOGUE_GAIN) == sensorControls.end()) { LOG(IPASoft, Error) << "Don't have gain control"; return -EINVAL; } @@ -198,6 +205,8 @@ int IPASoftSimple::configure(const IPAConfigInfo &configInfo) context_.activeState = {}; context_.frameContexts.clear(); + context_.configuration.agc.lineDuration = + context_.sensorInfo.minLineLength * 1.0s / context_.sensorInfo.pixelRate; context_.configuration.agc.exposureMin = exposureInfo.min().get(); context_.configuration.agc.exposureMax = exposureInfo.max().get(); if (!context_.configuration.agc.exposureMin) { diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index 6baa3fec..28e2a360 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -133,12 +133,20 @@ SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor, std::string ipaTuningFile = ipa_->configurationFile(sensor->model() + ".yaml", "uncalibrated.yaml"); - int ret = ipa_->init(IPASettings{ ipaTuningFile, sensor->model() }, - debayer_->getStatsFD(), - sharedParams_.fd(), - sensor->controls(), - ipaControls, - &ccmEnabled_); + IPACameraSensorInfo sensorInfo{}; + int ret = sensor->sensorInfo(&sensorInfo); + if (ret) { + LOG(SoftwareIsp, Error) << "Camera sensor information not available"; + return; + } + + ret = ipa_->init(IPASettings{ ipaTuningFile, sensor->model() }, + debayer_->getStatsFD(), + sharedParams_.fd(), + sensorInfo, + sensor->controls(), + ipaControls, + &ccmEnabled_); if (ret) { LOG(SoftwareIsp, Error) << "IPA init failed"; debayer_.reset();