From patchwork Wed Jun 22 10:20:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 16307 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 33663BD808 for ; Wed, 22 Jun 2022 10:21:02 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 700E16563A; Wed, 22 Jun 2022 12:21:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1655893261; bh=k3T80DdORNI2PylIheqo2ZV0WyCsLU4hQlEpLXcguDU=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=akeQG0AFFz0W8bcf7Ph+hJQ18+QPhqdvpxx6GvF2Vmqw02bGIQIEKxjRkZTXIgyD1 TC3Qi98xVQDHUIZbyWdHrEm+i0ZWy3zns4h5Qzq98g8W24a8ifYCiLWg63FpnaGjpM 5WRB6q0zCrxv5H9cw1T6icfi5prxic934aAX1/a+i4+Gw8FTXnep8MpSUGO+Qhy4sM 3ieBIBIU1wf0ptoO6NELOt5NW3r24mAaOWbKr7a1J8i3qlQNVOcaX383Y0+pkQ7750 GfHGu/B3Ldkn3VchV8E4hh1GwLkLTg1DQ9Zxm21KVHIjstRrhC0MA76Ij5ks/lu0Ru 9WK2M1yYfAxCQ== Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1EC1561FB2 for ; Wed, 22 Jun 2022 12:20:58 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="F06UQsCl"; dkim-atps=neutral Received: by mail-wr1-x42f.google.com with SMTP id g27so16061038wrb.10 for ; Wed, 22 Jun 2022 03:20:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HlMG9zb1PmYaoA6v2JC3+vcDPjbEt84nv5H1Rmzpq4A=; b=F06UQsClKXiCKv14uXQf7UFhotTfhuU/DeUAvcSmA9f6V0iyJL2nGtyFVkJ/vHWJms 5QgncYd5xE9nIjrW/VRJH9F50MrSaGsNgsrCBB4yMsvCLl7Awff0cqCMertp0AmE+eeT 7FNiOVxBzUAWcxgHzOBveAzut/CiJQwrwF0nFKBXlnsKPpNF4mWxqp0IKiYRRnDVfBrD YDB2kxJ31aZ0iCGe6XvMIDL1lIWWBwfrscI4EIsb9sf/Kjq4Ht9/rULK6C4ERJJTdLXA 2zjFaw+0GRaNARamRYOls1nZCcBjx2LciM86105a8pz4gOcurj7EnAbTnHS8GjyJ6qRN omaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HlMG9zb1PmYaoA6v2JC3+vcDPjbEt84nv5H1Rmzpq4A=; b=NCR3enqBzaoBMu+/duFjD5IlW55hkKog1kZ03tY+iU74eR1AbfvzEr93sfpThvgcRe rna2bhxfxBb/FA7md0e8TtTh6SsnbdXoJ7GozHKiFTSDnuNHiPdIQ+cf9QZzxWOJQqZ5 nEtsvI/tW2jLBWcdzRTO02jW80K7ZMLMuTSQtTtDsUzlSJzkpx0I20ySz+MRLPUR4mTa EuUQBlrdySJM5uCdm5aBajEHXlZQ2tOJ/f9hFVAertsIN54mCunTUoBKcLhCtzoAt3pf ylqHzao8/5h+AnRRTrSfGTnsogmrkGF6CJ1ae75fd5SaSq08nSxghBSV39H5bWVplVHP Z+fA== X-Gm-Message-State: AJIora/Ef71kHoTnv9djUFuA/33dtPbldsW09scyWqUFbU4rUQAjqolU RFx5XNuI3wWz1oab7bLHNzhg7WaBjDXl8Y36 X-Google-Smtp-Source: AGRyM1t9zV3nij/ox7UsMJgBw8bi7FfTzEGTwpXS4tk0Mfm47U2SE5XUs4SuzExZQYVIAx7k5QaH5g== X-Received: by 2002:adf:fa8f:0:b0:21b:9811:43e7 with SMTP id h15-20020adffa8f000000b0021b981143e7mr2587819wrr.275.1655893257387; Wed, 22 Jun 2022 03:20:57 -0700 (PDT) Received: from naush-laptop.localdomain ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id h204-20020a1c21d5000000b0039c693a54ecsm26849423wmh.23.2022.06.22.03.20.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Jun 2022 03:20:56 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Wed, 22 Jun 2022 11:20:45 +0100 Message-Id: <20220622102047.22492-2-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220622102047.22492-1-naush@raspberrypi.com> References: <20220622102047.22492-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 1/3] pipeline: ipa: raspberrypi: Move ControlInfoMap to the IPA X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Currently the pipeline handler advertises controls handled by the IPA from a static ControlInfoMap defined in the raspberrypi.h header. This change removes this header file, and instead the IPA returns the ControlInfoMap to the pipeline handler from the ipa::init() function. This is done to allow the IPA to adjust the limits of the controls based on the sensor mode in a subsequent change. Bug: https://bugs.libcamera.org/show_bug.cgi?id=83 Signed-off-by: Naushir Patuck Reviewed-by: Kieran Bingham Reviewed-by: David Plowman --- include/libcamera/ipa/raspberrypi.h | 55 ------------------- include/libcamera/ipa/raspberrypi.mojom | 7 ++- src/ipa/raspberrypi/raspberrypi.cpp | 39 ++++++++++--- .../pipeline/raspberrypi/raspberrypi.cpp | 30 +++++----- .../pipeline/raspberrypi/rpi_stream.h | 1 - 5 files changed, 53 insertions(+), 79 deletions(-) delete mode 100644 include/libcamera/ipa/raspberrypi.h diff --git a/include/libcamera/ipa/raspberrypi.h b/include/libcamera/ipa/raspberrypi.h deleted file mode 100644 index 6a56b0083b85..000000000000 --- a/include/libcamera/ipa/raspberrypi.h +++ /dev/null @@ -1,55 +0,0 @@ -/* SPDX-License-Identifier: LGPL-2.1-or-later */ -/* - * Copyright (C) 2019-2020, Raspberry Pi Ltd. - * - * raspberrypi.h - Image Processing Algorithm interface for Raspberry Pi - */ - -#pragma once - -#include - -#include -#include - -#ifndef __DOXYGEN__ - -namespace libcamera { - -namespace RPi { - -/* - * List of controls handled by the Raspberry Pi IPA - * - * \todo This list will need to be built dynamically from the control - * algorithms loaded by the json file, once this is supported. At that - * point applications should check first whether a control is supported, - * and the pipeline handler may be reverted so that it aborts when an - * unsupported control is encountered. - */ -static const ControlInfoMap Controls({ - { &controls::AeEnable, ControlInfo(false, true) }, - { &controls::ExposureTime, ControlInfo(0, 999999) }, - { &controls::AnalogueGain, ControlInfo(1.0f, 32.0f) }, - { &controls::AeMeteringMode, ControlInfo(controls::AeMeteringModeValues) }, - { &controls::AeConstraintMode, ControlInfo(controls::AeConstraintModeValues) }, - { &controls::AeExposureMode, ControlInfo(controls::AeExposureModeValues) }, - { &controls::ExposureValue, ControlInfo(-8.0f, 8.0f, 0.0f) }, - { &controls::AwbEnable, ControlInfo(false, true) }, - { &controls::ColourGains, ControlInfo(0.0f, 32.0f) }, - { &controls::AwbMode, ControlInfo(controls::AwbModeValues) }, - { &controls::Brightness, ControlInfo(-1.0f, 1.0f, 0.0f) }, - { &controls::Contrast, ControlInfo(0.0f, 32.0f, 1.0f) }, - { &controls::Saturation, ControlInfo(0.0f, 32.0f, 1.0f) }, - { &controls::Sharpness, ControlInfo(0.0f, 16.0f, 1.0f) }, - { &controls::ColourCorrectionMatrix, ControlInfo(-16.0f, 16.0f) }, - { &controls::ScalerCrop, ControlInfo(Rectangle{}, Rectangle(65535, 65535, 65535, 65535), Rectangle{}) }, - { &controls::FrameDurationLimits, ControlInfo(INT64_C(1000), INT64_C(1000000000)) }, - { &controls::draft::NoiseReductionMode, ControlInfo(controls::draft::NoiseReductionModeValues) } - }, controls::controls); - -} /* namespace RPi */ - -} /* namespace libcamera */ - -#endif /* __DOXYGEN__ */ diff --git a/include/libcamera/ipa/raspberrypi.mojom b/include/libcamera/ipa/raspberrypi.mojom index a60c3bb43d3c..77f52c282b0f 100644 --- a/include/libcamera/ipa/raspberrypi.mojom +++ b/include/libcamera/ipa/raspberrypi.mojom @@ -26,6 +26,11 @@ struct SensorConfig { uint32 sensorMetadata; }; +struct IPAInitResult { + SensorConfig sensorConfig; + libcamera.ControlInfoMap controlInfo; +}; + struct ISPConfig { uint32 embeddedBufferId; uint32 bayerBufferId; @@ -50,7 +55,7 @@ struct StartConfig { interface IPARPiInterface { init(libcamera.IPASettings settings) - => (int32 ret, SensorConfig sensorConfig); + => (int32 ret, IPAInitResult result); start(libcamera.ControlList controls) => (StartConfig startConfig); stop(); diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 3b126bb5175e..089528f5e126 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -24,7 +24,6 @@ #include #include #include -#include #include #include @@ -72,6 +71,28 @@ constexpr Duration defaultMaxFrameDuration = 250.0s; */ constexpr Duration controllerMinFrameDuration = 1.0s / 30.0; +/* List of controls handled by the Raspberry Pi IPA */ +static const ControlInfoMap::Map ipaControls{ + { &controls::AeEnable, ControlInfo(false, true) }, + { &controls::ExposureTime, ControlInfo(0, 999999) }, + { &controls::AnalogueGain, ControlInfo(1.0f, 32.0f) }, + { &controls::AeMeteringMode, ControlInfo(controls::AeMeteringModeValues) }, + { &controls::AeConstraintMode, ControlInfo(controls::AeConstraintModeValues) }, + { &controls::AeExposureMode, ControlInfo(controls::AeExposureModeValues) }, + { &controls::ExposureValue, ControlInfo(-8.0f, 8.0f, 0.0f) }, + { &controls::AwbEnable, ControlInfo(false, true) }, + { &controls::ColourGains, ControlInfo(0.0f, 32.0f) }, + { &controls::AwbMode, ControlInfo(controls::AwbModeValues) }, + { &controls::Brightness, ControlInfo(-1.0f, 1.0f, 0.0f) }, + { &controls::Contrast, ControlInfo(0.0f, 32.0f, 1.0f) }, + { &controls::Saturation, ControlInfo(0.0f, 32.0f, 1.0f) }, + { &controls::Sharpness, ControlInfo(0.0f, 16.0f, 1.0f) }, + { &controls::ColourCorrectionMatrix, ControlInfo(-16.0f, 16.0f) }, + { &controls::ScalerCrop, ControlInfo(Rectangle{}, Rectangle(65535, 65535, 65535, 65535), Rectangle{}) }, + { &controls::FrameDurationLimits, ControlInfo(INT64_C(1000), INT64_C(1000000000)) }, + { &controls::draft::NoiseReductionMode, ControlInfo(controls::draft::NoiseReductionModeValues) } +}; + LOG_DEFINE_CATEGORY(IPARPI) namespace ipa::RPi { @@ -91,7 +112,7 @@ public: munmap(lsTable_, MaxLsGridSize); } - int init(const IPASettings &settings, SensorConfig *sensorConfig) override; + int init(const IPASettings &settings, IPAInitResult *result) override; void start(const ControlList &controls, StartConfig *startConfig) override; void stop() override {} @@ -180,7 +201,7 @@ private: uint32_t maxSensorGainCode_; }; -int IPARPi::init(const IPASettings &settings, SensorConfig *sensorConfig) +int IPARPi::init(const IPASettings &settings, IPAInitResult *result) { /* * Load the "helper" for this sensor. This tells us all the device specific stuff @@ -202,15 +223,19 @@ int IPARPi::init(const IPASettings &settings, SensorConfig *sensorConfig) helper_->GetDelays(exposureDelay, gainDelay, vblankDelay); sensorMetadata = helper_->SensorEmbeddedDataPresent(); - sensorConfig->gainDelay = gainDelay; - sensorConfig->exposureDelay = exposureDelay; - sensorConfig->vblankDelay = vblankDelay; - sensorConfig->sensorMetadata = sensorMetadata; + result->sensorConfig.gainDelay = gainDelay; + result->sensorConfig.exposureDelay = exposureDelay; + result->sensorConfig.vblankDelay = vblankDelay; + result->sensorConfig.sensorMetadata = sensorMetadata; /* Load the tuning file for this sensor. */ controller_.Read(settings.configurationFile.c_str()); controller_.Initialise(); + /* Return the controls handled by the IPA */ + ControlInfoMap::Map ctrlMap = ipaControls; + result->controlInfo = ControlInfoMap(std::move(ctrlMap), controls::controls); + return 0; } diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index adc397e8aabd..d980c1a71dd8 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -20,7 +20,6 @@ #include #include #include -#include #include #include #include @@ -199,7 +198,7 @@ public: void freeBuffers(); void frameStarted(uint32_t sequence); - int loadIPA(ipa::RPi::SensorConfig *sensorConfig); + int loadIPA(ipa::RPi::IPAInitResult *result); int configureIPA(const CameraConfiguration *config, ipa::RPi::IPAConfigResult *result); void enumerateVideoDevices(MediaLink *link); @@ -1231,15 +1230,15 @@ int PipelineHandlerRPi::registerCamera(MediaDevice *unicam, MediaDevice *isp, Me data->sensorFormats_ = populateSensorFormats(data->sensor_); - ipa::RPi::SensorConfig sensorConfig; - if (data->loadIPA(&sensorConfig)) { + ipa::RPi::IPAInitResult result; + if (data->loadIPA(&result)) { LOG(RPI, Error) << "Failed to load a suitable IPA library"; return -EINVAL; } - if (sensorConfig.sensorMetadata ^ !!unicamEmbedded) { + if (result.sensorConfig.sensorMetadata ^ !!unicamEmbedded) { LOG(RPI, Warning) << "Mismatch between Unicam and CamHelper for embedded data usage!"; - sensorConfig.sensorMetadata = false; + result.sensorConfig.sensorMetadata = false; if (unicamEmbedded) data->unicam_[Unicam::Embedded].dev()->bufferReady.disconnect(); } @@ -1253,7 +1252,7 @@ int PipelineHandlerRPi::registerCamera(MediaDevice *unicam, MediaDevice *isp, Me * iterate over all streams in one go. */ data->streams_.push_back(&data->unicam_[Unicam::Image]); - if (sensorConfig.sensorMetadata) + if (result.sensorConfig.sensorMetadata) data->streams_.push_back(&data->unicam_[Unicam::Embedded]); for (auto &stream : data->isp_) @@ -1275,15 +1274,16 @@ int PipelineHandlerRPi::registerCamera(MediaDevice *unicam, MediaDevice *isp, Me * gain and exposure delays. Mark VBLANK for priority write. */ std::unordered_map params = { - { V4L2_CID_ANALOGUE_GAIN, { sensorConfig.gainDelay, false } }, - { V4L2_CID_EXPOSURE, { sensorConfig.exposureDelay, false } }, - { V4L2_CID_VBLANK, { sensorConfig.vblankDelay, true } } + { V4L2_CID_ANALOGUE_GAIN, { result.sensorConfig.gainDelay, false } }, + { V4L2_CID_EXPOSURE, { result.sensorConfig.exposureDelay, false } }, + { V4L2_CID_VBLANK, { result.sensorConfig.vblankDelay, true } } }; data->delayedCtrls_ = std::make_unique(data->sensor_->device(), params); - data->sensorMetadata_ = sensorConfig.sensorMetadata; + data->sensorMetadata_ = result.sensorConfig.sensorMetadata; + + /* Register initial controls that the Raspberry Pi IPA can handle. */ + data->controlInfo_ = std::move(result.controlInfo); - /* Register the controls that the Raspberry Pi IPA can handle. */ - data->controlInfo_ = RPi::Controls; /* Initialize the camera properties. */ data->properties_ = data->sensor_->properties(); @@ -1509,7 +1509,7 @@ void RPiCameraData::frameStarted(uint32_t sequence) delayedCtrls_->applyControls(sequence); } -int RPiCameraData::loadIPA(ipa::RPi::SensorConfig *sensorConfig) +int RPiCameraData::loadIPA(ipa::RPi::IPAInitResult *result) { ipa_ = IPAManager::createIPA(pipe(), 1, 1); @@ -1535,7 +1535,7 @@ int RPiCameraData::loadIPA(ipa::RPi::SensorConfig *sensorConfig) IPASettings settings(configurationFile, sensor_->model()); - return ipa_->init(settings, sensorConfig); + return ipa_->init(settings, result); } int RPiCameraData::configureIPA(const CameraConfiguration *config, ipa::RPi::IPAConfigResult *result) diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.h b/src/libcamera/pipeline/raspberrypi/rpi_stream.h index c37f7e82eef6..fe01110019b7 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.h +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.h @@ -12,7 +12,6 @@ #include #include -#include #include #include From patchwork Wed Jun 22 10:20:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 16308 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 85050BD808 for ; Wed, 22 Jun 2022 10:21:04 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5185B65640; Wed, 22 Jun 2022 12:21:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1655893264; bh=2tme2r25E6v8Hd+ZFkt14xUEx8o8WHikpVKgnqRQj7c=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=ga0Y1Yp5loKlACzqILVjGM7MQSj4kbB1WotjIf5JXU/w8HSF2NdBm1Boz1luF5UIl qg5/ygVFhGVfEJ5plu1q+6tI2Vh2Al3kPzn+1rHTYh8xSWpzNutKfvGOAro6UNXDIv bcHKJFr6D7Eu4NHkDVMxDY3GVfuCMY2MCu5/HzNtKRQUEPdzAIxA1UsFnw5vjMNbQf AvhD2JZCFNNh3+HWH4ZedBBasIR5Uvz3Zu7zA0LDAAB9bqCu/oEwPIZBCIW1SFVaSV wvk5zWJjMG/OxI2/gv6MgaYrdquP2t7VUIul1j+r9tA/BKWbhVLrWapRNuxKqeub2h G4nL/aTQRW6fA== Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9E97B65638 for ; Wed, 22 Jun 2022 12:20:59 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="aMlY+3Ae"; dkim-atps=neutral Received: by mail-wr1-x42f.google.com with SMTP id i10so18944992wrc.0 for ; Wed, 22 Jun 2022 03:20:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hEvVAIMRDdzIdQ1Up9tQJGX/dynhotDpdpBJ2vFJbEs=; b=aMlY+3Aed3/2vFAjbVhtfpj9gUD2OD3bXTXc99zuAs4VVQg2dFk4WOox4zs5124XZ9 FTUNH1tkQ4rcBwEpEtCSj+I0Iv77QMagUwgAi9PZu0opdhDNShWE3riGScgZRAsAd4jI Dt3TzxIiJ0AoLiQN7xeHQXyr3/ocJ9hZIqmqnjwkxcT1tLrN6+p/JL/4Uxevd5kWRf4c 5VRHI3ZXERGNVjEwHttjLESPYVFLfd7n7c8Ve56PFNXLOLt11nph2z7pKDrZUiYqJ7Xp lrVBCyKOAuclkxYWY0fRMxzXEImQIBt5zZYtCrjxyEyhnPiWQvjxn/oc848XvWeKVj9m yzgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hEvVAIMRDdzIdQ1Up9tQJGX/dynhotDpdpBJ2vFJbEs=; b=AoZQRLt0Da0Fl1ljeincaQQfuMlwsOHemfD6X8A3FzcSGccXCKJBih8Eufno0Kl0e3 Qljf44sA3sSMeqFqD9hvFLb2KweVBhXpaQPYxVBLPK8oe43kNzjfdtY5rQsD2BO7bOk3 26HrzRFEgvMeREGSjKTh9eG58hMIsqApaB9ge7dF3UqZzSqDj5STbfxLa2LqqdLjIGWt SybY/umtueJqx6e8faNS798wrAeN5P5bJbEfp+khIawzFLw2K/Om1+E1oizBwr9eT/Ks V8pNGv5BqByklagz61mDTgEsRJ7MPUYDIkFWa7E+TbWsTlvzNZF3Y1KjauBRLkT9iJAl s5Qg== X-Gm-Message-State: AJIora+JbPHygHJMJ2RqC2HFRgrMUxgqYP+0uxudKQD9ICRc3fr6Ne3v /6VaS8H0yszFp7B7odztQLBnL2Jvdticoof7 X-Google-Smtp-Source: AGRyM1sK6kD1Gkhy3ezL5g8h7MHj5r+9uXOP5VUfU7zfc8XMDKGDv7n4q/hn8dccsjgtStae0Q+rag== X-Received: by 2002:a5d:6d8f:0:b0:219:b5cd:6516 with SMTP id l15-20020a5d6d8f000000b00219b5cd6516mr2583514wrs.246.1655893258704; Wed, 22 Jun 2022 03:20:58 -0700 (PDT) Received: from naush-laptop.localdomain ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id h204-20020a1c21d5000000b0039c693a54ecsm26849423wmh.23.2022.06.22.03.20.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Jun 2022 03:20:58 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Wed, 22 Jun 2022 11:20:46 +0100 Message-Id: <20220622102047.22492-3-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220622102047.22492-1-naush@raspberrypi.com> References: <20220622102047.22492-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 2/3] pipeline: ipa: raspberrypi: Correctly report available control limits X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The ipa currently advertises a static ControlInfoMap to specify the available controls and their limits. Fix this limitation by having the IPA populate a new ControlInfoMap with updated limits for ExposureTime, AnalogueGain, and FrameDurationLimits controls based on the current sensor mode. This new ControlInfoMap is then returned back to the pipeline handler and available to the application after a successful Camera::configure() call. Before the first call to Camera::configure(), this ControlInfoMap provides some reasonable default limits for ExposureTime, AnalogueGain, and FrameDurationLimits. However, applications must not rely on these values, but obtain the correct limits after the call to Camera::configure(). Signed-off-by: Naushir Patuck Reviewed-by: Kieran Bingham Reviewed-by: David Plowman --- include/libcamera/ipa/raspberrypi.mojom | 1 + src/ipa/raspberrypi/raspberrypi.cpp | 33 +++++++++++++++++-- .../pipeline/raspberrypi/raspberrypi.cpp | 3 ++ 3 files changed, 34 insertions(+), 3 deletions(-) diff --git a/include/libcamera/ipa/raspberrypi.mojom b/include/libcamera/ipa/raspberrypi.mojom index 77f52c282b0f..c0de435b7b33 100644 --- a/include/libcamera/ipa/raspberrypi.mojom +++ b/include/libcamera/ipa/raspberrypi.mojom @@ -45,6 +45,7 @@ struct IPAConfig { struct IPAConfigResult { float modeSensitivity; + libcamera.ControlInfoMap controlInfo; }; struct StartConfig { diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 089528f5e126..8f57350878cf 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -74,8 +74,8 @@ constexpr Duration controllerMinFrameDuration = 1.0s / 30.0; /* List of controls handled by the Raspberry Pi IPA */ static const ControlInfoMap::Map ipaControls{ { &controls::AeEnable, ControlInfo(false, true) }, - { &controls::ExposureTime, ControlInfo(0, 999999) }, - { &controls::AnalogueGain, ControlInfo(1.0f, 32.0f) }, + { &controls::ExposureTime, ControlInfo(0, 66666) }, + { &controls::AnalogueGain, ControlInfo(1.0f, 16.0f) }, { &controls::AeMeteringMode, ControlInfo(controls::AeMeteringModeValues) }, { &controls::AeConstraintMode, ControlInfo(controls::AeConstraintModeValues) }, { &controls::AeExposureMode, ControlInfo(controls::AeExposureModeValues) }, @@ -89,7 +89,7 @@ static const ControlInfoMap::Map ipaControls{ { &controls::Sharpness, ControlInfo(0.0f, 16.0f, 1.0f) }, { &controls::ColourCorrectionMatrix, ControlInfo(-16.0f, 16.0f) }, { &controls::ScalerCrop, ControlInfo(Rectangle{}, Rectangle(65535, 65535, 65535, 65535), Rectangle{}) }, - { &controls::FrameDurationLimits, ControlInfo(INT64_C(1000), INT64_C(1000000000)) }, + { &controls::FrameDurationLimits, ControlInfo(INT64_C(33333), INT64_C(120000)) }, { &controls::draft::NoiseReductionMode, ControlInfo(controls::draft::NoiseReductionModeValues) } }; @@ -446,6 +446,33 @@ int IPARPi::configure(const IPACameraSensorInfo &sensorInfo, ASSERT(controls); *controls = std::move(ctrls); + /* + * Apply the correct limits to the exposure, gain and frame duration controls + * based on the current sensor mode. + */ + ControlInfoMap::Map ctrlMap = ipaControls; + const Duration minSensorFrameDuration = mode_.min_frame_length * mode_.line_length; + const Duration maxSensorFrameDuration = mode_.max_frame_length * mode_.line_length; + ctrlMap[&controls::FrameDurationLimits] = + ControlInfo(static_cast(minSensorFrameDuration.get()), + static_cast(maxSensorFrameDuration.get())); + + ctrlMap[&controls::AnalogueGain] = + ControlInfo(1.0f, static_cast(helper_->Gain(maxSensorGainCode_))); + + /* + * Calculate the max exposure limit from the frame duration limit as V4L2 + * will limit the maximum control value based on the current VBLANK value. + */ + Duration maxShutter = Duration::max(); + helper_->GetVBlanking(maxShutter, minSensorFrameDuration, maxSensorFrameDuration); + const uint32_t exposureMin = sensorCtrls_.at(V4L2_CID_EXPOSURE).min().get(); + + ctrlMap[&controls::ExposureTime] = + ControlInfo(static_cast(helper_->Exposure(exposureMin).get()), + static_cast(maxShutter.get())); + + result->controlInfo = ControlInfoMap(std::move(ctrlMap), controls::controls); return 0; } diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index d980c1a71dd8..4596f2babcea 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -940,6 +940,9 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) /* Store the mode sensitivity for the application. */ data->properties_.set(properties::SensorSensitivity, result.modeSensitivity); + /* Update the controls that the Raspberry Pi IPA can handle. */ + data->controlInfo_ = result.controlInfo; + /* Setup the Video Mux/Bridge entities. */ for (auto &[device, link] : data->bridgeDevices_) { /* From patchwork Wed Jun 22 10:20:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 16309 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6B80FBD808 for ; Wed, 22 Jun 2022 10:21:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 140B56563D; Wed, 22 Jun 2022 12:21:05 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1655893265; bh=R8W2IaKmrDLUH23lPmlzOIdyDArFwOsNlCOPxwtI32w=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=BPdfyIIXswJwP82nkjJf+FjChvYMHgJzSHBtWUPwBs0aqVuK3GhTLYHEoA2J8VwH2 zKoY77cneLsvv7zU4J51I1uxjVVEIaTD3IfzNDdWQ06wFjy+nzUb2VRJyjcnx7pPaW K/IZaFZOYXF1CBL/1MvaUHTjT1ct6jxvU3FjqnfYgxQtQp7QXq2byF/7cP4F9J9B3Q 9MOJ5wccQQ7z7LHk9FSggIvXIE/8rSveGKt9r3oiVWwAloUjhDXHvJcg1KjTLZttRd 8Th6jkm6Myxy1MpuZQZqyjQvwZR+gfH3kF/oUsiKbhybEKSajqyBMOm3xYNOOCx0eJ 24aDRvUVz7nLw== Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6F04C633A7 for ; Wed, 22 Jun 2022 12:21:00 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="qaNuaXEt"; dkim-atps=neutral Received: by mail-wr1-x432.google.com with SMTP id o8so22726051wro.3 for ; Wed, 22 Jun 2022 03:21:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=of8OcpUH49FJA/uG9bdSOgvolWS8eZrOGomkTlrVHdg=; b=qaNuaXEtimiC4yR4JtuWreDE7BOXcUaNKIsuvTexJa41KqCuN7S2UBKVbAhjhISwRD DNs1A06fMWfCz8TDdGgltcKuixfWLepQo22RBZRFUzLJO0NFvLftAeBrI1eATvwYwgwS NZn5PFWGQ1HFkKiZF5VWN3kFc0m2GIF00PjiCWehaNletYCEC3MOeklRykXPszhUClDZ bbCjpe8UQuS7Xu+Dv6PeMXTe+z7Vh5Byw3qRUSfiFmm8Sx70NtoLGyxTovt58zcxrSe3 XLsGOcCx4KY40oRDU7i62Ud/rLUk5cJ3ZA4DyCXrl0PFnEq83QzLq7UdbPtUaADrEnnQ Vwbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=of8OcpUH49FJA/uG9bdSOgvolWS8eZrOGomkTlrVHdg=; b=PR6WYjzkfCwfPJhkjOxzgJzneB4eK8I2rB5a2GZ/ocoXrL+CVvASapXenS+9W/Roox fZW8GucsGjIE2q8NodUVT987nrNxzprAM9k3Zjepi1oOVZJ9xVxUQg4GJyS/4Tu+Me9s 5DT24bZwv2tQVbcyb/8PlH6AIFgUsKfZz2/vBHrJQq6fNLFmu5WPyqzOQTJsUISXhX95 rKZxidn2pixtE5Fc42qS5FM+WjB6SXczhSYlT/xjoNBaFk6SmBk98WSvah+z0AyzdAS1 7du9Q8ywmMgYg8IkT4ELHPEBUvOQJhKLXypEbTAl/YU+RYIguxlfaqs548B+8wjv9jOU NbVQ== X-Gm-Message-State: AJIora9Rt9WsHSM9Ws3Pl0ohVNeOHEw4+UvX3ogO0DegBXQb3VP299+s vqo+gZ8nzSA3bmpwBAftLFb+uqYAvupfnD+m X-Google-Smtp-Source: AGRyM1s9stLD2YOEijmr9SFT/VKUJB2Pc5Jh85SYmdys7x0oeIDjPFmQYYNF9hpNM/5WRkhQIHl7XA== X-Received: by 2002:a05:6000:1548:b0:217:6480:e65 with SMTP id 8-20020a056000154800b0021764800e65mr2611386wry.381.1655893259869; Wed, 22 Jun 2022 03:20:59 -0700 (PDT) Received: from naush-laptop.localdomain ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id h204-20020a1c21d5000000b0039c693a54ecsm26849423wmh.23.2022.06.22.03.20.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Jun 2022 03:20:59 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Wed, 22 Jun 2022 11:20:47 +0100 Message-Id: <20220622102047.22492-4-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220622102047.22492-1-naush@raspberrypi.com> References: <20220622102047.22492-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 3/3] pipeline: raspberrypi: Advertise ScalerCrop from the pipeline handler X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The ScalerCrop control is handled directly by the pipeline handler. Remove the control from the IPA's static ControlInfoMap, and let the pipeline handler add it to the ControlInfoMap advertised to the application, ensuring the limits are set appropriately based on the current sensor mode. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Reviewed-by: Kieran Bingham --- src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 4596f2babcea..66a84b1dfb97 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -941,7 +941,15 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) data->properties_.set(properties::SensorSensitivity, result.modeSensitivity); /* Update the controls that the Raspberry Pi IPA can handle. */ - data->controlInfo_ = result.controlInfo; + ControlInfoMap::Map ctrlMap; + for (auto const &c : result.controlInfo) + ctrlMap.emplace(c.first, c.second); + + /* Add the ScalerCrop control limits based on the current mode. */ + ctrlMap.emplace(&controls::ScalerCrop, + ControlInfo(Rectangle(data->ispMinCropSize_), Rectangle(data->sensorInfo_.outputSize))); + + data->controlInfo_ = ControlInfoMap(std::move(ctrlMap), result.controlInfo.idmap()); /* Setup the Video Mux/Bridge entities. */ for (auto &[device, link] : data->bridgeDevices_) {