From patchwork Fri Jun 10 12:25:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 16194 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 77D3DBD160 for ; Fri, 10 Jun 2022 12:25:30 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BA0C365637; Fri, 10 Jun 2022 14:25:29 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1654863929; bh=I7i1DSsao3GzQ0OO/FrMaVtH+w6QDuQvYFdCBS0808Q=; h=To:Date:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=IObAGbWE9L8S7zDADnO8XM8DlOmCPzJptsz0x+9Q9Orqxa0HVIi7PfTK0jd87mG09 eJOdVwefEBvsRJpcWV207QAWpM6Ud0gmPADuPuksPcZGoQrnOubqk9aGHPfHXjajhy 0hGlFOZQGcCInvoZ+K317YDAt+8mMYpErb62dK/OvMzCCklrT92FmoyA4LTTvchlCb MMbDueqKdJG72BWpPAuY6iBPU0d7cvlYtZbzggEzBhHJT7Qc6f2DCdRiGrLqdxjuDx opu9UvMiBVqEa/KvnuhIItDbJ87KoyxB2JOHv8r/YnGlpIZvdAX9idwQwcO450A0Tn S2RH1ul0Ftxgw== Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9EE58633A4 for ; Fri, 10 Jun 2022 14:25:27 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="qcb04WnE"; dkim-atps=neutral Received: by mail-wm1-x32e.google.com with SMTP id j5-20020a05600c1c0500b0039c5dbbfa48so1057942wms.5 for ; Fri, 10 Jun 2022 05:25:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=qtIlzrnVwnl2drGulqi2KdcK9VR3lErOmNgq7bBW4L8=; b=qcb04WnESussviqlqPRXiwz0ZBrZlWfjGoOTUffrHTmDMaIsiVLuQRRTm2imgD+WRj O1WJ/TxQ4j/2DDguWLH4HYVjtd+oUhx79YG4nJ+J0cIGbnDKwMg4xzp5LWnPLytj+0Wb IF1j7PitUH1KRbNeNXVst4zSyzijC/mLcG1BTL+5BM7yloHWnOCxAwKucV4k3SYlhgmu M3whWKp1pqFTnXbRqrSg1NOrUnZOF3pjjVClsB7Tnftq8EVoW6zi+JC3fbLqZtAcw5x+ 8eeeLcVqr0Kgs0ejv+Qxe4I1z6tGGeSywolqhvKbDe2Qf+UXxH4YV8ZUy6fG7c17kHHa Xtdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=qtIlzrnVwnl2drGulqi2KdcK9VR3lErOmNgq7bBW4L8=; b=LCrdWhWUAiYdrXo2oCN8UTMxmkMe8LgfAsHWAw/jKPzU0Xcs87RJcDkUUXyTb/f2WI RYq1cDXXBoGGviRR/4OHAu8G16yKsqg30ZLjpLEKSZ43AfikYlCeIV2BPvth72xnocWW 6fv3o5tS3aQJQMABZepPIK1Eu/ZhdvNt1NQWiOQrRzNpKZi5XmgwL+BB8pj3ZKcDJRtS aQJNoNuibpuALNwZPQZe9ZUk03+5immyNIiaL8Vkl+SxxWqtgfDtF4SGUNIYfOhfKHOZ Kas+WSPYreMAro9jAsBs+/8z/EUQf4Fu9Jz1R15f8uB4XCORJkjnfRbkkIx3OP8c0xek ex3Q== X-Gm-Message-State: AOAM532j3Lpej3Z5TtPFLPR9LieTNEdR/QSjw02odF7UEVSqDm4lywgz CqENTCqOPO47C79hwtOz5J3NZ+sPvG37kQ== X-Google-Smtp-Source: ABdhPJzj0sberSyNPRJKTtbulrq6A4t68JmPgzdwjLCBQ86G9AOFYh2A+zJM6Vm3BPA90YO1lobZsA== X-Received: by 2002:a05:600c:25cd:b0:39c:6bc0:a1fb with SMTP id 13-20020a05600c25cd00b0039c6bc0a1fbmr8659510wml.179.1654863926778; Fri, 10 Jun 2022 05:25:26 -0700 (PDT) Received: from naush-laptop.localdomain ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id e29-20020a5d595d000000b00213b93cff5fsm23097909wri.98.2022.06.10.05.25.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Jun 2022 05:25:26 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Fri, 10 Jun 2022 13:25:31 +0100 Message-Id: <20220610122533.11888-1-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 1/3] pipeline: ipa: raspberrypi: Move ControlInfoMap to the IPA X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Currently the pipeline handler advertises controls handled by the IPA from a static ControlInfoMap defined in the raspberrypi.h header. This change removes this header file, and instead the IPA returns the ControlInfoMap to the pipeline handler from the ipa::init() function. This is done to allow the IPA to adjust the limits of the controls based on the sensor mode in a subsequent change. Signed-off-by: Naushir Patuck Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- include/libcamera/ipa/raspberrypi.h | 55 ------------------- include/libcamera/ipa/raspberrypi.mojom | 7 ++- src/ipa/raspberrypi/raspberrypi.cpp | 39 ++++++++++--- .../pipeline/raspberrypi/raspberrypi.cpp | 30 +++++----- .../pipeline/raspberrypi/rpi_stream.h | 1 - 5 files changed, 53 insertions(+), 79 deletions(-) delete mode 100644 include/libcamera/ipa/raspberrypi.h diff --git a/include/libcamera/ipa/raspberrypi.h b/include/libcamera/ipa/raspberrypi.h deleted file mode 100644 index 6a56b0083b85..000000000000 --- a/include/libcamera/ipa/raspberrypi.h +++ /dev/null @@ -1,55 +0,0 @@ -/* SPDX-License-Identifier: LGPL-2.1-or-later */ -/* - * Copyright (C) 2019-2020, Raspberry Pi Ltd. - * - * raspberrypi.h - Image Processing Algorithm interface for Raspberry Pi - */ - -#pragma once - -#include - -#include -#include - -#ifndef __DOXYGEN__ - -namespace libcamera { - -namespace RPi { - -/* - * List of controls handled by the Raspberry Pi IPA - * - * \todo This list will need to be built dynamically from the control - * algorithms loaded by the json file, once this is supported. At that - * point applications should check first whether a control is supported, - * and the pipeline handler may be reverted so that it aborts when an - * unsupported control is encountered. - */ -static const ControlInfoMap Controls({ - { &controls::AeEnable, ControlInfo(false, true) }, - { &controls::ExposureTime, ControlInfo(0, 999999) }, - { &controls::AnalogueGain, ControlInfo(1.0f, 32.0f) }, - { &controls::AeMeteringMode, ControlInfo(controls::AeMeteringModeValues) }, - { &controls::AeConstraintMode, ControlInfo(controls::AeConstraintModeValues) }, - { &controls::AeExposureMode, ControlInfo(controls::AeExposureModeValues) }, - { &controls::ExposureValue, ControlInfo(-8.0f, 8.0f, 0.0f) }, - { &controls::AwbEnable, ControlInfo(false, true) }, - { &controls::ColourGains, ControlInfo(0.0f, 32.0f) }, - { &controls::AwbMode, ControlInfo(controls::AwbModeValues) }, - { &controls::Brightness, ControlInfo(-1.0f, 1.0f, 0.0f) }, - { &controls::Contrast, ControlInfo(0.0f, 32.0f, 1.0f) }, - { &controls::Saturation, ControlInfo(0.0f, 32.0f, 1.0f) }, - { &controls::Sharpness, ControlInfo(0.0f, 16.0f, 1.0f) }, - { &controls::ColourCorrectionMatrix, ControlInfo(-16.0f, 16.0f) }, - { &controls::ScalerCrop, ControlInfo(Rectangle{}, Rectangle(65535, 65535, 65535, 65535), Rectangle{}) }, - { &controls::FrameDurationLimits, ControlInfo(INT64_C(1000), INT64_C(1000000000)) }, - { &controls::draft::NoiseReductionMode, ControlInfo(controls::draft::NoiseReductionModeValues) } - }, controls::controls); - -} /* namespace RPi */ - -} /* namespace libcamera */ - -#endif /* __DOXYGEN__ */ diff --git a/include/libcamera/ipa/raspberrypi.mojom b/include/libcamera/ipa/raspberrypi.mojom index a60c3bb43d3c..77f52c282b0f 100644 --- a/include/libcamera/ipa/raspberrypi.mojom +++ b/include/libcamera/ipa/raspberrypi.mojom @@ -26,6 +26,11 @@ struct SensorConfig { uint32 sensorMetadata; }; +struct IPAInitResult { + SensorConfig sensorConfig; + libcamera.ControlInfoMap controlInfo; +}; + struct ISPConfig { uint32 embeddedBufferId; uint32 bayerBufferId; @@ -50,7 +55,7 @@ struct StartConfig { interface IPARPiInterface { init(libcamera.IPASettings settings) - => (int32 ret, SensorConfig sensorConfig); + => (int32 ret, IPAInitResult result); start(libcamera.ControlList controls) => (StartConfig startConfig); stop(); diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 3b126bb5175e..089528f5e126 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -24,7 +24,6 @@ #include #include #include -#include #include #include @@ -72,6 +71,28 @@ constexpr Duration defaultMaxFrameDuration = 250.0s; */ constexpr Duration controllerMinFrameDuration = 1.0s / 30.0; +/* List of controls handled by the Raspberry Pi IPA */ +static const ControlInfoMap::Map ipaControls{ + { &controls::AeEnable, ControlInfo(false, true) }, + { &controls::ExposureTime, ControlInfo(0, 999999) }, + { &controls::AnalogueGain, ControlInfo(1.0f, 32.0f) }, + { &controls::AeMeteringMode, ControlInfo(controls::AeMeteringModeValues) }, + { &controls::AeConstraintMode, ControlInfo(controls::AeConstraintModeValues) }, + { &controls::AeExposureMode, ControlInfo(controls::AeExposureModeValues) }, + { &controls::ExposureValue, ControlInfo(-8.0f, 8.0f, 0.0f) }, + { &controls::AwbEnable, ControlInfo(false, true) }, + { &controls::ColourGains, ControlInfo(0.0f, 32.0f) }, + { &controls::AwbMode, ControlInfo(controls::AwbModeValues) }, + { &controls::Brightness, ControlInfo(-1.0f, 1.0f, 0.0f) }, + { &controls::Contrast, ControlInfo(0.0f, 32.0f, 1.0f) }, + { &controls::Saturation, ControlInfo(0.0f, 32.0f, 1.0f) }, + { &controls::Sharpness, ControlInfo(0.0f, 16.0f, 1.0f) }, + { &controls::ColourCorrectionMatrix, ControlInfo(-16.0f, 16.0f) }, + { &controls::ScalerCrop, ControlInfo(Rectangle{}, Rectangle(65535, 65535, 65535, 65535), Rectangle{}) }, + { &controls::FrameDurationLimits, ControlInfo(INT64_C(1000), INT64_C(1000000000)) }, + { &controls::draft::NoiseReductionMode, ControlInfo(controls::draft::NoiseReductionModeValues) } +}; + LOG_DEFINE_CATEGORY(IPARPI) namespace ipa::RPi { @@ -91,7 +112,7 @@ public: munmap(lsTable_, MaxLsGridSize); } - int init(const IPASettings &settings, SensorConfig *sensorConfig) override; + int init(const IPASettings &settings, IPAInitResult *result) override; void start(const ControlList &controls, StartConfig *startConfig) override; void stop() override {} @@ -180,7 +201,7 @@ private: uint32_t maxSensorGainCode_; }; -int IPARPi::init(const IPASettings &settings, SensorConfig *sensorConfig) +int IPARPi::init(const IPASettings &settings, IPAInitResult *result) { /* * Load the "helper" for this sensor. This tells us all the device specific stuff @@ -202,15 +223,19 @@ int IPARPi::init(const IPASettings &settings, SensorConfig *sensorConfig) helper_->GetDelays(exposureDelay, gainDelay, vblankDelay); sensorMetadata = helper_->SensorEmbeddedDataPresent(); - sensorConfig->gainDelay = gainDelay; - sensorConfig->exposureDelay = exposureDelay; - sensorConfig->vblankDelay = vblankDelay; - sensorConfig->sensorMetadata = sensorMetadata; + result->sensorConfig.gainDelay = gainDelay; + result->sensorConfig.exposureDelay = exposureDelay; + result->sensorConfig.vblankDelay = vblankDelay; + result->sensorConfig.sensorMetadata = sensorMetadata; /* Load the tuning file for this sensor. */ controller_.Read(settings.configurationFile.c_str()); controller_.Initialise(); + /* Return the controls handled by the IPA */ + ControlInfoMap::Map ctrlMap = ipaControls; + result->controlInfo = ControlInfoMap(std::move(ctrlMap), controls::controls); + return 0; } diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index adc397e8aabd..d980c1a71dd8 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -20,7 +20,6 @@ #include #include #include -#include #include #include #include @@ -199,7 +198,7 @@ public: void freeBuffers(); void frameStarted(uint32_t sequence); - int loadIPA(ipa::RPi::SensorConfig *sensorConfig); + int loadIPA(ipa::RPi::IPAInitResult *result); int configureIPA(const CameraConfiguration *config, ipa::RPi::IPAConfigResult *result); void enumerateVideoDevices(MediaLink *link); @@ -1231,15 +1230,15 @@ int PipelineHandlerRPi::registerCamera(MediaDevice *unicam, MediaDevice *isp, Me data->sensorFormats_ = populateSensorFormats(data->sensor_); - ipa::RPi::SensorConfig sensorConfig; - if (data->loadIPA(&sensorConfig)) { + ipa::RPi::IPAInitResult result; + if (data->loadIPA(&result)) { LOG(RPI, Error) << "Failed to load a suitable IPA library"; return -EINVAL; } - if (sensorConfig.sensorMetadata ^ !!unicamEmbedded) { + if (result.sensorConfig.sensorMetadata ^ !!unicamEmbedded) { LOG(RPI, Warning) << "Mismatch between Unicam and CamHelper for embedded data usage!"; - sensorConfig.sensorMetadata = false; + result.sensorConfig.sensorMetadata = false; if (unicamEmbedded) data->unicam_[Unicam::Embedded].dev()->bufferReady.disconnect(); } @@ -1253,7 +1252,7 @@ int PipelineHandlerRPi::registerCamera(MediaDevice *unicam, MediaDevice *isp, Me * iterate over all streams in one go. */ data->streams_.push_back(&data->unicam_[Unicam::Image]); - if (sensorConfig.sensorMetadata) + if (result.sensorConfig.sensorMetadata) data->streams_.push_back(&data->unicam_[Unicam::Embedded]); for (auto &stream : data->isp_) @@ -1275,15 +1274,16 @@ int PipelineHandlerRPi::registerCamera(MediaDevice *unicam, MediaDevice *isp, Me * gain and exposure delays. Mark VBLANK for priority write. */ std::unordered_map params = { - { V4L2_CID_ANALOGUE_GAIN, { sensorConfig.gainDelay, false } }, - { V4L2_CID_EXPOSURE, { sensorConfig.exposureDelay, false } }, - { V4L2_CID_VBLANK, { sensorConfig.vblankDelay, true } } + { V4L2_CID_ANALOGUE_GAIN, { result.sensorConfig.gainDelay, false } }, + { V4L2_CID_EXPOSURE, { result.sensorConfig.exposureDelay, false } }, + { V4L2_CID_VBLANK, { result.sensorConfig.vblankDelay, true } } }; data->delayedCtrls_ = std::make_unique(data->sensor_->device(), params); - data->sensorMetadata_ = sensorConfig.sensorMetadata; + data->sensorMetadata_ = result.sensorConfig.sensorMetadata; + + /* Register initial controls that the Raspberry Pi IPA can handle. */ + data->controlInfo_ = std::move(result.controlInfo); - /* Register the controls that the Raspberry Pi IPA can handle. */ - data->controlInfo_ = RPi::Controls; /* Initialize the camera properties. */ data->properties_ = data->sensor_->properties(); @@ -1509,7 +1509,7 @@ void RPiCameraData::frameStarted(uint32_t sequence) delayedCtrls_->applyControls(sequence); } -int RPiCameraData::loadIPA(ipa::RPi::SensorConfig *sensorConfig) +int RPiCameraData::loadIPA(ipa::RPi::IPAInitResult *result) { ipa_ = IPAManager::createIPA(pipe(), 1, 1); @@ -1535,7 +1535,7 @@ int RPiCameraData::loadIPA(ipa::RPi::SensorConfig *sensorConfig) IPASettings settings(configurationFile, sensor_->model()); - return ipa_->init(settings, sensorConfig); + return ipa_->init(settings, result); } int RPiCameraData::configureIPA(const CameraConfiguration *config, ipa::RPi::IPAConfigResult *result) diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.h b/src/libcamera/pipeline/raspberrypi/rpi_stream.h index c37f7e82eef6..fe01110019b7 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.h +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.h @@ -12,7 +12,6 @@ #include #include -#include #include #include From patchwork Fri Jun 10 12:25:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 16195 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 30506C326B for ; Fri, 10 Jun 2022 12:25:32 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3E034633A6; Fri, 10 Jun 2022 14:25:30 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1654863930; bh=fbvK/Ak27dtZl3TEwoLt4TTFR3XDE3kvqgGiqHUfnmI=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=l4ZmidhfeJZuETEjcBx9xqPHqpJbwtetAdSksnUO8MPutcXuA6+6PHXYi9XzaMGoU 3vcgB1krsigEng9T6utp6LyWO5Ndsg9B+mHEHxw+PTl1liCcuEVteSB6pI8/JEedIs vr2NI8ZVD7LYCnD1ppUIl6DtFqgWAYh58PbMFoe6JPzFwmFE97BwoSCLESK3KL0doH HSqNCLbtOMalv7BheeLOpz1VTF1AKjOw5v3c07TZVbafzMp7x5/gRAbDS7I0MgSYLb wKjDmiVympQCyyr3OCL5kYq6gwt/64m3RBgQW5VxrjxflIVyAOGO5+DNESzAJDGTI5 k0oH1U2VEJEDg== Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BE8FB633A6 for ; Fri, 10 Jun 2022 14:25:28 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="jKrkCgeF"; dkim-atps=neutral Received: by mail-wm1-x32c.google.com with SMTP id z17so8987850wmi.1 for ; Fri, 10 Jun 2022 05:25:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=wZA9/ttvFh+Mazt9dmmLKBj2nOFyAUFVT4YF0YKI6g0=; b=jKrkCgeFhsJLtd8e0I2gI2szkkjHT88MBFxskl6/lpCUjiPNMWUt6aW6SQoHRA4TXU iD1ZC8V1PK+LkJOyuhoth64+/pefTPFtsTOcEZogXhp3zWuPZDOHGr+oaFssw6vN27X6 2jpEpscLGPsLLk7fqdld+DrccDh4SqXZfRGLjTbX3RTMPRXBGF92xzhughFoFQV2Pd54 HUFvhjwkEBFr7yRA2cnEeK+kYAnI/niVYskDuZFKs5HO+n2euc18U06TGGcQbuyqj0YM TXGEvK5IVuazl//nxJy9+d6mp2ZNSVbi2JKJ3/6jKP5PLBCr8RzbS0njU8mQ4w0IpRIV dy/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wZA9/ttvFh+Mazt9dmmLKBj2nOFyAUFVT4YF0YKI6g0=; b=YwgrwO8D/Hx0s1HGS0glHhMr4tKHzF84/DWRcsWPZdFcjQOdhrsiGEyQg3V5gpsaRu Mc6KzEaU/iLMVtj0taOFVJ+e3LOMzFzGaMX/MEuNBhsTE+fOtX+e7mf+FcxiDtjz5nrW 8cq4p6TnV/Ej0o22A86eBSrbmqMQyMtB55Bz9GcF3qiDKKIuIFK8MErxiEpfBovBNkar qswiytZb/M8lPPg7Ow13kWSGSwGvRMBTN6Av1YImygNFyM7KdNGC4f2bk3W6Onj7FQsj 2kgdFbdsCvrvaYivn8A6nz7C4gf9iGABwdawF44QJ1pQ57K04zdVXT4TiVzWt5LKmEo1 eTbQ== X-Gm-Message-State: AOAM532qykvvPrX52D9ZuzXBxmGcETWGABQnAmSH7W9lS073h8KEk2cF HKTVKzsgght41ZS3UcJz8fa1hPpYSfKJCg== X-Google-Smtp-Source: ABdhPJzCB11T0jtE8TwFfuZ3t0o6D1Ak9qBWQyMi2krbFeVqj3UPmMyuhEAThevd6EXZFjlBdfVECA== X-Received: by 2002:a7b:c4cb:0:b0:39c:35b3:e8c1 with SMTP id g11-20020a7bc4cb000000b0039c35b3e8c1mr8413573wmk.183.1654863927995; Fri, 10 Jun 2022 05:25:27 -0700 (PDT) Received: from naush-laptop.localdomain ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id e29-20020a5d595d000000b00213b93cff5fsm23097909wri.98.2022.06.10.05.25.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Jun 2022 05:25:27 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Fri, 10 Jun 2022 13:25:32 +0100 Message-Id: <20220610122533.11888-2-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220610122533.11888-1-naush@raspberrypi.com> References: <20220610122533.11888-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 2/3] pipeline: ipa: raspberrypi: Correctly report available control limits X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The ipa currently advertises a static ControlInfoMap to specify the available controls and their limits. Fix this limitation by having the IPA populate a new ControlInfoMap with updated limits for ExposureTime, AnalogueGain, and FrameDurationLimits controls based on the current sensor mode. This new ControlInfoMap is then returned back to the pipeline handler and available to the application after a successful Camera::Configure() call. Signed-off-by: Naushir Patuck Reviewed-by: Kieran Bingham --- include/libcamera/ipa/raspberrypi.mojom | 1 + src/ipa/raspberrypi/raspberrypi.cpp | 24 ++++++++++++++++--- .../pipeline/raspberrypi/raspberrypi.cpp | 3 +++ 3 files changed, 25 insertions(+), 3 deletions(-) diff --git a/include/libcamera/ipa/raspberrypi.mojom b/include/libcamera/ipa/raspberrypi.mojom index 77f52c282b0f..c0de435b7b33 100644 --- a/include/libcamera/ipa/raspberrypi.mojom +++ b/include/libcamera/ipa/raspberrypi.mojom @@ -45,6 +45,7 @@ struct IPAConfig { struct IPAConfigResult { float modeSensitivity; + libcamera.ControlInfoMap controlInfo; }; struct StartConfig { diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 089528f5e126..295f6b735dc0 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -74,8 +74,6 @@ constexpr Duration controllerMinFrameDuration = 1.0s / 30.0; /* List of controls handled by the Raspberry Pi IPA */ static const ControlInfoMap::Map ipaControls{ { &controls::AeEnable, ControlInfo(false, true) }, - { &controls::ExposureTime, ControlInfo(0, 999999) }, - { &controls::AnalogueGain, ControlInfo(1.0f, 32.0f) }, { &controls::AeMeteringMode, ControlInfo(controls::AeMeteringModeValues) }, { &controls::AeConstraintMode, ControlInfo(controls::AeConstraintModeValues) }, { &controls::AeExposureMode, ControlInfo(controls::AeExposureModeValues) }, @@ -89,7 +87,6 @@ static const ControlInfoMap::Map ipaControls{ { &controls::Sharpness, ControlInfo(0.0f, 16.0f, 1.0f) }, { &controls::ColourCorrectionMatrix, ControlInfo(-16.0f, 16.0f) }, { &controls::ScalerCrop, ControlInfo(Rectangle{}, Rectangle(65535, 65535, 65535, 65535), Rectangle{}) }, - { &controls::FrameDurationLimits, ControlInfo(INT64_C(1000), INT64_C(1000000000)) }, { &controls::draft::NoiseReductionMode, ControlInfo(controls::draft::NoiseReductionModeValues) } }; @@ -446,6 +443,27 @@ int IPARPi::configure(const IPACameraSensorInfo &sensorInfo, ASSERT(controls); *controls = std::move(ctrls); + /* + * Apply the correct limits to the exposure, gain and frame duration controls + * based on the current sensor mode. + */ + ControlInfoMap::Map ctrlMap = ipaControls; + const Duration minSensorFrameDuration = mode_.min_frame_length * mode_.line_length; + const Duration maxSensorFrameDuration = mode_.max_frame_length * mode_.line_length; + ctrlMap.emplace(&controls::FrameDurationLimits, + ControlInfo(static_cast(minSensorFrameDuration.get()), + static_cast(maxSensorFrameDuration.get()))); + + ctrlMap.emplace(&controls::AnalogueGain, + ControlInfo(1.0f, static_cast(helper_->Gain(maxSensorGainCode_)))); + + const uint32_t exposureMin = sensorCtrls_.at(V4L2_CID_EXPOSURE).min().get(); + const uint32_t exposureMax = sensorCtrls_.at(V4L2_CID_EXPOSURE).max().get(); + ctrlMap.emplace(&controls::ExposureTime, + ControlInfo(static_cast(helper_->Exposure(exposureMin).get()), + static_cast(helper_->Exposure(exposureMax).get()))); + + result->controlInfo = ControlInfoMap(std::move(ctrlMap), controls::controls); return 0; } diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index d980c1a71dd8..4596f2babcea 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -940,6 +940,9 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) /* Store the mode sensitivity for the application. */ data->properties_.set(properties::SensorSensitivity, result.modeSensitivity); + /* Update the controls that the Raspberry Pi IPA can handle. */ + data->controlInfo_ = result.controlInfo; + /* Setup the Video Mux/Bridge entities. */ for (auto &[device, link] : data->bridgeDevices_) { /* From patchwork Fri Jun 10 12:25:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 16196 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 95082BD160 for ; Fri, 10 Jun 2022 12:25:33 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4D0AD65638; Fri, 10 Jun 2022 14:25:33 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1654863933; bh=N/iX+qgcTLfLzyQQp6bpZhymY/EBic0Gs0P6wYW3az0=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=a/8lRfLnbHEac9vbXCYIzRzhh2R/hOhMrgLS6YZMUPlshDYiSLofTNA6jEXUMI/Te x65KqqvGgV6BnkZR0DrRpievMD83qvqgFkNKHBdfblFUfPHy0VdO68/3nBBj5sYfxx fLY/fp3+OTgK/JrwfPe5a7g/M1fV5P9BFTdWi2vyHdNhNNAK+uHNHnNXsM5l21ZvpO OznLdnYkfwjo2OLDTrApf0MgShi+Zd/sm9qofJWzrSJj8H91e8WwnrXrCqZkVLTTDN aEXxdnEvNhusBMcyzfuLeLpP/Z2yIkhYcNPCvr4e7KhB/FqrOkAnfoQJPA5sRqDQze 1gOLvSPkVFRQA== Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9475065638 for ; Fri, 10 Jun 2022 14:25:30 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="VYVO65Gr"; dkim-atps=neutral Received: by mail-wm1-x32e.google.com with SMTP id i17-20020a7bc951000000b0039c4760ec3fso1977351wml.0 for ; Fri, 10 Jun 2022 05:25:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=VgFx3Qp83lIvR05neP8MTvYhfy0+XkOHzCIAe5UkSbs=; b=VYVO65Gri4cTNgtISLQCEW5NFJyWkkn9alDg7TuNoxOEgJ9DmBjTYOEoUN4riFixvZ 5OMkS8/64OuWiclDxH2H08UVa8D9HIDH5iwdL7Fq5gWMTOfjjF3CcuoQNuzCaas3uRwe SNXNuia6aHlfUU5gCVHuRb8XA5/6Ohw2SC1TtPA+xJVAQeoafcTpcoiKuNe8wHXtJugB amH1q6H9s+bWbW5VWWHBSuE7AVmS3DbPFJ5Mj7S6mZex0fMWKh0pqJD1isReqTVWkKbt OJkxgZQWSv0ULcm4S5o1G35265/Spt4TUXfBADlcqG36z5Mf136D7AbchiHTREMNoZ1S KjxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=VgFx3Qp83lIvR05neP8MTvYhfy0+XkOHzCIAe5UkSbs=; b=kw1U7m1YgqZwzQzVAtwZvNngjLMKYJC/GW9LMLUR+98Zo2chdjSjB5qYZdabBMpNg2 PdeAf/AnTwo6hxS3iWVdhJxM7K0mrlrRItWYrZdFff8ovPF5wMPRkrU1a0neyBBDzdb2 qwix1pnpNJS5zKOzxH/yTb4yhpl4vr3hmDe/EMNMDBYUkWO4Zh+CuFD/2dsd9igWJZSR 2EdlSiHx7dpRcJJEPxQAYW0tNp45QeEnrThIIfAKfFBJ3sHxui+GZpyL7+JQz7BDNgOg ooMi0ZEBd0SIndAOhLjicD+olGxY+4MGVbxqVHdVYsaMhf4xTTkWuJ8650w46mzh1HMX bR5g== X-Gm-Message-State: AOAM533ZegD8ze3yPStwyqkktlU9KzYSwZujOtKm8XwZzPeJ7kkakL6X rX+hI1B373Xu8uoVb7Ytdr/fvdCrmIjw6Q== X-Google-Smtp-Source: ABdhPJzubZvwCIdqwZnDjhITbwIZEvnxEJVgTSlGKnT+/2AP3krPn6MRczFBbiVHaka2C+Te2rDs1A== X-Received: by 2002:a1c:4c12:0:b0:39c:6750:be17 with SMTP id z18-20020a1c4c12000000b0039c6750be17mr8842247wmf.21.1654863929615; Fri, 10 Jun 2022 05:25:29 -0700 (PDT) Received: from naush-laptop.localdomain ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id e29-20020a5d595d000000b00213b93cff5fsm23097909wri.98.2022.06.10.05.25.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Jun 2022 05:25:28 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Fri, 10 Jun 2022 13:25:33 +0100 Message-Id: <20220610122533.11888-3-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220610122533.11888-1-naush@raspberrypi.com> References: <20220610122533.11888-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 3/3] pipeline: raspberrypi: Advertise ScalerCrop from the pipeline handler X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The ScalerCrop control is handled directly by the pipeline handler. Remove the control from the IPA's static ControlInfoMap, and let the pipeline handler add it to the ControlInfoMap advertised to the application, ensuring the limits are set appropriately based on the current sensor mode. Signed-off-by: Naushir Patuck --- src/ipa/raspberrypi/raspberrypi.cpp | 1 - src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 10 +++++++++- 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 295f6b735dc0..f46fccdd4177 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -86,7 +86,6 @@ static const ControlInfoMap::Map ipaControls{ { &controls::Saturation, ControlInfo(0.0f, 32.0f, 1.0f) }, { &controls::Sharpness, ControlInfo(0.0f, 16.0f, 1.0f) }, { &controls::ColourCorrectionMatrix, ControlInfo(-16.0f, 16.0f) }, - { &controls::ScalerCrop, ControlInfo(Rectangle{}, Rectangle(65535, 65535, 65535, 65535), Rectangle{}) }, { &controls::draft::NoiseReductionMode, ControlInfo(controls::draft::NoiseReductionModeValues) } }; diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 4596f2babcea..66a84b1dfb97 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -941,7 +941,15 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) data->properties_.set(properties::SensorSensitivity, result.modeSensitivity); /* Update the controls that the Raspberry Pi IPA can handle. */ - data->controlInfo_ = result.controlInfo; + ControlInfoMap::Map ctrlMap; + for (auto const &c : result.controlInfo) + ctrlMap.emplace(c.first, c.second); + + /* Add the ScalerCrop control limits based on the current mode. */ + ctrlMap.emplace(&controls::ScalerCrop, + ControlInfo(Rectangle(data->ispMinCropSize_), Rectangle(data->sensorInfo_.outputSize))); + + data->controlInfo_ = ControlInfoMap(std::move(ctrlMap), result.controlInfo.idmap()); /* Setup the Video Mux/Bridge entities. */ for (auto &[device, link] : data->bridgeDevices_) {