From patchwork Wed Jun 22 10:20:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 16307 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 33663BD808 for ; Wed, 22 Jun 2022 10:21:02 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 700E16563A; Wed, 22 Jun 2022 12:21:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1655893261; bh=k3T80DdORNI2PylIheqo2ZV0WyCsLU4hQlEpLXcguDU=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=akeQG0AFFz0W8bcf7Ph+hJQ18+QPhqdvpxx6GvF2Vmqw02bGIQIEKxjRkZTXIgyD1 TC3Qi98xVQDHUIZbyWdHrEm+i0ZWy3zns4h5Qzq98g8W24a8ifYCiLWg63FpnaGjpM 5WRB6q0zCrxv5H9cw1T6icfi5prxic934aAX1/a+i4+Gw8FTXnep8MpSUGO+Qhy4sM 3ieBIBIU1wf0ptoO6NELOt5NW3r24mAaOWbKr7a1J8i3qlQNVOcaX383Y0+pkQ7750 GfHGu/B3Ldkn3VchV8E4hh1GwLkLTg1DQ9Zxm21KVHIjstRrhC0MA76Ij5ks/lu0Ru 9WK2M1yYfAxCQ== Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1EC1561FB2 for ; Wed, 22 Jun 2022 12:20:58 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="F06UQsCl"; dkim-atps=neutral Received: by mail-wr1-x42f.google.com with SMTP id g27so16061038wrb.10 for ; Wed, 22 Jun 2022 03:20:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HlMG9zb1PmYaoA6v2JC3+vcDPjbEt84nv5H1Rmzpq4A=; b=F06UQsClKXiCKv14uXQf7UFhotTfhuU/DeUAvcSmA9f6V0iyJL2nGtyFVkJ/vHWJms 5QgncYd5xE9nIjrW/VRJH9F50MrSaGsNgsrCBB4yMsvCLl7Awff0cqCMertp0AmE+eeT 7FNiOVxBzUAWcxgHzOBveAzut/CiJQwrwF0nFKBXlnsKPpNF4mWxqp0IKiYRRnDVfBrD YDB2kxJ31aZ0iCGe6XvMIDL1lIWWBwfrscI4EIsb9sf/Kjq4Ht9/rULK6C4ERJJTdLXA 2zjFaw+0GRaNARamRYOls1nZCcBjx2LciM86105a8pz4gOcurj7EnAbTnHS8GjyJ6qRN omaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HlMG9zb1PmYaoA6v2JC3+vcDPjbEt84nv5H1Rmzpq4A=; b=NCR3enqBzaoBMu+/duFjD5IlW55hkKog1kZ03tY+iU74eR1AbfvzEr93sfpThvgcRe rna2bhxfxBb/FA7md0e8TtTh6SsnbdXoJ7GozHKiFTSDnuNHiPdIQ+cf9QZzxWOJQqZ5 nEtsvI/tW2jLBWcdzRTO02jW80K7ZMLMuTSQtTtDsUzlSJzkpx0I20ySz+MRLPUR4mTa EuUQBlrdySJM5uCdm5aBajEHXlZQ2tOJ/f9hFVAertsIN54mCunTUoBKcLhCtzoAt3pf ylqHzao8/5h+AnRRTrSfGTnsogmrkGF6CJ1ae75fd5SaSq08nSxghBSV39H5bWVplVHP Z+fA== X-Gm-Message-State: AJIora/Ef71kHoTnv9djUFuA/33dtPbldsW09scyWqUFbU4rUQAjqolU RFx5XNuI3wWz1oab7bLHNzhg7WaBjDXl8Y36 X-Google-Smtp-Source: AGRyM1t9zV3nij/ox7UsMJgBw8bi7FfTzEGTwpXS4tk0Mfm47U2SE5XUs4SuzExZQYVIAx7k5QaH5g== X-Received: by 2002:adf:fa8f:0:b0:21b:9811:43e7 with SMTP id h15-20020adffa8f000000b0021b981143e7mr2587819wrr.275.1655893257387; Wed, 22 Jun 2022 03:20:57 -0700 (PDT) Received: from naush-laptop.localdomain ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id h204-20020a1c21d5000000b0039c693a54ecsm26849423wmh.23.2022.06.22.03.20.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Jun 2022 03:20:56 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Wed, 22 Jun 2022 11:20:45 +0100 Message-Id: <20220622102047.22492-2-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220622102047.22492-1-naush@raspberrypi.com> References: <20220622102047.22492-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 1/3] pipeline: ipa: raspberrypi: Move ControlInfoMap to the IPA X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Currently the pipeline handler advertises controls handled by the IPA from a static ControlInfoMap defined in the raspberrypi.h header. This change removes this header file, and instead the IPA returns the ControlInfoMap to the pipeline handler from the ipa::init() function. This is done to allow the IPA to adjust the limits of the controls based on the sensor mode in a subsequent change. Bug: https://bugs.libcamera.org/show_bug.cgi?id=83 Signed-off-by: Naushir Patuck Reviewed-by: Kieran Bingham Reviewed-by: David Plowman --- include/libcamera/ipa/raspberrypi.h | 55 ------------------- include/libcamera/ipa/raspberrypi.mojom | 7 ++- src/ipa/raspberrypi/raspberrypi.cpp | 39 ++++++++++--- .../pipeline/raspberrypi/raspberrypi.cpp | 30 +++++----- .../pipeline/raspberrypi/rpi_stream.h | 1 - 5 files changed, 53 insertions(+), 79 deletions(-) delete mode 100644 include/libcamera/ipa/raspberrypi.h diff --git a/include/libcamera/ipa/raspberrypi.h b/include/libcamera/ipa/raspberrypi.h deleted file mode 100644 index 6a56b0083b85..000000000000 --- a/include/libcamera/ipa/raspberrypi.h +++ /dev/null @@ -1,55 +0,0 @@ -/* SPDX-License-Identifier: LGPL-2.1-or-later */ -/* - * Copyright (C) 2019-2020, Raspberry Pi Ltd. - * - * raspberrypi.h - Image Processing Algorithm interface for Raspberry Pi - */ - -#pragma once - -#include - -#include -#include - -#ifndef __DOXYGEN__ - -namespace libcamera { - -namespace RPi { - -/* - * List of controls handled by the Raspberry Pi IPA - * - * \todo This list will need to be built dynamically from the control - * algorithms loaded by the json file, once this is supported. At that - * point applications should check first whether a control is supported, - * and the pipeline handler may be reverted so that it aborts when an - * unsupported control is encountered. - */ -static const ControlInfoMap Controls({ - { &controls::AeEnable, ControlInfo(false, true) }, - { &controls::ExposureTime, ControlInfo(0, 999999) }, - { &controls::AnalogueGain, ControlInfo(1.0f, 32.0f) }, - { &controls::AeMeteringMode, ControlInfo(controls::AeMeteringModeValues) }, - { &controls::AeConstraintMode, ControlInfo(controls::AeConstraintModeValues) }, - { &controls::AeExposureMode, ControlInfo(controls::AeExposureModeValues) }, - { &controls::ExposureValue, ControlInfo(-8.0f, 8.0f, 0.0f) }, - { &controls::AwbEnable, ControlInfo(false, true) }, - { &controls::ColourGains, ControlInfo(0.0f, 32.0f) }, - { &controls::AwbMode, ControlInfo(controls::AwbModeValues) }, - { &controls::Brightness, ControlInfo(-1.0f, 1.0f, 0.0f) }, - { &controls::Contrast, ControlInfo(0.0f, 32.0f, 1.0f) }, - { &controls::Saturation, ControlInfo(0.0f, 32.0f, 1.0f) }, - { &controls::Sharpness, ControlInfo(0.0f, 16.0f, 1.0f) }, - { &controls::ColourCorrectionMatrix, ControlInfo(-16.0f, 16.0f) }, - { &controls::ScalerCrop, ControlInfo(Rectangle{}, Rectangle(65535, 65535, 65535, 65535), Rectangle{}) }, - { &controls::FrameDurationLimits, ControlInfo(INT64_C(1000), INT64_C(1000000000)) }, - { &controls::draft::NoiseReductionMode, ControlInfo(controls::draft::NoiseReductionModeValues) } - }, controls::controls); - -} /* namespace RPi */ - -} /* namespace libcamera */ - -#endif /* __DOXYGEN__ */ diff --git a/include/libcamera/ipa/raspberrypi.mojom b/include/libcamera/ipa/raspberrypi.mojom index a60c3bb43d3c..77f52c282b0f 100644 --- a/include/libcamera/ipa/raspberrypi.mojom +++ b/include/libcamera/ipa/raspberrypi.mojom @@ -26,6 +26,11 @@ struct SensorConfig { uint32 sensorMetadata; }; +struct IPAInitResult { + SensorConfig sensorConfig; + libcamera.ControlInfoMap controlInfo; +}; + struct ISPConfig { uint32 embeddedBufferId; uint32 bayerBufferId; @@ -50,7 +55,7 @@ struct StartConfig { interface IPARPiInterface { init(libcamera.IPASettings settings) - => (int32 ret, SensorConfig sensorConfig); + => (int32 ret, IPAInitResult result); start(libcamera.ControlList controls) => (StartConfig startConfig); stop(); diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 3b126bb5175e..089528f5e126 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -24,7 +24,6 @@ #include #include #include -#include #include #include @@ -72,6 +71,28 @@ constexpr Duration defaultMaxFrameDuration = 250.0s; */ constexpr Duration controllerMinFrameDuration = 1.0s / 30.0; +/* List of controls handled by the Raspberry Pi IPA */ +static const ControlInfoMap::Map ipaControls{ + { &controls::AeEnable, ControlInfo(false, true) }, + { &controls::ExposureTime, ControlInfo(0, 999999) }, + { &controls::AnalogueGain, ControlInfo(1.0f, 32.0f) }, + { &controls::AeMeteringMode, ControlInfo(controls::AeMeteringModeValues) }, + { &controls::AeConstraintMode, ControlInfo(controls::AeConstraintModeValues) }, + { &controls::AeExposureMode, ControlInfo(controls::AeExposureModeValues) }, + { &controls::ExposureValue, ControlInfo(-8.0f, 8.0f, 0.0f) }, + { &controls::AwbEnable, ControlInfo(false, true) }, + { &controls::ColourGains, ControlInfo(0.0f, 32.0f) }, + { &controls::AwbMode, ControlInfo(controls::AwbModeValues) }, + { &controls::Brightness, ControlInfo(-1.0f, 1.0f, 0.0f) }, + { &controls::Contrast, ControlInfo(0.0f, 32.0f, 1.0f) }, + { &controls::Saturation, ControlInfo(0.0f, 32.0f, 1.0f) }, + { &controls::Sharpness, ControlInfo(0.0f, 16.0f, 1.0f) }, + { &controls::ColourCorrectionMatrix, ControlInfo(-16.0f, 16.0f) }, + { &controls::ScalerCrop, ControlInfo(Rectangle{}, Rectangle(65535, 65535, 65535, 65535), Rectangle{}) }, + { &controls::FrameDurationLimits, ControlInfo(INT64_C(1000), INT64_C(1000000000)) }, + { &controls::draft::NoiseReductionMode, ControlInfo(controls::draft::NoiseReductionModeValues) } +}; + LOG_DEFINE_CATEGORY(IPARPI) namespace ipa::RPi { @@ -91,7 +112,7 @@ public: munmap(lsTable_, MaxLsGridSize); } - int init(const IPASettings &settings, SensorConfig *sensorConfig) override; + int init(const IPASettings &settings, IPAInitResult *result) override; void start(const ControlList &controls, StartConfig *startConfig) override; void stop() override {} @@ -180,7 +201,7 @@ private: uint32_t maxSensorGainCode_; }; -int IPARPi::init(const IPASettings &settings, SensorConfig *sensorConfig) +int IPARPi::init(const IPASettings &settings, IPAInitResult *result) { /* * Load the "helper" for this sensor. This tells us all the device specific stuff @@ -202,15 +223,19 @@ int IPARPi::init(const IPASettings &settings, SensorConfig *sensorConfig) helper_->GetDelays(exposureDelay, gainDelay, vblankDelay); sensorMetadata = helper_->SensorEmbeddedDataPresent(); - sensorConfig->gainDelay = gainDelay; - sensorConfig->exposureDelay = exposureDelay; - sensorConfig->vblankDelay = vblankDelay; - sensorConfig->sensorMetadata = sensorMetadata; + result->sensorConfig.gainDelay = gainDelay; + result->sensorConfig.exposureDelay = exposureDelay; + result->sensorConfig.vblankDelay = vblankDelay; + result->sensorConfig.sensorMetadata = sensorMetadata; /* Load the tuning file for this sensor. */ controller_.Read(settings.configurationFile.c_str()); controller_.Initialise(); + /* Return the controls handled by the IPA */ + ControlInfoMap::Map ctrlMap = ipaControls; + result->controlInfo = ControlInfoMap(std::move(ctrlMap), controls::controls); + return 0; } diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index adc397e8aabd..d980c1a71dd8 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -20,7 +20,6 @@ #include #include #include -#include #include #include #include @@ -199,7 +198,7 @@ public: void freeBuffers(); void frameStarted(uint32_t sequence); - int loadIPA(ipa::RPi::SensorConfig *sensorConfig); + int loadIPA(ipa::RPi::IPAInitResult *result); int configureIPA(const CameraConfiguration *config, ipa::RPi::IPAConfigResult *result); void enumerateVideoDevices(MediaLink *link); @@ -1231,15 +1230,15 @@ int PipelineHandlerRPi::registerCamera(MediaDevice *unicam, MediaDevice *isp, Me data->sensorFormats_ = populateSensorFormats(data->sensor_); - ipa::RPi::SensorConfig sensorConfig; - if (data->loadIPA(&sensorConfig)) { + ipa::RPi::IPAInitResult result; + if (data->loadIPA(&result)) { LOG(RPI, Error) << "Failed to load a suitable IPA library"; return -EINVAL; } - if (sensorConfig.sensorMetadata ^ !!unicamEmbedded) { + if (result.sensorConfig.sensorMetadata ^ !!unicamEmbedded) { LOG(RPI, Warning) << "Mismatch between Unicam and CamHelper for embedded data usage!"; - sensorConfig.sensorMetadata = false; + result.sensorConfig.sensorMetadata = false; if (unicamEmbedded) data->unicam_[Unicam::Embedded].dev()->bufferReady.disconnect(); } @@ -1253,7 +1252,7 @@ int PipelineHandlerRPi::registerCamera(MediaDevice *unicam, MediaDevice *isp, Me * iterate over all streams in one go. */ data->streams_.push_back(&data->unicam_[Unicam::Image]); - if (sensorConfig.sensorMetadata) + if (result.sensorConfig.sensorMetadata) data->streams_.push_back(&data->unicam_[Unicam::Embedded]); for (auto &stream : data->isp_) @@ -1275,15 +1274,16 @@ int PipelineHandlerRPi::registerCamera(MediaDevice *unicam, MediaDevice *isp, Me * gain and exposure delays. Mark VBLANK for priority write. */ std::unordered_map params = { - { V4L2_CID_ANALOGUE_GAIN, { sensorConfig.gainDelay, false } }, - { V4L2_CID_EXPOSURE, { sensorConfig.exposureDelay, false } }, - { V4L2_CID_VBLANK, { sensorConfig.vblankDelay, true } } + { V4L2_CID_ANALOGUE_GAIN, { result.sensorConfig.gainDelay, false } }, + { V4L2_CID_EXPOSURE, { result.sensorConfig.exposureDelay, false } }, + { V4L2_CID_VBLANK, { result.sensorConfig.vblankDelay, true } } }; data->delayedCtrls_ = std::make_unique(data->sensor_->device(), params); - data->sensorMetadata_ = sensorConfig.sensorMetadata; + data->sensorMetadata_ = result.sensorConfig.sensorMetadata; + + /* Register initial controls that the Raspberry Pi IPA can handle. */ + data->controlInfo_ = std::move(result.controlInfo); - /* Register the controls that the Raspberry Pi IPA can handle. */ - data->controlInfo_ = RPi::Controls; /* Initialize the camera properties. */ data->properties_ = data->sensor_->properties(); @@ -1509,7 +1509,7 @@ void RPiCameraData::frameStarted(uint32_t sequence) delayedCtrls_->applyControls(sequence); } -int RPiCameraData::loadIPA(ipa::RPi::SensorConfig *sensorConfig) +int RPiCameraData::loadIPA(ipa::RPi::IPAInitResult *result) { ipa_ = IPAManager::createIPA(pipe(), 1, 1); @@ -1535,7 +1535,7 @@ int RPiCameraData::loadIPA(ipa::RPi::SensorConfig *sensorConfig) IPASettings settings(configurationFile, sensor_->model()); - return ipa_->init(settings, sensorConfig); + return ipa_->init(settings, result); } int RPiCameraData::configureIPA(const CameraConfiguration *config, ipa::RPi::IPAConfigResult *result) diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.h b/src/libcamera/pipeline/raspberrypi/rpi_stream.h index c37f7e82eef6..fe01110019b7 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.h +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.h @@ -12,7 +12,6 @@ #include #include -#include #include #include