From patchwork Mon Jun 27 16:27:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 16391 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 52A46BD808 for ; Mon, 27 Jun 2022 16:27:59 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CAA0565653; Mon, 27 Jun 2022 18:27:58 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1656347279; bh=kOEVtb1pxIg4nalpzYb9LbdT3uzPqrarMrdtPenImMU=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=aY4+VdEswtT9JEd3fBJ7BOCcSYKcpqVwG9D0Yjn3JCDrtNHYEc7eSITRCtMEBxKS4 sLb/eXL8Uo/SNVWa7D0N1539zNPVbPAtU0BMZk+15KjsaUKwKGpIaFUmsKx14rMnlD WXqiCX3t60EMLgaQvGjYoiilgZKAXQOhhtbwXh7eYa2YfGLRn0N7//9yxE5bpVKvOu IqlIFhjwKJnw9pLQ/z8abPrdYa7Gzky06hOn5EZSy38jVnMuAcXXbOXase9tVWZl8u muXiJ4JtR5KoA2tKxBoHEj6CDw5r1KIyCVlYcCG2luUiGHA0LIboyWAm+QQfP9PgCp tvd9K9lpzdvDw== Received: from relay7-d.mail.gandi.net (relay7-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9EEC86564C for ; Mon, 27 Jun 2022 18:27:50 +0200 (CEST) Received: (Authenticated sender: jacopo@jmondi.org) by mail.gandi.net (Postfix) with ESMTPSA id 82A7620009; Mon, 27 Jun 2022 16:27:49 +0000 (UTC) To: libcamera-devel@lists.libcamera.org Date: Mon, 27 Jun 2022 18:27:27 +0200 Message-Id: <20220627162732.33160-11-jacopo@jmondi.org> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220627162732.33160-1-jacopo@jmondi.org> References: <20220627162732.33160-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 10/15] libcamera: ipu3: Initialize Camera controls in the PH X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Currently the map of V4L2 controls collected from the sensor is passed to the IPA module which initializes Camera controls by inspecting the sensor control values. As the CameraSensor class translates V4L2 controls into libcamera::internal controls, the Camera controls can be initialized directly in the pipeline handler, without sending them to the IPA which receives all the information it needs using the CameraSensorInfo. Remove from the IPA interface the list of V4L2 controls from the sensor sent to the IPA as it is not used anymore and initialize Camera controls in the pipeline directly. Signed-off-by: Jacopo Mondi --- include/libcamera/ipa/ipu3.mojom | 7 +-- src/ipa/ipu3/ipu3.cpp | 87 ++-------------------------- src/libcamera/pipeline/ipu3/ipu3.cpp | 37 +++++++----- 3 files changed, 31 insertions(+), 100 deletions(-) diff --git a/include/libcamera/ipa/ipu3.mojom b/include/libcamera/ipa/ipu3.mojom index d1b1c6b867da..5818cd3a6846 100644 --- a/include/libcamera/ipa/ipu3.mojom +++ b/include/libcamera/ipa/ipu3.mojom @@ -18,14 +18,13 @@ struct IPAConfigInfo { interface IPAIPU3Interface { init(libcamera.IPASettings settings, - libcamera.IPACameraSensorInfo sensorInfo, - libcamera.ControlInfoMap sensorControls) - => (int32 ret, libcamera.ControlInfoMap ipaControls); + libcamera.IPACameraSensorInfo sensorInfo) + => (int32 ret); start() => (int32 ret); stop(); configure(IPAConfigInfo configInfo) - => (int32 ret, libcamera.ControlInfoMap ipaControls); + => (int32 ret); mapBuffers(array buffers); unmapBuffers(array ids); diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 2f6bb672f7bb..7e8bf0feee8a 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -28,6 +28,7 @@ #include #include +#include "libcamera/internal/control_ids.h" #include "libcamera/internal/mapped_framebuffer.h" #include "algorithms/af.h" @@ -132,15 +133,12 @@ class IPAIPU3 : public IPAIPU3Interface { public: int init(const IPASettings &settings, - const IPACameraSensorInfo &sensorInfo, - const ControlInfoMap &sensorControls, - ControlInfoMap *ipaControls) override; + const IPACameraSensorInfo &sensorInfo) override; int start() override; void stop() override; - int configure(const IPAConfigInfo &configInfo, - ControlInfoMap *ipaControls) override; + int configure(const IPAConfigInfo &configInfo) override; void mapBuffers(const std::vector &buffers) override; void unmapBuffers(const std::vector &ids) override; @@ -151,9 +149,6 @@ public: const uint32_t bufferId, const ControlList &sensorControls) override; private: - void updateControls(const IPACameraSensorInfo &sensorInfo, - const ControlInfoMap &sensorControls, - ControlInfoMap *ipaControls); void updateSessionConfiguration(const ControlInfoMap &sensorControls); bool validateSensorControls(); @@ -208,68 +203,6 @@ void IPAIPU3::updateSessionConfiguration(const ControlInfoMap &sensorControls) context_.configuration.agc.maxAnalogueGain = camHelper_->gain(maxGain); } -/** - * \brief Compute camera controls using the sensor information and the sensor - * V4L2 controls - * - * Some of the camera controls are computed by the pipeline handler, some others - * by the IPA module which is in charge of handling, for example, the exposure - * time and the frame duration. - * - * This function computes: - * - controls::ExposureTime - * - controls::FrameDurationLimits - */ -void IPAIPU3::updateControls(const IPACameraSensorInfo &sensorInfo, - const ControlInfoMap &sensorControls, - ControlInfoMap *ipaControls) -{ - ControlInfoMap::Map controls{}; - double lineDuration = context_.configuration.sensor.lineDuration.get(); - - /* - * Compute exposure time limits by using line length and pixel rate - * converted to microseconds. Use the V4L2_CID_EXPOSURE control to get - * exposure min, max and default and convert it from lines to - * microseconds. - */ - const ControlInfo &v4l2Exposure = sensorControls.find(V4L2_CID_EXPOSURE)->second; - int32_t minExposure = v4l2Exposure.min().get() * lineDuration; - int32_t maxExposure = v4l2Exposure.max().get() * lineDuration; - int32_t defExposure = v4l2Exposure.def().get() * lineDuration; - controls[&controls::ExposureTime] = ControlInfo(minExposure, maxExposure, - defExposure); - - /* - * Compute the frame duration limits. - * - * The frame length is computed assuming a fixed line length combined - * with the vertical frame sizes. - */ - const ControlInfo &v4l2HBlank = sensorControls.find(V4L2_CID_HBLANK)->second; - uint32_t hblank = v4l2HBlank.def().get(); - uint32_t lineLength = sensorInfo.outputSize.width + hblank; - - const ControlInfo &v4l2VBlank = sensorControls.find(V4L2_CID_VBLANK)->second; - std::array frameHeights{ - v4l2VBlank.min().get() + sensorInfo.outputSize.height, - v4l2VBlank.max().get() + sensorInfo.outputSize.height, - v4l2VBlank.def().get() + sensorInfo.outputSize.height, - }; - - std::array frameDurations; - for (unsigned int i = 0; i < frameHeights.size(); ++i) { - uint64_t frameSize = lineLength * frameHeights[i]; - frameDurations[i] = frameSize / (sensorInfo.pixelRate / 1000000U); - } - - controls[&controls::FrameDurationLimits] = ControlInfo(frameDurations[0], - frameDurations[1], - frameDurations[2]); - - *ipaControls = ControlInfoMap(std::move(controls), controls::controls); -} - /** * \brief Validate that the sensor controls mandatory for the IPA exists */ @@ -300,9 +233,7 @@ bool IPAIPU3::validateSensorControls() * them in the \a ipaControls output parameter. */ int IPAIPU3::init(const IPASettings &settings, - const IPACameraSensorInfo &sensorInfo, - const ControlInfoMap &sensorControls, - ControlInfoMap *ipaControls) + const IPACameraSensorInfo &sensorInfo) { camHelper_ = CameraSensorHelperFactory::create(settings.sensorModel); if (camHelper_ == nullptr) { @@ -323,9 +254,6 @@ int IPAIPU3::init(const IPASettings &settings, algorithms_.push_back(std::make_unique()); algorithms_.push_back(std::make_unique()); - /* Initialize controls. */ - updateControls(sensorInfo, sensorControls, ipaControls); - return 0; } @@ -423,7 +351,6 @@ void IPAIPU3::calculateBdsGrid(const Size &bdsOutputSize) * \brief Configure the IPU3 IPA * \param[in] configInfo The IPA configuration data, received from the pipeline * handler - * \param[in] ipaControls The IPA controls to update * * Calculate the best grid for the statistics based on the pipeline handler BDS * output, and parse the minimum and maximum exposure and analogue gain control @@ -434,8 +361,7 @@ void IPAIPU3::calculateBdsGrid(const Size &bdsOutputSize) * All algorithm modules are called to allow them to prepare the * \a IPASessionConfiguration structure for the \a IPAContext. */ -int IPAIPU3::configure(const IPAConfigInfo &configInfo, - ControlInfoMap *ipaControls) +int IPAIPU3::configure(const IPAConfigInfo &configInfo) { if (configInfo.sensorControls.empty()) { LOG(IPAIPU3, Error) << "No sensor controls provided"; @@ -464,9 +390,6 @@ int IPAIPU3::configure(const IPAConfigInfo &configInfo, return -EINVAL; } - /* Update the camera controls using the new sensor settings. */ - updateControls(sensorInfo_, sensorCtrls_, ipaControls); - /* Update the IPASessionConfiguration using the sensor settings. */ updateSessionConfiguration(sensorCtrls_); diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 28442756b184..815530dd640b 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -26,6 +26,7 @@ #include "libcamera/internal/camera.h" #include "libcamera/internal/camera_lens.h" #include "libcamera/internal/camera_sensor.h" +#include "libcamera/internal/control_ids.h" #include "libcamera/internal/delayed_controls.h" #include "libcamera/internal/device_enumerator.h" #include "libcamera/internal/framebuffer.h" @@ -84,8 +85,6 @@ public: /* Requests queued to the CIO2 device but not yet processed by the ImgU. */ std::queue processingRequests_; - ControlInfoMap ipaControls_; - private: void metadataReady(unsigned int id, const ControlList &metadata); void paramsBufferReady(unsigned int id); @@ -672,7 +671,7 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) configInfo.bdsOutputSize = config->imguConfig().bds; configInfo.iif = config->imguConfig().iif; - ret = data->ipa_->configure(configInfo, &data->ipaControls_); + ret = data->ipa_->configure(configInfo); if (ret) { LOG(IPU3, Error) << "Failed to configure IPA: " << strerror(-ret); @@ -950,9 +949,6 @@ bool PipelineHandlerIPU3::match(DeviceEnumerator *enumerator) * Initialize the camera controls by calculating controls which the pipeline * is reponsible for and merge them with the controls computed by the IPA. * - * This function needs data->ipaControls_ to be initialized by the IPA init() - * function at camera creation time. Always call this function after IPA init(). - * * \return 0 on success or a negative error code otherwise */ int PipelineHandlerIPU3::initControls(IPU3CameraData *data) @@ -982,9 +978,6 @@ int PipelineHandlerIPU3::initControls(IPU3CameraData *data) * Compute the camera controls by calculating controls which the pipeline * is reponsible for and merge them with the controls computed by the IPA. * - * This function needs data->ipaControls_ to be refreshed when a new - * configuration is applied to the camera by the IPA configure() function. - * * Always call this function after IPA configure() to make sure to have a * properly refreshed IPA controls list. * @@ -1063,9 +1056,26 @@ int PipelineHandlerIPU3::updateControls(IPU3CameraData *data) controls[&controls::ScalerCrop] = ControlInfo(minCrop, maxCrop, maxCrop); - /* Add the IPA registered controls to list of camera controls. */ - for (const auto &ipaControl : data->ipaControls_) - controls[ipaControl.first] = ipaControl.second; + /* Translate the sensor controls to Camera controls. */ + const ControlInfoMap &sensorControls = sensor->controls(); + + /* Exposure time is reported directly from the sensor control limits. */ + if (sensorControls.count(&controls::internal::ExposureTime)) { + auto &[id, info] = *sensorControls.find(&controls::internal::ExposureTime); + + controls[&controls::ExposureTime] = ControlInfo(info.min().get(), + info.max().get(), + info.def().get()); + } + + /* Frame duration is reported directly from the sensor control limits. */ + if (sensorControls.count(&controls::internal::FrameDuration)) { + auto &[id, info] = *sensorControls.find(&controls::internal::FrameDuration); + + controls[&controls::FrameDurationLimits] = ControlInfo(info.min().get(), + info.max().get(), + info.def().get()); + } data->controlInfo_ = ControlInfoMap(std::move(controls), controls::controls); @@ -1262,8 +1272,7 @@ int IPU3CameraData::loadIPA() if (ret) return ret; - ret = ipa_->init(IPASettings{ "", sensor->model() }, sensorInfo, - sensor->v4l2Controls(), &ipaControls_); + ret = ipa_->init(IPASettings{ "", sensor->model() }, sensorInfo); if (ret) { LOG(IPU3, Error) << "Failed to initialise the IPU3 IPA"; return ret;