From patchwork Mon Jun 7 12:35:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 12500 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EA94EC3206 for ; Mon, 7 Jun 2021 12:35:39 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3AECE6892D; Mon, 7 Jun 2021 14:35:39 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Pij1Vy3s"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 93244602A1 for ; Mon, 7 Jun 2021 14:35:37 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:b035:2cc2:4506:d7d3]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 1FE328DB; Mon, 7 Jun 2021 14:35:37 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1623069337; bh=iBFlBXj9i0YtVRe4QlVkkiatzxpxQbUU24rny0ox4Ew=; h=From:To:Cc:Subject:Date:From; b=Pij1Vy3sUz4SbYY+QV3t6FjUDyMX+JvIXgcv6RuzAIEV8IDttBG2W998zjC5NHPU0 ALJ3cSJI9fOEVXkVpjk2zn7bnWd11FUgq98V76CeFS0V0mioJBfXgO0Ab7/MhmXe09 dAvvs40oR2p5awLwI9AntIa8eZHX3FgSCrSJqLLU= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Mon, 7 Jun 2021 14:35:32 +0200 Message-Id: <20210607123533.51198-1-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 1/2] ipa: Create a camera sensor helper class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Setting analogue gain for a specific sensor is not a straghtforward operation, as one needs to know how the gain is calculated for it. This commit introduces a new camera sensor helper in libipa which aims to solve this specific issue. It is based on the MIPI alliance Specification for Camera Command Set and implements, for now, only the analogue "Global gain" mode. Is makes it possible to only have 4 parameters to store per sensor, and the gain calculation is then done identically for all of them. This commit gives the gains for imx219 (based on RPi cam_helper), ov5670 and ov5693. Adding a new sensor is pretty straightfoward as one only needs to implement the sub-class for it and register that class to the CameraSensorHelperFactory. Signed-off-by: Jean-Michel Hautbois --- src/ipa/libipa/camera_sensor_helper.cpp | 361 ++++++++++++++++++++++++ src/ipa/libipa/camera_sensor_helper.h | 91 ++++++ src/ipa/libipa/meson.build | 2 + 3 files changed, 454 insertions(+) create mode 100644 src/ipa/libipa/camera_sensor_helper.cpp create mode 100644 src/ipa/libipa/camera_sensor_helper.h diff --git a/src/ipa/libipa/camera_sensor_helper.cpp b/src/ipa/libipa/camera_sensor_helper.cpp new file mode 100644 index 00000000..2bd82861 --- /dev/null +++ b/src/ipa/libipa/camera_sensor_helper.cpp @@ -0,0 +1,361 @@ +/* SPDX-License-Identifier: BSD-2-Clause */ +/* + * Copyright (C) 2021 Ideas On Board + * + * camera_sensor_helper.cpp - helper class providing camera informations + */ +#include "camera_sensor_helper.h" + +#include + +#include "libcamera/internal/log.h" + +/** + * \file camera_sensor_helper.h + * \brief Create helper class for each sensor + * + * Each camera sensor supported by libcamera may need specific informations to + * be sent (analogue gain is sensor dependant for instance). + * + * Every subclass of CameraSensorHelper shall be registered with libipa using + * the REGISTER_CAMERA_SENSOR_HELPER() macro. + */ + +namespace libcamera { + +LOG_DEFINE_CATEGORY(CameraSensorHelper) + +namespace ipa { + +/** + * \class CameraSensorHelper + * \brief Create and give helpers for each camera sensor in the pipeline + * + * CameraSensorHelper instances are unique and sensor dependant. + */ + +/** + * \brief Construct a CameraSensorHelper instance + * \param[in] name The sensor model name + * + * The CameraSensorHelper instances shall never be constructed manually, but always + * through the CameraSensorHelperFactory::create() method implemented by the + * respective factories. + */ + +CameraSensorHelper::CameraSensorHelper(const char *name) + : name_(name) +{ + analogueGain_ = { GlobalGain, 1, 1, 0, 0 }; +} + +CameraSensorHelper::~CameraSensorHelper() +{ +} + +/** + * \fn CameraSensorHelper::getGainCode(double gain) + * \brief Get the analogue gain code to pass to V4L2 subdev control + * \param[in] gain The real gain to pass + * + * This function aims to abstract the calculation of the gain letting the IPA + * use the real gain for its estimations. + * + * The parameters come from the MIPI Alliance Camera Specification for + * Camera Command Set (CCS). + */ +uint32_t CameraSensorHelper::getGainCode(double gain) const +{ + /* \todo we only support the global gain mode for now */ + if (analogueGain_.type != GlobalGain) + return UINT32_MAX; + + if (analogueGain_.m0 == 0) + return (analogueGain_.c0 - gain * analogueGain_.c1) / (gain * analogueGain_.m1); + if (analogueGain_.m1 == 0) + return (gain * analogueGain_.c1 - analogueGain_.c0) / analogueGain_.m0; + + LOG(CameraSensorHelper, Error) << "For any given image sensor either m0 or m1 shall be zero."; + return 1; +} + +/** + * \fn CameraSensorHelper::getGain + * \brief Get the real gain from the V4l2 subdev control gain + * \param[in] gainCode The V4l2 subdev control gain + * + * This function aims to abstract the calculation of the gain letting the IPA + * use the real gain for its estimations. It is the counterpart of the function + * CameraSensorHelper::getGainCode. + * + * The parameters come from the MIPI Alliance Camera Specification for + * Camera Command Set (CCS). + */ +double CameraSensorHelper::getGain(uint32_t gainCode) const +{ + if (analogueGain_.type != GlobalGain) + return UINT32_MAX; + + if (analogueGain_.m0 == 0) + return analogueGain_.c0 / (analogueGain_.m1 * gainCode + analogueGain_.c1); + if (analogueGain_.m1 == 0) + return (analogueGain_.m0 * gainCode + analogueGain_.c0) / analogueGain_.c1; + + LOG(CameraSensorHelper, Error) << "For any given image sensor either m0 or m1 shall be zero."; + return 1.0; +} + +/** + * \fn CameraSensorHelper::name() + * \brief Retrieve the camera sensor helper name + * \return The camera sensor helper name + */ + +/** + * \enum CameraSensorHelper::AnalogueGainCapability + * \brief Specify the Gain mode supported by the sensor + * + * Describes the image sensor analog Gain capabilities. + * Two modes are possible, depending on the sensor: Global and Alternate. + */ + +/** + * \var CameraSensorHelper::GlobalGain + * \brief Sensor supports Global gain + * + * The relationship between the integer Gain parameter and the resulting Gain + * multiplier is given by the following equation: + * + * \f$\frac{m0x+c0}{m1x+c1}\f$ + * + * Where 'x' is the gain control parameter, and m0, m1, c0 and c1 are + * image-sensor-specific constants exposed by the sensor. + * These constants are static parameters, and for any given image sensor either + * m0 or m1 shall be zero. + * + * The full Gain equation therefore reduces to either: + * + * \f$\frac{c0}{m1x+c1}\f$ or \f$\frac{m0x+c0}{c1}\f$ + */ + +/** + * \var CameraSensorHelper::AlternateGlobalGain + * \brief Sensor supports Analogue Global gain (introduced in CCS v1.1) + * + * Starting with CCS v1.1, Alternate Global Analog Gain is also available. + * If the image sensor supports it, then the global analog Gain can be controlled + * by linear and exponential gain formula: + * + * \f$gain = analogLinearGainGlobal * 2^{analogExponentialGainGlobal} \f$ + * \todo not implemented in libipa + */ + +/** + * \struct CameraSensorHelper::analogueGainConstants + * \brief Analogue gain constants used for gain calculation + */ + +/** + * \var CameraSensorHelper::analogueGainConstants::type + * \brief Analogue gain coding type + */ + +/** + * \var CameraSensorHelper::analogueGainConstants::m0 + * \brief Constant used in the analog Gain control coding/decoding. + * + * Note: either m0 or m1 shall be zero. + */ + +/** + * \var CameraSensorHelper::analogueGainConstants::c0 + * \brief Constant used in the analog Gain control coding/decoding. + */ + +/** + * \var CameraSensorHelper::analogueGainConstants::m1 + * \brief Constant used in the analog Gain control coding/decoding. + * + * Note: either m0 or m1 shall be zero. + */ + +/** + * \var CameraSensorHelper::analogueGainConstants::c1 + * \brief Constant used in the analog Gain control coding/decoding. + */ + +/** + * \var CameraSensorHelper::analogueGain_ + * \brief The analogue gain parameters used for calculation + * + * The analogue gain is calculated through a formula, and its parameters are + * sensor specific. Use this variable to store the values at init time. + * + */ + +/** + * \class CameraSensorHelperFactory + * \brief Registration of CameraSensorHelperFactory classes and creation of instances + * + * To facilitate discovery and instantiation of CameraSensorHelper classes, the + * CameraSensorHelperFactory class maintains a registry of camera sensor helper + * classes. Each CameraSensorHelper subclass shall register itself using the + * REGISTER_CAMERA_SENSOR_HELPER() macro, which will create a corresponding + * instance of a CameraSensorHelperFactory subclass and register it with the + * static list of factories. + */ + +/** + * \brief Construct a camera sensor helper factory + * \param[in] name Name of the camera sensor helper class + * + * Creating an instance of the factory registers it with the global list of + * factories, accessible through the factories() function. + * + * The factory \a name is used for debug purpose and shall be unique. + */ + +CameraSensorHelperFactory::CameraSensorHelperFactory(const char *name) + : name_(name) +{ + registerType(this); +} + +/** + * \brief Create an instance of the CameraSensorHelper corresponding to the factory + * + * \return A unique pointer to a new instance of the CameraSensorHelper subclass + * corresponding to the factory + */ +std::unique_ptr CameraSensorHelperFactory::create() +{ + CameraSensorHelper *handler = createInstance(); + return std::unique_ptr(handler); +} + +/** + * \fn CameraSensorHelperFactory::name() + * \brief Retrieve the factory name + * \return The factory name + */ + +/** + * \brief Add a camera sensor helper class to the registry + * \param[in] factory Factory to use to construct the camera sensor helper + * + * The caller is responsible to guarantee the uniqueness of the camera sensor helper + * name. + */ +void CameraSensorHelperFactory::registerType(CameraSensorHelperFactory *factory) +{ + std::vector &factories = CameraSensorHelperFactory::factories(); + + factories.push_back(factory); +} + +/** + * \brief Retrieve the list of all camera sensor helper factories + * + * The static factories map is defined inside the function to ensures it gets + * initialized on first use, without any dependency on link order. + * + * \return The list of camera sensor helper factories + */ +std::vector &CameraSensorHelperFactory::factories() +{ + static std::vector factories; + return factories; +} + +/** + * \fn CameraSensorHelperFactory::createInstance() + * \brief Create an instance of the CameraSensorHelper corresponding to the factory + * + * This virtual function is implemented by the REGISTER_CAMERA_SENSOR_HELPER() + * macro. It creates a camera sensor helper instance associated with the camera + * sensor model. + * + * \return A pointer to a newly constructed instance of the CameraSensorHelper + * subclass corresponding to the factory + */ + +/** + * \def REGISTER_CAMERA_SENSOR_HELPER + * \brief Register a camera sensor helper with the camera sensor helper factory + * \param[in] name Sensor model name used to register the class + * \param[in] handler Class name of CameraSensorHelper derived class to register + * + * Register a CameraSensorHelper subclass with the factory and make it available to + * try and match devices. + */ + +/** + * \class CameraSensorHelperImx219 + * \brief Create and give helpers for the imx219 sensor + */ + +/** + * \fn CameraSensorHelperImx219::CameraSensorHelperImx219 + * \brief Construct a CameraSensorHelperImx219 instance for imx219 + * \param[in] name The sensor model name + */ +class CameraSensorHelperImx219 : public CameraSensorHelper +{ +public: + CameraSensorHelperImx219(const char *name); +}; +REGISTER_CAMERA_SENSOR_HELPER("imx219", CameraSensorHelperImx219) +CameraSensorHelperImx219::CameraSensorHelperImx219(const char *name) + : CameraSensorHelper(name) +{ + analogueGain_ = { GlobalGain, 0, -1, 256, 256 }; +} + +/** + * \class CameraSensorHelperOv5670 + * \brief Create and give helpers for the ov5670 sensor + */ + +/** + * \fn CameraSensorHelperOv5670::CameraSensorHelperOv5670 + * \brief Construct a CameraSensorHelperOv5670 instance for ov5670 + * \param[in] name The sensor model name + */ +class CameraSensorHelperOv5670 : public CameraSensorHelper +{ +public: + CameraSensorHelperOv5670(const char *name); +}; +REGISTER_CAMERA_SENSOR_HELPER("ov5670", CameraSensorHelperOv5670) +CameraSensorHelperOv5670::CameraSensorHelperOv5670(const char *name) + : CameraSensorHelper(name) +{ + analogueGain_ = { GlobalGain, 1, 0, 0, 256 }; +} + +/** + * \class CameraSensorHelperOv5693 + * \brief Create and give helpers for the ov5693 sensor + */ + +/** + * \fn CameraSensorHelperOv5693::CameraSensorHelperOv5693 + * \brief Construct a CameraSensorHelperOv5693 instance for ov5693 + * \param[in] name The sensor model name + */ +class CameraSensorHelperOv5693 : public CameraSensorHelper +{ +public: + CameraSensorHelperOv5693(const char *name); +}; +REGISTER_CAMERA_SENSOR_HELPER("ov5693", CameraSensorHelperOv5693) +CameraSensorHelperOv5693::CameraSensorHelperOv5693(const char *name) + : CameraSensorHelper(name) +{ + analogueGain_ = { GlobalGain, 1, 0, 0, 16 }; +} + +} /* namespace ipa */ + +} /* namespace libcamera */ + diff --git a/src/ipa/libipa/camera_sensor_helper.h b/src/ipa/libipa/camera_sensor_helper.h new file mode 100644 index 00000000..63031902 --- /dev/null +++ b/src/ipa/libipa/camera_sensor_helper.h @@ -0,0 +1,91 @@ +/* SPDX-License-Identifier: BSD-2-Clause */ +/* + * Copyright (C) 2021 Ideas On Board + * + * camera_sensor_helper.h - helper class providing camera informations + */ +#ifndef __LIBCAMERA_IPA_LIBIPA_CAMERA_SENSOR_HELPER_H__ +#define __LIBCAMERA_IPA_LIBIPA_CAMERA_SENSOR_HELPER_H__ + +#include + +#include + +#include +#include + +namespace libcamera { + +namespace ipa { + +class CameraSensorHelper; + +class CameraSensorHelper : public Object +{ +public: + CameraSensorHelper(const char *name); + virtual ~CameraSensorHelper(); + + const std::string &name() const { return name_; } + uint32_t getGainCode(double gain) const; + double getGain(uint32_t gainCode) const; + +protected: + enum AnalogueGainCapability : uint16_t { + GlobalGain = 0, + AlternateGlobalGain = 2, + }; + + struct analogueGainConstants { + uint16_t type; + int16_t m0; + int16_t c0; + int16_t m1; + int16_t c1; + }; + analogueGainConstants analogueGain_; + +private: + std::string name_; + friend class CameraSensorHelperFactory; +}; + +class CameraSensorHelperFactory +{ +public: + CameraSensorHelperFactory(const char *name); + virtual ~CameraSensorHelperFactory() = default; + + std::unique_ptr create(); + + const std::string &name() const { return name_; } + + static void registerType(CameraSensorHelperFactory *factory); + static std::vector &factories(); + +private: + virtual CameraSensorHelper *createInstance() = 0; + + std::string name_; +}; + +#define REGISTER_CAMERA_SENSOR_HELPER(name, handler) \ + class handler##Factory final : public CameraSensorHelperFactory \ + { \ + public: \ + handler##Factory() : CameraSensorHelperFactory(#handler) {} \ + \ + private: \ + CameraSensorHelper *createInstance() \ + { \ + return new handler(name); \ + } \ + }; \ + static handler##Factory global_##handler##Factory; + +} /* namespace ipa */ + +} /* namespace libcamera */ + +#endif /* __LIBCAMERA_IPA_LIBIPA_CAMERA_SENSOR_HELPER_H__ */ + diff --git a/src/ipa/libipa/meson.build b/src/ipa/libipa/meson.build index 038fc490..dc90542c 100644 --- a/src/ipa/libipa/meson.build +++ b/src/ipa/libipa/meson.build @@ -2,11 +2,13 @@ libipa_headers = files([ 'algorithm.h', + 'camera_sensor_helper.h', 'histogram.h' ]) libipa_sources = files([ 'algorithm.cpp', + 'camera_sensor_helper.cpp', 'histogram.cpp' ]) From patchwork Mon Jun 7 12:35:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 12501 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 117F8C3206 for ; Mon, 7 Jun 2021 12:35:41 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2748568928; Mon, 7 Jun 2021 14:35:40 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="YAa+rGa1"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A3A0C602A7 for ; Mon, 7 Jun 2021 14:35:38 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:b035:2cc2:4506:d7d3]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 525BDE04; Mon, 7 Jun 2021 14:35:38 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1623069338; bh=vGypoHQDA7k+uHeBZo5sgiKPxCqMYf0iCN7RUIo1YCE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=YAa+rGa1kb69HjNze578j7tISEOZBq3yd6DwNzR+28oNLilfrk2HZrdufVq6C03eK YpKlE/mF7HTKjZh69Qro4iTaoStkMTNSwhs6xcu8LN86SBr0SguQKwQ7/3bucuQdqC /Yq7hFhyJSU444mXtV+P7RpeGIYcghmTJBt0UmyY= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Mon, 7 Jun 2021 14:35:33 +0200 Message-Id: <20210607123533.51198-2-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210607123533.51198-1-jeanmichel.hautbois@ideasonboard.com> References: <20210607123533.51198-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 2/2] ipa: ipu3: Initialize CameraSensorHelper at IPU3 init stage X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In order for the CameraSensorHelper to be instantianted, we need to find its factory using the camera sensor model name stored in IPASettings::sensorModel. As we don't need to do it at each configure call (the sensor is not changing in-between), implement the init call in IPAIPU3 to do that. Signed-off-by: Jean-Michel Hautbois --- src/ipa/ipu3/ipu3.cpp | 34 ++++++++++++++++++++++++++++++---- 1 file changed, 30 insertions(+), 4 deletions(-) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 700a5660..7b0d66ea 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -19,10 +19,12 @@ #include #include "libcamera/internal/buffer.h" +#include "libcamera/internal/camera_sensor_properties.h" #include "libcamera/internal/log.h" #include "ipu3_agc.h" #include "ipu3_awb.h" +#include "libipa/camera_sensor_helper.h" static constexpr uint32_t kMaxCellWidthPerSet = 160; static constexpr uint32_t kMaxCellHeightPerSet = 56; @@ -36,10 +38,7 @@ namespace ipa::ipu3 { class IPAIPU3 : public IPAIPU3Interface { public: - int init([[maybe_unused]] const IPASettings &settings) override - { - return 0; - } + int init(const IPASettings &settings) override; int start() override; void stop() override {} @@ -75,6 +74,8 @@ private: std::unique_ptr awbAlgo_; /* Interface to the AEC/AGC algorithm */ std::unique_ptr agcAlgo_; + /* Interface to the Camera Helper */ + std::unique_ptr camHelper_; /* Local parameter storage */ struct ipu3_uapi_params params_; @@ -82,6 +83,31 @@ private: struct ipu3_uapi_grid_config bdsGrid_; }; +int IPAIPU3::init(const IPASettings &settings) +{ + std::vector &factories = + CameraSensorHelperFactory::factories(); + + for (CameraSensorHelperFactory *factory : factories) { + LOG(IPAIPU3, Debug) + << "Found registered camera sensor helper '" + << factory->name() << "'"; + + std::unique_ptr camHelper = factory->create(); + + if (camHelper->name() != settings.sensorModel) + continue; + + LOG(IPAIPU3, Debug) + << "Camera sensor helper \"" << factory->name() + << "\" matched"; + + camHelper_ = std::move(camHelper); + } + + return 0; +} + int IPAIPU3::start() { setControls(0);