From patchwork Sun Sep 10 17:50:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 18980 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 157BABD160 for ; Sun, 10 Sep 2023 17:52:07 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C1AE5628E7; Sun, 10 Sep 2023 19:52:06 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1694368326; bh=0Ni4tPfL+muGpZfcUUKdL165fs8g8IRcdghvK7bFy9M=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=tId0GfcMCtZJgpJO2iyxb031WROokOzaY/DAtRBxw0qF6IVTBw0+pnq591RhOngyg +JgQ4E7WrtKOg4ecVtFNzGRtRIZd/9ch2EZzjTvK1xNk6+6Ks3yqWJuUgkL3MedY7q U5XXJwC0vR9dpKg/5dFfXoc2Yk5fWBCje17EC1knVBhBo9FiLIRk2SVuyYZyNulZan s7YyI/jewyB6OlFt4F5/4tuPDRo35guJ5HE9TPjEIxNJzPEGz1UUJZ+FPRIpjtkSHX erGQst3LSo0tCjf2wOhPGOfIeJ1AbqGK+mlvRy8udcuwWuSJpmIcGdNFboJLasoafT NtlyKOSvQnwGQ== Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com [IPv6:2a00:1450:4864:20::631]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0AD7D61DF6 for ; Sun, 10 Sep 2023 19:52:05 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="xFQuR7D7"; dkim-atps=neutral Received: by mail-ej1-x631.google.com with SMTP id a640c23a62f3a-99bcc0adab4so461186866b.2 for ; Sun, 10 Sep 2023 10:52:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1694368324; x=1694973124; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=WC7htGSn6OAf1Z0vtHghGElAC2Fw5PErptZHPMh/4r0=; b=xFQuR7D7+wgBmfA4FWdzOSb5qKw6DfBT4Xk7shtsDpo2NWgUlGOrUpn0NplI0R7SXi qm8hmokf0SE1qAwJ93GmVxabAHiHkZvf6T54W/eN59AF49Zx5HY6S6z6HvsfHH5Djz+q FOFaSSxVmtxwR3RU4kThEhKdclD1h4LkyVs1YjPcyNmzI8U7D/cVFBvt7dokZCWfJTKw tcvE1/p5XCN1Zqs2mHGbmTfpuniZLXi8QvF3Opr5dbQmPKY1+jqaeA3eLpM/o7axB+Ma FreUBWeVJLEcY9RnzelGE5ciQlhIqp/DGpYAEbfkANC3FaGQ2RgqSOG03Na2gh75JPxC JVLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694368324; x=1694973124; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WC7htGSn6OAf1Z0vtHghGElAC2Fw5PErptZHPMh/4r0=; b=DaVLGnTTMiH4GKMEUbSWWlb5pTUlr811x+aW/1OOE0KdzRLxXW0acY2kfK+fICC7Cf +IOmvQCFEm6opRzCbSmFq6/FiwGVwwxi7DNC5KNqNPDWDtqfodOYdJi2Fy/meYwsP2Tk JRnftYHCu2Ljb+aqZN2rLCEBdNCsY6Sc8wO5e9nhsxSRztiwITRD1aJQnitBXMJ9FXER m2Xum/kThLlfoamHp0DphlxrfwO/rYlCIoFl+FhO8TMQLBWcmhkEG9PlmtkBdaiwIDae XDJ+TEbjYOESRQeL677onP/+Lts9SDgkGcj29oi7lTIAglzSb87Rr5znRutLJ133TFgo nC5A== X-Gm-Message-State: AOJu0Yz8Ub/MYaFBHfn6RoZqwAqI1MUK20q75Hh1nIDQT2M47Ew+ZpG5 +MhdWXy/CzY9pFXseyqtQ5YhuzREqbmhPQOJnDM= X-Google-Smtp-Source: AGHT+IESCMh+5YUMsj348CxUl9KVntTuYBpjdENdS3/EP4bTpsC/8wAKPD7V2vyUM/Zfphd1czdY4Q== X-Received: by 2002:a17:907:2c61:b0:9a6:3d19:df7 with SMTP id ib1-20020a1709072c6100b009a63d190df7mr6367897ejc.17.1694368324593; Sun, 10 Sep 2023 10:52:04 -0700 (PDT) Received: from Lat-5310.. ([87.116.166.126]) by smtp.gmail.com with ESMTPSA id kd27-20020a17090798db00b009a57d30df89sm4090089ejc.132.2023.09.10.10.52.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 10 Sep 2023 10:52:04 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Sun, 10 Sep 2023 20:50:25 +0300 Message-Id: <20230910175027.23384-2-andrey.konovalov@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230910175027.23384-1-andrey.konovalov@linaro.org> References: <20230910175027.23384-1-andrey.konovalov@linaro.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 1/3] libcamera: converter: split ConverterMD (media device) out of Converter X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Andrey Konovalov via libcamera-devel From: Andrey Konovalov Reply-To: Andrey Konovalov Cc: jacopo.mondi@ideasonboard.com, bryan.odonoghue@linaro.org, srinivas.kandagatla@linaro.org Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Make Converter a bit more generic by not requiring it to use a media device. For this split out ConverterMD from Converter class, and rename ConverterFactoryBase / ConverterFactory to ConverterMDFactoryBase / ConverterMDFactory for consistency. Signed-off-by: Andrey Konovalov --- include/libcamera/internal/converter.h | 49 +--- .../internal/converter/converter_v4l2_m2m.h | 4 +- include/libcamera/internal/converter_media.h | 86 +++++++ include/libcamera/internal/meson.build | 1 + src/libcamera/converter.cpp | 191 +------------- .../converter/converter_v4l2_m2m.cpp | 4 +- src/libcamera/converter_media.cpp | 241 ++++++++++++++++++ src/libcamera/meson.build | 1 + src/libcamera/pipeline/simple/simple.cpp | 4 +- 9 files changed, 337 insertions(+), 244 deletions(-) create mode 100644 include/libcamera/internal/converter_media.h create mode 100644 src/libcamera/converter_media.cpp diff --git a/include/libcamera/internal/converter.h b/include/libcamera/internal/converter.h index 834ec5ab..da1f5d73 100644 --- a/include/libcamera/internal/converter.h +++ b/include/libcamera/internal/converter.h @@ -24,14 +24,13 @@ namespace libcamera { class FrameBuffer; -class MediaDevice; class PixelFormat; struct StreamConfiguration; class Converter { public: - Converter(MediaDevice *media); + Converter(); virtual ~Converter(); virtual int loadConfiguration(const std::string &filename) = 0; @@ -57,52 +56,6 @@ public: Signal inputBufferReady; Signal outputBufferReady; - - const std::string &deviceNode() const { return deviceNode_; } - -private: - std::string deviceNode_; }; -class ConverterFactoryBase -{ -public: - ConverterFactoryBase(const std::string name, std::initializer_list compatibles); - virtual ~ConverterFactoryBase() = default; - - const std::vector &compatibles() const { return compatibles_; } - - static std::unique_ptr create(MediaDevice *media); - static std::vector &factories(); - static std::vector names(); - -private: - LIBCAMERA_DISABLE_COPY_AND_MOVE(ConverterFactoryBase) - - static void registerType(ConverterFactoryBase *factory); - - virtual std::unique_ptr createInstance(MediaDevice *media) const = 0; - - std::string name_; - std::vector compatibles_; -}; - -template -class ConverterFactory : public ConverterFactoryBase -{ -public: - ConverterFactory(const char *name, std::initializer_list compatibles) - : ConverterFactoryBase(name, compatibles) - { - } - - std::unique_ptr createInstance(MediaDevice *media) const override - { - return std::make_unique<_Converter>(media); - } -}; - -#define REGISTER_CONVERTER(name, converter, compatibles) \ - static ConverterFactory global_##converter##Factory(name, compatibles); - } /* namespace libcamera */ diff --git a/include/libcamera/internal/converter/converter_v4l2_m2m.h b/include/libcamera/internal/converter/converter_v4l2_m2m.h index 815916d0..aeb8c0e9 100644 --- a/include/libcamera/internal/converter/converter_v4l2_m2m.h +++ b/include/libcamera/internal/converter/converter_v4l2_m2m.h @@ -20,7 +20,7 @@ #include -#include "libcamera/internal/converter.h" +#include "libcamera/internal/converter_media.h" namespace libcamera { @@ -31,7 +31,7 @@ class SizeRange; struct StreamConfiguration; class V4L2M2MDevice; -class V4L2M2MConverter : public Converter +class V4L2M2MConverter : public ConverterMD { public: V4L2M2MConverter(MediaDevice *media); diff --git a/include/libcamera/internal/converter_media.h b/include/libcamera/internal/converter_media.h new file mode 100644 index 00000000..2b56ebdb --- /dev/null +++ b/include/libcamera/internal/converter_media.h @@ -0,0 +1,86 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Laurent Pinchart + * Copyright 2022 NXP + * + * converter_media.h - Generic media device based format converter interface + */ + +#pragma once + +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +#include + +#include "libcamera/internal/converter.h" + +namespace libcamera { + +class FrameBuffer; +class MediaDevice; +class PixelFormat; +struct StreamConfiguration; + +class ConverterMD : public Converter +{ +public: + ConverterMD(MediaDevice *media); + ~ConverterMD(); + + const std::string &deviceNode() const { return deviceNode_; } + +private: + std::string deviceNode_; +}; + +class ConverterMDFactoryBase +{ +public: + ConverterMDFactoryBase(const std::string name, std::initializer_list compatibles); + virtual ~ConverterMDFactoryBase() = default; + + const std::vector &compatibles() const { return compatibles_; } + + static std::unique_ptr create(MediaDevice *media); + static std::vector &factories(); + static std::vector names(); + +private: + LIBCAMERA_DISABLE_COPY_AND_MOVE(ConverterMDFactoryBase) + + static void registerType(ConverterMDFactoryBase *factory); + + virtual std::unique_ptr createInstance(MediaDevice *media) const = 0; + + std::string name_; + std::vector compatibles_; +}; + +template +class ConverterMDFactory : public ConverterMDFactoryBase +{ +public: + ConverterMDFactory(const char *name, std::initializer_list compatibles) + : ConverterMDFactoryBase(name, compatibles) + { + } + + std::unique_ptr createInstance(MediaDevice *media) const override + { + return std::make_unique<_ConverterMD>(media); + } +}; + +#define REGISTER_CONVERTER_MD(name, converter, compatibles) \ + static ConverterMDFactory global_##converter##MDFactory(name, compatibles); + +} /* namespace libcamera */ diff --git a/include/libcamera/internal/meson.build b/include/libcamera/internal/meson.build index 7f1f3440..e9c41346 100644 --- a/include/libcamera/internal/meson.build +++ b/include/libcamera/internal/meson.build @@ -21,6 +21,7 @@ libcamera_internal_headers = files([ 'control_serializer.h', 'control_validator.h', 'converter.h', + 'converter_media.h', 'delayed_controls.h', 'device_enumerator.h', 'device_enumerator_sysfs.h', diff --git a/src/libcamera/converter.cpp b/src/libcamera/converter.cpp index fa0f1ec8..92dcdc03 100644 --- a/src/libcamera/converter.cpp +++ b/src/libcamera/converter.cpp @@ -38,26 +38,9 @@ LOG_DEFINE_CATEGORY(Converter) /** * \brief Construct a Converter instance - * \param[in] media The media device implementing the converter - * - * This searches for the entity implementing the data streaming function in the - * media graph entities and use its device node as the converter device node. */ -Converter::Converter(MediaDevice *media) +Converter::Converter() { - const std::vector &entities = media->entities(); - auto it = std::find_if(entities.begin(), entities.end(), - [](MediaEntity *entity) { - return entity->function() == MEDIA_ENT_F_IO_V4L; - }); - if (it == entities.end()) { - LOG(Converter, Error) - << "No entity suitable for implementing a converter in " - << media->driver() << " entities list."; - return; - } - - deviceNode_ = (*it)->deviceNode(); } Converter::~Converter() @@ -159,176 +142,4 @@ Converter::~Converter() * \brief A signal emitted on each frame buffer completion of the output queue */ -/** - * \fn Converter::deviceNode() - * \brief The converter device node attribute accessor - * \return The converter device node string - */ - -/** - * \class ConverterFactoryBase - * \brief Base class for converter factories - * - * The ConverterFactoryBase class is the base of all specializations of the - * ConverterFactory class template. It implements the factory registration, - * maintains a registry of factories, and provides access to the registered - * factories. - */ - -/** - * \brief Construct a converter factory base - * \param[in] name Name of the converter class - * \param[in] compatibles Name aliases of the converter class - * - * Creating an instance of the factory base registers it with the global list of - * factories, accessible through the factories() function. - * - * The factory \a name is used as unique identifier. If the converter - * implementation fully relies on a generic framework, the name should be the - * same as the framework. Otherwise, if the implementation is specialized, the - * factory name should match the driver name implementing the function. - * - * The factory \a compatibles holds a list of driver names implementing a generic - * subsystem without any personalizations. - */ -ConverterFactoryBase::ConverterFactoryBase(const std::string name, std::initializer_list compatibles) - : name_(name), compatibles_(compatibles) -{ - registerType(this); -} - -/** - * \fn ConverterFactoryBase::compatibles() - * \return The names compatibles - */ - -/** - * \brief Create an instance of the converter corresponding to a named factory - * \param[in] media Name of the factory - * - * \return A unique pointer to a new instance of the converter subclass - * corresponding to the named factory or one of its alias. Otherwise a null - * pointer if no such factory exists - */ -std::unique_ptr ConverterFactoryBase::create(MediaDevice *media) -{ - const std::vector &factories = - ConverterFactoryBase::factories(); - - for (const ConverterFactoryBase *factory : factories) { - const std::vector &compatibles = factory->compatibles(); - auto it = std::find(compatibles.begin(), compatibles.end(), media->driver()); - - if (it == compatibles.end() && media->driver() != factory->name_) - continue; - - LOG(Converter, Debug) - << "Creating converter from " - << factory->name_ << " factory with " - << (it == compatibles.end() ? "no" : media->driver()) << " alias."; - - std::unique_ptr converter = factory->createInstance(media); - if (converter->isValid()) - return converter; - } - - return nullptr; -} - -/** - * \brief Add a converter class to the registry - * \param[in] factory Factory to use to construct the converter class - * - * The caller is responsible to guarantee the uniqueness of the converter name. - */ -void ConverterFactoryBase::registerType(ConverterFactoryBase *factory) -{ - std::vector &factories = - ConverterFactoryBase::factories(); - - factories.push_back(factory); -} - -/** - * \brief Retrieve the list of all converter factory names - * \return The list of all converter factory names - */ -std::vector ConverterFactoryBase::names() -{ - std::vector list; - - std::vector &factories = - ConverterFactoryBase::factories(); - - for (ConverterFactoryBase *factory : factories) { - list.push_back(factory->name_); - for (auto alias : factory->compatibles()) - list.push_back(alias); - } - - return list; -} - -/** - * \brief Retrieve the list of all converter factories - * \return The list of converter factories - */ -std::vector &ConverterFactoryBase::factories() -{ - /* - * The static factories map is defined inside the function to ensure - * it gets initialized on first use, without any dependency on link - * order. - */ - static std::vector factories; - return factories; -} - -/** - * \var ConverterFactoryBase::name_ - * \brief The name of the factory - */ - -/** - * \var ConverterFactoryBase::compatibles_ - * \brief The list holding the factory compatibles - */ - -/** - * \class ConverterFactory - * \brief Registration of ConverterFactory classes and creation of instances - * \param _Converter The converter class type for this factory - * - * To facilitate discovery and instantiation of Converter classes, the - * ConverterFactory class implements auto-registration of converter helpers. - * Each Converter subclass shall register itself using the REGISTER_CONVERTER() - * macro, which will create a corresponding instance of a ConverterFactory - * subclass and register it with the static list of factories. - */ - -/** - * \fn ConverterFactory::ConverterFactory(const char *name, std::initializer_list compatibles) - * \brief Construct a converter factory - * \details \copydetails ConverterFactoryBase::ConverterFactoryBase - */ - -/** - * \fn ConverterFactory::createInstance() const - * \brief Create an instance of the Converter corresponding to the factory - * \param[in] media Media device pointer - * \return A unique pointer to a newly constructed instance of the Converter - * subclass corresponding to the factory - */ - -/** - * \def REGISTER_CONVERTER - * \brief Register a converter with the Converter factory - * \param[in] name Converter name used to register the class - * \param[in] converter Class name of Converter derived class to register - * \param[in] compatibles List of compatible names - * - * Register a Converter subclass with the factory and make it available to try - * and match converters. - */ - } /* namespace libcamera */ diff --git a/src/libcamera/converter/converter_v4l2_m2m.cpp b/src/libcamera/converter/converter_v4l2_m2m.cpp index 2a4d1d99..d0a5e3bf 100644 --- a/src/libcamera/converter/converter_v4l2_m2m.cpp +++ b/src/libcamera/converter/converter_v4l2_m2m.cpp @@ -194,7 +194,7 @@ void V4L2M2MConverter::Stream::captureBufferReady(FrameBuffer *buffer) */ V4L2M2MConverter::V4L2M2MConverter(MediaDevice *media) - : Converter(media) + : ConverterMD(media) { if (deviceNode().empty()) return; @@ -448,6 +448,6 @@ static std::initializer_list compatibles = { "pxp", }; -REGISTER_CONVERTER("v4l2_m2m", V4L2M2MConverter, compatibles) +REGISTER_CONVERTER_MD("v4l2_m2m", V4L2M2MConverter, compatibles) } /* namespace libcamera */ diff --git a/src/libcamera/converter_media.cpp b/src/libcamera/converter_media.cpp new file mode 100644 index 00000000..f5024764 --- /dev/null +++ b/src/libcamera/converter_media.cpp @@ -0,0 +1,241 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright 2022 NXP + * + * converter.cpp - Generic format converter interface + */ + +#include "libcamera/internal/converter_media.h" + +#include + +#include + +#include "libcamera/internal/media_device.h" + +#include "linux/media.h" + +/** + * \file internal/converter_media.h + * \brief Abstract media device based converter + */ + +namespace libcamera { + +LOG_DECLARE_CATEGORY(Converter) + +/** + * \class ConverterMD + * \brief Abstract Base Class for media device based converter + * + * The ConverterMD class is an Abstract Base Class defining the interfaces of + * media device based converter implementations. + * + * Converters offer scaling and pixel format conversion services on an input + * stream. The converter can output multiple streams with individual conversion + * parameters from the same input stream. + */ + +/** + * \brief Construct a ConverterMD instance + * \param[in] media The media device implementing the converter + * + * This searches for the entity implementing the data streaming function in the + * media graph entities and use its device node as the converter device node. + */ +ConverterMD::ConverterMD(MediaDevice *media) +{ + const std::vector &entities = media->entities(); + auto it = std::find_if(entities.begin(), entities.end(), + [](MediaEntity *entity) { + return entity->function() == MEDIA_ENT_F_IO_V4L; + }); + if (it == entities.end()) { + LOG(Converter, Error) + << "No entity suitable for implementing a converter in " + << media->driver() << " entities list."; + return; + } + + deviceNode_ = (*it)->deviceNode(); +} + +ConverterMD::~ConverterMD() +{ +} + +/** + * \fn ConverterMD::deviceNode() + * \brief The converter device node attribute accessor + * \return The converter device node string + */ + +/** + * \class ConverterMDFactoryBase + * \brief Base class for media device based converter factories + * + * The ConverterMDFactoryBase class is the base of all specializations of the + * ConverterMDFactory class template. It implements the factory registration, + * maintains a registry of factories, and provides access to the registered + * factories. + */ + +/** + * \brief Construct a media device based converter factory base + * \param[in] name Name of the converter class + * \param[in] compatibles Name aliases of the converter class + * + * Creating an instance of the factory base registers it with the global list of + * factories, accessible through the factories() function. + * + * The factory \a name is used as unique identifier. If the converter + * implementation fully relies on a generic framework, the name should be the + * same as the framework. Otherwise, if the implementation is specialized, the + * factory name should match the driver name implementing the function. + * + * The factory \a compatibles holds a list of driver names implementing a generic + * subsystem without any personalizations. + */ +ConverterMDFactoryBase::ConverterMDFactoryBase(const std::string name, std::initializer_list compatibles) + : name_(name), compatibles_(compatibles) +{ + registerType(this); +} + +/** + * \fn ConverterMDFactoryBase::compatibles() + * \return The names compatibles + */ + +/** + * \brief Create an instance of the converter corresponding to a named factory + * \param[in] media Name of the factory + * + * \return A unique pointer to a new instance of the media device based + * converter subclass corresponding to the named factory or one of its alias. + * Otherwise a null pointer if no such factory exists. + */ +std::unique_ptr ConverterMDFactoryBase::create(MediaDevice *media) +{ + const std::vector &factories = + ConverterMDFactoryBase::factories(); + + for (const ConverterMDFactoryBase *factory : factories) { + const std::vector &compatibles = factory->compatibles(); + auto it = std::find(compatibles.begin(), compatibles.end(), media->driver()); + + if (it == compatibles.end() && media->driver() != factory->name_) + continue; + + LOG(Converter, Debug) + << "Creating converter from " + << factory->name_ << " factory with " + << (it == compatibles.end() ? "no" : media->driver()) << " alias."; + + std::unique_ptr converter = factory->createInstance(media); + if (converter->isValid()) + return converter; + } + + return nullptr; +} + +/** + * \brief Add a media device based converter class to the registry + * \param[in] factory Factory to use to construct the media device based + * converter class + * + * The caller is responsible to guarantee the uniqueness of the converter name. + */ +void ConverterMDFactoryBase::registerType(ConverterMDFactoryBase *factory) +{ + std::vector &factories = + ConverterMDFactoryBase::factories(); + + factories.push_back(factory); +} + +/** + * \brief Retrieve the list of all media device based converter factory names + * \return The list of all media device based converter factory names + */ +std::vector ConverterMDFactoryBase::names() +{ + std::vector list; + + std::vector &factories = + ConverterMDFactoryBase::factories(); + + for (ConverterMDFactoryBase *factory : factories) { + list.push_back(factory->name_); + for (auto alias : factory->compatibles()) + list.push_back(alias); + } + + return list; +} + +/** + * \brief Retrieve the list of all media device based converter factories + * \return The list of media device based converter factories + */ +std::vector &ConverterMDFactoryBase::factories() +{ + /* + * The static factories map is defined inside the function to ensure + * it gets initialized on first use, without any dependency on link + * order. + */ + static std::vector factories; + return factories; +} + +/** + * \var ConverterMDFactoryBase::name_ + * \brief The name of the factory + */ + +/** + * \var ConverterMDFactoryBase::compatibles_ + * \brief The list holding the factory compatibles + */ + +/** + * \class ConverterMDFactory + * \brief Registration of ConverterMDFactory classes and creation of instances + * \param _Converter The converter class type for this factory + * + * To facilitate discovery and instantiation of ConverterMD classes, the + * ConverterMDFactory class implements auto-registration of converter helpers. + * Each ConverterMD subclass shall register itself using the + * REGISTER_CONVERTER() macro, which will create a corresponding instance of a + * ConverterMDFactory subclass and register it with the static list of + * factories. + */ + +/** + * \fn ConverterMDFactory::ConverterMDFactory(const char *name, std::initializer_list compatibles) + * \brief Construct a media device converter factory + * \details \copydetails ConverterMDFactoryBase::ConverterMDFactoryBase + */ + +/** + * \fn ConverterMDFactory::createInstance() const + * \brief Create an instance of the ConverterMD corresponding to the factory + * \param[in] media Media device pointer + * \return A unique pointer to a newly constructed instance of the ConverterMD + * subclass corresponding to the factory + */ + +/** + * \def REGISTER_CONVERTER_MD + * \brief Register a media device based converter with the ConverterMD factory + * \param[in] name ConverterMD name used to register the class + * \param[in] converter Class name of ConverterMD derived class to register + * \param[in] compatibles List of compatible names + * + * Register a ConverterMD subclass with the factory and make it available to try + * and match converters. + */ + +} /* namespace libcamera */ diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build index b24f8296..af8b1812 100644 --- a/src/libcamera/meson.build +++ b/src/libcamera/meson.build @@ -14,6 +14,7 @@ libcamera_sources = files([ 'control_serializer.cpp', 'control_validator.cpp', 'converter.cpp', + 'converter_media.cpp', 'delayed_controls.cpp', 'device_enumerator.cpp', 'device_enumerator_sysfs.cpp', diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 05ba76bc..f0622a74 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -30,7 +30,7 @@ #include "libcamera/internal/camera.h" #include "libcamera/internal/camera_sensor.h" -#include "libcamera/internal/converter.h" +#include "libcamera/internal/converter_media.h" #include "libcamera/internal/device_enumerator.h" #include "libcamera/internal/media_device.h" #include "libcamera/internal/pipeline_handler.h" @@ -497,7 +497,7 @@ int SimpleCameraData::init() /* Open the converter, if any. */ MediaDevice *converter = pipe->converter(); if (converter) { - converter_ = ConverterFactoryBase::create(converter); + converter_ = ConverterMDFactoryBase::create(converter); if (!converter_) { LOG(SimplePipeline, Warning) << "Failed to create converter, disabling format conversion"; From patchwork Sun Sep 10 17:50:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 18981 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2CFE0C3260 for ; Sun, 10 Sep 2023 17:52:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5E97B628F1; Sun, 10 Sep 2023 19:52:11 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1694368331; bh=l7PfTNOR6AaHQ3ZDSpXlVuPFfR+LlKy+UHMxbQHeioI=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=fukOmB+lB/LAENYO5Qd3QR/L5h8i0xsLaXa5d/A6kk22fpRGLmrclh62nwZMDdvaJ M4OFGY8z/jc3BHCh+hqQbFuaTo7K2kPNfgm+pF5VZSpBlcy1XndPjlkZCo7qNGC7o5 KKpMw2o+umGJ/nA6uOUlJ9/mPTjFD2Np6BPwkREeecVG7daXNC5RA4jaFiny90lnXM Gl1KAO0zfXztfSgy4bKHb2vREok1/xEACXcnqXhAHvPhM1DdMlmto/DAmrtAj7rENW z7W9Tkrf+ORTkvL9/DxfOXfm3njN/wrfYZvDkuo6/opNuOoaHPlt8he68hTtYDA1iO haDUfEUDBFRtA== Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com [IPv6:2a00:1450:4864:20::633]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8B792628E7 for ; Sun, 10 Sep 2023 19:52:10 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="GsMLlNJE"; dkim-atps=neutral Received: by mail-ej1-x633.google.com with SMTP id a640c23a62f3a-9a63b2793ecso461580966b.2 for ; Sun, 10 Sep 2023 10:52:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1694368330; x=1694973130; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xvWNpNkJSK8RV8XrpOdADEp8LJKAmGEfU5DdngUjYV0=; b=GsMLlNJEiyBAjCmAlKuK3MyHoBPOCEy725Anu+1cWCUu2vrL+C9bXFecm7rdGgOWxg DohSZuGh7CYqd+wFbEo+xonRmoKtWmvGsqwk3QR/hNhPRcsmnab8Scl9Ht9EOOdguMgZ PSHV3i59RS6tiTyUHik9WKOx+uqWEMLgSOyEc8cw1ysJkbiNrRcX+jjUBaQs8SdCMtjg dhA780Olol/rB+haqXna2bPPpvhwRfiRM1AsqNc3dSCdWyUnApcFKGQJtT8DrtpiKm5M i+Au9zEGfrxvSW5JGNHUroflkMrg4IXrT7vUqiWzBdfEg28GrqAFg8SootcuDyFWxpfw jopA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694368330; x=1694973130; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xvWNpNkJSK8RV8XrpOdADEp8LJKAmGEfU5DdngUjYV0=; b=mZ0VKgBi5HBQsMCNnM5L+fJVRvy5RMNdrTFgwegmmMRTWVxYuhq1b+NIBKsr3Gm6ta IJxm1LxDDSc1v6YK81u4yZIvtRCkKBWvr+QC53yOfI9Bd2z1GcOxfJ3XxP829JsnOsVu GU1t4UYXwsubzl08itvwTl2gdus86E/rnAAa7ZcU4Jbipnat5Cd3inXbAGjrFXe1nkz3 0o7TVfZklEsW/J3ZYTuYvLX64dex16Rc2LxjE9xsQBRlKi/30HcS2syyygb8+ZvyjnqZ Ic6GehtNgD0GvL+QSrSpCnNPslWA1bvcb187QETplWJN4X2DFzd0a1cqEBeIxtertwAR SUXw== X-Gm-Message-State: AOJu0Yz0OOqz64m2XxWiL4CGhzJK8o70NERG7oExE3Nc6g5z1nbr1GVA IngJ0I1bHPkFBvdj/gAzPeQMaNNKFc3CggBzjCs= X-Google-Smtp-Source: AGHT+IGoGtTq8Ale+IbJWOJAQbxJ70dyNhzkoMMSbNOup/PI5oaNHgPRbhn7ENaxskvD0DzUAb9zOA== X-Received: by 2002:a17:906:10dc:b0:9a5:852f:10ac with SMTP id v28-20020a17090610dc00b009a5852f10acmr6853976ejv.53.1694368330109; Sun, 10 Sep 2023 10:52:10 -0700 (PDT) Received: from Lat-5310.. ([87.116.166.126]) by smtp.gmail.com with ESMTPSA id kd27-20020a17090798db00b009a57d30df89sm4090089ejc.132.2023.09.10.10.52.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 10 Sep 2023 10:52:09 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Sun, 10 Sep 2023 20:50:26 +0300 Message-Id: <20230910175027.23384-3-andrey.konovalov@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230910175027.23384-1-andrey.konovalov@linaro.org> References: <20230910175027.23384-1-andrey.konovalov@linaro.org> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 2/3] libcamera: converter: add software converter X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Andrey Konovalov via libcamera-devel From: Andrey Konovalov Reply-To: Andrey Konovalov Cc: jacopo.mondi@ideasonboard.com, bryan.odonoghue@linaro.org, srinivas.kandagatla@linaro.org Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Use the Converter interface to implement bilinear Bayer demosaic filtering and very simple AWB on CPU. The only output format currently supported is RGB888. Signed-off-by: Andrey Konovalov --- .../internal/converter/converter_softw.h | 88 ++++ .../libcamera/internal/converter/meson.build | 1 + src/libcamera/converter/converter_softw.cpp | 412 ++++++++++++++++++ src/libcamera/converter/meson.build | 3 +- 4 files changed, 503 insertions(+), 1 deletion(-) create mode 100644 include/libcamera/internal/converter/converter_softw.h create mode 100644 src/libcamera/converter/converter_softw.cpp diff --git a/include/libcamera/internal/converter/converter_softw.h b/include/libcamera/internal/converter/converter_softw.h new file mode 100644 index 00000000..aab1a650 --- /dev/null +++ b/include/libcamera/internal/converter/converter_softw.h @@ -0,0 +1,88 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2023, Linaro Ltd + * + * converter_softw.h - interface of software converter (runs 100% on CPU) + */ + +#include +#include +#include + +#include +#include +#include + +#include + +#include "libcamera/internal/converter.h" + +namespace libcamera { + +class FrameBuffer; +class Size; +class SizeRange; +struct StreamConfiguration; + +class SwConverter : public Converter +{ +public: + SwConverter() : Converter() {} + + int loadConfiguration([[maybe_unused]] const std::string &filename) { return 0; } + bool isValid() const { return true; } + + std::vector formats(PixelFormat input); + SizeRange sizes(const Size &input); + + std::tuple + strideAndFrameSize(const PixelFormat &pixelFormat, const Size &size); + + int configure(const StreamConfiguration &inputCfg, + const std::vector> &outputCfg); + int exportBuffers(unsigned int ouput, unsigned int count, + std::vector> *buffers); + + void process(FrameBuffer *input, FrameBuffer *output); + int start(); + void stop(); + + int queueBuffers(FrameBuffer *input, + const std::map &outputs); + +private: + class Isp : public Object + { + public: + Isp(SwConverter *converter) + : converter_(converter) {} + + int configure(const StreamConfiguration &inputCfg, + const StreamConfiguration &outputCfg); + int exportBuffers(unsigned int count, + std::vector> *buffers); + void process(FrameBuffer *input, FrameBuffer *output); + int start(); + void stop(); + + private: + void debayer(uint8_t *dst, const uint8_t *src); + + SwConverter *converter_; + + Thread thread_; + + unsigned int width_; + unsigned int height_; + unsigned int stride_; + Point red_shift_; + + unsigned long rNumerat_, rDenomin_; /* red gain for AWB */ + unsigned long bNumerat_, bDenomin_; /* blue gain for AWB */ + unsigned long gNumerat_, gDenomin_; /* green gain for AWB */ + }; + + std::unique_ptr isp_; +}; + +} /* namespace libcamera */ diff --git a/include/libcamera/internal/converter/meson.build b/include/libcamera/internal/converter/meson.build index 891e79e7..843a0483 100644 --- a/include/libcamera/internal/converter/meson.build +++ b/include/libcamera/internal/converter/meson.build @@ -1,5 +1,6 @@ # SPDX-License-Identifier: CC0-1.0 libcamera_internal_headers += files([ + 'converter_softw.h', 'converter_v4l2_m2m.h', ]) diff --git a/src/libcamera/converter/converter_softw.cpp b/src/libcamera/converter/converter_softw.cpp new file mode 100644 index 00000000..c66400b0 --- /dev/null +++ b/src/libcamera/converter/converter_softw.cpp @@ -0,0 +1,412 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2023, Linaro Ltd + * + * converter_softw.h - interface of software converter (runs 100% on CPU) + */ + +#include "libcamera/internal/converter/converter_softw.h" + +#include +#include +#include +#include +#include + +#include +#include +#include + +#include +#include +#include +#include + +#include "libcamera/internal/bayer_format.h" +#include "libcamera/internal/framebuffer.h" +#include "libcamera/internal/mapped_framebuffer.h" + +namespace libcamera { + +LOG_DECLARE_CATEGORY(Converter) + +std::vector SwConverter::formats(PixelFormat input) +{ + std::vector pixelFormats; + BayerFormat inputFormat = BayerFormat::fromPixelFormat(input); + + /* Only RAW10P is currently supported */ + if (inputFormat.bitDepth == 10 && inputFormat.packing == BayerFormat::Packing::CSI2) + pixelFormats.push_back(formats::RGB888); + + if (pixelFormats.empty()) + LOG(Converter, Info) + << "Unsupported input format " << input.toString(); + + return pixelFormats; +} + +SizeRange SwConverter::sizes(const Size &input) +{ + if (input.width < 2 || input.height < 2) { + LOG(Converter, Error) + << "Input format size too small: " << input.toString(); + return {}; + } + + return SizeRange(Size(input.width - 2, input.height - 2)); +} + +std::tuple +SwConverter::strideAndFrameSize(const PixelFormat &pixelFormat, + const Size &size) +{ + /* Only RGB888 output is currently supported */ + if (pixelFormat == formats::RGB888) { + int stride = size.width * 3; + return std::make_tuple(stride, stride * (size.height)); + } + + return std::make_tuple(0, 0); +} + +int SwConverter::configure(const StreamConfiguration &inputCfg, + const std::vector> &outputCfgs) +{ + if (outputCfgs.size() != 1) { + LOG(Converter, Error) + << "Unsupported number of output streams: " + << outputCfgs.size(); + return -EINVAL; + } + + isp_ = std::make_unique(this); + + return isp_->configure(inputCfg, outputCfgs[0]); +} + +int SwConverter::Isp::configure(const StreamConfiguration &inputCfg, + const StreamConfiguration &outputCfg) +{ + BayerFormat bayerFormat = + BayerFormat::fromPixelFormat(inputCfg.pixelFormat); + width_ = inputCfg.size.width; + height_ = inputCfg.size.height; + stride_ = inputCfg.stride; + + if (bayerFormat.bitDepth != 10 || + bayerFormat.packing != BayerFormat::Packing::CSI2 || + width_ < 2 || height_ < 2) { + LOG(Converter, Error) << "Input format " + << inputCfg.size << "-" + << inputCfg.pixelFormat + << "not supported"; + return -EINVAL; + } + + switch (bayerFormat.order) { + case BayerFormat::BGGR: + red_shift_ = Point(0, 0); + break; + case BayerFormat::GBRG: + red_shift_ = Point(1, 0); + break; + case BayerFormat::GRBG: + red_shift_ = Point(0, 1); + break; + case BayerFormat::RGGB: + default: + red_shift_ = Point(1, 1); + break; + } + + if (outputCfg.size.width != width_ - 2 || + outputCfg.size.height != height_ - 2 || + outputCfg.stride != (width_ - 2) * 3 || + outputCfg.pixelFormat != formats::RGB888) { + LOG(Converter, Error) + << "Output format not supported"; + return -EINVAL; + } + + LOG(Converter, Info) << "SwConverter configuration: " + << inputCfg.size << "-" << inputCfg.pixelFormat + << " -> " + << outputCfg.size << "-" << outputCfg.pixelFormat; + + /* set r/g/b gains to 1.0 until frame data collected */ + rNumerat_ = rDenomin_ = 1; + bNumerat_ = bDenomin_ = 1; + gNumerat_ = gDenomin_ = 1; + + return 0; +} + +int SwConverter::exportBuffers(unsigned int output, unsigned int count, + std::vector> *buffers) +{ + /* single output for now */ + if (output >= 1) + return -EINVAL; + + return isp_->exportBuffers(count, buffers); +} + +int SwConverter::Isp::exportBuffers(unsigned int count, + std::vector> *buffers) +{ + /* V4L2_PIX_FMT_BGR24 aka 'BGR3' for output: */ + unsigned int bufSize = (height_ - 2) * (width_ - 2) * 3; + + for (unsigned int i = 0; i < count; i++) { + std::string name = "frame-" + std::to_string(i); + + const int ispFd = memfd_create(name.c_str(), 0); + int ret = ftruncate(ispFd, bufSize); + if (ret < 0) { + LOG(Converter, Error) << "ftruncate() for memfd failed " + << strerror(-ret); + return ret; + } + + FrameBuffer::Plane outPlane; + outPlane.fd = SharedFD(std::move(ispFd)); + outPlane.offset = 0; + outPlane.length = bufSize; + + std::vector planes{ outPlane }; + buffers->emplace_back(std::make_unique(std::move(planes))); + } + + return count; +} + +int SwConverter::Isp::start() +{ + moveToThread(&thread_); + thread_.start(); + return 0; +} + +void SwConverter::Isp::stop() +{ + thread_.exit(); + thread_.wait(); +} + +int SwConverter::start() +{ + return isp_->start(); +} + +void SwConverter::stop() +{ + return isp_->stop(); +} + +int SwConverter::queueBuffers(FrameBuffer *input, + const std::map &outputs) +{ + unsigned int mask = 0; + + /* + * Validate the outputs as a sanity check: at least one output is + * required, all outputs must reference a valid stream and no two + * outputs can reference the same stream. + */ + if (outputs.empty()) + return -EINVAL; + + for (auto [index, buffer] : outputs) { + if (!buffer) + return -EINVAL; + if (index >= 1) /* only single stream atm */ + return -EINVAL; + if (mask & (1 << index)) + return -EINVAL; + + mask |= 1 << index; + } + + process(input, outputs.at(0)); + + return 0; +} + +void SwConverter::process(FrameBuffer *input, FrameBuffer *output) +{ + isp_->invokeMethod(&SwConverter::Isp::process, + ConnectionTypeQueued, input, output); +} + +void SwConverter::Isp::process(FrameBuffer *input, FrameBuffer *output) +{ + /* Copy metadata from the input buffer */ + FrameMetadata &metadata = output->_d()->metadata(); + metadata.status = input->metadata().status; + metadata.sequence = input->metadata().sequence; + metadata.timestamp = input->metadata().timestamp; + + MappedFrameBuffer in(input, MappedFrameBuffer::MapFlag::Read); + MappedFrameBuffer out(output, MappedFrameBuffer::MapFlag::Write); + if (!in.isValid() || !out.isValid()) { + LOG(Converter, Error) << "mmap-ing buffer(s) failed"; + metadata.status = FrameMetadata::FrameError; + converter_->outputBufferReady.emit(output); + converter_->inputBufferReady.emit(input); + return; + } + + debayer(out.planes()[0].data(), in.planes()[0].data()); + metadata.planes()[0].bytesused = out.planes()[0].size(); + + converter_->outputBufferReady.emit(output); + converter_->inputBufferReady.emit(input); +} + +void SwConverter::Isp::debayer(uint8_t *dst, const uint8_t *src) +{ + /* RAW10P input format is assumed */ + + /* output buffer is in BGR24 format and is of (width-2)*(height-2) */ + + int w_out = width_ - 2; + int h_out = height_ - 2; + + unsigned long sumR = 0; + unsigned long sumB = 0; + unsigned long sumG = 0; + + for (int y = 0; y < h_out; y++) { + const uint8_t *pin_base = src + (y + 1) * stride_; + uint8_t *pout = dst + y * w_out * 3; + int phase_y = (y + red_shift_.y) % 2; + + for (int x = 0; x < w_out; x++) { + int phase_x = (x + red_shift_.x) % 2; + int phase = 2 * phase_y + phase_x; + + /* x part of the offset in the input buffer: */ + int x_m1 = x + x / 4; /* offset for (x-1) */ + int x_0 = x + 1 + (x + 1) / 4; /* offset for x */ + int x_p1 = x + 2 + (x + 2) / 4; /* offset for (x+1) */ + /* the colour component value to write to the output */ + unsigned val; + + switch (phase) { + case 0: /* at R pixel */ + /* blue: ((-1,-1)+(1,-1)+(-1,1)+(1,1)) / 4 */ + val = ( *(pin_base + x_m1 - stride_) + + *(pin_base + x_p1 - stride_) + + *(pin_base + x_m1 + stride_) + + *(pin_base + x_p1 + stride_) ) >> 2; + val = val * bNumerat_ / bDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* green: ((0,-1)+(-1,0)+(1,0)+(0,1)) / 4 */ + val = ( *(pin_base + x_0 - stride_) + + *(pin_base + x_p1) + + *(pin_base + x_m1) + + *(pin_base + x_0 + stride_) ) >> 2; + val = val * gNumerat_ / gDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* red: (0,0) */ + val = *(pin_base + x_0); + sumR += val; + val = val * rNumerat_ / rDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + break; + case 1: /* at Gr pixel */ + /* blue: ((0,-1) + (0,1)) / 2 */ + val = ( *(pin_base + x_0 - stride_) + + *(pin_base + x_0 + stride_) ) >> 1; + val = val * bNumerat_ / bDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* green: (0,0) */ + val = *(pin_base + x_0); + sumG += val; + val = val * gNumerat_ / gDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* red: ((-1,0) + (1,0)) / 2 */ + val = ( *(pin_base + x_m1) + + *(pin_base + x_p1) ) >> 1; + val = val * rNumerat_ / rDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + break; + case 2: /* at Gb pixel */ + /* blue: ((-1,0) + (1,0)) / 2 */ + val = ( *(pin_base + x_m1) + + *(pin_base + x_p1) ) >> 1; + val = val * bNumerat_ / bDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* green: (0,0) */ + val = *(pin_base + x_0); + sumG += val; + val = val * gNumerat_ / gDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* red: ((0,-1) + (0,1)) / 2 */ + val = ( *(pin_base + x_0 - stride_) + + *(pin_base + x_0 + stride_) ) >> 1; + val = val * rNumerat_ / rDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + break; + default: /* at B pixel */ + /* blue: (0,0) */ + val = *(pin_base + x_0); + sumB += val; + val = val * bNumerat_ / bDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* green: ((0,-1)+(-1,0)+(1,0)+(0,1)) / 4 */ + val = ( *(pin_base + x_0 - stride_) + + *(pin_base + x_p1) + + *(pin_base + x_m1) + + *(pin_base + x_0 + stride_) ) >> 2; + val = val * gNumerat_ / gDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* red: ((-1,-1)+(1,-1)+(-1,1)+(1,1)) / 4 */ + val = ( *(pin_base + x_m1 - stride_) + + *(pin_base + x_p1 - stride_) + + *(pin_base + x_m1 + stride_) + + *(pin_base + x_p1 + stride_) ) >> 2; + val = val * rNumerat_ / rDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + } + } + } + + /* calculate red and blue gains for simple AWB */ + LOG(Converter, Debug) << "sumR = " << sumR + << ", sumB = " << sumB << ", sumG = " << sumG; + + sumG /= 2; /* the number of G pixels is twice as big vs R and B ones */ + + /* normalize red, blue, and green sums to fit into 22-bit value */ + unsigned long fRed = sumR / 0x400000; + unsigned long fBlue = sumB / 0x400000; + unsigned long fGreen = sumG / 0x400000; + unsigned long fNorm = std::max({ 1UL, fRed, fBlue, fGreen }); + sumR /= fNorm; + sumB /= fNorm; + sumG /= fNorm; + + LOG(Converter, Debug) << "fNorm = " << fNorm; + LOG(Converter, Debug) << "Normalized: sumR = " << sumR + << ", sumB= " << sumB << ", sumG = " << sumG; + + /* make sure red/blue gains never exceed approximately 256 */ + unsigned long minDenom; + rNumerat_ = (sumR + sumB + sumG) / 3; + minDenom = rNumerat_ / 0x100; + rDenomin_ = std::max(minDenom, sumR); + bNumerat_ = rNumerat_; + bDenomin_ = std::max(minDenom, sumB); + gNumerat_ = rNumerat_; + gDenomin_ = std::max(minDenom, sumG); + + LOG(Converter, Debug) << "rGain = [ " + << rNumerat_ << " / " << rDenomin_ + << " ], bGain = [ " << bNumerat_ << " / " << bDenomin_ + << " ], gGain = [ " << gNumerat_ << " / " << gDenomin_ + << " (minDenom = " << minDenom << ")"; +} + +} /* namespace libcamera */ diff --git a/src/libcamera/converter/meson.build b/src/libcamera/converter/meson.build index 2aa72fe4..1f1e0ea4 100644 --- a/src/libcamera/converter/meson.build +++ b/src/libcamera/converter/meson.build @@ -1,5 +1,6 @@ # SPDX-License-Identifier: CC0-1.0 libcamera_sources += files([ - 'converter_v4l2_m2m.cpp' + 'converter_softw.cpp', + 'converter_v4l2_m2m.cpp', ]) From patchwork Sun Sep 10 17:50:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 18982 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 707E4C32B0 for ; Sun, 10 Sep 2023 17:52:17 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E7705628EC; Sun, 10 Sep 2023 19:52:16 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1694368336; bh=srqHSL6BE4Q26xBPeIWB8/NIW+CA8LoFsqhyI5OBVSk=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=PUgsRf4BulCwlcmBeTqrKDsZR5xLioKX4EWY9Or8MoxgMEbO1QmajPghj90+fF2sJ viqOle++oWDHT2hnlkJZaBbtweT99OTT0smzdGRTfk7J8nyXGudlHirWnipAUtum3k tvBnOXg9HyckSeFa0ZyBwpl+n4EMsR7l4y27pDu2hyUWfnNAI4v2rVIcbTOoSwE4Gm GnBmxTW0a9fJksphZ0vE3qx0+39nTGfEbBRVn8UZq6DmGw8VfMmqpwUcccmU5xpoMb 4CzO4nhtBgnbU4zLnaW04mlagJzpSkDYbq1pem6y4vfHxfDT5k167epkpKWrchgEmt 13XzyMZlxvPFA== Received: from mail-lf1-x132.google.com (mail-lf1-x132.google.com [IPv6:2a00:1450:4864:20::132]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9AAED628EA for ; Sun, 10 Sep 2023 19:52:15 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="Zt8bbMNg"; dkim-atps=neutral Received: by mail-lf1-x132.google.com with SMTP id 2adb3069b0e04-50091b91a83so5971759e87.3 for ; Sun, 10 Sep 2023 10:52:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1694368335; x=1694973135; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bKl/MCKY+YXVhV69y4zaeOVIHgeFK1crRaAAOozssiY=; b=Zt8bbMNgfzHyrIIIGs5+lWifKvMKLWxqBdwMd7EVK5V7NWzbiMfKqAzw9qb2Hdgakq 2mqhQ7Ug+UT/ycb5+TnLDK1hCYuOEknaXdawZ80jSqrbaJaRh3++3dg6jbOPcZC8AXBn Ay6rGXHczj4DgOT/KPYWaxqpdNJ4LsQ//0uL1j8RIJpBoINduV05aTVDrP7dPlH0jQnM bWJEsSiIuCbUUMMFhHfLEg+DhOYY47eOi4HGcdJmwZsjRO9ZgGjKD5fRZWG+GBWSnMEd xfJIjy36IzNZ1mehFtymuExvbu9dSv2Xn53yoHTqfLYkiXhW4EP1nioZzwgMo+zAuQka Hl/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694368335; x=1694973135; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bKl/MCKY+YXVhV69y4zaeOVIHgeFK1crRaAAOozssiY=; b=b0bbbfs/41zUunFBgTHB7Wy4Zabe1Dn7Rg5Ky/RU91ZNHpI/IN531FJwHpwRCdS0k/ dFqLmfo66dEJ9ftRnUjI1mMdJ4ahNp+h/jU+EFjaFM0s5/11JF9e5OM+7iGWtV2/mE1V ip4O69OLAgtGmCyMqfCNXbz7YKI7L3yT6+vdUQFAucHwho6D3q3hlswLlQnjaSmheC4X xHLu3W9T/31+GGGUaBaIZv5lw1hUz15t/ZP6b9spW2O0fB15peBKF4mSIrnVoroJyCZx CjSUgnGDPjpXAtB0viZhpxYDMmg3GrLBVL9bHTfmSFfvx3H/rbkVTesgompgmSONGJvN BHfg== X-Gm-Message-State: AOJu0Yyx4znwmok5TRpWcLwVi/LVflhnvV5zAnDTZjLR3JCPsx3DJGEr L+VIqhx6MTaHupYjwyF4lAw6umVllI/fDhV3qtw= X-Google-Smtp-Source: AGHT+IGZ4Ff4GvgJUemXD30hwIntc0sQEIPde8d57wYVaCd+lHa3rsMEyVdJ1Ezt28DTkFOaXf3xVA== X-Received: by 2002:a05:6512:2256:b0:500:9d6c:913e with SMTP id i22-20020a056512225600b005009d6c913emr6594559lfu.52.1694368334719; Sun, 10 Sep 2023 10:52:14 -0700 (PDT) Received: from Lat-5310.. ([87.116.166.126]) by smtp.gmail.com with ESMTPSA id kd27-20020a17090798db00b009a57d30df89sm4090089ejc.132.2023.09.10.10.52.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 10 Sep 2023 10:52:14 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Sun, 10 Sep 2023 20:50:27 +0300 Message-Id: <20230910175027.23384-4-andrey.konovalov@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230910175027.23384-1-andrey.konovalov@linaro.org> References: <20230910175027.23384-1-andrey.konovalov@linaro.org> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 3/3] libcamera: pipeline: simple: add software converter support X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Andrey Konovalov via libcamera-devel From: Andrey Konovalov Reply-To: Andrey Konovalov Cc: jacopo.mondi@ideasonboard.com, bryan.odonoghue@linaro.org, srinivas.kandagatla@linaro.org Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Signed-off-by: Andrey Konovalov --- src/libcamera/pipeline/simple/simple.cpp | 34 +++++++++++++++++------- 1 file changed, 25 insertions(+), 9 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index f0622a74..ee9b672c 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -37,6 +37,7 @@ #include "libcamera/internal/v4l2_subdevice.h" #include "libcamera/internal/v4l2_videodevice.h" +#include "libcamera/internal/converter/converter_softw.h" namespace libcamera { @@ -185,17 +186,21 @@ struct SimplePipelineInfo { * and the number of streams it supports. */ std::vector> converters; + /* + * Nonzero value enables creating software converter + */ + bool enable_sw_converter; }; namespace { static const SimplePipelineInfo supportedDevices[] = { - { "dcmipp", {} }, - { "imx7-csi", { { "pxp", 1 } } }, - { "j721e-csi2rx", {} }, - { "mxc-isi", {} }, - { "qcom-camss", {} }, - { "sun6i-csi", {} }, + { "dcmipp", {}, false }, + { "imx7-csi", { { "pxp", 1 } }, false }, + { "j721e-csi2rx", {}, false }, + { "mxc-isi", {}, false }, + { "qcom-camss", {}, true }, + { "sun6i-csi", {}, false }, }; } /* namespace */ @@ -331,6 +336,7 @@ public: V4L2VideoDevice *video(const MediaEntity *entity); V4L2Subdevice *subdev(const MediaEntity *entity); MediaDevice *converter() { return converter_; } + bool sw_converter_enabled() { return enable_sw_converter_; } protected: int queueRequestDevice(Camera *camera, Request *request) override; @@ -359,6 +365,8 @@ private: std::map entities_; MediaDevice *converter_; + + bool enable_sw_converter_; }; /* ----------------------------------------------------------------------------- @@ -495,6 +503,7 @@ int SimpleCameraData::init() int ret; /* Open the converter, if any. */ + converter_ = nullptr; MediaDevice *converter = pipe->converter(); if (converter) { converter_ = ConverterMDFactoryBase::create(converter); @@ -502,11 +511,16 @@ int SimpleCameraData::init() LOG(SimplePipeline, Warning) << "Failed to create converter, disabling format conversion"; converter_.reset(); - } else { - converter_->inputBufferReady.connect(this, &SimpleCameraData::converterInputDone); - converter_->outputBufferReady.connect(this, &SimpleCameraData::converterOutputDone); } } + /* Don't create sw converter if media device converter is created. */ + if (!converter_ && pipe->sw_converter_enabled()) { + converter_ = std::make_unique(); + } + if (converter_) { + converter_->inputBufferReady.connect(this, &SimpleCameraData::converterInputDone); + converter_->outputBufferReady.connect(this, &SimpleCameraData::converterOutputDone); + } video_ = pipe->video(entities_.back().entity); ASSERT(video_); @@ -1418,6 +1432,8 @@ bool SimplePipelineHandler::match(DeviceEnumerator *enumerator) } } + enable_sw_converter_ = info->enable_sw_converter; + /* Locate the sensors. */ std::vector sensors = locateSensors(); if (sensors.empty()) {