From patchwork Tue Apr 16 09:13:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 19877 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9237EBE08B for ; Tue, 16 Apr 2024 09:14:49 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 30B2E63372; Tue, 16 Apr 2024 11:14:49 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="IQ19ofHy"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3CC3863370 for ; Tue, 16 Apr 2024 11:14:45 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1713258884; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fSszTJQB5gPNmvBm4TcPhdmDraKFa4Z75NzFnB31gjA=; b=IQ19ofHyfuFUOe1bzhp3CpLoSwELTVtHqOoSC4M3dRNVD0xSXBbYSjo7Eoq098/QHqc4xm neq4SBOy4nQcbgGzXQXpvNjEKQlDqVX5xmyB7jddL3nyDg2WYzCW7O8Hxj3A0wkNi/fZW6 tEpqv1404bozlMtfETHzpTl1LCmfJkE= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-649-mXI4ayF5Mty1sA7m40SXLQ-1; Tue, 16 Apr 2024 05:14:41 -0400 X-MC-Unique: mXI4ayF5Mty1sA7m40SXLQ-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 540F080B518; Tue, 16 Apr 2024 09:14:41 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.245]) by smtp.corp.redhat.com (Postfix) with ESMTP id D023C2026962; Tue, 16 Apr 2024 09:14:38 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Hans de Goede , Milan Zamazal , Andrei Konovalov , Bryan O'Donoghue , Maxime Ripard , Pavel Machek , Kieran Bingham , Laurent Pinchart , Dennis Bonke , Andrey Konovalov Subject: [PATCH v8 08/18] libcamera: software_isp: Add Debayer base class Date: Tue, 16 Apr 2024 11:13:44 +0200 Message-ID: <20240416091357.211951-9-mzamazal@redhat.com> In-Reply-To: <20240416091357.211951-1-mzamazal@redhat.com> References: <20240416091357.211951-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Hans de Goede Add a base class for debayer implementations. This is intended to be suitable for both GPU (or otherwise) accelerated debayer implementations as well as CPU based debayering. Doxygen documentation by Dennis Bonke. Tested-by: Bryan O'Donoghue # sc8280xp Lenovo x13s Tested-by: Pavel Machek Reviewed-by: Pavel Machek Reviewed-by: Milan Zamazal Co-developed-by: Dennis Bonke Signed-off-by: Dennis Bonke Co-developed-by: Andrey Konovalov Signed-off-by: Andrey Konovalov Signed-off-by: Hans de Goede --- .../internal/software_isp/debayer_params.h | 25 ++++ .../internal/software_isp/meson.build | 1 + src/libcamera/software_isp/TODO | 31 ++++ src/libcamera/software_isp/debayer.cpp | 132 ++++++++++++++++++ src/libcamera/software_isp/debayer.h | 54 +++++++ src/libcamera/software_isp/meson.build | 1 + 6 files changed, 244 insertions(+) create mode 100644 include/libcamera/internal/software_isp/debayer_params.h create mode 100644 src/libcamera/software_isp/debayer.cpp create mode 100644 src/libcamera/software_isp/debayer.h diff --git a/include/libcamera/internal/software_isp/debayer_params.h b/include/libcamera/internal/software_isp/debayer_params.h new file mode 100644 index 00000000..c818ca3a --- /dev/null +++ b/include/libcamera/internal/software_isp/debayer_params.h @@ -0,0 +1,25 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2023, Red Hat Inc. + * + * Authors: + * Hans de Goede + * + * debayer_params.h - DebayerParams header + */ + +#pragma once + +namespace libcamera { + +struct DebayerParams { + static constexpr unsigned int kGain10 = 256; + + unsigned int gainR; + unsigned int gainG; + unsigned int gainB; + + float gamma; +}; + +} /* namespace libcamera */ diff --git a/include/libcamera/internal/software_isp/meson.build b/include/libcamera/internal/software_isp/meson.build index 66c9c3fb..a620e16d 100644 --- a/include/libcamera/internal/software_isp/meson.build +++ b/include/libcamera/internal/software_isp/meson.build @@ -1,5 +1,6 @@ # SPDX-License-Identifier: CC0-1.0 libcamera_internal_headers += files([ + 'debayer_params.h', 'swisp_stats.h', ]) diff --git a/src/libcamera/software_isp/TODO b/src/libcamera/software_isp/TODO index 1c28fc36..7929e73e 100644 --- a/src/libcamera/software_isp/TODO +++ b/src/libcamera/software_isp/TODO @@ -69,3 +69,34 @@ Removing the signal and refactoring those classes doesn't have to be addressed now, I think it would be part of a larger refactoring (possibly also considering platforms that have no ISP but can produce stats in hardware, such as the i.MX7), but please keep it on your radar. + +--- + +4. Hide internal representation of gains from callers + +> struct DebayerParams { +> static constexpr unsigned int kGain10 = 256; + +Forcing the caller to deal with the internal representation of gains +isn't nice, especially given that it precludes implementing gains of +different precisions in different backend. Wouldn't it be better to pass +the values as floating point numbers, and convert them to the internal +representation in the implementation of process() before using them ? + +--- + +5. Store ISP parameters in per-frame buffers + +> /** +> * \fn void Debayer::process(FrameBuffer *input, FrameBuffer *output, DebayerParams params) +> * \brief Process the bayer data into the requested format. +> * \param[in] input The input buffer. +> * \param[in] output The output buffer. +> * \param[in] params The parameters to be used in debayering. +> * +> * \note DebayerParams is passed by value deliberately so that a copy is passed +> * when this is run in another thread by invokeMethod(). +> */ + +Possibly something to address later, by storing ISP parameters in +per-frame buffers like we do for hardware ISPs. diff --git a/src/libcamera/software_isp/debayer.cpp b/src/libcamera/software_isp/debayer.cpp new file mode 100644 index 00000000..1c035e9b --- /dev/null +++ b/src/libcamera/software_isp/debayer.cpp @@ -0,0 +1,132 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2023, Linaro Ltd + * Copyright (C) 2023, Red Hat Inc. + * + * Authors: + * Hans de Goede + * + * debayer.cpp - debayer base class + */ + +#include "debayer.h" + +namespace libcamera { + +/** + * \struct DebayerParams + * \brief Struct to hold the debayer parameters. + */ + +/** + * \var DebayerParams::kGain10 + * \brief const value for 1.0 gain + */ + +/** + * \var DebayerParams::gainR + * \brief Red gain + * + * 128 = 0.5, 256 = 1.0, 512 = 2.0, etc. + */ + +/** + * \var DebayerParams::gainG + * \brief Green gain + * + * 128 = 0.5, 256 = 1.0, 512 = 2.0, etc. + */ + +/** + * \var DebayerParams::gainB + * \brief Blue gain + * + * 128 = 0.5, 256 = 1.0, 512 = 2.0, etc. + */ + +/** + * \var DebayerParams::gamma + * \brief Gamma correction, 1.0 is no correction + */ + +/** + * \class Debayer + * \brief Base debayering class + * + * Base class that provides functions for setting up the debayering process. + */ + +LOG_DEFINE_CATEGORY(Debayer) + +Debayer::~Debayer() +{ +} + +/** + * \fn int Debayer::configure(const StreamConfiguration &inputCfg, const std::vector> &outputCfgs) + * \brief Configure the debayer object according to the passed in parameters. + * \param[in] inputCfg The input configuration. + * \param[in] outputCfgs The output configurations. + * + * \return 0 on success, a negative errno on failure. + */ + +/** + * \fn Size Debayer::patternSize(PixelFormat inputFormat) + * \brief Get the width and height at which the bayer pattern repeats. + * \param[in] inputFormat The input format. + * + * Valid sizes are: 2x2, 4x2 or 4x4. + * + * \return Pattern size or an empty size for unsupported inputFormats. + */ + +/** + * \fn std::vector Debayer::formats(PixelFormat inputFormat) + * \brief Get the supported output formats. + * \param[in] inputFormat The input format. + * + * \return All supported output formats or an empty vector if there are none. + */ + +/** + * \fn std::tuple Debayer::strideAndFrameSize(const PixelFormat &outputFormat, const Size &size) + * \brief Get the stride and the frame size. + * \param[in] outputFormat The output format. + * \param[in] size The output size. + * + * \return A tuple of the stride and the frame size, or a tuple with 0,0 if + * there is no valid output config. + */ + +/** + * \fn void Debayer::process(FrameBuffer *input, FrameBuffer *output, DebayerParams params) + * \brief Process the bayer data into the requested format. + * \param[in] input The input buffer. + * \param[in] output The output buffer. + * \param[in] params The parameters to be used in debayering. + * + * \note DebayerParams is passed by value deliberately so that a copy is passed + * when this is run in another thread by invokeMethod(). + */ + +/** + * \fn virtual SizeRange Debayer::sizes(PixelFormat inputFormat, const Size &inputSize) + * \brief Get the supported output sizes for the given input format and size. + * \param[in] inputFormat The input format. + * \param[in] inputSize The input size. + * + * \return The valid size ranges or an empty range if there are none. + */ + +/** + * \var Signal Debayer::inputBufferReady + * \brief Signals when the input buffer is ready. + */ + +/** + * \var Signal Debayer::outputBufferReady + * \brief Signals when the output buffer is ready. + */ + +} /* namespace libcamera */ diff --git a/src/libcamera/software_isp/debayer.h b/src/libcamera/software_isp/debayer.h new file mode 100644 index 00000000..42ae58ab --- /dev/null +++ b/src/libcamera/software_isp/debayer.h @@ -0,0 +1,54 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2023, Linaro Ltd + * Copyright (C) 2023, Red Hat Inc. + * + * Authors: + * Hans de Goede + * + * debayer.h - debayering base class + */ + +#pragma once + +#include + +#include +#include + +#include +#include + +#include "libcamera/internal/software_isp/debayer_params.h" + +namespace libcamera { + +class FrameBuffer; + +LOG_DECLARE_CATEGORY(Debayer) + +class Debayer +{ +public: + virtual ~Debayer() = 0; + + virtual int configure(const StreamConfiguration &inputCfg, + const std::vector> &outputCfgs) = 0; + + virtual std::vector formats(PixelFormat inputFormat) = 0; + + virtual std::tuple + strideAndFrameSize(const PixelFormat &outputFormat, const Size &size) = 0; + + virtual void process(FrameBuffer *input, FrameBuffer *output, DebayerParams params) = 0; + + virtual SizeRange sizes(PixelFormat inputFormat, const Size &inputSize) = 0; + + Signal inputBufferReady; + Signal outputBufferReady; + +private: + virtual Size patternSize(PixelFormat inputFormat) = 0; +}; + +} /* namespace libcamera */ diff --git a/src/libcamera/software_isp/meson.build b/src/libcamera/software_isp/meson.build index 281bbf0e..6068a418 100644 --- a/src/libcamera/software_isp/meson.build +++ b/src/libcamera/software_isp/meson.build @@ -8,5 +8,6 @@ if not softisp_enabled endif libcamera_sources += files([ + 'debayer.cpp', 'swstats_cpu.cpp', ])