From patchwork Sat Jan 13 14:22:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hans de Goede X-Patchwork-Id: 19403 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EB177C32BF for ; Sat, 13 Jan 2024 14:23:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6B48D6293F; Sat, 13 Jan 2024 15:23:05 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1705155785; bh=BfJ5Q9Abhzz6JEp7fS1+QoCoBqrqe1324HyjxPDD+bw=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=rC68iBokerstVOkyLRtdvJwQ8Xu0J++2nT0/Gef4F9mVeLUQzvFHQyjt8X6PGlqzv XA+SU0tU844JmBCv8y+GSdolxuD74IHZhWzoatd5SZQBdauAwgLoJpi+YpcqI+r/RW HkNTNfQ3e+H44fbRHwX0agdd0gSvu6+rXIGibzNDpExeAe8JCPrq+DvdeGFRfzzf5P thwVRBe46d4kzs+NQbEXT8BirQrTJsltGKFcoF5LdaqFj6Yle2P+irmrBVhN8shqyO g3lS/U/JIFz64cKIVA2eonp+01jaSwoV823yl4M7ETNKrv1c+Ejyg2ITPlm7c2N5wX dHRfR+tCo1HTQ== Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DDACA6293E for ; Sat, 13 Jan 2024 15:23:03 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="iwhK1vGH"; dkim-atps=neutral DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1705155783; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sQaXLJ5xMtn4atOohd1+oTGyXpXD11nrK5SpNOQcEJU=; b=iwhK1vGHjntjkCFGj+5veOFwkofd/INM+D+P5zH+n9t/FORVyOiu2mYNsMbledRygk+Tbb ASWD1wjUYmkSQuNe/r8dMnWHeTiaTBV4mbfZ8IU1tA2xxCGrbF9yxQYbF70lT2Z+S5gplm Bo1wbs5dGitdyJns2JIAkPmbh1cHq30= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-582-75DdyCoGPv2GTZxhoWEFDA-1; Sat, 13 Jan 2024 09:22:58 -0500 X-MC-Unique: 75DdyCoGPv2GTZxhoWEFDA-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 4A6AF82A6C3; Sat, 13 Jan 2024 14:22:58 +0000 (UTC) Received: from localhost.localdomain (unknown [10.39.192.58]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5715A3C25; Sat, 13 Jan 2024 14:22:56 +0000 (UTC) To: libcamera-devel@lists.libcamera.org, Andrey Konovalov Date: Sat, 13 Jan 2024 15:22:13 +0100 Message-ID: <20240113142218.28063-14-hdegoede@redhat.com> In-Reply-To: <20240113142218.28063-1-hdegoede@redhat.com> References: <20240113142218.28063-1-hdegoede@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [libcamera-devel] [PATCH v2 13/18] libcamera: software_isp: add Simple SoftwareIsp implementation X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Hans de Goede via libcamera-devel From: Hans de Goede Reply-To: Hans de Goede Cc: Maxime Ripard , g.martti@gmail.com, t.langendam@gmail.com, srinivas.kandagatla@linaro.org, Pavel Machek , Bryan O'Donoghue , admin@dennisbonke.com Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Andrey Konovalov The implementation of SoftwareIsp handles creation of Soft IPA and interactions with it, so that the pipeline handler wouldn't need to care about the Soft IPA. Doxygen documentation by Dennis Bonke. Signed-off-by: Andrey Konovalov Co-authored-by: Dennis Bonke Signed-off-by: Dennis Bonke Co-authored-by: Hans de Goede Signed-off-by: Hans de Goede Tested-by: Bryan O'Donoghue # sc8280xp Lenovo x13s Tested-by: Pavel Machek --- .../internal/software_isp/meson.build | 1 + .../internal/software_isp/swisp_simple.h | 163 ++++++++++++ src/libcamera/software_isp/meson.build | 19 ++ src/libcamera/software_isp/swisp_simple.cpp | 238 ++++++++++++++++++ 4 files changed, 421 insertions(+) create mode 100644 include/libcamera/internal/software_isp/swisp_simple.h create mode 100644 src/libcamera/software_isp/swisp_simple.cpp diff --git a/include/libcamera/internal/software_isp/meson.build b/include/libcamera/internal/software_isp/meson.build index b5a0d737..cf21ace5 100644 --- a/include/libcamera/internal/software_isp/meson.build +++ b/include/libcamera/internal/software_isp/meson.build @@ -4,6 +4,7 @@ libcamera_internal_headers += files([ 'debayer.h', 'debayer_cpu.h', 'debayer_params.h', + 'swisp_simple.h', 'swisp_stats.h', 'swstats.h', 'swstats_cpu.h', diff --git a/include/libcamera/internal/software_isp/swisp_simple.h b/include/libcamera/internal/software_isp/swisp_simple.h new file mode 100644 index 00000000..87613c23 --- /dev/null +++ b/include/libcamera/internal/software_isp/swisp_simple.h @@ -0,0 +1,163 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2023, Linaro Ltd + * + * swisp_simple.h - Simple software ISP implementation + */ + +#pragma once + +#include +#include + +#include + +#include +#include + +#include "libcamera/internal/dma_heaps.h" +#include "libcamera/internal/software_isp.h" +#include "libcamera/internal/software_isp/debayer_cpu.h" + +namespace libcamera { + +/** + * \brief Class for the Simple Software ISP. + * + * Implementation of the SoftwareIsp interface. + */ +class SwIspSimple : public SoftwareIsp +{ +public: + /** + * \brief Constructor for the SwIspSimple object. + * + * \param[in] pipe The pipeline handler in use. + * \param[in] sensorControls The sensor controls. + */ + SwIspSimple(PipelineHandler *pipe, const ControlInfoMap &sensorControls); + ~SwIspSimple() {} + + /** + * \brief Load a configuration from a file. + * \param[in] filename The file to load from. + * + * \return 0 on success. + */ + int loadConfiguration([[maybe_unused]] const std::string &filename) { return 0; } + + /** + * \brief Gets if there is a valid debayer object. + * + * \returns true if there is, false otherwise. + */ + bool isValid() const; + + /** + * \brief Get the supported output formats. + * \param[in] input The input format. + * + * \return all supported output formats or an empty vector if there are none. + */ + std::vector formats(PixelFormat input); + + /** + * \brief Get the supported output sizes for the given input format and size. + * \param[in] inputFormat The input format. + * \param[in] inputSize The input size. + * + * \return The valid size ranges or an empty range if there are none. + */ + SizeRange sizes(PixelFormat inputFormat, const Size &inputSize); + + /** + * \brief Get the stride and the frame size. + * \param[in] outputFormat The output format. + * \param[in] size The output size. + * + * \return a tuple of the stride and the frame size, or a tuple with 0,0 if there is no valid output config. + */ + std::tuple + strideAndFrameSize(const PixelFormat &outputFormat, const Size &size); + + /** + * \brief Configure the SwIspSimple object according to the passed in parameters. + * \param[in] inputCfg The input configuration. + * \param[in] outputCfgs The output configurations. + * \param[in] sensorControls The sensor controls. + * + * \return 0 on success, a negative errno on failure. + */ + int configure(const StreamConfiguration &inputCfg, + const std::vector> &outputCfgs, + const ControlInfoMap &sensorControls); + + /** + * \brief Exports the buffers for use in processing. + * \param[in] output The number of outputs requested. + * \param[in] count The number of planes. + * \param[out] buffers The exported buffers. + * + * \return count when successful, a negative return value if an error occurred. + */ + int exportBuffers(unsigned int output, unsigned int count, + std::vector> *buffers); + + /** + * \brief Process the statistics gathered. + * \param[in] sensorControls The sensor controls. + */ + void processStats(const ControlList &sensorControls); + + /** + * \brief Starts the Software ISP worker. + * + * \return 0 on success, any other value indicates an error. + */ + int start(); + + /** + * \brief Stops the Software ISP worker. + */ + void stop(); + + /** + * \brief Queues buffers for processing. + * \param[in] input The input framebuffer. + * \param[in] outputs The output framebuffers. + * + * \return 0 on success, a negative errno on failure + */ + int queueBuffers(FrameBuffer *input, + const std::map &outputs); + + /** + * \brief Get the signal for when the sensor controls are set. + * + * \return The control list of the sensor controls. + */ + Signal &getSignalSetSensorControls(); + + /** + * \brief Process the input framebuffer. + * \param[in] input The input framebuffer. + * \param[out] output The output framebuffer. + */ + void process(FrameBuffer *input, FrameBuffer *output); + +private: + void saveIspParams(int dummy); + void statsReady(int dummy); + void inputReady(FrameBuffer *input); + void outputReady(FrameBuffer *output); + + std::unique_ptr debayer_; + Thread ispWorkerThread_; + SharedMemObject sharedParams_; + DebayerParams debayerParams_; + DmaHeap dmaHeap_; + + std::unique_ptr ipa_; +}; + +} /* namespace libcamera */ diff --git a/src/libcamera/software_isp/meson.build b/src/libcamera/software_isp/meson.build index 6d7a44d7..9464f2fd 100644 --- a/src/libcamera/software_isp/meson.build +++ b/src/libcamera/software_isp/meson.build @@ -6,3 +6,22 @@ libcamera_sources += files([ 'swstats.cpp', 'swstats_cpu.cpp', ]) + +# Software ISP is enabled for 'simple' pipeline handler. +# The 'pipelines' option value should be +# 'simple/' e.g.: +# -Dpipelines=simple/simple +# The source file should be named swisp_.cpp, +# e.g. 'swisp_simple.cpp'. + +foreach pipeline : pipelines + pipeline = pipeline.split('/') + if pipeline.length() == 2 and pipeline[0] == 'simple' + libcamera_sources += files([ + 'swisp_' + pipeline[1] + '.cpp', + ]) + # the 'break' below can be removed if/when multiple + # Software ISP implementations are allowed in single build + break + endif +endforeach diff --git a/src/libcamera/software_isp/swisp_simple.cpp b/src/libcamera/software_isp/swisp_simple.cpp new file mode 100644 index 00000000..0884166e --- /dev/null +++ b/src/libcamera/software_isp/swisp_simple.cpp @@ -0,0 +1,238 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2023, Linaro Ltd + * + * swisp_simple.cpp - Simple software ISP implementation + */ + +#include "libcamera/internal/software_isp/swisp_simple.h" + +#include +#include +#include + +#include +#include + +#include "libcamera/internal/bayer_format.h" +#include "libcamera/internal/framebuffer.h" +#include "libcamera/internal/ipa_manager.h" +#include "libcamera/internal/mapped_framebuffer.h" + +namespace libcamera { + +SwIspSimple::SwIspSimple(PipelineHandler *pipe, const ControlInfoMap &sensorControls) + : SoftwareIsp(pipe, sensorControls), debayer_(nullptr), debayerParams_{ 256, 256, 256, 0.5f }, + dmaHeap_(DmaHeap::DmaHeapFlag::Cma | DmaHeap::DmaHeapFlag::System) +{ + if (!dmaHeap_.isValid()) { + LOG(SoftwareIsp, Error) << "Failed to create DmaHeap object"; + return; + } + + sharedParams_ = SharedMemObject("softIsp_params"); + if (!sharedParams_.fd().isValid()) { + LOG(SoftwareIsp, Error) << "Failed to create shared memory for parameters"; + return; + } + + std::unique_ptr stats; + + stats = std::make_unique(); + if (!stats) { + LOG(SoftwareIsp, Error) << "Failed to create SwStatsCpu object"; + return; + } + + stats->statsReady.connect(this, &SwIspSimple::statsReady); + + debayer_ = std::make_unique(std::move(stats)); + if (!debayer_) { + LOG(SoftwareIsp, Error) << "Failed to create DebayerCpu object"; + return; + } + + debayer_->inputBufferReady.connect(this, &SwIspSimple::inputReady); + debayer_->outputBufferReady.connect(this, &SwIspSimple::outputReady); + + ipa_ = IPAManager::createIPA(pipe, 0, 0); + if (!ipa_) { + LOG(SoftwareIsp, Error) + << "Creating IPA for software ISP failed"; + debayer_.reset(); + return; + } + + int ret = ipa_->init(IPASettings{ "No cfg file", "No sensor model" }, + debayer_->getStatsFD(), + sharedParams_.fd(), + sensorControls); + if (ret) { + LOG(SoftwareIsp, Error) << "IPA init failed"; + debayer_.reset(); + return; + } + + ipa_->setIspParams.connect(this, &SwIspSimple::saveIspParams); + + debayer_->moveToThread(&ispWorkerThread_); +} + +void SwIspSimple::processStats(const ControlList &sensorControls) +{ + ASSERT(ipa_); + ipa_->processStats(sensorControls); +} + +Signal &SwIspSimple::getSignalSetSensorControls() +{ + ASSERT(ipa_); + return ipa_->setSensorControls; +} + +bool SwIspSimple::isValid() const +{ + return !!debayer_; +} + +std::vector SwIspSimple::formats(PixelFormat inputFormat) +{ + ASSERT(debayer_ != nullptr); + + return debayer_->formats(inputFormat); +} + +SizeRange SwIspSimple::sizes(PixelFormat inputFormat, const Size &inputSize) +{ + ASSERT(debayer_ != nullptr); + + return debayer_->sizes(inputFormat, inputSize); +} + +std::tuple +SwIspSimple::strideAndFrameSize(const PixelFormat &outputFormat, const Size &size) +{ + ASSERT(debayer_ != nullptr); + + return debayer_->strideAndFrameSize(outputFormat, size); +} + +int SwIspSimple::configure(const StreamConfiguration &inputCfg, + const std::vector> &outputCfgs, + const ControlInfoMap &sensorControls) +{ + ASSERT(ipa_ != nullptr && debayer_ != nullptr); + + int ret = ipa_->configure(sensorControls); + if (ret < 0) + return ret; + + return debayer_->configure(inputCfg, outputCfgs); +} + +int SwIspSimple::exportBuffers(unsigned int output, unsigned int count, + std::vector> *buffers) +{ + ASSERT(debayer_ != nullptr); + + /* single output for now */ + if (output >= 1) + return -EINVAL; + + for (unsigned int i = 0; i < count; i++) { + const std::string name = "frame-" + std::to_string(i); + const size_t frameSize = debayer_->frameSize(); + + FrameBuffer::Plane outPlane; + outPlane.fd = SharedFD(dmaHeap_.alloc(name.c_str(), frameSize)); + if (!outPlane.fd.isValid()) { + LOG(SoftwareIsp, Error) + << "failed to allocate a dma_buf"; + return -ENOMEM; + } + outPlane.offset = 0; + outPlane.length = frameSize; + + std::vector planes{ outPlane }; + buffers->emplace_back(std::make_unique(std::move(planes))); + } + + return count; +} + +int SwIspSimple::queueBuffers(FrameBuffer *input, + const std::map &outputs) +{ + unsigned int mask = 0; + + /* + * Validate the outputs as a sanity check: at least one output is + * required, all outputs must reference a valid stream and no two + * outputs can reference the same stream. + */ + if (outputs.empty()) + return -EINVAL; + + for (auto [index, buffer] : outputs) { + if (!buffer) + return -EINVAL; + if (index >= 1) /* only single stream atm */ + return -EINVAL; + if (mask & (1 << index)) + return -EINVAL; + + mask |= 1 << index; + } + + process(input, outputs.at(0)); + + return 0; +} + +int SwIspSimple::start() +{ + int ret = ipa_->start(); + if (ret) + return ret; + + ispWorkerThread_.start(); + return 0; +} + +void SwIspSimple::stop() +{ + ispWorkerThread_.exit(); + ispWorkerThread_.wait(); + + ipa_->stop(); +} + +void SwIspSimple::process(FrameBuffer *input, FrameBuffer *output) +{ + debayer_->invokeMethod(&DebayerCpu::process, + ConnectionTypeQueued, input, output, debayerParams_); +} + +void SwIspSimple::saveIspParams([[maybe_unused]] int dummy) +{ + debayerParams_ = *sharedParams_; +} + +void SwIspSimple::statsReady(int dummy) +{ + ispStatsReady.emit(dummy); +} + +void SwIspSimple::inputReady(FrameBuffer *input) +{ + inputBufferReady.emit(input); +} + +void SwIspSimple::outputReady(FrameBuffer *output) +{ + outputBufferReady.emit(output); +} + +REGISTER_SOFTWAREISP(SwIspSimple) + +} /* namespace libcamera */