From patchwork Thu Dec 14 12:13:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hans de Goede X-Patchwork-Id: 19324 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8F67BC3292 for ; Thu, 14 Dec 2023 12:14:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4611F62B61; Thu, 14 Dec 2023 13:14:12 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1702556052; bh=AzPbf99kmRo4L7Zn43l8oEDzEJoEC/Bv9o1ZyvIB0LY=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=3KobMLIQepuMLByMytZyjHMV6AF5ieDNdhtyIruya5OV3bRHw9tf4z1R0suUBWDHz /k2H8I5SyA0Wngpg5LL4mFdwXq6WFz8WaS5g1Zox+s3R0tez0dC9BUkP5/3dlA95/6 Ii5kFItYCjtcZkobD4kO4MowsVVNrmm7RepriQp9/MwWszTRuk1ekNx7pQsgLX4o/w HLS4z84aQzoMP2nynX+AJOvJrmMahudqOx+EnyjctgiKyl8RELPZqD30AbRdGR/wSG LS+2oEUjCNMhXNcKRnKNSVlefGJ4hirvOxVi3HYl2uQ5pWS27hsA4eXD9s5hdCJPXW 3GzBdh/VwG2xw== Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C60AF62B61 for ; Thu, 14 Dec 2023 13:14:10 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="LoQiLexx"; dkim-atps=neutral DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1702556049; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yDVEm3pYLfHh77GPI/fz95A+9YFc6sPF3hsMkf+AWCM=; b=LoQiLexx2KsQ9baRu8cVFT0wo5SC+SMECWXq0xKYMJ5t9XiVdIkACMXjsvj6XpB+dI1vu9 9UtdUa0eqE8hEIgFh2xOBK7uFPp6ApQ9FPcmonZ2wXMh5RAg7TZop/YokXOkl3egeqtZgJ OkcBdCa72wC9blTFZn43wFLXyZOOMQI= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-173-vGhwkzODNaWFO5mdrr15ZA-1; Thu, 14 Dec 2023 07:14:06 -0500 X-MC-Unique: vGhwkzODNaWFO5mdrr15ZA-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0D7043C23FE8; Thu, 14 Dec 2023 12:14:06 +0000 (UTC) Received: from shalem.redhat.com (unknown [10.39.194.72]) by smtp.corp.redhat.com (Postfix) with ESMTP id BC30D1121306; Thu, 14 Dec 2023 12:14:04 +0000 (UTC) To: libcamera-devel@lists.libcamera.org, Andrey Konovalov Date: Thu, 14 Dec 2023 13:13:47 +0100 Message-ID: <20231214121350.206015-10-hdegoede@redhat.com> In-Reply-To: <20231214121350.206015-1-hdegoede@redhat.com> References: <20231214121350.206015-1-hdegoede@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.3 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [libcamera-devel] [PATCH 09/11] libcamera: software_isp: add Simple SoftwareIsp implementation X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Hans de Goede via libcamera-devel From: Hans de Goede Reply-To: Hans de Goede Cc: Maxime Ripard , g.martti@gmail.com, t.langendam@gmail.com, srinivas.kandagatla@linaro.org, Bryan O'Donoghue , admin@dennisbonke.com Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Andrey Konovalov The implementation of SoftwareIsp handles creation of Soft IPA and interactions with it, so that the pipeline handler wouldn't need to care about the Soft IPA. Signed-off-by: Andrey Konovalov Co-authored-by: Hans de Goede Signed-off-by: Hans de Goede --- include/libcamera/internal/meson.build | 1 + .../internal/software_isp/meson.build | 1 + .../internal/software_isp/swisp_simple.h | 67 ++++++ src/libcamera/meson.build | 1 + src/libcamera/software_isp/meson.build | 19 ++ src/libcamera/software_isp/swisp_simple.cpp | 219 ++++++++++++++++++ 6 files changed, 308 insertions(+) create mode 100644 include/libcamera/internal/software_isp/swisp_simple.h create mode 100644 src/libcamera/software_isp/swisp_simple.cpp diff --git a/include/libcamera/internal/meson.build b/include/libcamera/internal/meson.build index b780777c..eeae801c 100644 --- a/include/libcamera/internal/meson.build +++ b/include/libcamera/internal/meson.build @@ -50,3 +50,4 @@ libcamera_internal_headers = files([ ]) subdir('converter') +subdir('software_isp') diff --git a/include/libcamera/internal/software_isp/meson.build b/include/libcamera/internal/software_isp/meson.build index b5a0d737..cf21ace5 100644 --- a/include/libcamera/internal/software_isp/meson.build +++ b/include/libcamera/internal/software_isp/meson.build @@ -4,6 +4,7 @@ libcamera_internal_headers += files([ 'debayer.h', 'debayer_cpu.h', 'debayer_params.h', + 'swisp_simple.h', 'swisp_stats.h', 'swstats.h', 'swstats_cpu.h', diff --git a/include/libcamera/internal/software_isp/swisp_simple.h b/include/libcamera/internal/software_isp/swisp_simple.h new file mode 100644 index 00000000..ee66e193 --- /dev/null +++ b/include/libcamera/internal/software_isp/swisp_simple.h @@ -0,0 +1,67 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2023, Linaro Ltd + * + * swisp_simple.h - Simple software ISP implementation + */ + +#pragma once + +#include +#include + +#include + +#include +#include + +#include "libcamera/internal/software_isp.h" +#include "libcamera/internal/software_isp/debayer_cpu.h" + +namespace libcamera { + +class SwIspSimple : public SoftwareIsp +{ +public: + SwIspSimple(PipelineHandler *pipe, const ControlInfoMap &sensorControls); + ~SwIspSimple() {} + + int loadConfiguration([[maybe_unused]] const std::string &filename) { return 0; } + bool isValid() const; + + std::vector formats(PixelFormat input); + SizeRange sizes(PixelFormat inputFormat, const Size &inputSize); + + std::tuple + strideAndFrameSize(const PixelFormat &outputFormat, const Size &size); + + int configure(const StreamConfiguration &inputCfg, + const std::vector> &outputCfgs); + int exportBuffers(unsigned int output, unsigned int count, + std::vector> *buffers); + void processStats(const ControlList &sensorControls); + + int start(); + void stop(); + + int queueBuffers(FrameBuffer *input, + const std::map &outputs); + + Signal &getSignalSetSensorControls(); + + void process(FrameBuffer *input, FrameBuffer *output); + +private: + void statsReady(int dummy); + void inputReady(FrameBuffer *input); + void outputReady(FrameBuffer *output); + + std::unique_ptr debayer_; + Thread ispWorkerThread_; + /* FIXME debayerParams should come from the IPA */ + DebayerParams debayerParams_; + + std::unique_ptr ipa_; +}; + +} /* namespace libcamera */ diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build index b3606969..e758ac9c 100644 --- a/src/libcamera/meson.build +++ b/src/libcamera/meson.build @@ -70,6 +70,7 @@ subdir('converter') subdir('ipa') subdir('pipeline') subdir('proxy') +subdir('software_isp') null_dep = dependency('', required : false) diff --git a/src/libcamera/software_isp/meson.build b/src/libcamera/software_isp/meson.build index 6d7a44d7..9464f2fd 100644 --- a/src/libcamera/software_isp/meson.build +++ b/src/libcamera/software_isp/meson.build @@ -6,3 +6,22 @@ libcamera_sources += files([ 'swstats.cpp', 'swstats_cpu.cpp', ]) + +# Software ISP is enabled for 'simple' pipeline handler. +# The 'pipelines' option value should be +# 'simple/' e.g.: +# -Dpipelines=simple/simple +# The source file should be named swisp_.cpp, +# e.g. 'swisp_simple.cpp'. + +foreach pipeline : pipelines + pipeline = pipeline.split('/') + if pipeline.length() == 2 and pipeline[0] == 'simple' + libcamera_sources += files([ + 'swisp_' + pipeline[1] + '.cpp', + ]) + # the 'break' below can be removed if/when multiple + # Software ISP implementations are allowed in single build + break + endif +endforeach diff --git a/src/libcamera/software_isp/swisp_simple.cpp b/src/libcamera/software_isp/swisp_simple.cpp new file mode 100644 index 00000000..ff05b6fe --- /dev/null +++ b/src/libcamera/software_isp/swisp_simple.cpp @@ -0,0 +1,219 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2023, Linaro Ltd + * + * swisp_simple.cpp - Simple software ISP implementation + */ + +#include "libcamera/internal/software_isp/swisp_simple.h" + +#include +#include +#include + +#include +#include + +#include "libcamera/internal/bayer_format.h" +#include "libcamera/internal/framebuffer.h" +#include "libcamera/internal/ipa_manager.h" +#include "libcamera/internal/mapped_framebuffer.h" + +namespace libcamera { + +SwIspSimple::SwIspSimple(PipelineHandler *pipe, const ControlInfoMap &sensorControls) + : SoftwareIsp(pipe, sensorControls), debayer_(nullptr), debayerParams_{256, 256, 0.5f} +{ + std::unique_ptr stats; + + stats = std::make_unique(); + if (!stats) { + LOG(SoftwareIsp, Error) << "Failed to create SwStatsCpu object"; + return; + } + + stats->statsReady.connect(this, &SwIspSimple::statsReady); + + debayer_ = std::make_unique(std::move(stats)); + if (!debayer_) { + LOG(SoftwareIsp, Error) << "Failed to create DebayerCpu object"; + return; + } + + debayer_->inputBufferReady.connect(this, &SwIspSimple::inputReady); + debayer_->outputBufferReady.connect(this, &SwIspSimple::outputReady); + + ipa_ = IPAManager::createIPA(pipe, 0, 0); + if (!ipa_) { + LOG(SoftwareIsp, Error) + << "Creating IPA for software ISP failed"; + debayer_.reset(); + return; + } + + int ret = ipa_->init(IPASettings{ "No cfg file", "No sensor model" }, + debayer_->getStatsFD(), + sensorControls); + if (ret) { + LOG(SoftwareIsp, Error) << "IPA init failed"; + debayer_.reset(); + return; + } + + ipa_->configure(sensorControls); + + debayer_->moveToThread(&ispWorkerThread_); +} + +void SwIspSimple::processStats(const ControlList &sensorControls) +{ + ASSERT(ipa_); + ipa_->processStats(sensorControls); +} + +Signal &SwIspSimple::getSignalSetSensorControls() +{ + ASSERT(ipa_); + return ipa_->setSensorControls; +} + +bool SwIspSimple::isValid() const +{ + return !!debayer_; +} + +std::vector SwIspSimple::formats(PixelFormat inputFormat) +{ + ASSERT(debayer_ != nullptr); + + return debayer_->formats(inputFormat); +} + +SizeRange SwIspSimple::sizes(PixelFormat inputFormat, const Size &inputSize) +{ + ASSERT(debayer_ != nullptr); + + return debayer_->sizes(inputFormat, inputSize); +} + +std::tuple +SwIspSimple::strideAndFrameSize(const PixelFormat &outputFormat, const Size &size) +{ + ASSERT(debayer_ != nullptr); + + return debayer_->strideAndFrameSize(outputFormat, size); +} + +int SwIspSimple::configure(const StreamConfiguration &inputCfg, + const std::vector> &outputCfgs) +{ + ASSERT(debayer_ != nullptr); + + return debayer_->configure(inputCfg, outputCfgs); +} + +int SwIspSimple::exportBuffers(unsigned int output, unsigned int count, + std::vector> *buffers) +{ + ASSERT(debayer_ != nullptr); + + /* single output for now */ + if (output >= 1) + return -EINVAL; + + /* TODO: allocate from dma_heap; memfd buffs aren't allowed in FrameBuffer */ + for (unsigned int i = 0; i < count; i++) { + std::string name = "frame-" + std::to_string(i); + + const int ispFd = memfd_create(name.c_str(), 0); + int ret = ftruncate(ispFd, debayer_->frameSize()); + if (ret < 0) { + LOG(SoftwareIsp, Error) + << "ftruncate() for memfd failed " + << strerror(-ret); + return ret; + } + + FrameBuffer::Plane outPlane; + outPlane.fd = SharedFD(std::move(ispFd)); + outPlane.offset = 0; + outPlane.length = debayer_->frameSize(); + + std::vector planes{ outPlane }; + buffers->emplace_back(std::make_unique(std::move(planes))); + } + + return count; +} + +int SwIspSimple::queueBuffers(FrameBuffer *input, + const std::map &outputs) +{ + unsigned int mask = 0; + + /* + * Validate the outputs as a sanity check: at least one output is + * required, all outputs must reference a valid stream and no two + * outputs can reference the same stream. + */ + if (outputs.empty()) + return -EINVAL; + + for (auto [index, buffer] : outputs) { + if (!buffer) + return -EINVAL; + if (index >= 1) /* only single stream atm */ + return -EINVAL; + if (mask & (1 << index)) + return -EINVAL; + + mask |= 1 << index; + } + + process(input, outputs.at(0)); + + return 0; +} + +int SwIspSimple::start() +{ + int ret = ipa_->start(); + if (ret) + return ret; + + ispWorkerThread_.start(); + return 0; +} + +void SwIspSimple::stop() +{ + ispWorkerThread_.exit(); + ispWorkerThread_.wait(); + + ipa_->stop(); +} + +void SwIspSimple::process(FrameBuffer *input, FrameBuffer *output) +{ + debayer_->invokeMethod(&DebayerCpu::process, + ConnectionTypeQueued, input, output, debayerParams_); +} + +void SwIspSimple::statsReady(int dummy) +{ + ispStatsReady.emit(dummy); +} + +void SwIspSimple::inputReady(FrameBuffer *input) +{ + inputBufferReady.emit(input); +} + +void SwIspSimple::outputReady(FrameBuffer *output) +{ + outputBufferReady.emit(output); +} + +REGISTER_SOFTWAREISP(SwIspSimple) + +} /* namespace libcamera */