From patchwork Tue Dec 12 11:50:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 19306 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D2A6DBD87C for ; Tue, 12 Dec 2023 11:51:20 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 881D362B2E; Tue, 12 Dec 2023 12:51:20 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1702381880; bh=+DHVvdlkAeBB23ioXjBk7DxfCC50ZttBLeVX6IWs+w4=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=h5OsJ5x9PJ02HZyOVbS41o1Xtj6Ed7mkdTd3vUKzxJosgtF/ndKKesUmVf5z5lthb ffNg38PdcST7tiYIAhlaeGyDutvj7lijKs5jiizdnh0dnzoFr9thW5eCI8UA6RQGwC ieHOt39I4kbCRA+dM61113hGb/kWcIbub7GbAK2O0vKncYD/zCS+2ZQBkQLIMU3w1O x7zbho0qfPspOueT1dQTly/aUjJmdqtLpeTkulR260coQXygcepp3wP226bXbUyg1D n23tKQ0KufCBMzEkbyh4bagLKjcRWTjRUer8+ruNApTGLijkDswlmKZwizIN/gVaRs ZexloDROffbig== Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com [IPv6:2a00:1450:4864:20::534]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E02F6629E1 for ; Tue, 12 Dec 2023 12:51:18 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="rcI/l7CI"; dkim-atps=neutral Received: by mail-ed1-x534.google.com with SMTP id 4fb4d7f45d1cf-54c79968ffbso7341146a12.3 for ; Tue, 12 Dec 2023 03:51:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1702381878; x=1702986678; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=uIstvMk8lbSTkr39GEaKHG4p3SI0k5/yPhecGjI4+ac=; b=rcI/l7CIqt7LKlj0CbiPb18puU1srndB4JIzeZ+Fyx0CpNBiFMXrHOIJEl+vWzmh1D ziiukPj7kdmSFvgFTI1p4Ova27w5ZcQLBYi9csJsojgR9f7HlJoO5yYTdNZa9noEtl4l oLMUhRoWGTTDc3+Q3eKcLz5ty+gnXqKPjy/m/+7dfH4YhZPbt6W3JeLxAEnm5XTpl7FS GD3o/1NrbNqHPSVYiJlSu4vmCFCzUbpypJXGX26WzJyIi+wnsVO8JWs+BTrAvPBtIE5/ hPsb8G26+/PUaxtZ6Is85WOcCVHyJyR+pf5fbj/SQY5WwAeb0LO1l2orFSBc/GHSUTRX 93rQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702381878; x=1702986678; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=uIstvMk8lbSTkr39GEaKHG4p3SI0k5/yPhecGjI4+ac=; b=SyXKpI/7ZbRvFPhYeE4ulKUXS6XGfjGOalvloVUzMC4+5/e5ikAmzCGEMirbcP0eIA BZbjVfA9+2a5hztIn7YVfKY7ib7K00FV8IM+/zqcWmr71M6ws8HBN6jUjysxJUz7FMMY bPg1Cv4uWG2SnOiWHluuGT9MzyVG/Rusqq5lu3UyoNQsG43zvbHSPb1dkI1rlXnzDujP rG0lss13izG0DYm6FAhM2DyHCmzd7T1Dd4vEA3VTMfKDioy5fOj3ANmZZK+CLWZ/nkW5 JVXhu3Jd8sjfoTemzaLks99JBJqxwf2GpiFLwkuw3huKMNDzGbGziWmISahtMXvug3Vj Iu6w== X-Gm-Message-State: AOJu0YzPbDyyLnaz7qLWCqSqi5S1LvxB7FzTCpoyHgyJx/UDEdHVSDiP SxG3hrP/EQcCD82mcAGs+ohcnfVPgPN8aQtpQjI= X-Google-Smtp-Source: AGHT+IGVqvwTNKFDvnw/9QVhpxsIXLRCjPG7DqRB5s5IXKdwv++pAsPzcsLA2EofB9EOXF/UhS57kQ== X-Received: by 2002:a17:906:3f13:b0:a1e:7683:4da4 with SMTP id c19-20020a1709063f1300b00a1e76834da4mr2495514ejj.11.1702381878410; Tue, 12 Dec 2023 03:51:18 -0800 (PST) Received: from Lat-5310.. ([87.116.161.153]) by smtp.gmail.com with ESMTPSA id tx17-20020a1709078e9100b00a1b75e0e061sm6213188ejc.130.2023.12.12.03.51.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Dec 2023 03:51:18 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Tue, 12 Dec 2023 14:50:40 +0300 Message-Id: <20231212115046.102726-2-andrey.konovalov@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231212115046.102726-1-andrey.konovalov@linaro.org> References: <20231212115046.102726-1-andrey.konovalov@linaro.org> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH v2 1/7] libcamera: introduce SoftwareIsp class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Andrey Konovalov via libcamera-devel From: Andrey Konovalov Reply-To: Andrey Konovalov Cc: mripard@redhat.com, g.martti@gmail.com, t.langendam@gmail.com, srinivas.kandagatla@linaro.org, pavel@ucw.cz, bryan.odonoghue@linaro.org, admin@dennisbonke.com Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Signed-off-by: Andrey Konovalov --- include/libcamera/internal/meson.build | 1 + include/libcamera/internal/software_isp.h | 106 ++++++++++++++++++++++ src/libcamera/meson.build | 1 + src/libcamera/software_isp.cpp | 62 +++++++++++++ 4 files changed, 170 insertions(+) create mode 100644 include/libcamera/internal/software_isp.h create mode 100644 src/libcamera/software_isp.cpp diff --git a/include/libcamera/internal/meson.build b/include/libcamera/internal/meson.build index 7f1f3440..0f725cfd 100644 --- a/include/libcamera/internal/meson.build +++ b/include/libcamera/internal/meson.build @@ -38,6 +38,7 @@ libcamera_internal_headers = files([ 'process.h', 'pub_key.h', 'request.h', + 'software_isp.h', 'source_paths.h', 'sysfs.h', 'v4l2_device.h', diff --git a/include/libcamera/internal/software_isp.h b/include/libcamera/internal/software_isp.h new file mode 100644 index 00000000..14511017 --- /dev/null +++ b/include/libcamera/internal/software_isp.h @@ -0,0 +1,106 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2023, Linaro Ltd + * + * software_isp.h - Interface for a software implementation of an ISP + */ + +#pragma once + +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +#include + +#include "libcamera/internal/pipeline_handler.h" + +namespace libcamera { + +class FrameBuffer; +class PixelFormat; +struct StreamConfiguration; + +class SoftwareIsp +{ +public: + SoftwareIsp(PipelineHandler *pipe, const ControlInfoMap &sensorControls); + virtual ~SoftwareIsp(); + + virtual int loadConfiguration(const std::string &filename) = 0; + + virtual bool isValid() const = 0; + + virtual std::vector formats(PixelFormat input) = 0; + virtual SizeRange sizes(PixelFormat inputFormat, const Size &inputSize) = 0; + + virtual std::tuple + strideAndFrameSize(const PixelFormat &pixelFormat, const Size &size) = 0; + + virtual int configure(const StreamConfiguration &inputCfg, + const std::vector> &outputCfgs) = 0; + virtual int exportBuffers(unsigned int output, unsigned int count, + std::vector> *buffers) = 0; + + virtual int start() = 0; + virtual void stop() = 0; + + virtual int queueBuffers(FrameBuffer *input, + const std::map &outputs) = 0; + + virtual void processStats(const ControlList &sensorControls) = 0; // rather merge with queueBuffers()? + + virtual Signal &getSignalSetSensorControls() = 0; + + Signal inputBufferReady; + Signal outputBufferReady; + + /* The int parameter isn't actually used */ + Signal ispStatsReady; +}; + +class SoftwareIspFactoryBase +{ +public: + SoftwareIspFactoryBase(); + virtual ~SoftwareIspFactoryBase() = default; + + static std::unique_ptr create(PipelineHandler *pipe, + const ControlInfoMap &sensorControls); + static SoftwareIspFactoryBase *&factory(); + +private: + LIBCAMERA_DISABLE_COPY_AND_MOVE(SoftwareIspFactoryBase) + + static void registerType(SoftwareIspFactoryBase *factory); + virtual std::unique_ptr createInstance(PipelineHandler *pipe, + const ControlInfoMap &sensorControls) const = 0; +}; + +template +class SoftwareIspFactory : public SoftwareIspFactoryBase +{ +public: + SoftwareIspFactory() + : SoftwareIspFactoryBase() + { + } + + std::unique_ptr createInstance(PipelineHandler *pipe, + const ControlInfoMap &sensorControls) const override + { + return std::make_unique<_SoftwareIsp>(pipe, sensorControls); + } +}; + +#define REGISTER_SOFTWAREISP(softwareIsp) \ + static SoftwareIspFactory global_##softwareIsp##Factory; + +} /* namespace libcamera */ diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build index 45f63e93..b3606969 100644 --- a/src/libcamera/meson.build +++ b/src/libcamera/meson.build @@ -40,6 +40,7 @@ libcamera_sources = files([ 'process.cpp', 'pub_key.cpp', 'request.cpp', + 'software_isp.cpp', 'source_paths.cpp', 'stream.cpp', 'sysfs.cpp', diff --git a/src/libcamera/software_isp.cpp b/src/libcamera/software_isp.cpp new file mode 100644 index 00000000..2ff97d70 --- /dev/null +++ b/src/libcamera/software_isp.cpp @@ -0,0 +1,62 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2023, Linaro Ltd + * + * software_isp.cpp - Interface for a software implementation of an ISP + */ + +#include "libcamera/internal/software_isp.h" + +#include + +namespace libcamera { + +LOG_DEFINE_CATEGORY(SoftwareIsp) + +SoftwareIsp::SoftwareIsp([[maybe_unused]] PipelineHandler *pipe, + [[maybe_unused]] const ControlInfoMap &sensorControls) +{ +} + +SoftwareIsp::~SoftwareIsp() +{ +} + +/* SoftwareIspFactoryBase */ + +SoftwareIspFactoryBase::SoftwareIspFactoryBase() +{ + registerType(this); +} + +void SoftwareIspFactoryBase::registerType(SoftwareIspFactoryBase *factory) +{ + SoftwareIspFactoryBase *®istered = + SoftwareIspFactoryBase::factory(); + + ASSERT(!registered && factory); + registered = factory; +} + +SoftwareIspFactoryBase *&SoftwareIspFactoryBase::factory() +{ + static SoftwareIspFactoryBase *factory; + return factory; +} + +std::unique_ptr +SoftwareIspFactoryBase::create(PipelineHandler *pipe, + const ControlInfoMap &sensorControls) +{ + SoftwareIspFactoryBase *factory = SoftwareIspFactoryBase::factory(); + if (!factory) + return nullptr; + + std::unique_ptr swIsp = factory->createInstance(pipe, sensorControls); + if (swIsp->isValid()) + return swIsp; + + return nullptr; +} + +} /* namespace libcamera */ From patchwork Tue Dec 12 11:50:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 19307 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7C0F1BD87C for ; Tue, 12 Dec 2023 11:51:23 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2C3CD62B49; Tue, 12 Dec 2023 12:51:23 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1702381883; bh=oRBIY8kOi0gKYV5IPe6DVH6UPAc8qzG43Lgmf4uwB34=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=stA0/DEGFaa8f6XMcomT0hlegGMZls1wo2kavHX3rdz7CtVIy3lCgRsRHOPL1MsDX dIo8b2LyKGErGFwhygJAfQmYNJmz0S4KWmhJ5zoOSqLzOn+dZPm4eGmK/GISiygxN7 SmFSKpWYC2qTsTo+WaFXmu5poKQbp6L2s1ChNeO33hP479VXT/L7picyEJ5whCNeai aRYD+a5OMv3nwpx3nFVOhrz2QfHbT4miFglyJFyZK28zLSLjEM/wPqC4nyFAs89hB3 Bf/k0gJpGo/+46ZPqIGCABdOuFb0xvlmmxGr4Zo2bq0jtravqJzy5AmySVTeSA8KPd 9Emyn33TBlSJA== Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com [IPv6:2a00:1450:4864:20::632]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 42E93629E1 for ; Tue, 12 Dec 2023 12:51:21 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="iRu61vP9"; dkim-atps=neutral Received: by mail-ej1-x632.google.com with SMTP id a640c23a62f3a-a00cbb83c80so636150566b.0 for ; Tue, 12 Dec 2023 03:51:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1702381881; x=1702986681; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=pFJAoSA9Nwu/bV7xUFK2GgTlAP4DOK35M5GoMj00EiU=; b=iRu61vP9BuGQLeQvrp10obvknYuPYoHW8BrZKmERHEJlXAoS46+x7X1oy7rG7oTiB2 A1d4x6/1EFwc4RJZHrI0hOc6JQOgD1f/ow7TVECITU9xYppUsx0MHi214XagIQnPbuTy 9UtxNmFS/rCjrLVhv7JrKWzMnEyaNfsxK7Nbai8j8vZTzYVcyP1NmOYoCkV37NGIkhGg QPop0EWA53W7xuaeAt7ITHRz+wlq46eytS02ruyejfd44CXrXP2gpoORUS40r4SaMqOh ZfFuMQ8IoARxMuwOfF8ETOW7xjOW4EdULEhcnV69/uktT+QCUZLps87XrRkhRCj/8IPu H49w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702381881; x=1702986681; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pFJAoSA9Nwu/bV7xUFK2GgTlAP4DOK35M5GoMj00EiU=; b=oSfCHWDzM8e8rY6AeAvHUZ5eTZlORgOVqwvnyw86j7wiWScfK+w+JYfRdWGaOQi6WF uzhfKo+nO9Rs67bgT1auwbUHRlE4FULW0mFpySbWyOyY40YVTaJysNOvsCtJ1bXJrm9c IeMvGddrdjMygzEDqJeqdWhka3wcZUbXBQ336wCEv0Pvo+oaHnRJaG6NsDxGzke0g5If KtN2N/bW2VnUtody2SVeaFFX9dF3mohh8gV5nKwaRtcEtymJDYezkEsxrp8Nej+guzP2 VYut1xXPJ4/FjLMGDTZgxfOjZvtr0smBJ94c9TBZ51CTRJ9pKgWyO6bZX6hvkP1H9lUB ARGQ== X-Gm-Message-State: AOJu0YypRrqflg/fPu9nexmWNtVSeh8LWAk0ROs2vxvbmBEY3GzQA3WP a9XI/ADyvEUhQIT52YLD5fA3z5HVPDLTjY4C63Y= X-Google-Smtp-Source: AGHT+IGUcWnaflIsAT1p4lLw8JO6YrriyT5HqbBf0QqyopyLHQ+Z7pLu0DV+AM24kZb05DUk/xosYw== X-Received: by 2002:a17:907:7f88:b0:9be:7b67:1674 with SMTP id qk8-20020a1709077f8800b009be7b671674mr3384289ejc.3.1702381880873; Tue, 12 Dec 2023 03:51:20 -0800 (PST) Received: from Lat-5310.. ([87.116.161.153]) by smtp.gmail.com with ESMTPSA id tx17-20020a1709078e9100b00a1b75e0e061sm6213188ejc.130.2023.12.12.03.51.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Dec 2023 03:51:20 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Tue, 12 Dec 2023 14:50:41 +0300 Message-Id: <20231212115046.102726-3-andrey.konovalov@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231212115046.102726-1-andrey.konovalov@linaro.org> References: <20231212115046.102726-1-andrey.konovalov@linaro.org> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH v2 2/7] libcamera: internal: Move SharedMemObject class to a common directory X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Andrey Konovalov via libcamera-devel From: Andrey Konovalov Reply-To: Andrey Konovalov Cc: mripard@redhat.com, g.martti@gmail.com, t.langendam@gmail.com, srinivas.kandagatla@linaro.org, pavel@ucw.cz, bryan.odonoghue@linaro.org, admin@dennisbonke.com Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Move SharedMemObject class out of RPi namespace and put it into include/libcamera/internal so that everyone could use it. Signed-off-by: Andrey Konovalov --- include/libcamera/internal/meson.build | 1 + .../common => include/libcamera/internal}/shared_mem_object.h | 4 ---- 2 files changed, 1 insertion(+), 4 deletions(-) rename {src/libcamera/pipeline/rpi/common => include/libcamera/internal}/shared_mem_object.h (98%) diff --git a/include/libcamera/internal/meson.build b/include/libcamera/internal/meson.build index 0f725cfd..b780777c 100644 --- a/include/libcamera/internal/meson.build +++ b/include/libcamera/internal/meson.build @@ -38,6 +38,7 @@ libcamera_internal_headers = files([ 'process.h', 'pub_key.h', 'request.h', + 'shared_mem_object.h', 'software_isp.h', 'source_paths.h', 'sysfs.h', diff --git a/src/libcamera/pipeline/rpi/common/shared_mem_object.h b/include/libcamera/internal/shared_mem_object.h similarity index 98% rename from src/libcamera/pipeline/rpi/common/shared_mem_object.h rename to include/libcamera/internal/shared_mem_object.h index aa56c220..bfb639ee 100644 --- a/src/libcamera/pipeline/rpi/common/shared_mem_object.h +++ b/include/libcamera/internal/shared_mem_object.h @@ -19,8 +19,6 @@ namespace libcamera { -namespace RPi { - template class SharedMemObject { @@ -123,6 +121,4 @@ private: T *obj_; }; -} /* namespace RPi */ - } /* namespace libcamera */ From patchwork Tue Dec 12 11:50:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 19308 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 54DE3C3292 for ; Tue, 12 Dec 2023 11:51:25 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E699062B52; Tue, 12 Dec 2023 12:51:24 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1702381884; bh=CJI8FQq0mtavduac21Ymmof0RKxobTxU+1R/iZoaOig=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=AX9iC/Wvoy55Ma34FUozOGuB2ofBHck4PUQ4IQu+vR1i2eh+/Il9+v4zL9XgkSTnI WWkfRWvcofE8rFp/VAwg4pCfDp76f6xxsCtyyjnPfdHAkazgV6UtEUmXf7GnnTAW8X PXlQAzJed8KzXaAxyQ9/stWR8VYyy9bHabkHb6nDWMzD2KIi1opBmqSy8RhQZprawj aAy/v8DezaV7LA8V33q5Xif6SDS2+JwmdzMhzQ5q26Cu4qfXaEIc9VpS5dwWywzilx w9OUdiXX40JR409yynu9gFI1iGfBF/I7AvwJTQB90e1B08sICNFw0vTNwT0MbMByLB nHj5PyM6OcfDA== Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com [IPv6:2a00:1450:4864:20::52e]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2AA8662B41 for ; Tue, 12 Dec 2023 12:51:23 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="yr9a8pMn"; dkim-atps=neutral Received: by mail-ed1-x52e.google.com with SMTP id 4fb4d7f45d1cf-55193d5e8cdso511841a12.1 for ; Tue, 12 Dec 2023 03:51:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1702381882; x=1702986682; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=UMHEt2nX/fX21DwzK+lb5B5tJv/czMRUFMKk7pQX9aQ=; b=yr9a8pMnlbo2KMa9e09SmdDP7UNVlw1QyP8fv3MX0re7QHhJYKe+Ex01KqGTTbFFff XlUDYUO7r489gUp5JOZdZmnR5oR1kekV79+3IDwetGYJcyuyNDwPAr0YyCC12SrRab6n Wf0a5Zw8Z9WR010HiSoSIx7+XfzvGDZdQR1ufaZnmmiUx2Kv8OZsN7FiCLEG+7JW+3w8 OZKipKr1ckX4S+bBGZixtnPzNkG9LoHB/PVUElT/bR9Fwjh7PZi2XbdsL//KwLLYYdXX zTKWw2/kjuEAfWBJa3fhHG1cVVlnTQUr1ZpP8l8nHeTJ0S+GCr0HcXz5y0B2CYm9Rm/0 AHnw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702381882; x=1702986682; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UMHEt2nX/fX21DwzK+lb5B5tJv/czMRUFMKk7pQX9aQ=; b=LUFkfF4xLoVGsLDnqimc3LbJ9E74CwZPceDpAk1YtPkUaSAte6mhNOekw8nCQ18Qyy OrciP8UiV9P2oFtHCPBeoLKYegg91lG46j047Pt18vS5RKdik/s7ZxZcP3KbsmPnieDK 5W54drGeVfpDFg9EivLPxx9ErK08ayr5VV3sdSGIxY3PPHfaRrt95GVS9OdEhxk5x3tj w238aq4x/0Nn+iifOmhzBMcX3DgEJllC2x9h7WGzbeM1JntW9UusE3ZUukUEQOcjWRBd gk+p+SLOo0Ae5pWOz8+y3j8Cw65l6XPHE8v3bhybMuQPTecyrknfh3F/eQcqFvdb3fCi +lVQ== X-Gm-Message-State: AOJu0Ywk7wOb0UX/eqC4Y4adN1m6p5zMhMrnF5HRer6RChQD0RPZjO3N 9NyjtUaU0+qNhCeAzG3fuKtQZi97Vgk1nsGxbfk= X-Google-Smtp-Source: AGHT+IGMqLximQEeYHKJnpqgG+iwSG6I868OH7WHfbl1p214rixjj6lGbk6f5N0Zh3DBSQZtjsCRBQ== X-Received: by 2002:a17:906:1db:b0:a1d:932f:9098 with SMTP id 27-20020a17090601db00b00a1d932f9098mr2876607ejj.97.1702381882708; Tue, 12 Dec 2023 03:51:22 -0800 (PST) Received: from Lat-5310.. ([87.116.161.153]) by smtp.gmail.com with ESMTPSA id tx17-20020a1709078e9100b00a1b75e0e061sm6213188ejc.130.2023.12.12.03.51.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Dec 2023 03:51:22 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Tue, 12 Dec 2023 14:50:42 +0300 Message-Id: <20231212115046.102726-4-andrey.konovalov@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231212115046.102726-1-andrey.konovalov@linaro.org> References: <20231212115046.102726-1-andrey.konovalov@linaro.org> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH v2 3/7] libcamera: ipa: add Soft IPA common files X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Andrey Konovalov via libcamera-devel From: Andrey Konovalov Reply-To: Andrey Konovalov Cc: mripard@redhat.com, g.martti@gmail.com, t.langendam@gmail.com, srinivas.kandagatla@linaro.org, pavel@ucw.cz, bryan.odonoghue@linaro.org, admin@dennisbonke.com Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Define the Soft IPA main and event interfaces, add IPASoftBase class the Soft IPA implementation inherit from. Signed-off-by: Andrey Konovalov --- include/libcamera/ipa/meson.build | 1 + include/libcamera/ipa/soft.mojom | 27 ++++++++++++ src/ipa/simple/common/meson.build | 17 ++++++++ src/ipa/simple/common/soft_base.cpp | 66 +++++++++++++++++++++++++++++ src/ipa/simple/common/soft_base.h | 47 ++++++++++++++++++++ src/ipa/simple/meson.build | 3 ++ 6 files changed, 161 insertions(+) create mode 100644 include/libcamera/ipa/soft.mojom create mode 100644 src/ipa/simple/common/meson.build create mode 100644 src/ipa/simple/common/soft_base.cpp create mode 100644 src/ipa/simple/common/soft_base.h create mode 100644 src/ipa/simple/meson.build diff --git a/include/libcamera/ipa/meson.build b/include/libcamera/ipa/meson.build index f3b4881c..aaee5cbf 100644 --- a/include/libcamera/ipa/meson.build +++ b/include/libcamera/ipa/meson.build @@ -65,6 +65,7 @@ pipeline_ipa_mojom_mapping = { 'ipu3': 'ipu3.mojom', 'rkisp1': 'rkisp1.mojom', 'rpi/vc4': 'raspberrypi.mojom', + 'simple/linaro': 'soft.mojom', 'vimc': 'vimc.mojom', } diff --git a/include/libcamera/ipa/soft.mojom b/include/libcamera/ipa/soft.mojom new file mode 100644 index 00000000..c3449188 --- /dev/null +++ b/include/libcamera/ipa/soft.mojom @@ -0,0 +1,27 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ + +/* + * \todo Document the interface and remove the related EXCLUDE_PATTERNS entry. + * \todo Add a way to tell SoftIPA the list of params SoftISP accepts? + */ + +module ipa.soft; + +import "include/libcamera/ipa/core.mojom"; + +interface IPASoftInterface { + init(libcamera.IPASettings settings, + libcamera.SharedFD fdStats, + libcamera.ControlInfoMap sensorCtrlInfoMap) + => (int32 ret); + start() => (int32 ret); + stop(); + configure(libcamera.ControlInfoMap sensorCtrlInfoMap) + => (int32 ret); + + [async] processStats(libcamera.ControlList sensorControls); +}; + +interface IPASoftEventInterface { + setSensorControls(libcamera.ControlList sensorControls); +}; diff --git a/src/ipa/simple/common/meson.build b/src/ipa/simple/common/meson.build new file mode 100644 index 00000000..023e617b --- /dev/null +++ b/src/ipa/simple/common/meson.build @@ -0,0 +1,17 @@ +# SPDX-License-Identifier: CC0-1.0 + +soft_ipa_common_sources = files([ + 'soft_base.cpp', +]) + +soft_ipa_common_includes = [ + include_directories('..'), +] + +soft_ipa_common_deps = [ + libcamera_private, +] + +soft_ipa_common_lib = static_library('soft_ipa_common', soft_ipa_common_sources, + include_directories : soft_ipa_common_includes, + dependencies : soft_ipa_common_deps) diff --git a/src/ipa/simple/common/soft_base.cpp b/src/ipa/simple/common/soft_base.cpp new file mode 100644 index 00000000..7bd9b8de --- /dev/null +++ b/src/ipa/simple/common/soft_base.cpp @@ -0,0 +1,66 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2023, Linaro Ltd + * + * soft-base.cpp - Software IPA base class + */ + +#include "soft_base.h" + +#include + +#include +#include + +#include + +namespace libcamera { + +LOG_DEFINE_CATEGORY(IPASoft) + +namespace ipa::soft { + +IPASoftBase::IPASoftBase() +{ +} + +IPASoftBase::~IPASoftBase() +{ +} + +int IPASoftBase::init([[maybe_unused]] const IPASettings &settings, + const SharedFD &fdStats, + const ControlInfoMap &sensorInfoMap) +{ + fdStats_ = std::move(fdStats); + if (!fdStats_.isValid()) { + LOG(IPASoft, Error) << "Invalid Statistics handle"; + return -ENODEV; + } + + return platformInit(sensorInfoMap); +} + +int IPASoftBase::configure(const ControlInfoMap &sensorInfoMap) +{ + return platformConfigure(sensorInfoMap); +} + +int IPASoftBase::start() +{ + return platformStart(); +} + +void IPASoftBase::stop() +{ + return platformStop(); +} + +void IPASoftBase::processStats(const ControlList &sensorControls) +{ + return platformProcessStats(sensorControls); +} + +} /* namespace ipa::soft */ + +} /* namespace libcamera */ diff --git a/src/ipa/simple/common/soft_base.h b/src/ipa/simple/common/soft_base.h new file mode 100644 index 00000000..bff53713 --- /dev/null +++ b/src/ipa/simple/common/soft_base.h @@ -0,0 +1,47 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2023, Linaro Ltd + * + * soft-base.h - Software IPA base class + */ +#pragma once + +#include +#include + +#include + +namespace libcamera { + +namespace ipa::soft { + +class IPASoftBase : public ipa::soft::IPASoftInterface +{ +public: + IPASoftBase(); + ~IPASoftBase(); + + int init(const IPASettings &settings, + const SharedFD &fdStats, + const ControlInfoMap &sensorInfoMap) override; + int configure(const ControlInfoMap &sensorInfoMap) override; + + int start() override; + void stop() override; + + void processStats(const ControlList &sensorControls) override; + +protected: + SharedFD fdStats_; + +private: + virtual int platformInit(const ControlInfoMap &sensorInfoMap) = 0; + virtual int platformConfigure(const ControlInfoMap &sensorInfoMap) = 0; + virtual int platformStart() = 0; + virtual void platformStop() = 0; + virtual void platformProcessStats(const ControlList &sensorControls) = 0; +}; + +} /* namespace ipa::soft */ + +} /* namespace libcamera */ diff --git a/src/ipa/simple/meson.build b/src/ipa/simple/meson.build new file mode 100644 index 00000000..9688bbdb --- /dev/null +++ b/src/ipa/simple/meson.build @@ -0,0 +1,3 @@ +# SPDX-License-Identifier: CC0-1.0 + +subdir('common') From patchwork Tue Dec 12 11:50:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 19309 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id F37FABD87C for ; Tue, 12 Dec 2023 11:51:27 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8204262B40; Tue, 12 Dec 2023 12:51:27 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1702381887; bh=9k6HitU/ugcAxc68F/Fkbvwoe/i4WopB9FGBeglF/sA=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=MEho6vq2tFmjPHjYF9BBwBFdsFbUL4xmNuW6F0n3MDrDYJGNdR9rz/Oe9ZTrfTe16 q5mU6q5+kwZSEsV/MROwGMwQwpkknX4JzSp1mWIbRNEficviFxII9cC20YvQ8+5kDr XXsibXpxFTQy6sOEjPKu6wIQDludL78h+Muyc3GsP6nonEKv84tnBR+AZom2PtODXS IDD/EAsAGwAJqebbgV5Eg4V/ZCkIQtslLyWWCTlD18c0UjtlDhnqZktgKzLatV69pL ZZDdXRJeFD8biR3l7XO9SNHDphfIm6NN5AGuOjtKrRNwOU2a4GdEcciTJIMWLzWgWR Gcys3i79E8fXw== Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com [IPv6:2a00:1450:4864:20::629]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CFA5762B40 for ; Tue, 12 Dec 2023 12:51:24 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="m//Yj3Sl"; dkim-atps=neutral Received: by mail-ej1-x629.google.com with SMTP id a640c23a62f3a-a1f6433bc1eso851525966b.1 for ; Tue, 12 Dec 2023 03:51:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1702381884; x=1702986684; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=d7Yka9JlQsGu3JwP/jcS1RWJP5f9p2Jdc7n6scdb/kI=; b=m//Yj3SlmWYGBj4L5iIkqkbiGnS93M+C7dn8QcKZ00l2EQqCm0Mg4Zem0AgB/FVOsw uQ3PHwUXW5xZVFWurNKyM1MKmIUyubRFSKTouQV7GMN4AWsdYNR/adsfHJPfrkeMf8La bZ/3wY2hVonSgpGw3sFut2DkqzsJ6ivgoEyLRENvJdozjoM4pETPimPMaymj3bPGVUoY prPCgP5XFPDB1gHs5jz2SPMKMZIeMatWsAoI4bVyXRN3/ABbE3hkf3/BaI4zD34RD3Ix lluqE661Vw10EbRqrnbtc7d9H0XObdL92689CQzftWYwtctx4tG+P6ReMvwDsvaQRaiC sBqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702381884; x=1702986684; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=d7Yka9JlQsGu3JwP/jcS1RWJP5f9p2Jdc7n6scdb/kI=; b=GqwllAh/Pc6DqihSnZuMlo/xp5yA03ckW91aVaaIgKiG0FqKQqTIjM9Z2pzXU3budQ NIwsuxDscZsNpTv0WwLHshv1RnxvaIQTqtF1Bx++2qpozCfsMYkiBhDc8RXcNML2H1Dk cNxc2zvo/rtGup/odD4YGmwgUaKZfDBYhNOc6exUeBECXlaJs0fVSiB0B7uxsZZH9gVc AGUfCTQzmJoU5rDt3hNNg0X359oYsXoXjwCUIrpe47/bhzWicsyiIiE6gDLjoYl/0iqU G3r0fPfxfvBBqyDNLxC/9mhrjEVnSrixla9P5hLEQFrfAhM8khDIjIwVe+b6csZIyoHR Sxow== X-Gm-Message-State: AOJu0YysYQLXKNN0vOOaSAbhK5XB0FBVzig14KKwY4G/ZkpPa/rfcWOp GqangkirLg7NKVFG48lEseXwiqdltJhCSpkgi0Y= X-Google-Smtp-Source: AGHT+IFDjZPYk8VuCtB4QIYn01OAcicXWkIJFrlySRAY/9ZSujzhwa5TEzDfVM3OSMUvKIUeQFYFLQ== X-Received: by 2002:a17:906:3f12:b0:a1f:7065:1efc with SMTP id c18-20020a1709063f1200b00a1f70651efcmr5043134ejj.5.1702381884452; Tue, 12 Dec 2023 03:51:24 -0800 (PST) Received: from Lat-5310.. ([87.116.161.153]) by smtp.gmail.com with ESMTPSA id tx17-20020a1709078e9100b00a1b75e0e061sm6213188ejc.130.2023.12.12.03.51.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Dec 2023 03:51:24 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Tue, 12 Dec 2023 14:50:43 +0300 Message-Id: <20231212115046.102726-5-andrey.konovalov@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231212115046.102726-1-andrey.konovalov@linaro.org> References: <20231212115046.102726-1-andrey.konovalov@linaro.org> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH v2 4/7] libcamera: ipa: Soft IPA: add a Soft IPA implementation X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Andrey Konovalov via libcamera-devel From: Andrey Konovalov Reply-To: Andrey Konovalov Cc: mripard@redhat.com, g.martti@gmail.com, t.langendam@gmail.com, srinivas.kandagatla@linaro.org, pavel@ucw.cz, bryan.odonoghue@linaro.org, admin@dennisbonke.com Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Signed-off-by: Andrey Konovalov Signed-off-by: Hans de Goede --- .../internal/software_isp/meson.build | 5 + .../internal/software_isp/statistics-linaro.h | 17 ++ meson_options.txt | 3 +- src/ipa/simple/linaro/data/meson.build | 9 + src/ipa/simple/linaro/data/soft.conf | 3 + src/ipa/simple/linaro/meson.build | 26 +++ src/ipa/simple/linaro/soft_linaro.cpp | 210 ++++++++++++++++++ src/ipa/simple/meson.build | 9 + 8 files changed, 281 insertions(+), 1 deletion(-) create mode 100644 include/libcamera/internal/software_isp/meson.build create mode 100644 include/libcamera/internal/software_isp/statistics-linaro.h create mode 100644 src/ipa/simple/linaro/data/meson.build create mode 100644 src/ipa/simple/linaro/data/soft.conf create mode 100644 src/ipa/simple/linaro/meson.build create mode 100644 src/ipa/simple/linaro/soft_linaro.cpp diff --git a/include/libcamera/internal/software_isp/meson.build b/include/libcamera/internal/software_isp/meson.build new file mode 100644 index 00000000..a24fe7df --- /dev/null +++ b/include/libcamera/internal/software_isp/meson.build @@ -0,0 +1,5 @@ +# SPDX-License-Identifier: CC0-1.0 + +libcamera_internal_headers += files([ + 'statistics-linaro.h', +]) diff --git a/include/libcamera/internal/software_isp/statistics-linaro.h b/include/libcamera/internal/software_isp/statistics-linaro.h new file mode 100644 index 00000000..20c64e44 --- /dev/null +++ b/include/libcamera/internal/software_isp/statistics-linaro.h @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2023, Linaro Ltd + * + * statistics.h - Statistics data format used by the software ISP + */ + +#pragma once + +namespace libcamera { + +struct SwIspStats { + float bright_ratio; + float too_bright_ratio; +}; + +} /* namespace libcamera */ diff --git a/meson_options.txt b/meson_options.txt index 5fdc7be8..49939421 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -27,7 +27,7 @@ option('gstreamer', option('ipas', type : 'array', - choices : ['ipu3', 'rkisp1', 'rpi/vc4', 'vimc'], + choices : ['ipu3', 'rkisp1', 'rpi/vc4', 'simple/linaro', 'vimc'], description : 'Select which IPA modules to build') option('lc-compliance', @@ -46,6 +46,7 @@ option('pipelines', 'rkisp1', 'rpi/vc4', 'simple', + 'simple/linaro', 'uvcvideo', 'vimc' ], diff --git a/src/ipa/simple/linaro/data/meson.build b/src/ipa/simple/linaro/data/meson.build new file mode 100644 index 00000000..33548cc6 --- /dev/null +++ b/src/ipa/simple/linaro/data/meson.build @@ -0,0 +1,9 @@ +# SPDX-License-Identifier: CC0-1.0 + +conf_files = files([ + 'soft.conf', +]) + +install_data(conf_files, + install_dir : ipa_data_dir / 'soft', + install_tag : 'runtime') diff --git a/src/ipa/simple/linaro/data/soft.conf b/src/ipa/simple/linaro/data/soft.conf new file mode 100644 index 00000000..0c70e7c0 --- /dev/null +++ b/src/ipa/simple/linaro/data/soft.conf @@ -0,0 +1,3 @@ +# SPDX-License-Identifier: LGPL-2.1-or-later +# +# Dummy configuration file for the soft IPA. diff --git a/src/ipa/simple/linaro/meson.build b/src/ipa/simple/linaro/meson.build new file mode 100644 index 00000000..97bf5d6f --- /dev/null +++ b/src/ipa/simple/linaro/meson.build @@ -0,0 +1,26 @@ +# SPDX-License-Identifier: CC0-1.0 + +ipa_name = 'ipa_soft_linaro' + +mod = shared_module(ipa_name, + ['soft_linaro.cpp', libcamera_generated_ipa_headers], + name_prefix : '', + include_directories : [ipa_includes, libipa_includes, '..'], + dependencies : libcamera_private, + link_with : libipa, + link_whole : soft_ipa_common_lib, + install : true, + install_dir : ipa_install_dir) + +if ipa_sign_module + custom_target(ipa_name + '.so.sign', + input : mod, + output : ipa_name + '.so.sign', + command : [ipa_sign, ipa_priv_key, '@INPUT@', '@OUTPUT@'], + install : false, + build_by_default : true) +endif + +subdir('data') + +ipa_names += ipa_name diff --git a/src/ipa/simple/linaro/soft_linaro.cpp b/src/ipa/simple/linaro/soft_linaro.cpp new file mode 100644 index 00000000..0b2e83bf --- /dev/null +++ b/src/ipa/simple/linaro/soft_linaro.cpp @@ -0,0 +1,210 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2023, Linaro Ltd + * + * soft.cpp - Software Image Processing Algorithm module + */ + +#include + +#include +#include + +#include + +#include +#include + +#include "libcamera/internal/camera_sensor.h" +#include "libcamera/internal/software_isp/statistics-linaro.h" + +#include "common/soft_base.h" + +namespace libcamera { + +LOG_DECLARE_CATEGORY(IPASoft) + +namespace ipa::soft { + +class IPASoftLinaro final : public IPASoftBase +{ +public: + IPASoftLinaro() + : IPASoftBase(), ignore_updates_(0) + { + } + + ~IPASoftLinaro() + { + if (stats_) + munmap(stats_, sizeof(SwIspStats)); + } + + int platformInit(const ControlInfoMap &sensorInfoMap) override; + int platformConfigure(const ControlInfoMap &sensorInfoMap) override; + int platformStart() override; + void platformStop() override; + void platformProcessStats(const ControlList &sensorControls) override; + +private: + void update_exposure(double ev_adjustment); + + SwIspStats *stats_; + int exposure_min_, exposure_max_; + int again_min_, again_max_; + int again_, exposure_; + int ignore_updates_; +}; + +int IPASoftLinaro::platformInit(const ControlInfoMap &sensorInfoMap) +{ + stats_ = static_cast(mmap(nullptr, sizeof(SwIspStats), + PROT_READ | PROT_WRITE, MAP_SHARED, + fdStats_.get(), 0)); + if (!stats_) { + LOG(IPASoft, Error) << "Unable to map Statistics"; + return -ENODEV; + } + + if (sensorInfoMap.find(V4L2_CID_EXPOSURE) == sensorInfoMap.end()) { + LOG(IPASoft, Error) << "Don't have exposure control"; + return -EINVAL; + } + + if (sensorInfoMap.find(V4L2_CID_ANALOGUE_GAIN) == sensorInfoMap.end()) { + LOG(IPASoft, Error) << "Don't have gain control"; + return -EINVAL; + } + + return 0; +} + +int IPASoftLinaro::platformConfigure(const ControlInfoMap &sensorInfoMap) +{ + const ControlInfo &exposure_info = sensorInfoMap.find(V4L2_CID_EXPOSURE)->second; + const ControlInfo &gain_info = sensorInfoMap.find(V4L2_CID_ANALOGUE_GAIN)->second; + + exposure_min_ = exposure_info.min().get(); + if (!exposure_min_) { + LOG(IPASoft, Warning) << "Minimum exposure is zero, that can't be linear"; + exposure_min_ = 1; + } + exposure_max_ = exposure_info.max().get(); + again_min_ = gain_info.min().get(); + if (!again_min_) { + LOG(IPASoft, Warning) << "Minimum gain is zero, that can't be linear"; + again_min_ = 100; + } + again_max_ = gain_info.max().get(); + + LOG(IPASoft, Info) << "Exposure " << exposure_min_ << "-" << exposure_max_ + << ", gain " << again_min_ << "-" << again_max_; + + return 0; +} + +int IPASoftLinaro::platformStart() +{ + return 0; +} + +void IPASoftLinaro::platformStop() +{ +} + +void IPASoftLinaro::platformProcessStats(const ControlList &sensorControls) +{ + double ev_adjustment = 0.0; + ControlList ctrls(sensorControls); + + /* + * Use 2 frames delay to make sure that the exposure and the gain set + * have applied to the camera sensor + */ + if (ignore_updates_ > 0) { + LOG(IPASoft, Debug) << "Skipping exposure update: " + << ignore_updates_; + --ignore_updates_; + return; + } + + if (stats_->bright_ratio < 0.01) + ev_adjustment = 1.1; + if (stats_->too_bright_ratio > 0.04) + ev_adjustment = 0.9; + + if (ev_adjustment != 0.0) { + /* sanity check */ + if (!sensorControls.contains(V4L2_CID_EXPOSURE) || + !sensorControls.contains(V4L2_CID_ANALOGUE_GAIN)) { + LOG(IPASoft, Error) << "Control(s) missing"; + return; + } + + exposure_ = ctrls.get(V4L2_CID_EXPOSURE).get(); + again_ = ctrls.get(V4L2_CID_ANALOGUE_GAIN).get(); + + update_exposure(ev_adjustment); + + ctrls.set(V4L2_CID_EXPOSURE, exposure_); + ctrls.set(V4L2_CID_ANALOGUE_GAIN, again_); + + ignore_updates_ = 2; + + setSensorControls.emit(ctrls); + } +} + +void IPASoftLinaro::update_exposure(double ev_adjustment) +{ + double exp = (double)exposure_; + double gain = (double)again_; + double ev = ev_adjustment * exp * gain; + + /* + * Try to use the minimal possible analogue gain. + * The exposure can be any value from exposure_min_ to exposure_max_, + * and normally this should keep the frame rate intact. + */ + + exp = ev / again_min_; + if (exp > exposure_max_) + exposure_ = exposure_max_; + else if (exp < exposure_min_) + exposure_ = exposure_min_; + else + exposure_ = (int)exp; + + gain = ev / exposure_; + if (gain > again_max_) + again_ = again_max_; + else if (gain < again_min_) + again_ = again_min_; + else + again_ = (int)gain; + + LOG(IPASoft, Debug) << "Desired EV = " << ev + << ", real EV = " << (double)again_ * exposure_; +} + +} /* namespace ipa::soft */ + +/* + * External IPA module interface + */ +extern "C" { +const struct IPAModuleInfo ipaModuleInfo = { + IPA_MODULE_API_VERSION, + 0, + "SimplePipelineHandler", + "soft/linaro", +}; + +IPAInterface *ipaCreate() +{ + return new ipa::soft::IPASoftLinaro(); +} + +} /* extern "C" */ + +} /* namespace libcamera */ diff --git a/src/ipa/simple/meson.build b/src/ipa/simple/meson.build index 9688bbdb..14be5dc2 100644 --- a/src/ipa/simple/meson.build +++ b/src/ipa/simple/meson.build @@ -1,3 +1,12 @@ # SPDX-License-Identifier: CC0-1.0 subdir('common') + +foreach pipeline : pipelines + pipeline = pipeline.split('/') + if pipeline.length() < 2 or pipeline[0] != 'simple' + continue + endif + + subdir(pipeline[1]) +endforeach From patchwork Tue Dec 12 11:50:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 19310 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 884E0C3292 for ; Tue, 12 Dec 2023 11:51:28 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2A44562B49; Tue, 12 Dec 2023 12:51:28 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1702381888; bh=ZzgDbG5BAr3rpe3OXUbPePZWBKIIm0NVxT0vKz0m8uA=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=cEdDqZgMFMXGA5uKEPRYCeu0TnCNfwNorbSbu8C+FmDlLI7VVBktHB2mItkO0FODK 3ofBOWHwB/Gi32v+qWB2hgZ9FSK8tTd8Iil4tVHL7KPMQjxCxCwSufzqelUJH+UpqE VAyw4/hgMbzDOdXr0KWQgQX5y/gq1UaMwe7iiDQfaqVh2xh/o/fvGVmjDo4ZJh3rUh WjgLEkxqD+Lwc71dhYkq6bigI19T1Ced9mF7Fe7F4zRCuy8cUJxo7b4qCc1DgpN6gv M80E1N2X/UtyMtaa6P510iJ0tOoTRGWnZr0TtFJRAG86RjuTHEHWnAK84x439LjcOO cUz9iT4ndTr9w== Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com [IPv6:2a00:1450:4864:20::633]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BBDB062B3D for ; Tue, 12 Dec 2023 12:51:26 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="MQbcgskt"; dkim-atps=neutral Received: by mail-ej1-x633.google.com with SMTP id a640c23a62f3a-a1e2f34467aso520418066b.2 for ; Tue, 12 Dec 2023 03:51:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1702381886; x=1702986686; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hKQg9mAih69LodQyOzERLWYq7VTf2Z8iii77nqI92wQ=; b=MQbcgsktGHugdvuLnzatzQPDUpvY6YIc7U96EkUoW6jyl8zPg1VJaOsyfX7Cjhh3ML mG/Gd76rUSBhfohZiSYnFG7WYrsqmZHGT8JgGMshxduYmBKfUAY55n9GGkgOBb3lPdhi FOtVVaertFSTrrCr5ImRA2fZzEgso0Qwur2SgzurR1PoBrJCk5DGQ9qEh6Y477MWoI7V 8kkKeEJMkZjo6zq2Q5MqmZj3uSuvWt2FMvvXtwIZrD3Ig2YnAJC5opGUe1h3p5YGUuLa WoyvwqaX42UNgjltj165ZDAElpBJzdtvnD0+t7JLTYi5iYl0TLSCq+O0c1vfKjYV074N tYtg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702381886; x=1702986686; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hKQg9mAih69LodQyOzERLWYq7VTf2Z8iii77nqI92wQ=; b=tc1HGO5S9ZvOLumREAC5bUjh3iztsHBfY1jxJ/Upl2ghCGJ5ewsQ7I4bSpTKFODkJ9 Dbf86ChedALCGy/MErZd7DOvcWfA4UdXnDnX0zjSgBkLD6BL4GqOmkT8AY1UHje66qXI 2iVQPBPi8rLrLCQTiNavv90kXE2isT4z7OaRp5cpb+mYNuGD/FHcB5yUS7aX4Ao6Q0Lt wDxvtIM+tsZJv6h4YCcCXj7CtpKB7pKcRlWSH7D0UK3aljJw+r+3kGmyHwdluS5/igpJ MGASw6uD9GlPrZTlr9ExLYrJCt/uu8X/KKkMiWVuh6PTT5Abvo+yaPkt3mbpEMgfwZ8t cbrw== X-Gm-Message-State: AOJu0YwTiRd5NneqN5x58T0HEcBeb1Ng8Az4xXDubxXp50ahFIOCWNOB t92lbb8LAoC2jvCDJKjxlDsB0W1UMxvOBwsltLs= X-Google-Smtp-Source: AGHT+IGr398qcNcUU3H9InVX2CeGeOektYuF0P83RVZ5lKt68QTBifG8yfqWpP7dwY/fw8F1bLVOkw== X-Received: by 2002:a17:906:fcb5:b0:9be:2b53:ac4d with SMTP id qw21-20020a170906fcb500b009be2b53ac4dmr2976674ejb.74.1702381886220; Tue, 12 Dec 2023 03:51:26 -0800 (PST) Received: from Lat-5310.. ([87.116.161.153]) by smtp.gmail.com with ESMTPSA id tx17-20020a1709078e9100b00a1b75e0e061sm6213188ejc.130.2023.12.12.03.51.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Dec 2023 03:51:25 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Tue, 12 Dec 2023 14:50:44 +0300 Message-Id: <20231212115046.102726-6-andrey.konovalov@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231212115046.102726-1-andrey.konovalov@linaro.org> References: <20231212115046.102726-1-andrey.konovalov@linaro.org> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH v2 5/7] libcamera: software_isp: add SwIspLinaro implementation of SoftwareIsp X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Andrey Konovalov via libcamera-devel From: Andrey Konovalov Reply-To: Andrey Konovalov Cc: mripard@redhat.com, g.martti@gmail.com, t.langendam@gmail.com, srinivas.kandagatla@linaro.org, pavel@ucw.cz, bryan.odonoghue@linaro.org, admin@dennisbonke.com Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The implementation of SoftwareIsp handles creation of Soft IPA and interactions with it, so that the pipeline handler wouldn't need to care about the Soft IPA. Signed-off-by: Andrey Konovalov --- include/libcamera/internal/meson.build | 1 + .../internal/software_isp/meson.build | 1 + .../internal/software_isp/swisp_linaro.h | 117 ++++ src/libcamera/meson.build | 1 + src/libcamera/software_isp/meson.build | 20 + src/libcamera/software_isp/swisp_linaro.cpp | 589 ++++++++++++++++++ 6 files changed, 729 insertions(+) create mode 100644 include/libcamera/internal/software_isp/swisp_linaro.h create mode 100644 src/libcamera/software_isp/meson.build create mode 100644 src/libcamera/software_isp/swisp_linaro.cpp diff --git a/include/libcamera/internal/meson.build b/include/libcamera/internal/meson.build index b780777c..eeae801c 100644 --- a/include/libcamera/internal/meson.build +++ b/include/libcamera/internal/meson.build @@ -50,3 +50,4 @@ libcamera_internal_headers = files([ ]) subdir('converter') +subdir('software_isp') diff --git a/include/libcamera/internal/software_isp/meson.build b/include/libcamera/internal/software_isp/meson.build index a24fe7df..9f84f00e 100644 --- a/include/libcamera/internal/software_isp/meson.build +++ b/include/libcamera/internal/software_isp/meson.build @@ -2,4 +2,5 @@ libcamera_internal_headers += files([ 'statistics-linaro.h', + 'swisp_linaro.h', ]) diff --git a/include/libcamera/internal/software_isp/swisp_linaro.h b/include/libcamera/internal/software_isp/swisp_linaro.h new file mode 100644 index 00000000..c0df7863 --- /dev/null +++ b/include/libcamera/internal/software_isp/swisp_linaro.h @@ -0,0 +1,117 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2023, Linaro Ltd + * + * swisp_linaro.h - software ISP implementation by Linaro + */ + +#pragma once + +#include +#include + +#include + +#include +#include + +#include "libcamera/internal/shared_mem_object.h" +#include "libcamera/internal/software_isp.h" +#include "libcamera/internal/software_isp/statistics-linaro.h" + +namespace libcamera { + +class SwIspLinaro : public SoftwareIsp +{ +public: + SwIspLinaro(PipelineHandler *pipe, const ControlInfoMap &sensorControls); + ~SwIspLinaro() {} + + int loadConfiguration([[maybe_unused]] const std::string &filename) { return 0; } + bool isValid() const; + + std::vector formats(PixelFormat input); + SizeRange sizes(PixelFormat inputFormat, const Size &inputSize); + + std::tuple + strideAndFrameSize(const PixelFormat &outputFormat, const Size &size); + + int configure(const StreamConfiguration &inputCfg, + const std::vector> &outputCfgs); + int exportBuffers(unsigned int output, unsigned int count, + std::vector> *buffers); + void processStats(const ControlList &sensorControls); + + int start(); + void stop(); + + int queueBuffers(FrameBuffer *input, + const std::map &outputs); + + Signal &getSignalSetSensorControls(); + + void process(FrameBuffer *input, FrameBuffer *output); + +private: + SharedMemObject sharedStats_; + + class IspWorker : public Object + { + public: + IspWorker(SwIspLinaro *swIsp); + + std::vector formats(PixelFormat input); + SizeRange sizes(PixelFormat inputFormat, const Size &inputSize); + unsigned int outStride(const PixelFormat &outputFormat, + const Size &outSize); + + int configure(const StreamConfiguration &inputCfg, + const StreamConfiguration &outputCfg); + unsigned int outBufferSize(); + void process(FrameBuffer *input, FrameBuffer *output); + + private: + SwIspLinaro *swIsp_; + + typedef void (SwIspLinaro::IspWorker::*debayerFn)(uint8_t *dst, const uint8_t *src); + typedef SizeRange (*outSizesFn)(const Size &inSize); + typedef unsigned int (*outStrideFn)(const Size &outSize); + struct debayerInfo { + PixelFormat outPixelFmt; + debayerFn debayer; + outSizesFn getOutSizes; + outStrideFn getOutStride; + }; + // TODO: use inputFormat+outputFormat as the map key + // to enable multiple output formats + // TODO: use BayerFormat instead of PixelFormat as inputFormat + std::map debayerInfos_; + int setDebayerInfo(PixelFormat format); + debayerInfo *debayerInfo_; + + /* CSI-2 packed 10-bit raw bayer format (all the 4 orders) */ + void debayerRaw10P(uint8_t *dst, const uint8_t *src); + static SizeRange outSizesRaw10P(const Size &inSize); + static unsigned int outStrideRaw10P(const Size &outSize); + + unsigned int width_; + unsigned int height_; + unsigned int stride_; + Point redShift_; + unsigned int outHeight_; + unsigned int outStride_; + + unsigned long rNumerat_, rDenomin_; /* red gain for AWB */ + unsigned long bNumerat_, bDenomin_; /* blue gain for AWB */ + unsigned long gNumerat_, gDenomin_; /* green gain for AWB */ + + SwIspStats stats_; + }; + + std::unique_ptr ispWorker_; + Thread ispWorkerThread_; + + std::unique_ptr ipa_; +}; + +} /* namespace libcamera */ diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build index b3606969..e758ac9c 100644 --- a/src/libcamera/meson.build +++ b/src/libcamera/meson.build @@ -70,6 +70,7 @@ subdir('converter') subdir('ipa') subdir('pipeline') subdir('proxy') +subdir('software_isp') null_dep = dependency('', required : false) diff --git a/src/libcamera/software_isp/meson.build b/src/libcamera/software_isp/meson.build new file mode 100644 index 00000000..d4a8d499 --- /dev/null +++ b/src/libcamera/software_isp/meson.build @@ -0,0 +1,20 @@ +# SPDX-License-Identifier: CC0-1.0 + +# Software ISP is enabled for 'simple' pipeline handler. +# The 'pipelines' option value should be +# 'simple/' e.g.: +# -Dpipelines=simple/linaro +# The source file should be named swisp_.cpp, +# e.g. 'swisp_linaro.cpp'. + +foreach pipeline : pipelines + pipeline = pipeline.split('/') + if pipeline.length() == 2 and pipeline[0] == 'simple' + libcamera_sources += files([ + 'swisp_' + pipeline[1] + '.cpp', + ]) + # the 'break' below can be removed if/when multiple + # Software ISP implementations are allowed in single build + break + endif +endforeach diff --git a/src/libcamera/software_isp/swisp_linaro.cpp b/src/libcamera/software_isp/swisp_linaro.cpp new file mode 100644 index 00000000..b7f36db1 --- /dev/null +++ b/src/libcamera/software_isp/swisp_linaro.cpp @@ -0,0 +1,589 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2023, Linaro Ltd + * + * swisp_linaro.cpp - software ISP implementation by Linaro + */ + +#include "libcamera/internal/software_isp/swisp_linaro.h" + +#include +#include +#include + +#include +#include + +#include "libcamera/internal/bayer_format.h" +#include "libcamera/internal/framebuffer.h" +#include "libcamera/internal/ipa_manager.h" +#include "libcamera/internal/mapped_framebuffer.h" + +namespace libcamera { + +LOG_DECLARE_CATEGORY(SoftwareIsp) + +SwIspLinaro::SwIspLinaro(PipelineHandler *pipe, const ControlInfoMap &sensorControls) + : SoftwareIsp(pipe, sensorControls) +{ + ispWorker_ = std::make_unique(this); + if (!ispWorker_) { + LOG(SoftwareIsp, Error) + << "Failed to create ISP worker"; + return; + } + + sharedStats_ = SharedMemObject("softIsp_stats"); + if (!sharedStats_.fd().isValid()) { + LOG(SoftwareIsp, Error) + << "Failed to create shared memory for statistics"; + ispWorker_.reset(); + return; + } + + ipa_ = IPAManager::createIPA(pipe, 0, 0); + if (!ipa_) { + LOG(SoftwareIsp, Error) + << "Creating IPA for software ISP failed"; + ispWorker_.reset(); + return; + } + + int ret = ipa_->init(IPASettings{ "No cfg file", "No sensor model" }, + sharedStats_.fd(), + sensorControls); + if (ret) { + LOG(SoftwareIsp, Error) << "IPA init failed"; + ispWorker_.reset(); + return; + } + + ipa_->configure(sensorControls); + + ispWorker_->moveToThread(&ispWorkerThread_); +} + +void SwIspLinaro::processStats(const ControlList &sensorControls) +{ + ASSERT(ipa_); + ipa_->processStats(sensorControls); +} + +Signal &SwIspLinaro::getSignalSetSensorControls() +{ + ASSERT(ipa_); + return ipa_->setSensorControls; +} + +bool SwIspLinaro::isValid() const +{ + return !!ispWorker_; +} + +/* + * Demosaic Raw10P frame into RGB888 format. + * Two LS bits of the RAW10P's total 10 are ignored. + * Also this function performs the statistics calculations, and has a fairly + * naive grey world AWB algorithm implementation. + * \todo Split the stats calculations out of this function. + * \todo Move the AWB algorithm into the IPA module. + */ +void SwIspLinaro::IspWorker::debayerRaw10P(uint8_t *dst, const uint8_t *src) +{ + /* for brightness values in the 0 to 255 range: */ + static const unsigned int BRIGHT_LVL = 200U << 8; + static const unsigned int TOO_BRIGHT_LVL = 240U << 8; + + static const unsigned int RED_Y_MUL = 77; /* 0.30 * 256 */ + static const unsigned int GREEN_Y_MUL = 150; /* 0.59 * 256 */ + static const unsigned int BLUE_Y_MUL = 29; /* 0.11 * 256 */ + + int w_out = width_ - 2; + int h_out = height_ - 2; + + unsigned long sumR = 0; + unsigned long sumB = 0; + unsigned long sumG = 0; + + unsigned long bright_sum = 0; + unsigned long too_bright_sum = 0; + + for (int y = 0; y < h_out; y++) { + const uint8_t *pin_base = src + (y + 1) * stride_; + uint8_t *pout = dst + y * w_out * 3; + int phase_y = (y + redShift_.y) % 2; + + for (int x = 0; x < w_out; x++) { + int phase_x = (x + redShift_.x) % 2; + int phase = 2 * phase_y + phase_x; + + /* x part of the offset in the input buffer: */ + int x_m1 = x + x / 4; /* offset for (x-1) */ + int x_0 = x + 1 + (x + 1) / 4; /* offset for x */ + int x_p1 = x + 2 + (x + 2) / 4; /* offset for (x+1) */ + /* the colour component value to write to the output */ + unsigned val; + /* Y value times 256 */ + unsigned y_val; + + switch (phase) { + case 0: /* at R pixel */ + /* blue: ((-1,-1)+(1,-1)+(-1,1)+(1,1)) / 4 */ + val = ( *(pin_base + x_m1 - stride_) + + *(pin_base + x_p1 - stride_) + + *(pin_base + x_m1 + stride_) + + *(pin_base + x_p1 + stride_) ) >> 2; + y_val = BLUE_Y_MUL * val; + val = val * bNumerat_ / bDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* green: ((0,-1)+(-1,0)+(1,0)+(0,1)) / 4 */ + val = ( *(pin_base + x_0 - stride_) + + *(pin_base + x_p1) + + *(pin_base + x_m1) + + *(pin_base + x_0 + stride_) ) >> 2; + val = val * gNumerat_ / gDenomin_; + y_val += GREEN_Y_MUL * val; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* red: (0,0) */ + val = *(pin_base + x_0); + sumR += val; + y_val += RED_Y_MUL * val; + if (y_val > BRIGHT_LVL) ++bright_sum; + if (y_val > TOO_BRIGHT_LVL) ++too_bright_sum; + val = val * rNumerat_ / rDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + break; + case 1: /* at Gr pixel */ + /* blue: ((0,-1) + (0,1)) / 2 */ + val = ( *(pin_base + x_0 - stride_) + + *(pin_base + x_0 + stride_) ) >> 1; + y_val = BLUE_Y_MUL * val; + val = val * bNumerat_ / bDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* green: (0,0) */ + val = *(pin_base + x_0); + sumG += val; + y_val += GREEN_Y_MUL * val; + val = val * gNumerat_ / gDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* red: ((-1,0) + (1,0)) / 2 */ + val = ( *(pin_base + x_m1) + + *(pin_base + x_p1) ) >> 1; + y_val += RED_Y_MUL * val; + if (y_val > BRIGHT_LVL) ++bright_sum; + if (y_val > TOO_BRIGHT_LVL) ++too_bright_sum; + val = val * rNumerat_ / rDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + break; + case 2: /* at Gb pixel */ + /* blue: ((-1,0) + (1,0)) / 2 */ + val = ( *(pin_base + x_m1) + + *(pin_base + x_p1) ) >> 1; + y_val = BLUE_Y_MUL * val; + val = val * bNumerat_ / bDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* green: (0,0) */ + val = *(pin_base + x_0); + sumG += val; + y_val += GREEN_Y_MUL * val; + val = val * gNumerat_ / gDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* red: ((0,-1) + (0,1)) / 2 */ + val = ( *(pin_base + x_0 - stride_) + + *(pin_base + x_0 + stride_) ) >> 1; + y_val += RED_Y_MUL * val; + if (y_val > BRIGHT_LVL) ++bright_sum; + if (y_val > TOO_BRIGHT_LVL) ++too_bright_sum; + val = val * rNumerat_ / rDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + break; + default: /* at B pixel */ + /* blue: (0,0) */ + val = *(pin_base + x_0); + sumB += val; + y_val = BLUE_Y_MUL * val; + val = val * bNumerat_ / bDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* green: ((0,-1)+(-1,0)+(1,0)+(0,1)) / 4 */ + val = ( *(pin_base + x_0 - stride_) + + *(pin_base + x_p1) + + *(pin_base + x_m1) + + *(pin_base + x_0 + stride_) ) >> 2; + y_val += GREEN_Y_MUL * val; + val = val * gNumerat_ / gDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* red: ((-1,-1)+(1,-1)+(-1,1)+(1,1)) / 4 */ + val = ( *(pin_base + x_m1 - stride_) + + *(pin_base + x_p1 - stride_) + + *(pin_base + x_m1 + stride_) + + *(pin_base + x_p1 + stride_) ) >> 2; + y_val += RED_Y_MUL * val; + if (y_val > BRIGHT_LVL) ++bright_sum; + if (y_val > TOO_BRIGHT_LVL) ++too_bright_sum; + val = val * rNumerat_ / rDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + } + } + } + + /* calculate the fractions of "bright" and "too bright" pixels */ + stats_.bright_ratio = (float)bright_sum / (h_out * w_out); + stats_.too_bright_ratio = (float)too_bright_sum / (h_out * w_out); + + /* calculate red and blue gains for simple AWB */ + LOG(SoftwareIsp, Debug) + << "sumR = " << sumR << ", sumB = " << sumB << ", sumG = " << sumG; + + sumG /= 2; /* the number of G pixels is twice as big vs R and B ones */ + + /* normalize red, blue, and green sums to fit into 22-bit value */ + unsigned long fRed = sumR / 0x400000; + unsigned long fBlue = sumB / 0x400000; + unsigned long fGreen = sumG / 0x400000; + unsigned long fNorm = std::max({ 1UL, fRed, fBlue, fGreen }); + sumR /= fNorm; + sumB /= fNorm; + sumG /= fNorm; + + LOG(SoftwareIsp, Debug) << "fNorm = " << fNorm; + LOG(SoftwareIsp, Debug) + << "Normalized: sumR = " << sumR + << ", sumB= " << sumB << ", sumG = " << sumG; + + /* make sure red/blue gains never exceed approximately 256 */ + unsigned long minDenom; + rNumerat_ = (sumR + sumB + sumG) / 3; + minDenom = rNumerat_ / 0x100; + rDenomin_ = std::max(minDenom, sumR); + bNumerat_ = rNumerat_; + bDenomin_ = std::max(minDenom, sumB); + gNumerat_ = rNumerat_; + gDenomin_ = std::max(minDenom, sumG); + + LOG(SoftwareIsp, Debug) + << "rGain = [ " << rNumerat_ << " / " << rDenomin_ + << " ], bGain = [ " << bNumerat_ << " / " << bDenomin_ + << " ], gGain = [ " << gNumerat_ << " / " << gDenomin_ + << " (minDenom = " << minDenom << ")"; +} + +SizeRange SwIspLinaro::IspWorker::outSizesRaw10P(const Size &inSize) +{ + if (inSize.width < 2 || inSize.height < 2) { + LOG(SoftwareIsp, Error) + << "Input format size too small: " << inSize.toString(); + return {}; + } + + return SizeRange(Size(inSize.width - 2, inSize.height - 2)); +} + +unsigned int SwIspLinaro::IspWorker::outStrideRaw10P(const Size &outSize) +{ + return outSize.width * 3; +} + +SwIspLinaro::IspWorker::IspWorker(SwIspLinaro *swIsp) + : swIsp_(swIsp) +{ + debayerInfos_[formats::SBGGR10_CSI2P] = { formats::RGB888, + &SwIspLinaro::IspWorker::debayerRaw10P, + &SwIspLinaro::IspWorker::outSizesRaw10P, + &SwIspLinaro::IspWorker::outStrideRaw10P }; + debayerInfos_[formats::SGBRG10_CSI2P] = { formats::RGB888, + &SwIspLinaro::IspWorker::debayerRaw10P, + &SwIspLinaro::IspWorker::outSizesRaw10P, + &SwIspLinaro::IspWorker::outStrideRaw10P }; + debayerInfos_[formats::SGRBG10_CSI2P] = { formats::RGB888, + &SwIspLinaro::IspWorker::debayerRaw10P, + &SwIspLinaro::IspWorker::outSizesRaw10P, + &SwIspLinaro::IspWorker::outStrideRaw10P }; + debayerInfos_[formats::SRGGB10_CSI2P] = { formats::RGB888, + &SwIspLinaro::IspWorker::debayerRaw10P, + &SwIspLinaro::IspWorker::outSizesRaw10P, + &SwIspLinaro::IspWorker::outStrideRaw10P }; +} + +int SwIspLinaro::IspWorker::setDebayerInfo(PixelFormat format) +{ + const auto it = debayerInfos_.find(format); + if (it == debayerInfos_.end()) + return -1; + + debayerInfo_ = &it->second; + return 0; +} + +std::vector SwIspLinaro::IspWorker::formats(PixelFormat input) +{ + std::vector pixelFormats; + + const auto it = debayerInfos_.find(input); + if (it == debayerInfos_.end()) + LOG(SoftwareIsp, Info) + << "Unsupported input format " << input.toString(); + else + pixelFormats.push_back(it->second.outPixelFmt); + + return pixelFormats; +} + +SizeRange SwIspLinaro::IspWorker::sizes(PixelFormat inputFormat, + const Size &inputSize) +{ + const auto it = debayerInfos_.find(inputFormat); + if (it == debayerInfos_.end()) { + LOG(SoftwareIsp, Info) + << "Unsupported input format " << inputFormat.toString(); + return {}; + } + + return (*it->second.getOutSizes)(inputSize); +} + +unsigned int SwIspLinaro::IspWorker::outStride(const PixelFormat &outputFormat, + const Size &outSize) +{ + /* + * Assuming that the output stride depends only on the outputFormat, + * we use the first debayerInfos_ entry with the matching output format + */ + for (auto it = debayerInfos_.begin(); it != debayerInfos_.end(); it++) { + if (it->second.outPixelFmt == outputFormat) + return (*it->second.getOutStride)(outSize); + } + + return 0; +} + +int SwIspLinaro::IspWorker::configure(const StreamConfiguration &inputCfg, + const StreamConfiguration &outputCfg) +{ + if (setDebayerInfo(inputCfg.pixelFormat) != 0) { + LOG(SoftwareIsp, Error) + << "Input format " << inputCfg.pixelFormat + << "not supported"; + return -EINVAL; + } + + /* check that: + * - output format is valid + * - output size matches the input size and is valid */ + SizeRange outSizeRange = (*debayerInfo_->getOutSizes)(inputCfg.size); + if (debayerInfo_->outPixelFmt != outputCfg.pixelFormat || + outputCfg.size.isNull() || !outSizeRange.contains(outputCfg.size) || + (*debayerInfo_->getOutStride)(outputCfg.size) != outputCfg.stride) { + LOG(SoftwareIsp, Error) + << "Invalid output format/size/stride: " + << "\n " << outputCfg.pixelFormat << " (" + << debayerInfo_->outPixelFmt << ")" + << "\n " << outputCfg.size << " (" + << outSizeRange << ")" + << "\n " << outputCfg.stride << " (" + << (*debayerInfo_->getOutStride)(outputCfg.size) << ")"; + return -EINVAL; + } + + width_ = inputCfg.size.width; + height_ = inputCfg.size.height; + stride_ = inputCfg.stride; + + BayerFormat bayerFormat = + BayerFormat::fromPixelFormat(inputCfg.pixelFormat); + switch (bayerFormat.order) { + case BayerFormat::BGGR: + redShift_ = Point(0, 0); + break; + case BayerFormat::GBRG: + redShift_ = Point(1, 0); + break; + case BayerFormat::GRBG: + redShift_ = Point(0, 1); + break; + case BayerFormat::RGGB: + default: + redShift_ = Point(1, 1); + break; + } + + outStride_ = outputCfg.stride; + outHeight_ = outputCfg.size.height; + + LOG(SoftwareIsp, Info) + << "SoftwareISP configuration: " + << inputCfg.size << "-" << inputCfg.pixelFormat << " -> " + << outputCfg.size << "-" << outputCfg.pixelFormat; + + /* set r/g/b gains to 1.0 until frame data collected */ + rNumerat_ = rDenomin_ = 1; + bNumerat_ = bDenomin_ = 1; + gNumerat_ = gDenomin_ = 1; + + return 0; +} + +/* May not be called before SwIspLinaro::IspWorker::configure() */ +unsigned int SwIspLinaro::IspWorker::outBufferSize() +{ + return outHeight_ * outStride_; +} + +std::vector SwIspLinaro::formats(PixelFormat inputFormat) +{ + ASSERT(ispWorker_ != nullptr); + + return ispWorker_->formats(inputFormat); +} + +SizeRange SwIspLinaro::sizes(PixelFormat inputFormat, const Size &inputSize) +{ + ASSERT(ispWorker_ != nullptr); + + return ispWorker_->sizes(inputFormat, inputSize); +} + +std::tuple +SwIspLinaro::strideAndFrameSize(const PixelFormat &outputFormat, const Size &size) +{ + ASSERT(ispWorker_ != nullptr); + + unsigned int stride = ispWorker_->outStride(outputFormat, size); + + return std::make_tuple(stride, stride * size.height); +} + +int SwIspLinaro::configure(const StreamConfiguration &inputCfg, + const std::vector> &outputCfgs) +{ + ASSERT(ispWorker_ != nullptr); + + if (outputCfgs.size() != 1) { + LOG(SoftwareIsp, Error) + << "Unsupported number of output streams: " + << outputCfgs.size(); + return -EINVAL; + } + + return ispWorker_->configure(inputCfg, outputCfgs[0]); +} + +int SwIspLinaro::exportBuffers(unsigned int output, unsigned int count, + std::vector> *buffers) +{ + ASSERT(ispWorker_ != nullptr); + + /* single output for now */ + if (output >= 1) + return -EINVAL; + + unsigned int bufSize = ispWorker_->outBufferSize(); + + /* TODO: allocate from dma_heap; memfd buffs aren't allowed in FrameBuffer */ + for (unsigned int i = 0; i < count; i++) { + std::string name = "frame-" + std::to_string(i); + + const int ispFd = memfd_create(name.c_str(), 0); + int ret = ftruncate(ispFd, bufSize); + if (ret < 0) { + LOG(SoftwareIsp, Error) + << "ftruncate() for memfd failed " + << strerror(-ret); + return ret; + } + + FrameBuffer::Plane outPlane; + outPlane.fd = SharedFD(std::move(ispFd)); + outPlane.offset = 0; + outPlane.length = bufSize; + + std::vector planes{ outPlane }; + buffers->emplace_back(std::make_unique(std::move(planes))); + } + + return count; +} + +int SwIspLinaro::queueBuffers(FrameBuffer *input, + const std::map &outputs) +{ + unsigned int mask = 0; + + /* + * Validate the outputs as a sanity check: at least one output is + * required, all outputs must reference a valid stream and no two + * outputs can reference the same stream. + */ + if (outputs.empty()) + return -EINVAL; + + for (auto [index, buffer] : outputs) { + if (!buffer) + return -EINVAL; + if (index >= 1) /* only single stream atm */ + return -EINVAL; + if (mask & (1 << index)) + return -EINVAL; + + mask |= 1 << index; + } + + process(input, outputs.at(0)); + + return 0; +} + +int SwIspLinaro::start() +{ + int ret = ipa_->start(); + if (ret) + return ret; + + ispWorkerThread_.start(); + return 0; +} + +void SwIspLinaro::stop() +{ + ispWorkerThread_.exit(); + ispWorkerThread_.wait(); + + ipa_->stop(); +} + +void SwIspLinaro::IspWorker::process(FrameBuffer *input, FrameBuffer *output) +{ + /* Copy metadata from the input buffer */ + FrameMetadata &metadata = output->_d()->metadata(); + metadata.status = input->metadata().status; + metadata.sequence = input->metadata().sequence; + metadata.timestamp = input->metadata().timestamp; + + MappedFrameBuffer in(input, MappedFrameBuffer::MapFlag::Read); + MappedFrameBuffer out(output, MappedFrameBuffer::MapFlag::Write); + if (!in.isValid() || !out.isValid()) { + LOG(SoftwareIsp, Error) << "mmap-ing buffer(s) failed"; + metadata.status = FrameMetadata::FrameError; + swIsp_->outputBufferReady.emit(output); + swIsp_->inputBufferReady.emit(input); + return; + } + + (this->*debayerInfo_->debayer)(out.planes()[0].data(), in.planes()[0].data()); + metadata.planes()[0].bytesused = out.planes()[0].size(); + + *swIsp_->sharedStats_ = stats_; + swIsp_->ispStatsReady.emit(0); + + swIsp_->outputBufferReady.emit(output); + swIsp_->inputBufferReady.emit(input); +} + +void SwIspLinaro::process(FrameBuffer *input, FrameBuffer *output) +{ + ispWorker_->invokeMethod(&SwIspLinaro::IspWorker::process, + ConnectionTypeQueued, input, output); +} + +REGISTER_SOFTWAREISP(SwIspLinaro) + +} /* namespace libcamera */ From patchwork Tue Dec 12 11:50:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 19311 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 750B7C32BC for ; Tue, 12 Dec 2023 11:51:30 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0BD7262B40; Tue, 12 Dec 2023 12:51:30 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1702381890; bh=fjl8sMSFfkzTBOcgHmEeNYc0wONBoMHUPlxMl7V61oA=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=txiHxC+jWVFfRbiII+OIJhvPZ/DYJVCdsQzTJbjRKQIQixmTebVrKXyzUVsq5MSal DK/gBHmBYc1s+MmYGmACQlR9Z8c6rU6Ivo24xVeDwsGglJHDPHBlYLZWO6f12feBE2 en1x6TRXRz32dZyDG9eErukJwQy8+y0bTR89EHnHmnFbhAvtEIuoOSOSEnjJucKcCf jDAgRIM652DgmYpNtdK5vU+5aDYg49eWiewMCunbXFyDPsQio7WeQlvc3Xxuuuy3Kx uEoEI1VP+b4Od6LDGzBq0RA06pmXi9wq8IIspFXh6ychUklznoLZNeEoUWfDlKbYYh yGq0sCeZKxxDg== Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com [IPv6:2a00:1450:4864:20::62b]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2A4EA62B5A for ; Tue, 12 Dec 2023 12:51:28 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="lZzsReAe"; dkim-atps=neutral Received: by mail-ej1-x62b.google.com with SMTP id a640c23a62f3a-a1e7971db2aso633736966b.3 for ; Tue, 12 Dec 2023 03:51:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1702381887; x=1702986687; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Ul3YjVRAwDCJplrzfkmo/zpiwDH9NzQCnUUAZU0wH4I=; b=lZzsReAekcvS5huqzSj0D2Q2ygWy/ZHjqzJi1eHdF1znfuHLCgmjNLt+UzUSKSBVVR wKsBm0BZxhwhiqYdur/50vy2NAyvy+hcxwfwfQc17O5VdennsGfnNqZ4AZ5mlOhLjluF j8yvhXJ0QanC8+eyWBPzU8mwV99L8gRNtzFY6Ptt6ZSP6jSxP37Bq44h8W0ZF7SIsolN ES4AQhZDtyqN1WepGZ9PGUUxZqUie1nQozjiaBMLp2b5Du3YJRmukJ2TMjS6lRzkjrVV 27G36aeGGMl7g8nVs1lxQt7vtIjv8fL/B14ZcmHvNNSgsk+m2dsEXQA76LxfjeOcs9vo OIxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702381887; x=1702986687; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ul3YjVRAwDCJplrzfkmo/zpiwDH9NzQCnUUAZU0wH4I=; b=RA1E855LbZBxKWBi9GmpNxID5O6ynp9mbDD/Hsx9n2zNUKZp6dv5OFBVNZ5Rcx9LZj XtO+OEsRvuSRkyTQl/u1XqkJahdPOphY/vqD/1JX1TpaNK6azGxowCknsQonA9dwm/9p vaVVLaoNX6V7frxKUDchNxBbC54cTnnOIBMc46Re8VT1q2mXFPEhiOPnD2A4r9QI/Ewv ahEA7n7wYOQfukSPJx9Rv8L6G9Fv7JUAVQFzFSjnGO/fsgKIKhJXdLZ6o7tEI1aXnWGZ 4w2VzJv0of9YvaOKha5kyMredPxL7EzW0JlRUVeUq92KYQoocwy/Do5NvZ0mRNpNDkdN 8z5A== X-Gm-Message-State: AOJu0YxET7XltRZLQWJH/NvRvQpBiOX9UXh/Znxx02AiBAnvBMkiDeMp CTI3q/zLuViQq5oU8sOmYrORGXef8WeCKIWfBJU= X-Google-Smtp-Source: AGHT+IHmQRBAp5lBCqIYWdgmNStRBhA3JMNlz+lz0Pha1070up+4Bf2aTmG3BQ9GJrx0Db1jS4uhWg== X-Received: by 2002:a17:907:3f10:b0:a1f:750e:a285 with SMTP id hq16-20020a1709073f1000b00a1f750ea285mr3494293ejc.39.1702381887761; Tue, 12 Dec 2023 03:51:27 -0800 (PST) Received: from Lat-5310.. ([87.116.161.153]) by smtp.gmail.com with ESMTPSA id tx17-20020a1709078e9100b00a1b75e0e061sm6213188ejc.130.2023.12.12.03.51.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Dec 2023 03:51:27 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Tue, 12 Dec 2023 14:50:45 +0300 Message-Id: <20231212115046.102726-7-andrey.konovalov@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231212115046.102726-1-andrey.konovalov@linaro.org> References: <20231212115046.102726-1-andrey.konovalov@linaro.org> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH v2 6/7] libcamera: pipeline: simple: rename converterBuffers_ and related vars X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Andrey Konovalov via libcamera-devel From: Andrey Konovalov Reply-To: Andrey Konovalov Cc: mripard@redhat.com, g.martti@gmail.com, t.langendam@gmail.com, srinivas.kandagatla@linaro.org, pavel@ucw.cz, bryan.odonoghue@linaro.org, admin@dennisbonke.com Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The converterBuffers_ and the converterQueue_ are not that specific to the Converter, and could be used by another entity doing the format conversion. Rename converterBuffers_, converterQueue_, and useConverter_ to conversionBuffers_, conversionQueue_ and useConversion_ to disassociate them from the Converter. Signed-off-by: Andrey Konovalov --- src/libcamera/pipeline/simple/simple.cpp | 63 ++++++++++++------------ 1 file changed, 32 insertions(+), 31 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 911051b2..878794f6 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -269,17 +269,18 @@ public: std::vector configs_; std::map> formats_; + std::vector> conversionBuffers_; + std::queue> conversionQueue_; + bool useConversion_; + std::unique_ptr converter_; - std::vector> converterBuffers_; - bool useConverter_; - std::queue> converterQueue_; private: void tryPipeline(unsigned int code, const Size &size); static std::vector routedSourcePads(MediaPad *sink); - void converterInputDone(FrameBuffer *buffer); - void converterOutputDone(FrameBuffer *buffer); + void conversionInputDone(FrameBuffer *buffer); + void conversionOutputDone(FrameBuffer *buffer); }; class SimpleCameraConfiguration : public CameraConfiguration @@ -503,8 +504,8 @@ int SimpleCameraData::init() << "Failed to create converter, disabling format conversion"; converter_.reset(); } else { - converter_->inputBufferReady.connect(this, &SimpleCameraData::converterInputDone); - converter_->outputBufferReady.connect(this, &SimpleCameraData::converterOutputDone); + converter_->inputBufferReady.connect(this, &SimpleCameraData::conversionInputDone); + converter_->outputBufferReady.connect(this, &SimpleCameraData::conversionOutputDone); } } @@ -740,7 +741,7 @@ void SimpleCameraData::bufferReady(FrameBuffer *buffer) * point converting an erroneous buffer. */ if (buffer->metadata().status != FrameMetadata::FrameSuccess) { - if (!useConverter_) { + if (!useConversion_) { /* No conversion, just complete the request. */ Request *request = buffer->request(); pipe->completeBuffer(request, buffer); @@ -756,16 +757,16 @@ void SimpleCameraData::bufferReady(FrameBuffer *buffer) if (buffer->metadata().status != FrameMetadata::FrameCancelled) video_->queueBuffer(buffer); - if (converterQueue_.empty()) + if (conversionQueue_.empty()) return; Request *request = nullptr; - for (auto &item : converterQueue_.front()) { + for (auto &item : conversionQueue_.front()) { FrameBuffer *outputBuffer = item.second; request = outputBuffer->request(); pipe->completeBuffer(request, outputBuffer); } - converterQueue_.pop(); + conversionQueue_.pop(); if (request) pipe->completeRequest(request); @@ -782,9 +783,9 @@ void SimpleCameraData::bufferReady(FrameBuffer *buffer) */ Request *request = buffer->request(); - if (useConverter_ && !converterQueue_.empty()) { + if (useConversion_ && !conversionQueue_.empty()) { const std::map &outputs = - converterQueue_.front(); + conversionQueue_.front(); if (!outputs.empty()) { FrameBuffer *outputBuffer = outputs.begin()->second; if (outputBuffer) @@ -801,14 +802,14 @@ void SimpleCameraData::bufferReady(FrameBuffer *buffer) * conversion is needed. If there's no queued request, just requeue the * captured buffer for capture. */ - if (useConverter_) { - if (converterQueue_.empty()) { + if (useConversion_) { + if (conversionQueue_.empty()) { video_->queueBuffer(buffer); return; } - converter_->queueBuffers(buffer, converterQueue_.front()); - converterQueue_.pop(); + converter_->queueBuffers(buffer, conversionQueue_.front()); + conversionQueue_.pop(); return; } @@ -817,13 +818,13 @@ void SimpleCameraData::bufferReady(FrameBuffer *buffer) pipe->completeRequest(request); } -void SimpleCameraData::converterInputDone(FrameBuffer *buffer) +void SimpleCameraData::conversionInputDone(FrameBuffer *buffer) { /* Queue the input buffer back for capture. */ video_->queueBuffer(buffer); } -void SimpleCameraData::converterOutputDone(FrameBuffer *buffer) +void SimpleCameraData::conversionOutputDone(FrameBuffer *buffer) { SimplePipelineHandler *pipe = SimpleCameraData::pipe(); @@ -1156,14 +1157,14 @@ int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c) /* Configure the converter if needed. */ std::vector> outputCfgs; - data->useConverter_ = config->needConversion(); + data->useConversion_ = config->needConversion(); for (unsigned int i = 0; i < config->size(); ++i) { StreamConfiguration &cfg = config->at(i); cfg.setStream(&data->streams_[i]); - if (data->useConverter_) + if (data->useConversion_) outputCfgs.push_back(cfg); } @@ -1189,7 +1190,7 @@ int SimplePipelineHandler::exportFrameBuffers(Camera *camera, Stream *stream, * Export buffers on the converter or capture video node, depending on * whether the converter is used or not. */ - if (data->useConverter_) + if (data->useConversion_) return data->converter_->exportBuffers(data->streamIndex(stream), count, buffers); else @@ -1210,13 +1211,13 @@ int SimplePipelineHandler::start(Camera *camera, [[maybe_unused]] const ControlL return -EBUSY; } - if (data->useConverter_) { + if (data->useConversion_) { /* * When using the converter allocate a fixed number of internal * buffers. */ ret = video->allocateBuffers(kNumInternalBuffers, - &data->converterBuffers_); + &data->conversionBuffers_); } else { /* Otherwise, prepare for using buffers from the only stream. */ Stream *stream = &data->streams_[0]; @@ -1235,7 +1236,7 @@ int SimplePipelineHandler::start(Camera *camera, [[maybe_unused]] const ControlL return ret; } - if (data->useConverter_) { + if (data->useConversion_) { ret = data->converter_->start(); if (ret < 0) { stop(camera); @@ -1243,7 +1244,7 @@ int SimplePipelineHandler::start(Camera *camera, [[maybe_unused]] const ControlL } /* Queue all internal buffers for capture. */ - for (std::unique_ptr &buffer : data->converterBuffers_) + for (std::unique_ptr &buffer : data->conversionBuffers_) video->queueBuffer(buffer.get()); } @@ -1255,7 +1256,7 @@ void SimplePipelineHandler::stopDevice(Camera *camera) SimpleCameraData *data = cameraData(camera); V4L2VideoDevice *video = data->video_; - if (data->useConverter_) + if (data->useConversion_) data->converter_->stop(); video->streamOff(); @@ -1263,7 +1264,7 @@ void SimplePipelineHandler::stopDevice(Camera *camera) video->bufferReady.disconnect(data, &SimpleCameraData::bufferReady); - data->converterBuffers_.clear(); + data->conversionBuffers_.clear(); releasePipeline(data); } @@ -1281,7 +1282,7 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request) * queue, it will be handed to the converter in the capture * completion handler. */ - if (data->useConverter_) { + if (data->useConversion_) { buffers.emplace(data->streamIndex(stream), buffer); } else { ret = data->video_->queueBuffer(buffer); @@ -1290,8 +1291,8 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request) } } - if (data->useConverter_) - data->converterQueue_.push(std::move(buffers)); + if (data->useConversion_) + data->conversionQueue_.push(std::move(buffers)); return 0; } From patchwork Tue Dec 12 11:50:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 19312 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id ED333BD87C for ; Tue, 12 Dec 2023 11:51:33 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9871462B33; Tue, 12 Dec 2023 12:51:33 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1702381893; bh=2+oVwBiLoDG1CDdrv0bdoRdovwtqIl7Ba33ZLynjJBQ=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=yfxoTOwL6wtnVYF5uFsmnTyWz+0W7lYlm3ff8nb8piIrtodd+kWOdBxuko3FMnEnb wfVOvxeJTybLuHAX+Vs6w8ykmXhWYHvnpMRHh09fWPH1Hfd6BFui/BxhGfCYNX/8Hy p9Jo+FrSSHBt6DP+ROMy5e4ctaYNxq3Arnb4zguLDTPH++3LQCDJsGerrxZmfI//cF Lz1/HQytZLekMi8BuZvnaqe3hyjh8OOKuwcUQxaVjWZ+8rYWumUJ5iPIqMm/8OjBm2 BJXyXd4hGmRWfC2gxS6xhmg9dvCGq2fKkI3ri0g3oufWsqLvivpDE45cg3hMRexRrT TGe9bEyBbBIgQ== Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com [IPv6:2a00:1450:4864:20::635]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 70BEE62B2E for ; Tue, 12 Dec 2023 12:51:31 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="T9UiCMrj"; dkim-atps=neutral Received: by mail-ej1-x635.google.com with SMTP id a640c23a62f3a-a1da1017a09so676275566b.3 for ; Tue, 12 Dec 2023 03:51:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1702381891; x=1702986691; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=DOT0/3W2904w/yNWAzyag1CLj3C5QSiWMpVEzLHWH2Q=; b=T9UiCMrjzG1aJcOOJAjvP5wOHE5x4K56eA/qs9e5rZe7nNcIsRSTBVjV2uSHbZOqc3 rxF4glrti8EtNiSq3LZX+zHmj1rcG8cN56IoxEj+fupSq9jzq7rBu4bAqjXmvKzMug/c cg3mWJaZDtFqXAf1vUapYnakro42npZJ1G2xtuQYUrZ15vWEgFf5nZpXxBslEs47P49K +jnWeGkedFmHiX6PfMod8U25yfseJ8nhP/BszEaf+IX9j25rK0RT5XCKL9ZFF0h6E2gz JxgNCx77Nv+bxy4dWxXr24yrQvw3rtDrvrrguEHRCdb7TQVhk0xfDByNQEO8+vzZ5gVL aRew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702381891; x=1702986691; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DOT0/3W2904w/yNWAzyag1CLj3C5QSiWMpVEzLHWH2Q=; b=Qk/jyHFgzC4NKua122wUCRTVyhvMd7sFIfQ2ZVhXQ9PZI227KBUfH0U//i+PWQGeRp SilC5c6X6+GwCo46QiDbn+NDzgyix3M2ifaXaF2Z65XPygOLdOlTJpwXy+7vyge7RpOy 3PFqGJ8R9/p04OkIz4djeOAmkUFcrZbOGwmXwWzjPU9fp8hS88NRoDfi+wSoBgdV7fYK ucBS8rzW6UlVecQJ0fdVQWDspMcAOrrOucESWKloC8gfMiyzaut2OHGm15EUj75jhQjR HJjN4PVM1FoDNofH6d0TEzP3Ww05kfLlG5B/A3NHZGYtspZ75XI+CJy6WnfvJxpTow/y 4uAA== X-Gm-Message-State: AOJu0YwLO/P8sL1VU+W5ti4uhEYwatW2FECCqWiyKsWVn1tX2qWDighI 1s2JrLXNRPOLC2t78PcYOmGgdeDiScbTuZO0LUU= X-Google-Smtp-Source: AGHT+IGe2oAaT+rnR+kYKUmHpjoF6xyDshVKL4aTIMMapCtQSCTSleeQF4RpzMjmRK8dUVdryUZZfQ== X-Received: by 2002:a17:907:510:b0:a19:a19b:78a5 with SMTP id wj16-20020a170907051000b00a19a19b78a5mr2699050ejb.104.1702381889790; Tue, 12 Dec 2023 03:51:29 -0800 (PST) Received: from Lat-5310.. ([87.116.161.153]) by smtp.gmail.com with ESMTPSA id tx17-20020a1709078e9100b00a1b75e0e061sm6213188ejc.130.2023.12.12.03.51.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Dec 2023 03:51:29 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Tue, 12 Dec 2023 14:50:46 +0300 Message-Id: <20231212115046.102726-8-andrey.konovalov@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231212115046.102726-1-andrey.konovalov@linaro.org> References: <20231212115046.102726-1-andrey.konovalov@linaro.org> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH v2 7/7] libcamera: pipeline: simple: enable use of Soft ISP and Soft IPA X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Andrey Konovalov via libcamera-devel From: Andrey Konovalov Reply-To: Andrey Konovalov Cc: mripard@redhat.com, g.martti@gmail.com, t.langendam@gmail.com, srinivas.kandagatla@linaro.org, pavel@ucw.cz, bryan.odonoghue@linaro.org, admin@dennisbonke.com Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" To enable the Soft ISP and Soft IPA for simple pipeline handler configure the build with: -Dpipelines=simple/linaro -Dipas=simple/linaro If the pipeline uses Converter, Soft ISP and Soft IPA aren't available. Configuring the build with just: -Dpipelines=simple makes it to work like before - Soft ISP and Soft IPA aren't used. Signed-off-by: Andrey Konovalov --- src/libcamera/pipeline/simple/simple.cpp | 106 ++++++++++++++++++----- 1 file changed, 83 insertions(+), 23 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 878794f6..276e9d72 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -34,10 +34,10 @@ #include "libcamera/internal/device_enumerator.h" #include "libcamera/internal/media_device.h" #include "libcamera/internal/pipeline_handler.h" +#include "libcamera/internal/software_isp.h" #include "libcamera/internal/v4l2_subdevice.h" #include "libcamera/internal/v4l2_videodevice.h" - namespace libcamera { LOG_DEFINE_CATEGORY(SimplePipeline) @@ -274,6 +274,7 @@ public: bool useConversion_; std::unique_ptr converter_; + std::unique_ptr swIsp_; private: void tryPipeline(unsigned int code, const Size &size); @@ -281,6 +282,9 @@ private: void conversionInputDone(FrameBuffer *buffer); void conversionOutputDone(FrameBuffer *buffer); + + void ispStatsReady(int dummy); + void setSensorControls(const ControlList &sensorControls); }; class SimpleCameraConfiguration : public CameraConfiguration @@ -509,6 +513,25 @@ int SimpleCameraData::init() } } + /* + * Create Linaro's implementation of SoftwareIsp unconditionally if no converter is used + * - to be revisited + */ + if (!converter_) { + swIsp_ = SoftwareIspFactoryBase::create(pipe, sensor_->controls()); + if (!swIsp_) { + LOG(SimplePipeline, Warning) + << "Failed to create software ISP, disabling software debayering"; + swIsp_.reset(); + } else { + swIsp_->inputBufferReady.connect(this, &SimpleCameraData::conversionInputDone); + swIsp_->outputBufferReady.connect(this, &SimpleCameraData::conversionOutputDone); + swIsp_->ispStatsReady.connect(this, &SimpleCameraData::ispStatsReady); + + swIsp_->getSignalSetSensorControls().connect(this, &SimpleCameraData::setSensorControls); + } + } + video_ = pipe->video(entities_.back().entity); ASSERT(video_); @@ -599,12 +622,15 @@ void SimpleCameraData::tryPipeline(unsigned int code, const Size &size) config.captureFormat = pixelFormat; config.captureSize = format.size; - if (!converter_) { - config.outputFormats = { pixelFormat }; - config.outputSizes = config.captureSize; - } else { + if (converter_) { config.outputFormats = converter_->formats(pixelFormat); config.outputSizes = converter_->sizes(format.size); + } else if (swIsp_) { + config.outputFormats = swIsp_->formats(pixelFormat); + config.outputSizes = swIsp_->sizes(pixelFormat, format.size); + } else { + config.outputFormats = { pixelFormat }; + config.outputSizes = config.captureSize; } configs_.push_back(config); @@ -750,9 +776,9 @@ void SimpleCameraData::bufferReady(FrameBuffer *buffer) } /* - * The converter is in use. Requeue the internal buffer for - * capture (unless the stream is being stopped), and complete - * the request with all the user-facing buffers. + * The converter or Software ISP is in use. Requeue the internal + * buffer for capture (unless the stream is being stopped), and + * complete the request with all the user-facing buffers. */ if (buffer->metadata().status != FrameMetadata::FrameCancelled) video_->queueBuffer(buffer); @@ -798,9 +824,9 @@ void SimpleCameraData::bufferReady(FrameBuffer *buffer) buffer->metadata().timestamp); /* - * Queue the captured and the request buffer to the converter if format - * conversion is needed. If there's no queued request, just requeue the - * captured buffer for capture. + * Queue the captured and the request buffer to the converter or Software + * ISP if format conversion is needed. If there's no queued request, just + * requeue the captured buffer for capture. */ if (useConversion_) { if (conversionQueue_.empty()) { @@ -808,7 +834,11 @@ void SimpleCameraData::bufferReady(FrameBuffer *buffer) return; } - converter_->queueBuffers(buffer, conversionQueue_.front()); + if (converter_) + converter_->queueBuffers(buffer, conversionQueue_.front()); + else + swIsp_->queueBuffers(buffer, conversionQueue_.front()); + conversionQueue_.pop(); return; } @@ -834,6 +864,18 @@ void SimpleCameraData::conversionOutputDone(FrameBuffer *buffer) pipe->completeRequest(request); } +void SimpleCameraData::ispStatsReady([[maybe_unused]] int dummy) +{ + swIsp_->processStats(sensor_->getControls({ V4L2_CID_ANALOGUE_GAIN, + V4L2_CID_EXPOSURE })); +} + +void SimpleCameraData::setSensorControls(const ControlList &sensorControls) +{ + ControlList ctrls(sensorControls); + sensor_->setControls(&ctrls); +} + /* Retrieve all source pads connected to a sink pad through active routes. */ std::vector SimpleCameraData::routedSourcePads(MediaPad *sink) { @@ -1013,8 +1055,10 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() /* Set the stride, frameSize and bufferCount. */ if (needConversion_) { std::tie(cfg.stride, cfg.frameSize) = - data_->converter_->strideAndFrameSize(cfg.pixelFormat, - cfg.size); + (data_->converter_) ? data_->converter_->strideAndFrameSize(cfg.pixelFormat, + cfg.size) + : data_->swIsp_->strideAndFrameSize(cfg.pixelFormat, + cfg.size); if (cfg.stride == 0) return Invalid; } else { @@ -1177,7 +1221,8 @@ int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c) inputCfg.stride = captureFormat.planes[0].bpl; inputCfg.bufferCount = kNumInternalBuffers; - return data->converter_->configure(inputCfg, outputCfgs); + return (data->converter_) ? data->converter_->configure(inputCfg, outputCfgs) + : data->swIsp_->configure(inputCfg, outputCfgs); } int SimplePipelineHandler::exportFrameBuffers(Camera *camera, Stream *stream, @@ -1191,8 +1236,10 @@ int SimplePipelineHandler::exportFrameBuffers(Camera *camera, Stream *stream, * whether the converter is used or not. */ if (data->useConversion_) - return data->converter_->exportBuffers(data->streamIndex(stream), - count, buffers); + return (data->converter_) ? data->converter_->exportBuffers(data->streamIndex(stream), + count, buffers) + : data->swIsp_->exportBuffers(data->streamIndex(stream), + count, buffers); else return data->video_->exportBuffers(count, buffers); } @@ -1237,10 +1284,18 @@ int SimplePipelineHandler::start(Camera *camera, [[maybe_unused]] const ControlL } if (data->useConversion_) { - ret = data->converter_->start(); - if (ret < 0) { - stop(camera); - return ret; + if (data->converter_) { + ret = data->converter_->start(); + if (ret < 0) { + stop(camera); + return ret; + } + } else if (data->swIsp_) { + ret = data->swIsp_->start(); + if (ret < 0) { + stop(camera); + return ret; + } } /* Queue all internal buffers for capture. */ @@ -1256,8 +1311,13 @@ void SimplePipelineHandler::stopDevice(Camera *camera) SimpleCameraData *data = cameraData(camera); V4L2VideoDevice *video = data->video_; - if (data->useConversion_) - data->converter_->stop(); + if (data->useConversion_) { + if (data->converter_) + data->converter_->stop(); + else if (data->swIsp_) { + data->swIsp_->stop(); + } + } video->streamOff(); video->releaseBuffers();