From patchwork Mon Dec 4 00:10:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 19263 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 26C94C3226 for ; Mon, 4 Dec 2023 00:10:43 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C7837629D2; Mon, 4 Dec 2023 01:10:42 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1701648642; bh=Hs6qsED+J4nvWplDsdRC9lZLlUivMZkNy1c3J8GcY4s=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=mO1s0Yz5bZDGTNwNa5Lyg5i37hKB1PxQr6L+NquDuWOHhhILqKvFc7eWZ6jms27wU k87Ba8zcp1cGlAnb0M6BQcHp05e4OZVz3DiAUX/C5023dr1+2OB8GPRl/u7kUuvJH+ YhgJ9pbeFAzf1hIJIO4SqMIE8Kie+ia4qrUUXxEQddSRA/d8x2oy876Yac+ikFJZ+Z 00YgSX5xSOX/khctDoiA6jkCMpGFlkQpg8jJK6NTYEbgtHxN7Sm87vYxlYhKHkyOxh CKEVMNCfObGd0ncLO68mFRoLIt76LmkoHNvt5jv2wcR3w6FKZBLufbNhPIV9QQJCSQ TtQ9Q8fcEpM3g== Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D305A629C2 for ; Mon, 4 Dec 2023 01:10:40 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="hunj3/mj"; dkim-atps=neutral Received: by mail-wr1-x432.google.com with SMTP id ffacd0b85a97d-33339d843b9so1290077f8f.0 for ; Sun, 03 Dec 2023 16:10:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1701648640; x=1702253440; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Kd7bj0qh+i1DezuNBO0exd5KVwqH+RukxtsQuZTDKZg=; b=hunj3/mjE2kUUPUYupTWOUTd4LP1YwmCXbwqjnkqIKhEpSky+p3bpRRJw/LNz44pRk uxbuEtHQyswbXyx1xVBdsOv/iywhmFdaemRd+TquFjWIq+riF0ghflA1rrELZD5x1SkG 016H177zpMDVmkEmLsXancHPG20w3pDo8lewh/cPrwa1eWTY71Ajlvz2loSMaWu8kqQZ UEZS76YGmppKIb1gz3quxESoiojwWvfT5T1iP1YOkH1HZMlsrOoBgUUnxpRZJrVWGYcY cegoB6O8yoBdacRoVQKOkkXY1IjeGsYjNCQTTbCtzXorLMLcbP9xsM01gz3fRFw9mFgc gxuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701648640; x=1702253440; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Kd7bj0qh+i1DezuNBO0exd5KVwqH+RukxtsQuZTDKZg=; b=fos1FQDlakfCTVwlx6BaNu/oqgexF2C2UPAfSEHsQOwb7AQ6MR6v9THvkekIxM+XL/ 5/XXu6rqdAymK+AVBCEs2b7elvdq3P76CeYmoOU7W3D8qwAxAhIsHU8CI8ToldJpRTLw fqWB4KuQoVH2pgSeoxPQc1IvjhGxWhw+Drn3z76rPbaIjeX3fO2pN+2cnahIKMj5lcSR BzTQVVmcnOorSQ1AUhyTJ3QaCFMcKjztkvQCCdaFTWhZIp+4ah/larKkJDFzVC3c+xds Cuv+EhLzVXbU0qYHP9S5OY+jtNTLXVpCE3x3QEi6HSoppYcwbbYi6QvPsCjTmcRXaJrR +oRw== X-Gm-Message-State: AOJu0Yy0ZyEUXX9cBDg4zTxL0o6l2inGEQfwxJ1n5kSBuxDZEvsljcLz CgaRInGhHh/Vy9LDQRTcDYTQMkyMuAsopxD7g1Q= X-Google-Smtp-Source: AGHT+IGe4tUY3HRpL0V+K5xUbo1HhDeMXr/5xzLoA/Zz2NHajHg3dKuKdfzVb6L25kZxaPKQsh4I4A== X-Received: by 2002:adf:f4d2:0:b0:333:6ce:4da9 with SMTP id h18-20020adff4d2000000b0033306ce4da9mr2613368wrp.58.1701648640407; Sun, 03 Dec 2023 16:10:40 -0800 (PST) Received: from Lat-5310.. ([87.116.165.212]) by smtp.gmail.com with ESMTPSA id u29-20020adfa19d000000b003332db7d91dsm8835997wru.39.2023.12.03.16.10.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 03 Dec 2023 16:10:40 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Mon, 4 Dec 2023 03:10:07 +0300 Message-Id: <20231204001013.404720-2-andrey.konovalov@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231204001013.404720-1-andrey.konovalov@linaro.org> References: <20231204001013.404720-1-andrey.konovalov@linaro.org> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 1/7] libcamera: introduce SoftwareIsp class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Andrey Konovalov via libcamera-devel From: Andrey Konovalov Reply-To: Andrey Konovalov Cc: mripard@redhat.com, g.martti@gmail.com, t.langendam@gmail.com, srinivas.kandagatla@linaro.org, bryan.odonoghue@linaro.org, admin@dennisbonke.com Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Signed-off-by: Andrey Konovalov --- include/libcamera/internal/meson.build | 1 + include/libcamera/internal/software_isp.h | 67 +++++++++++++++++++++++ src/libcamera/meson.build | 1 + src/libcamera/software_isp.cpp | 25 +++++++++ 4 files changed, 94 insertions(+) create mode 100644 include/libcamera/internal/software_isp.h create mode 100644 src/libcamera/software_isp.cpp diff --git a/include/libcamera/internal/meson.build b/include/libcamera/internal/meson.build index 7f1f3440..0f725cfd 100644 --- a/include/libcamera/internal/meson.build +++ b/include/libcamera/internal/meson.build @@ -38,6 +38,7 @@ libcamera_internal_headers = files([ 'process.h', 'pub_key.h', 'request.h', + 'software_isp.h', 'source_paths.h', 'sysfs.h', 'v4l2_device.h', diff --git a/include/libcamera/internal/software_isp.h b/include/libcamera/internal/software_isp.h new file mode 100644 index 00000000..556203d9 --- /dev/null +++ b/include/libcamera/internal/software_isp.h @@ -0,0 +1,67 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2023, Linaro Ltd + * + * software_isp.h - Interface for a software implementation of an ISP + */ + +#pragma once + +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +#include + +namespace libcamera { + +class FrameBuffer; +class PixelFormat; +struct StreamConfiguration; + +class SoftwareIsp +{ +public: + SoftwareIsp(const std::string &name); + virtual ~SoftwareIsp(); + + virtual int loadConfiguration(const std::string &filename) = 0; + + virtual bool isValid() const = 0; + + virtual std::vector formats(PixelFormat input) = 0; + virtual SizeRange sizes(PixelFormat inputFormat, const Size &inputSize) = 0; + + virtual std::tuple + strideAndFrameSize(const PixelFormat &pixelFormat, const Size &size) = 0; + + virtual int configure(const StreamConfiguration &inputCfg, + const std::vector> &outputCfgs) = 0; + virtual int exportBuffers(unsigned int output, unsigned int count, + std::vector> *buffers) = 0; + + virtual int start() = 0; + virtual void stop() = 0; + + virtual int queueBuffers(FrameBuffer *input, + const std::map &outputs) = 0; + + Signal inputBufferReady; + Signal outputBufferReady; + + /* The int parameter isn't actually used */ + Signal ispStatsReady; + + const std::string &name() const { return name_; } +private: + std::string name_; +}; + +} /* namespace libcamera */ diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build index d0e26f6b..9d26a87f 100644 --- a/src/libcamera/meson.build +++ b/src/libcamera/meson.build @@ -40,6 +40,7 @@ libcamera_sources = files([ 'process.cpp', 'pub_key.cpp', 'request.cpp', + 'software_isp.cpp', 'source_paths.cpp', 'stream.cpp', 'sysfs.cpp', diff --git a/src/libcamera/software_isp.cpp b/src/libcamera/software_isp.cpp new file mode 100644 index 00000000..3d81c27a --- /dev/null +++ b/src/libcamera/software_isp.cpp @@ -0,0 +1,25 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2023, Linaro Ltd + * + * software_isp.cpp - Interface for a software implementation of an ISP + */ + +#include "libcamera/internal/software_isp.h" + +#include + +namespace libcamera { + +LOG_DEFINE_CATEGORY(SoftwareIsp) + +SoftwareIsp::SoftwareIsp(const std::string &name) + : name_(name) +{ +} + +SoftwareIsp::~SoftwareIsp() +{ +} + +} /* namespace libcamera */ From patchwork Mon Dec 4 00:10:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 19264 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 083BDC322E for ; Mon, 4 Dec 2023 00:10:44 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 85BDD629D3; Mon, 4 Dec 2023 01:10:44 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1701648644; bh=oRBIY8kOi0gKYV5IPe6DVH6UPAc8qzG43Lgmf4uwB34=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=FHW8UY1Oh+Rn6pOWH323GllOFHCQeI8qllNbkPaEO/stgH8uA/H/ecPTsE2lm8X8l 8QF7bOj2O0zWnEGCsqE5DBP+HY3UmshL+tfksWj018troxXnBdFhnzS9Qg3iqPZIv+ fiADG+CRNbG7y+hm07LMStKVTGLYmqjRPKFR73aSH0wQvDXrHNuvNg4GnPrbrpNznr lnmmhBFdvXRveONNH8LNQycakRTyjRRUWZJPxQ5xkpmcicx9A+QdiUCofT9yOdM56C +ClLj8fap9IKzXa87PduS8aa1+ZO14GLSiR3b3bz5VwxwplFdbFwacYti0+DU4ZL92 t8hjjLr3xGHQA== Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E5F04629D3 for ; Mon, 4 Dec 2023 01:10:42 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="L6n+Zikn"; dkim-atps=neutral Received: by mail-wr1-x429.google.com with SMTP id ffacd0b85a97d-3316bb1303bso3028563f8f.0 for ; Sun, 03 Dec 2023 16:10:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1701648642; x=1702253442; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=pFJAoSA9Nwu/bV7xUFK2GgTlAP4DOK35M5GoMj00EiU=; b=L6n+ZiknGX4m7jbMUJpO2aY0E4WMl9208MpAP3w+GZrJ9LVudbrZiMWtdM3VYvWzSQ 3+aMXMRhIPMQ5kGEKS6RRzQ9teUSScU1v29lh6i9F5+sRAXYb9aW1EHwdWUisasK968w B2gBNvMJgueRVIKo0shRCNW77HJBwfk1FdDYEUjSttIv+XSVdUO8sU80lssX6N3Y9wC0 FAkdrICB+iOqHG5Imw6CgK3STCDaDGn5WHIC/+wxLEMA9YmdsmloSGwd++QV7WOUtqxQ GnlDWTkm3K6gU3OuG3TwnDd/NKOLuxsqynAFZ++f5HqrXfkzpgqZxo3QJPuVq3bqhQvz 02Lg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701648642; x=1702253442; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pFJAoSA9Nwu/bV7xUFK2GgTlAP4DOK35M5GoMj00EiU=; b=bfHujq5Rxha3t/J3hDPsudVpWhGhLtJYacohWlhP6gY5Dfs/FIvA4kVGoPBRLVKfAN a6HNbBjaa9Re2HYaycOLAVcTmdiRWkqsIK/eUc5+M+5plKr3mHIlOratJ35G+YB9sebq I2tRUG27QkUgjFvo77UTvDBHgbOO82szJZ/5EgUj6M4ieE0aQNs+NYlN/j+6fwSp0ZQe PEPCCtUgLqlJWUfX3z/GwmZRBwCjMWhBONy4Lq7G1HdBXS4xki2eQiVbozFrF5Ph1rMs 8lcTAejF8l2ztTCcptFkejsT6RqxsWQU5jub97ANArA4W7xr3iRS5VrPBOTKhveAQbuo Ovow== X-Gm-Message-State: AOJu0YwjkMOyQzIy3iJoyReY/4AxOsNjLqouGYp1w6tjZJDdNgk1A9/z b00EALBUQeWfqKRY7txNAVCHR4rzSm1nHACKG5w= X-Google-Smtp-Source: AGHT+IEFHvqqtebCZhYG5PVXfhYrdKwdoDUW62eNL+1EvUJHrWqNK2j2whHdzDwDd7HNrxSbauvjgQ== X-Received: by 2002:a5d:63cb:0:b0:333:13cf:dd15 with SMTP id c11-20020a5d63cb000000b0033313cfdd15mr2868893wrw.55.1701648642646; Sun, 03 Dec 2023 16:10:42 -0800 (PST) Received: from Lat-5310.. ([87.116.165.212]) by smtp.gmail.com with ESMTPSA id u29-20020adfa19d000000b003332db7d91dsm8835997wru.39.2023.12.03.16.10.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 03 Dec 2023 16:10:42 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Mon, 4 Dec 2023 03:10:08 +0300 Message-Id: <20231204001013.404720-3-andrey.konovalov@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231204001013.404720-1-andrey.konovalov@linaro.org> References: <20231204001013.404720-1-andrey.konovalov@linaro.org> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 2/7] libcamera: internal: Move SharedMemObject class to a common directory X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Andrey Konovalov via libcamera-devel From: Andrey Konovalov Reply-To: Andrey Konovalov Cc: mripard@redhat.com, g.martti@gmail.com, t.langendam@gmail.com, srinivas.kandagatla@linaro.org, bryan.odonoghue@linaro.org, admin@dennisbonke.com Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Move SharedMemObject class out of RPi namespace and put it into include/libcamera/internal so that everyone could use it. Signed-off-by: Andrey Konovalov --- include/libcamera/internal/meson.build | 1 + .../common => include/libcamera/internal}/shared_mem_object.h | 4 ---- 2 files changed, 1 insertion(+), 4 deletions(-) rename {src/libcamera/pipeline/rpi/common => include/libcamera/internal}/shared_mem_object.h (98%) diff --git a/include/libcamera/internal/meson.build b/include/libcamera/internal/meson.build index 0f725cfd..b780777c 100644 --- a/include/libcamera/internal/meson.build +++ b/include/libcamera/internal/meson.build @@ -38,6 +38,7 @@ libcamera_internal_headers = files([ 'process.h', 'pub_key.h', 'request.h', + 'shared_mem_object.h', 'software_isp.h', 'source_paths.h', 'sysfs.h', diff --git a/src/libcamera/pipeline/rpi/common/shared_mem_object.h b/include/libcamera/internal/shared_mem_object.h similarity index 98% rename from src/libcamera/pipeline/rpi/common/shared_mem_object.h rename to include/libcamera/internal/shared_mem_object.h index aa56c220..bfb639ee 100644 --- a/src/libcamera/pipeline/rpi/common/shared_mem_object.h +++ b/include/libcamera/internal/shared_mem_object.h @@ -19,8 +19,6 @@ namespace libcamera { -namespace RPi { - template class SharedMemObject { @@ -123,6 +121,4 @@ private: T *obj_; }; -} /* namespace RPi */ - } /* namespace libcamera */ From patchwork Mon Dec 4 00:10:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 19265 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7F553C3226 for ; Mon, 4 Dec 2023 00:10:46 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 26DBC629D6; Mon, 4 Dec 2023 01:10:46 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1701648646; bh=D5fo1V8p+jrOhg6enCbL+FVy5lwayRKFVeW+Z0Sc0Vw=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=l1aghxnhB8X8ffzi6Y0iyIAONQDxsOja02kNr6W9zBcxze/NBLDTN/zbK1UyqRD62 fL+5beWmfapt+loHT1BNVCPXqhiPsyX5RCPrVuA9CTtBW0HIK9pm+31fEL/IIpHsre 4MVCN5dAS+PL0hLIc8YfOlJnVcurHF8DaBrY0nTG7NHyeNgt4+dhfAIzquNNSsfs3s 1PcKT9ZgQrI5Y880tZ72c6q/ihWflwR6TKkWQmtBpzilNlPTVdztgEMJv/5ArSYpXS 8m+alFGtXwF18rr5Zwo4XhtxPBevM4D9XmpUtettEktaWthC4Zpdq1y8V9xRRwAzSD pQpYchqzogncQ== Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 34EBB629D5 for ; Mon, 4 Dec 2023 01:10:45 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="Db0P4occ"; dkim-atps=neutral Received: by mail-wm1-x333.google.com with SMTP id 5b1f17b1804b1-40c09fcfa9fso6866105e9.2 for ; Sun, 03 Dec 2023 16:10:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1701648645; x=1702253445; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Gh+koOeAw51LTI6Xbzy0kR5L2rjRgitnCI0TM1zcDoc=; b=Db0P4occALNC46LbFoUsC9dXHMvF79u5k1R18W3AuMKo+ASRueNk2hDCk7CfzDnbBD p8C3krvvbIAEsrbjlPTrK1Ge/XW0L6+03xana4k6Zp1jfVKlLU8rTWIkiJJTXPtB7WrN n4YqPQqIi2KdrFbbBMDLHjs/cEkUlDj/2f8tgiLu4AWgQwksWCSxRpbiQuww02VXfkJP /Ggbw5IYPp1QyvwJGCi4n1eSNTlAsnaKQkbHyroQzRXbvUfsWcWNhAXONlClkYgpEuSq LoSAygNSQlkIG8Zd5pSXLAioc7ID3D6+2At81hkOI0Oytw7csSLxiZR/lsaLnnrUHEdI nMcQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701648645; x=1702253445; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Gh+koOeAw51LTI6Xbzy0kR5L2rjRgitnCI0TM1zcDoc=; b=EZleO5SmRlm/1650XskN41PHo34UiDq/qtm5Rsw6BMMvBfLpy5ZHVM5r0XeBbxmzH/ o+7/w5yhsdq6b8gd1zjhkbftxllwC0E6JhOeCjIkMlq9BswC1FZG2izEhXU9DH9A5xlQ cCOrwAUL43tVcNvtGynDV1yAeXpj9/7+v0StQyh36ipRwMN8bAXCKy9fMVkFEL/LTw8X 4MAaPdxRBrlzB1Aklphf4/q3Jywy9u4gLqt0CgvVhRFZ7rDXVUhLQwWRr0lS6dXcKkzL UXCiCD6tKjOILQ5oi5OveWRv7f1xpsO9/8ckQU8ntBqmdZgY541+h2rgAej9+JEwuxwk CRQg== X-Gm-Message-State: AOJu0YxWZeuFWXw3AodJNMGgeWAdJ0RL/e5SMHZSXuoyBOKrrQEf1akD sSPbAz69GdFKFK4ZZEFjzdNCUJF09kcfJgupfeQ= X-Google-Smtp-Source: AGHT+IGpcZBI3Dp7n/wOZVnsuJQh9IAQIZAYSNqzKHeBZr2NtXA2FBK4JBVm+aT0jPOqmMGyxd0Iww== X-Received: by 2002:a5d:58e1:0:b0:333:2fd2:766d with SMTP id f1-20020a5d58e1000000b003332fd2766dmr1624495wrd.94.1701648644838; Sun, 03 Dec 2023 16:10:44 -0800 (PST) Received: from Lat-5310.. ([87.116.165.212]) by smtp.gmail.com with ESMTPSA id u29-20020adfa19d000000b003332db7d91dsm8835997wru.39.2023.12.03.16.10.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 03 Dec 2023 16:10:44 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Mon, 4 Dec 2023 03:10:09 +0300 Message-Id: <20231204001013.404720-4-andrey.konovalov@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231204001013.404720-1-andrey.konovalov@linaro.org> References: <20231204001013.404720-1-andrey.konovalov@linaro.org> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 3/7] libcamera: software_isp: add SwIspLinaro implementation of SoftwareIsp X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Andrey Konovalov via libcamera-devel From: Andrey Konovalov Reply-To: Andrey Konovalov Cc: mripard@redhat.com, g.martti@gmail.com, t.langendam@gmail.com, srinivas.kandagatla@linaro.org, bryan.odonoghue@linaro.org, admin@dennisbonke.com Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Signed-off-by: Andrey Konovalov --- include/libcamera/internal/meson.build | 1 + .../internal/software_isp/meson.build | 6 + .../internal/software_isp/statistics-linaro.h | 17 + .../internal/software_isp/swisp_linaro.h | 109 ++++ src/libcamera/meson.build | 1 + src/libcamera/software_isp/meson.build | 5 + src/libcamera/software_isp/swisp_linaro.cpp | 539 ++++++++++++++++++ 7 files changed, 678 insertions(+) create mode 100644 include/libcamera/internal/software_isp/meson.build create mode 100644 include/libcamera/internal/software_isp/statistics-linaro.h create mode 100644 include/libcamera/internal/software_isp/swisp_linaro.h create mode 100644 src/libcamera/software_isp/meson.build create mode 100644 src/libcamera/software_isp/swisp_linaro.cpp diff --git a/include/libcamera/internal/meson.build b/include/libcamera/internal/meson.build index b780777c..eeae801c 100644 --- a/include/libcamera/internal/meson.build +++ b/include/libcamera/internal/meson.build @@ -50,3 +50,4 @@ libcamera_internal_headers = files([ ]) subdir('converter') +subdir('software_isp') diff --git a/include/libcamera/internal/software_isp/meson.build b/include/libcamera/internal/software_isp/meson.build new file mode 100644 index 00000000..9f84f00e --- /dev/null +++ b/include/libcamera/internal/software_isp/meson.build @@ -0,0 +1,6 @@ +# SPDX-License-Identifier: CC0-1.0 + +libcamera_internal_headers += files([ + 'statistics-linaro.h', + 'swisp_linaro.h', +]) diff --git a/include/libcamera/internal/software_isp/statistics-linaro.h b/include/libcamera/internal/software_isp/statistics-linaro.h new file mode 100644 index 00000000..20c64e44 --- /dev/null +++ b/include/libcamera/internal/software_isp/statistics-linaro.h @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2023, Linaro Ltd + * + * statistics.h - Statistics data format used by the software ISP + */ + +#pragma once + +namespace libcamera { + +struct SwIspStats { + float bright_ratio; + float too_bright_ratio; +}; + +} /* namespace libcamera */ diff --git a/include/libcamera/internal/software_isp/swisp_linaro.h b/include/libcamera/internal/software_isp/swisp_linaro.h new file mode 100644 index 00000000..0d8a31a6 --- /dev/null +++ b/include/libcamera/internal/software_isp/swisp_linaro.h @@ -0,0 +1,109 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2023, Linaro Ltd + * + * swisp_linaro.h - software ISP implementation by Linaro + */ + +#pragma once + +#include +#include + +#include + +#include "libcamera/internal/shared_mem_object.h" +#include "libcamera/internal/software_isp/statistics-linaro.h" +#include "libcamera/internal/software_isp.h" + +namespace libcamera { + +class SwIspLinaro : public SoftwareIsp +{ +public: + SwIspLinaro(const std::string &name); + ~SwIspLinaro() {} + + int loadConfiguration([[maybe_unused]] const std::string &filename) { return 0; } + bool isValid() const; + + std::vector formats(PixelFormat input); + SizeRange sizes(PixelFormat inputFormat, const Size &inputSize); + + std::tuple + strideAndFrameSize(const PixelFormat &outputFormat, const Size &size); + + int configure(const StreamConfiguration &inputCfg, + const std::vector> &outputCfgs); + int exportBuffers(unsigned int output, unsigned int count, + std::vector> *buffers); + + int start(); + void stop(); + + int queueBuffers(FrameBuffer *input, + const std::map &outputs); + + void process(FrameBuffer *input, FrameBuffer *output); + +private: + SharedMemObject sharedStats_; + + class IspWorker : public Object + { + public: + IspWorker(SwIspLinaro *swIsp); + + std::vector formats(PixelFormat input); + SizeRange sizes(PixelFormat inputFormat, const Size &inputSize); + unsigned int outStride(const PixelFormat &outputFormat, + const Size &outSize); + + int configure(const StreamConfiguration &inputCfg, + const StreamConfiguration &outputCfg); + unsigned int outBufferSize(); + void process(FrameBuffer *input, FrameBuffer *output); + + private: + SwIspLinaro *swIsp_; + + typedef void (SwIspLinaro::IspWorker::*debayerFn)(uint8_t *dst, const uint8_t *src); + typedef SizeRange (*outSizesFn)(const Size &inSize); + typedef unsigned int (*outStrideFn)(const Size &outSize); + struct debayerInfo { + PixelFormat outPixelFmt; + debayerFn debayer; + outSizesFn getOutSizes; + outStrideFn getOutStride; + }; + // TODO: use inputFormat+outputFormat as the map key + // to enable multiple output formats + // TODO: use BayerFormat instead of PixelFormat as inputFormat + std::map debayerInfos_; + int setDebayerInfo(PixelFormat format); + debayerInfo *debayerInfo_; + + /* CSI-2 packed 10-bit raw bayer format (all the 4 orders) */ + void debayerRaw10P(uint8_t *dst, const uint8_t *src); + static SizeRange outSizesRaw10P(const Size &inSize); + static unsigned int outStrideRaw10P(const Size &outSize); + + unsigned int width_; + unsigned int height_; + unsigned int stride_; + Point redShift_; + unsigned int outHeight_; + unsigned int outStride_; + + unsigned long rNumerat_, rDenomin_; /* red gain for AWB */ + unsigned long bNumerat_, bDenomin_; /* blue gain for AWB */ + unsigned long gNumerat_, gDenomin_; /* green gain for AWB */ + + SwIspStats stats_; + }; + + std::unique_ptr ispWorker_; + Thread ispWorkerThread_; +}; + +} /* namespace libcamera */ diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build index 9d26a87f..395235ed 100644 --- a/src/libcamera/meson.build +++ b/src/libcamera/meson.build @@ -70,6 +70,7 @@ subdir('converter') subdir('ipa') subdir('pipeline') subdir('proxy') +subdir('software_isp') null_dep = dependency('', required : false) diff --git a/src/libcamera/software_isp/meson.build b/src/libcamera/software_isp/meson.build new file mode 100644 index 00000000..05eda181 --- /dev/null +++ b/src/libcamera/software_isp/meson.build @@ -0,0 +1,5 @@ +# SPDX-License-Identifier: CC0-1.0 + +libcamera_sources += files([ + 'swisp_linaro.cpp', +]) diff --git a/src/libcamera/software_isp/swisp_linaro.cpp b/src/libcamera/software_isp/swisp_linaro.cpp new file mode 100644 index 00000000..79ceda4d --- /dev/null +++ b/src/libcamera/software_isp/swisp_linaro.cpp @@ -0,0 +1,539 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2023, Linaro Ltd + * + * swisp_linaro.cpp - software ISP implementation by Linaro + */ + +#include "libcamera/internal/software_isp/swisp_linaro.h" + +#include +#include +#include + +#include +#include + +#include "libcamera/internal/bayer_format.h" +#include "libcamera/internal/framebuffer.h" +#include "libcamera/internal/mapped_framebuffer.h" + +namespace libcamera { + +LOG_DECLARE_CATEGORY(SoftwareIsp) + +SwIspLinaro::SwIspLinaro(const std::string &name) + : SoftwareIsp(name) +{ + ispWorker_ = std::make_unique(this); + if (!ispWorker_) { + LOG(SoftwareIsp, Error) + << "Failed to create ISP worker"; + } else { + sharedStats_ = SharedMemObject("softIsp_stats"); + if (!sharedStats_.fd().isValid()) { + LOG(SoftwareIsp, Error) + << "Failed to create shared memory for statistics"; + ispWorker_.reset(); + } else { + ispWorker_->moveToThread(&ispWorkerThread_); + } + } +} + +bool SwIspLinaro::isValid() const +{ + return !!ispWorker_; +} + +void SwIspLinaro::IspWorker::debayerRaw10P(uint8_t *dst, const uint8_t *src) +{ + /* for brightness values in the 0 to 255 range: */ + static const unsigned int BRIGHT_LVL = 200U << 8; + static const unsigned int TOO_BRIGHT_LVL = 240U << 8; + + static const unsigned int RED_Y_MUL = 77; /* 0.30 * 256 */ + static const unsigned int GREEN_Y_MUL = 150; /* 0.59 * 256 */ + static const unsigned int BLUE_Y_MUL = 29; /* 0.11 * 256 */ + + int w_out = width_ - 2; + int h_out = height_ - 2; + + unsigned long sumR = 0; + unsigned long sumB = 0; + unsigned long sumG = 0; + + unsigned long bright_sum = 0; + unsigned long too_bright_sum = 0; + + for (int y = 0; y < h_out; y++) { + const uint8_t *pin_base = src + (y + 1) * stride_; + uint8_t *pout = dst + y * w_out * 3; + int phase_y = (y + redShift_.y) % 2; + + for (int x = 0; x < w_out; x++) { + int phase_x = (x + redShift_.x) % 2; + int phase = 2 * phase_y + phase_x; + + /* x part of the offset in the input buffer: */ + int x_m1 = x + x / 4; /* offset for (x-1) */ + int x_0 = x + 1 + (x + 1) / 4; /* offset for x */ + int x_p1 = x + 2 + (x + 2) / 4; /* offset for (x+1) */ + /* the colour component value to write to the output */ + unsigned val; + /* Y value times 256 */ + unsigned y_val; + + switch (phase) { + case 0: /* at R pixel */ + /* blue: ((-1,-1)+(1,-1)+(-1,1)+(1,1)) / 4 */ + val = ( *(pin_base + x_m1 - stride_) + + *(pin_base + x_p1 - stride_) + + *(pin_base + x_m1 + stride_) + + *(pin_base + x_p1 + stride_) ) >> 2; + y_val = BLUE_Y_MUL * val; + val = val * bNumerat_ / bDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* green: ((0,-1)+(-1,0)+(1,0)+(0,1)) / 4 */ + val = ( *(pin_base + x_0 - stride_) + + *(pin_base + x_p1) + + *(pin_base + x_m1) + + *(pin_base + x_0 + stride_) ) >> 2; + val = val * gNumerat_ / gDenomin_; + y_val += GREEN_Y_MUL * val; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* red: (0,0) */ + val = *(pin_base + x_0); + sumR += val; + y_val += RED_Y_MUL * val; + if (y_val > BRIGHT_LVL) ++bright_sum; + if (y_val > TOO_BRIGHT_LVL) ++too_bright_sum; + val = val * rNumerat_ / rDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + break; + case 1: /* at Gr pixel */ + /* blue: ((0,-1) + (0,1)) / 2 */ + val = ( *(pin_base + x_0 - stride_) + + *(pin_base + x_0 + stride_) ) >> 1; + y_val = BLUE_Y_MUL * val; + val = val * bNumerat_ / bDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* green: (0,0) */ + val = *(pin_base + x_0); + sumG += val; + y_val += GREEN_Y_MUL * val; + val = val * gNumerat_ / gDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* red: ((-1,0) + (1,0)) / 2 */ + val = ( *(pin_base + x_m1) + + *(pin_base + x_p1) ) >> 1; + y_val += RED_Y_MUL * val; + if (y_val > BRIGHT_LVL) ++bright_sum; + if (y_val > TOO_BRIGHT_LVL) ++too_bright_sum; + val = val * rNumerat_ / rDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + break; + case 2: /* at Gb pixel */ + /* blue: ((-1,0) + (1,0)) / 2 */ + val = ( *(pin_base + x_m1) + + *(pin_base + x_p1) ) >> 1; + y_val = BLUE_Y_MUL * val; + val = val * bNumerat_ / bDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* green: (0,0) */ + val = *(pin_base + x_0); + sumG += val; + y_val += GREEN_Y_MUL * val; + val = val * gNumerat_ / gDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* red: ((0,-1) + (0,1)) / 2 */ + val = ( *(pin_base + x_0 - stride_) + + *(pin_base + x_0 + stride_) ) >> 1; + y_val += RED_Y_MUL * val; + if (y_val > BRIGHT_LVL) ++bright_sum; + if (y_val > TOO_BRIGHT_LVL) ++too_bright_sum; + val = val * rNumerat_ / rDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + break; + default: /* at B pixel */ + /* blue: (0,0) */ + val = *(pin_base + x_0); + sumB += val; + y_val = BLUE_Y_MUL * val; + val = val * bNumerat_ / bDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* green: ((0,-1)+(-1,0)+(1,0)+(0,1)) / 4 */ + val = ( *(pin_base + x_0 - stride_) + + *(pin_base + x_p1) + + *(pin_base + x_m1) + + *(pin_base + x_0 + stride_) ) >> 2; + y_val += GREEN_Y_MUL * val; + val = val * gNumerat_ / gDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* red: ((-1,-1)+(1,-1)+(-1,1)+(1,1)) / 4 */ + val = ( *(pin_base + x_m1 - stride_) + + *(pin_base + x_p1 - stride_) + + *(pin_base + x_m1 + stride_) + + *(pin_base + x_p1 + stride_) ) >> 2; + y_val += RED_Y_MUL * val; + if (y_val > BRIGHT_LVL) ++bright_sum; + if (y_val > TOO_BRIGHT_LVL) ++too_bright_sum; + val = val * rNumerat_ / rDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + } + } + } + + /* calculate the fractions of "bright" and "too bright" pixels */ + stats_.bright_ratio = (float)bright_sum / (h_out * w_out); + stats_.too_bright_ratio = (float)too_bright_sum / (h_out * w_out); + + /* calculate red and blue gains for simple AWB */ + LOG(SoftwareIsp, Debug) + << "sumR = " << sumR << ", sumB = " << sumB << ", sumG = " << sumG; + + sumG /= 2; /* the number of G pixels is twice as big vs R and B ones */ + + /* normalize red, blue, and green sums to fit into 22-bit value */ + unsigned long fRed = sumR / 0x400000; + unsigned long fBlue = sumB / 0x400000; + unsigned long fGreen = sumG / 0x400000; + unsigned long fNorm = std::max({ 1UL, fRed, fBlue, fGreen }); + sumR /= fNorm; + sumB /= fNorm; + sumG /= fNorm; + + LOG(SoftwareIsp, Debug) << "fNorm = " << fNorm; + LOG(SoftwareIsp, Debug) + << "Normalized: sumR = " << sumR + << ", sumB= " << sumB << ", sumG = " << sumG; + + /* make sure red/blue gains never exceed approximately 256 */ + unsigned long minDenom; + rNumerat_ = (sumR + sumB + sumG) / 3; + minDenom = rNumerat_ / 0x100; + rDenomin_ = std::max(minDenom, sumR); + bNumerat_ = rNumerat_; + bDenomin_ = std::max(minDenom, sumB); + gNumerat_ = rNumerat_; + gDenomin_ = std::max(minDenom, sumG); + + LOG(SoftwareIsp, Debug) + << "rGain = [ " << rNumerat_ << " / " << rDenomin_ + << " ], bGain = [ " << bNumerat_ << " / " << bDenomin_ + << " ], gGain = [ " << gNumerat_ << " / " << gDenomin_ + << " (minDenom = " << minDenom << ")"; +} + +SizeRange SwIspLinaro::IspWorker::outSizesRaw10P(const Size &inSize) +{ + if (inSize.width < 2 || inSize.height < 2) { + LOG(SoftwareIsp, Error) + << "Input format size too small: " << inSize.toString(); + return {}; + } + + return SizeRange(Size(inSize.width - 2, inSize.height - 2)); +} + +unsigned int SwIspLinaro::IspWorker::outStrideRaw10P(const Size &outSize) +{ + return outSize.width * 3; +} + +SwIspLinaro::IspWorker::IspWorker(SwIspLinaro *swIsp) + : swIsp_(swIsp) +{ + debayerInfos_[formats::SBGGR10_CSI2P] = { formats::RGB888, + &SwIspLinaro::IspWorker::debayerRaw10P, + &SwIspLinaro::IspWorker::outSizesRaw10P, + &SwIspLinaro::IspWorker::outStrideRaw10P }; + debayerInfos_[formats::SGBRG10_CSI2P] = { formats::RGB888, + &SwIspLinaro::IspWorker::debayerRaw10P, + &SwIspLinaro::IspWorker::outSizesRaw10P, + &SwIspLinaro::IspWorker::outStrideRaw10P }; + debayerInfos_[formats::SGRBG10_CSI2P] = { formats::RGB888, + &SwIspLinaro::IspWorker::debayerRaw10P, + &SwIspLinaro::IspWorker::outSizesRaw10P, + &SwIspLinaro::IspWorker::outStrideRaw10P }; + debayerInfos_[formats::SRGGB10_CSI2P] = { formats::RGB888, + &SwIspLinaro::IspWorker::debayerRaw10P, + &SwIspLinaro::IspWorker::outSizesRaw10P, + &SwIspLinaro::IspWorker::outStrideRaw10P }; +} + +int SwIspLinaro::IspWorker::setDebayerInfo(PixelFormat format) +{ + const auto it = debayerInfos_.find(format); + if (it == debayerInfos_.end()) + return -1; + + debayerInfo_ = &it->second; + return 0; +} + +std::vector SwIspLinaro::IspWorker::formats(PixelFormat input) +{ + std::vector pixelFormats; + + const auto it = debayerInfos_.find(input); + if (it == debayerInfos_.end()) + LOG(SoftwareIsp, Info) + << "Unsupported input format " << input.toString(); + else + pixelFormats.push_back(it->second.outPixelFmt); + + return pixelFormats; +} + +SizeRange SwIspLinaro::IspWorker::sizes(PixelFormat inputFormat, + const Size &inputSize) +{ + const auto it = debayerInfos_.find(inputFormat); + if (it == debayerInfos_.end()) { + LOG(SoftwareIsp, Info) + << "Unsupported input format " << inputFormat.toString(); + return {}; + } + + return (*it->second.getOutSizes)(inputSize); +} + +unsigned int SwIspLinaro::IspWorker::outStride(const PixelFormat &outputFormat, + const Size &outSize) +{ + /* + * Assuming that the output stride depends only on the outputFormat, + * we use the first debayerInfos_ entry with the matching output format + */ + for (auto it = debayerInfos_.begin(); it != debayerInfos_.end(); it++) { + if (it->second.outPixelFmt == outputFormat) + return (*it->second.getOutStride)(outSize); + } + + return 0; +} + +int SwIspLinaro::IspWorker::configure(const StreamConfiguration &inputCfg, + const StreamConfiguration &outputCfg) +{ + if (setDebayerInfo(inputCfg.pixelFormat) != 0) { + LOG(SoftwareIsp, Error) + << "Input format " << inputCfg.pixelFormat + << "not supported"; + return -EINVAL; + } + + /* check that: + * - output format is valid + * - output size matches the input size and is valid */ + SizeRange outSizeRange = (*debayerInfo_->getOutSizes)(inputCfg.size); + if (debayerInfo_->outPixelFmt != outputCfg.pixelFormat || + outputCfg.size.isNull() || !outSizeRange.contains(outputCfg.size) || + (*debayerInfo_->getOutStride)(outputCfg.size) != outputCfg.stride) { + LOG(SoftwareIsp, Error) + << "Invalid output format/size/stride: " + << "\n " << outputCfg.pixelFormat << " (" + << debayerInfo_->outPixelFmt << ")" + << "\n " << outputCfg.size << " (" + << outSizeRange << ")" + << "\n " << outputCfg.stride << " (" + << (*debayerInfo_->getOutStride)(outputCfg.size) << ")"; + return -EINVAL; + } + + width_ = inputCfg.size.width; + height_ = inputCfg.size.height; + stride_ = inputCfg.stride; + + BayerFormat bayerFormat = + BayerFormat::fromPixelFormat(inputCfg.pixelFormat); + switch (bayerFormat.order) { + case BayerFormat::BGGR: + redShift_ = Point(0, 0); + break; + case BayerFormat::GBRG: + redShift_ = Point(1, 0); + break; + case BayerFormat::GRBG: + redShift_ = Point(0, 1); + break; + case BayerFormat::RGGB: + default: + redShift_ = Point(1, 1); + break; + } + + outStride_ = outputCfg.stride; + outHeight_ = outputCfg.size.height; + + LOG(SoftwareIsp, Info) + << "SoftwareISP configuration: " + << inputCfg.size << "-" << inputCfg.pixelFormat << " -> " + << outputCfg.size << "-" << outputCfg.pixelFormat; + + /* set r/g/b gains to 1.0 until frame data collected */ + rNumerat_ = rDenomin_ = 1; + bNumerat_ = bDenomin_ = 1; + gNumerat_ = gDenomin_ = 1; + + return 0; +} + +/* May not be called before SwIspLinaro::IspWorker::configure() */ +unsigned int SwIspLinaro::IspWorker::outBufferSize() +{ + return outHeight_ * outStride_; +} + +std::vector SwIspLinaro::formats(PixelFormat inputFormat) +{ + ASSERT(ispWorker_ != nullptr); + + return ispWorker_->formats(inputFormat); +} + +SizeRange SwIspLinaro::sizes(PixelFormat inputFormat, const Size &inputSize) +{ + ASSERT(ispWorker_ != nullptr); + + return ispWorker_->sizes(inputFormat, inputSize); +} + +std::tuple +SwIspLinaro::strideAndFrameSize(const PixelFormat &outputFormat, const Size &size) +{ + ASSERT(ispWorker_ != nullptr); + + unsigned int stride = ispWorker_->outStride(outputFormat, size); + + return std::make_tuple(stride, stride * size.height); +} + +int SwIspLinaro::configure(const StreamConfiguration &inputCfg, + const std::vector> &outputCfgs) +{ + ASSERT(ispWorker_ != nullptr); + + if (outputCfgs.size() != 1) { + LOG(SoftwareIsp, Error) + << "Unsupported number of output streams: " + << outputCfgs.size(); + return -EINVAL; + } + + return ispWorker_->configure(inputCfg, outputCfgs[0]); +} + +int SwIspLinaro::exportBuffers(unsigned int output, unsigned int count, + std::vector> *buffers) +{ + ASSERT(ispWorker_ != nullptr); + + /* single output for now */ + if (output >= 1) + return -EINVAL; + + unsigned int bufSize = ispWorker_->outBufferSize(); + + /* TODO: allocate from dma_heap; memfd buffs aren't allowed in FrameBuffer */ + for (unsigned int i = 0; i < count; i++) { + std::string name = "frame-" + std::to_string(i); + + const int ispFd = memfd_create(name.c_str(), 0); + int ret = ftruncate(ispFd, bufSize); + if (ret < 0) { + LOG(SoftwareIsp, Error) + << "ftruncate() for memfd failed " + << strerror(-ret); + return ret; + } + + FrameBuffer::Plane outPlane; + outPlane.fd = SharedFD(std::move(ispFd)); + outPlane.offset = 0; + outPlane.length = bufSize; + + std::vector planes{ outPlane }; + buffers->emplace_back(std::make_unique(std::move(planes))); + } + + return count; +} + +int SwIspLinaro::queueBuffers(FrameBuffer *input, + const std::map &outputs) +{ + unsigned int mask = 0; + + /* + * Validate the outputs as a sanity check: at least one output is + * required, all outputs must reference a valid stream and no two + * outputs can reference the same stream. + */ + if (outputs.empty()) + return -EINVAL; + + for (auto [index, buffer] : outputs) { + if (!buffer) + return -EINVAL; + if (index >= 1) /* only single stream atm */ + return -EINVAL; + if (mask & (1 << index)) + return -EINVAL; + + mask |= 1 << index; + } + + process(input, outputs.at(0)); + + return 0; +} + +int SwIspLinaro::start() +{ + ispWorkerThread_.start(); + return 0; +} + +void SwIspLinaro::stop() +{ + ispWorkerThread_.exit(); + ispWorkerThread_.wait(); +} + +void SwIspLinaro::IspWorker::process(FrameBuffer *input, FrameBuffer *output) +{ + /* Copy metadata from the input buffer */ + FrameMetadata &metadata = output->_d()->metadata(); + metadata.status = input->metadata().status; + metadata.sequence = input->metadata().sequence; + metadata.timestamp = input->metadata().timestamp; + + MappedFrameBuffer in(input, MappedFrameBuffer::MapFlag::Read); + MappedFrameBuffer out(output, MappedFrameBuffer::MapFlag::Write); + if (!in.isValid() || !out.isValid()) { + LOG(SoftwareIsp, Error) << "mmap-ing buffer(s) failed"; + metadata.status = FrameMetadata::FrameError; + swIsp_->outputBufferReady.emit(output); + swIsp_->inputBufferReady.emit(input); + return; + } + + (this->*debayerInfo_->debayer)(out.planes()[0].data(), in.planes()[0].data()); + metadata.planes()[0].bytesused = out.planes()[0].size(); + + *swIsp_->sharedStats_ = stats_; + swIsp_->ispStatsReady.emit(0); + + swIsp_->outputBufferReady.emit(output); + swIsp_->inputBufferReady.emit(input); +} + +void SwIspLinaro::process(FrameBuffer *input, FrameBuffer *output) +{ + ispWorker_->invokeMethod(&SwIspLinaro::IspWorker::process, + ConnectionTypeQueued, input, output); +} + +} /* namespace libcamera */ From patchwork Mon Dec 4 00:10:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 19266 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1ED62C322E for ; Mon, 4 Dec 2023 00:10:49 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BDC14629DA; Mon, 4 Dec 2023 01:10:48 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1701648648; bh=TH/uA5CaxgtfBx74YKHme0Y3mbcInfDdz+cHB5vzdpI=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=JObBKND85HLbsUyVOQLmb9IFijvOmpzfxARHuN0euHUDIXhZBVXZQZ9ltr+FVr920 0+D79OWb/w7NEQSqlJWQBcB6YOq1Gdk2zzhAOyTOqarQRIPJVnHaYR4jKkEVF7cN8J jlQUj2AKG7Z5CwpSzvR/ZOGkx2YduVQKEbC0t69ZiwtqSv9ie9QT8G03aoDEaeFX8p oCdGzp1WMPZx0wy1zFByCJLmkeAEuCa2ROrSAxGO/ZwddfbYqbmjlmxAYuW3ccSHPN h0n8wFP63id9klejMYQh0MzXi9g9M3ADDZyqUexIttVw0S8YFva6PZUZyxSNNmZ/So ssVHBtQOJ/tQg== Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C51B7629CE for ; Mon, 4 Dec 2023 01:10:46 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="VtyaEUw4"; dkim-atps=neutral Received: by mail-wr1-x42a.google.com with SMTP id ffacd0b85a97d-3332ad5b3e3so2595480f8f.2 for ; Sun, 03 Dec 2023 16:10:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1701648646; x=1702253446; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=I05whO6yD/81OplpogdCagQbY76H+d64aUOxjzpYx3I=; b=VtyaEUw41yJnGnWBKKOn52AB46pUVpsL053y+XtSh2aiyIc+YfJ3XOnTXnjkrK7sLA /wzapK22w5KMbecK5OTzBC/jtyP6os37VCCII7N2hdpc2mh1JBMVmZwim7ncK5Dj992X Pc21/fHoy+qkjvOUTDNxf7ojwrtLpefKNmzytO2u4o2wfc7oKDucAYd8uV+FGz4rm9b3 e2yrUaP6Xd9VtCzDiMgw49x80/zqoialNVBEeEtsnFQ6uBlBMLQlhAoSyKfxboISCERp v85rNp1l1UsFcTPmRz83Et29GLntbX7zHmy4rU77snljmMpPaQz/SHbP/FcnvL+UXMEX iBUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701648646; x=1702253446; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=I05whO6yD/81OplpogdCagQbY76H+d64aUOxjzpYx3I=; b=CweaKPGj4+8vD3DcV+OP5iRlwVzdbtRBbqfmJRvpN6hdDzFXejRO1Fa1oee2rOuBJT RyzRSlO15rimmsh1xjQ3YgK35YzvAKeU56liktD/PXy4Zj6vwfjvdWaM4myLBauebpXC qWix1JU41+fzqoKpVpr6VneNdupLgG+2RCN7WwJ2u7RKCNGoFea0ltFeNzH4D/+SzC7c 9Z9yinyPqYx35ioYELbdhQ4aw51LWJKaQDFAevyTOq06Xi4BDPvV0jWbauWreGt1uUsZ 04A640y76iFKem+u6t1M656WbFsp34/vKg7jeMqMkuIre4La9uxh5ueQ6LCqTCdjKuM0 8yFw== X-Gm-Message-State: AOJu0Ywyw7kg16TE5kWOZvEJ6dwuenN/gadMa/FKFOhy2DRn7sgwz2Qy /PRg1FEzJ1QXG4rDF48HotMqqNA9lsT3/TdbaRo= X-Google-Smtp-Source: AGHT+IFKSb1MpVQS82/j3nc49GV7icqpt4l8esiYXmiIliACWxjhOX7YKKDxg+fU855wM+DMeM13iQ== X-Received: by 2002:a05:6000:c43:b0:333:4f67:774a with SMTP id do3-20020a0560000c4300b003334f67774amr21377wrb.65.1701648646548; Sun, 03 Dec 2023 16:10:46 -0800 (PST) Received: from Lat-5310.. ([87.116.165.212]) by smtp.gmail.com with ESMTPSA id u29-20020adfa19d000000b003332db7d91dsm8835997wru.39.2023.12.03.16.10.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 03 Dec 2023 16:10:46 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Mon, 4 Dec 2023 03:10:10 +0300 Message-Id: <20231204001013.404720-5-andrey.konovalov@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231204001013.404720-1-andrey.konovalov@linaro.org> References: <20231204001013.404720-1-andrey.konovalov@linaro.org> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 4/7] libcamera: ipa: add Soft IPA X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Andrey Konovalov via libcamera-devel From: Andrey Konovalov Reply-To: Andrey Konovalov Cc: mripard@redhat.com, g.martti@gmail.com, t.langendam@gmail.com, srinivas.kandagatla@linaro.org, bryan.odonoghue@linaro.org, admin@dennisbonke.com Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Signed-off-by: Andrey Konovalov --- include/libcamera/ipa/meson.build | 1 + include/libcamera/ipa/soft.mojom | 27 ++++ meson_options.txt | 3 +- src/ipa/simple/common/meson.build | 17 ++ src/ipa/simple/common/soft_base.cpp | 66 ++++++++ src/ipa/simple/common/soft_base.h | 47 ++++++ src/ipa/simple/linaro/data/meson.build | 8 + src/ipa/simple/linaro/data/soft.conf | 3 + src/ipa/simple/linaro/meson.build | 26 +++ src/ipa/simple/linaro/soft_linaro.cpp | 210 +++++++++++++++++++++++++ src/ipa/simple/meson.build | 12 ++ 11 files changed, 419 insertions(+), 1 deletion(-) create mode 100644 include/libcamera/ipa/soft.mojom create mode 100644 src/ipa/simple/common/meson.build create mode 100644 src/ipa/simple/common/soft_base.cpp create mode 100644 src/ipa/simple/common/soft_base.h create mode 100644 src/ipa/simple/linaro/data/meson.build create mode 100644 src/ipa/simple/linaro/data/soft.conf create mode 100644 src/ipa/simple/linaro/meson.build create mode 100644 src/ipa/simple/linaro/soft_linaro.cpp create mode 100644 src/ipa/simple/meson.build diff --git a/include/libcamera/ipa/meson.build b/include/libcamera/ipa/meson.build index f3b4881c..aaee5cbf 100644 --- a/include/libcamera/ipa/meson.build +++ b/include/libcamera/ipa/meson.build @@ -65,6 +65,7 @@ pipeline_ipa_mojom_mapping = { 'ipu3': 'ipu3.mojom', 'rkisp1': 'rkisp1.mojom', 'rpi/vc4': 'raspberrypi.mojom', + 'simple/linaro': 'soft.mojom', 'vimc': 'vimc.mojom', } diff --git a/include/libcamera/ipa/soft.mojom b/include/libcamera/ipa/soft.mojom new file mode 100644 index 00000000..c3449188 --- /dev/null +++ b/include/libcamera/ipa/soft.mojom @@ -0,0 +1,27 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ + +/* + * \todo Document the interface and remove the related EXCLUDE_PATTERNS entry. + * \todo Add a way to tell SoftIPA the list of params SoftISP accepts? + */ + +module ipa.soft; + +import "include/libcamera/ipa/core.mojom"; + +interface IPASoftInterface { + init(libcamera.IPASettings settings, + libcamera.SharedFD fdStats, + libcamera.ControlInfoMap sensorCtrlInfoMap) + => (int32 ret); + start() => (int32 ret); + stop(); + configure(libcamera.ControlInfoMap sensorCtrlInfoMap) + => (int32 ret); + + [async] processStats(libcamera.ControlList sensorControls); +}; + +interface IPASoftEventInterface { + setSensorControls(libcamera.ControlList sensorControls); +}; diff --git a/meson_options.txt b/meson_options.txt index fad928af..2fb51ff4 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -27,7 +27,7 @@ option('gstreamer', option('ipas', type : 'array', - choices : ['ipu3', 'rkisp1', 'rpi/vc4', 'vimc'], + choices : ['ipu3', 'rkisp1', 'rpi/vc4', 'simple/linaro', 'vimc'], description : 'Select which IPA modules to build') option('lc-compliance', @@ -46,6 +46,7 @@ option('pipelines', 'rkisp1', 'rpi/vc4', 'simple', + 'simple/linaro', 'uvcvideo', 'vimc' ], diff --git a/src/ipa/simple/common/meson.build b/src/ipa/simple/common/meson.build new file mode 100644 index 00000000..023e617b --- /dev/null +++ b/src/ipa/simple/common/meson.build @@ -0,0 +1,17 @@ +# SPDX-License-Identifier: CC0-1.0 + +soft_ipa_common_sources = files([ + 'soft_base.cpp', +]) + +soft_ipa_common_includes = [ + include_directories('..'), +] + +soft_ipa_common_deps = [ + libcamera_private, +] + +soft_ipa_common_lib = static_library('soft_ipa_common', soft_ipa_common_sources, + include_directories : soft_ipa_common_includes, + dependencies : soft_ipa_common_deps) diff --git a/src/ipa/simple/common/soft_base.cpp b/src/ipa/simple/common/soft_base.cpp new file mode 100644 index 00000000..7bd9b8de --- /dev/null +++ b/src/ipa/simple/common/soft_base.cpp @@ -0,0 +1,66 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2023, Linaro Ltd + * + * soft-base.cpp - Software IPA base class + */ + +#include "soft_base.h" + +#include + +#include +#include + +#include + +namespace libcamera { + +LOG_DEFINE_CATEGORY(IPASoft) + +namespace ipa::soft { + +IPASoftBase::IPASoftBase() +{ +} + +IPASoftBase::~IPASoftBase() +{ +} + +int IPASoftBase::init([[maybe_unused]] const IPASettings &settings, + const SharedFD &fdStats, + const ControlInfoMap &sensorInfoMap) +{ + fdStats_ = std::move(fdStats); + if (!fdStats_.isValid()) { + LOG(IPASoft, Error) << "Invalid Statistics handle"; + return -ENODEV; + } + + return platformInit(sensorInfoMap); +} + +int IPASoftBase::configure(const ControlInfoMap &sensorInfoMap) +{ + return platformConfigure(sensorInfoMap); +} + +int IPASoftBase::start() +{ + return platformStart(); +} + +void IPASoftBase::stop() +{ + return platformStop(); +} + +void IPASoftBase::processStats(const ControlList &sensorControls) +{ + return platformProcessStats(sensorControls); +} + +} /* namespace ipa::soft */ + +} /* namespace libcamera */ diff --git a/src/ipa/simple/common/soft_base.h b/src/ipa/simple/common/soft_base.h new file mode 100644 index 00000000..bff53713 --- /dev/null +++ b/src/ipa/simple/common/soft_base.h @@ -0,0 +1,47 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2023, Linaro Ltd + * + * soft-base.h - Software IPA base class + */ +#pragma once + +#include +#include + +#include + +namespace libcamera { + +namespace ipa::soft { + +class IPASoftBase : public ipa::soft::IPASoftInterface +{ +public: + IPASoftBase(); + ~IPASoftBase(); + + int init(const IPASettings &settings, + const SharedFD &fdStats, + const ControlInfoMap &sensorInfoMap) override; + int configure(const ControlInfoMap &sensorInfoMap) override; + + int start() override; + void stop() override; + + void processStats(const ControlList &sensorControls) override; + +protected: + SharedFD fdStats_; + +private: + virtual int platformInit(const ControlInfoMap &sensorInfoMap) = 0; + virtual int platformConfigure(const ControlInfoMap &sensorInfoMap) = 0; + virtual int platformStart() = 0; + virtual void platformStop() = 0; + virtual void platformProcessStats(const ControlList &sensorControls) = 0; +}; + +} /* namespace ipa::soft */ + +} /* namespace libcamera */ diff --git a/src/ipa/simple/linaro/data/meson.build b/src/ipa/simple/linaro/data/meson.build new file mode 100644 index 00000000..f3464375 --- /dev/null +++ b/src/ipa/simple/linaro/data/meson.build @@ -0,0 +1,8 @@ +# SPDX-License-Identifier: CC0-1.0 + +conf_files = files([ + 'soft.conf', +]) + +install_data(conf_files, + install_dir : ipa_data_dir / 'soft') diff --git a/src/ipa/simple/linaro/data/soft.conf b/src/ipa/simple/linaro/data/soft.conf new file mode 100644 index 00000000..0c70e7c0 --- /dev/null +++ b/src/ipa/simple/linaro/data/soft.conf @@ -0,0 +1,3 @@ +# SPDX-License-Identifier: LGPL-2.1-or-later +# +# Dummy configuration file for the soft IPA. diff --git a/src/ipa/simple/linaro/meson.build b/src/ipa/simple/linaro/meson.build new file mode 100644 index 00000000..97bf5d6f --- /dev/null +++ b/src/ipa/simple/linaro/meson.build @@ -0,0 +1,26 @@ +# SPDX-License-Identifier: CC0-1.0 + +ipa_name = 'ipa_soft_linaro' + +mod = shared_module(ipa_name, + ['soft_linaro.cpp', libcamera_generated_ipa_headers], + name_prefix : '', + include_directories : [ipa_includes, libipa_includes, '..'], + dependencies : libcamera_private, + link_with : libipa, + link_whole : soft_ipa_common_lib, + install : true, + install_dir : ipa_install_dir) + +if ipa_sign_module + custom_target(ipa_name + '.so.sign', + input : mod, + output : ipa_name + '.so.sign', + command : [ipa_sign, ipa_priv_key, '@INPUT@', '@OUTPUT@'], + install : false, + build_by_default : true) +endif + +subdir('data') + +ipa_names += ipa_name diff --git a/src/ipa/simple/linaro/soft_linaro.cpp b/src/ipa/simple/linaro/soft_linaro.cpp new file mode 100644 index 00000000..0b2e83bf --- /dev/null +++ b/src/ipa/simple/linaro/soft_linaro.cpp @@ -0,0 +1,210 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2023, Linaro Ltd + * + * soft.cpp - Software Image Processing Algorithm module + */ + +#include + +#include +#include + +#include + +#include +#include + +#include "libcamera/internal/camera_sensor.h" +#include "libcamera/internal/software_isp/statistics-linaro.h" + +#include "common/soft_base.h" + +namespace libcamera { + +LOG_DECLARE_CATEGORY(IPASoft) + +namespace ipa::soft { + +class IPASoftLinaro final : public IPASoftBase +{ +public: + IPASoftLinaro() + : IPASoftBase(), ignore_updates_(0) + { + } + + ~IPASoftLinaro() + { + if (stats_) + munmap(stats_, sizeof(SwIspStats)); + } + + int platformInit(const ControlInfoMap &sensorInfoMap) override; + int platformConfigure(const ControlInfoMap &sensorInfoMap) override; + int platformStart() override; + void platformStop() override; + void platformProcessStats(const ControlList &sensorControls) override; + +private: + void update_exposure(double ev_adjustment); + + SwIspStats *stats_; + int exposure_min_, exposure_max_; + int again_min_, again_max_; + int again_, exposure_; + int ignore_updates_; +}; + +int IPASoftLinaro::platformInit(const ControlInfoMap &sensorInfoMap) +{ + stats_ = static_cast(mmap(nullptr, sizeof(SwIspStats), + PROT_READ | PROT_WRITE, MAP_SHARED, + fdStats_.get(), 0)); + if (!stats_) { + LOG(IPASoft, Error) << "Unable to map Statistics"; + return -ENODEV; + } + + if (sensorInfoMap.find(V4L2_CID_EXPOSURE) == sensorInfoMap.end()) { + LOG(IPASoft, Error) << "Don't have exposure control"; + return -EINVAL; + } + + if (sensorInfoMap.find(V4L2_CID_ANALOGUE_GAIN) == sensorInfoMap.end()) { + LOG(IPASoft, Error) << "Don't have gain control"; + return -EINVAL; + } + + return 0; +} + +int IPASoftLinaro::platformConfigure(const ControlInfoMap &sensorInfoMap) +{ + const ControlInfo &exposure_info = sensorInfoMap.find(V4L2_CID_EXPOSURE)->second; + const ControlInfo &gain_info = sensorInfoMap.find(V4L2_CID_ANALOGUE_GAIN)->second; + + exposure_min_ = exposure_info.min().get(); + if (!exposure_min_) { + LOG(IPASoft, Warning) << "Minimum exposure is zero, that can't be linear"; + exposure_min_ = 1; + } + exposure_max_ = exposure_info.max().get(); + again_min_ = gain_info.min().get(); + if (!again_min_) { + LOG(IPASoft, Warning) << "Minimum gain is zero, that can't be linear"; + again_min_ = 100; + } + again_max_ = gain_info.max().get(); + + LOG(IPASoft, Info) << "Exposure " << exposure_min_ << "-" << exposure_max_ + << ", gain " << again_min_ << "-" << again_max_; + + return 0; +} + +int IPASoftLinaro::platformStart() +{ + return 0; +} + +void IPASoftLinaro::platformStop() +{ +} + +void IPASoftLinaro::platformProcessStats(const ControlList &sensorControls) +{ + double ev_adjustment = 0.0; + ControlList ctrls(sensorControls); + + /* + * Use 2 frames delay to make sure that the exposure and the gain set + * have applied to the camera sensor + */ + if (ignore_updates_ > 0) { + LOG(IPASoft, Debug) << "Skipping exposure update: " + << ignore_updates_; + --ignore_updates_; + return; + } + + if (stats_->bright_ratio < 0.01) + ev_adjustment = 1.1; + if (stats_->too_bright_ratio > 0.04) + ev_adjustment = 0.9; + + if (ev_adjustment != 0.0) { + /* sanity check */ + if (!sensorControls.contains(V4L2_CID_EXPOSURE) || + !sensorControls.contains(V4L2_CID_ANALOGUE_GAIN)) { + LOG(IPASoft, Error) << "Control(s) missing"; + return; + } + + exposure_ = ctrls.get(V4L2_CID_EXPOSURE).get(); + again_ = ctrls.get(V4L2_CID_ANALOGUE_GAIN).get(); + + update_exposure(ev_adjustment); + + ctrls.set(V4L2_CID_EXPOSURE, exposure_); + ctrls.set(V4L2_CID_ANALOGUE_GAIN, again_); + + ignore_updates_ = 2; + + setSensorControls.emit(ctrls); + } +} + +void IPASoftLinaro::update_exposure(double ev_adjustment) +{ + double exp = (double)exposure_; + double gain = (double)again_; + double ev = ev_adjustment * exp * gain; + + /* + * Try to use the minimal possible analogue gain. + * The exposure can be any value from exposure_min_ to exposure_max_, + * and normally this should keep the frame rate intact. + */ + + exp = ev / again_min_; + if (exp > exposure_max_) + exposure_ = exposure_max_; + else if (exp < exposure_min_) + exposure_ = exposure_min_; + else + exposure_ = (int)exp; + + gain = ev / exposure_; + if (gain > again_max_) + again_ = again_max_; + else if (gain < again_min_) + again_ = again_min_; + else + again_ = (int)gain; + + LOG(IPASoft, Debug) << "Desired EV = " << ev + << ", real EV = " << (double)again_ * exposure_; +} + +} /* namespace ipa::soft */ + +/* + * External IPA module interface + */ +extern "C" { +const struct IPAModuleInfo ipaModuleInfo = { + IPA_MODULE_API_VERSION, + 0, + "SimplePipelineHandler", + "soft/linaro", +}; + +IPAInterface *ipaCreate() +{ + return new ipa::soft::IPASoftLinaro(); +} + +} /* extern "C" */ + +} /* namespace libcamera */ diff --git a/src/ipa/simple/meson.build b/src/ipa/simple/meson.build new file mode 100644 index 00000000..14be5dc2 --- /dev/null +++ b/src/ipa/simple/meson.build @@ -0,0 +1,12 @@ +# SPDX-License-Identifier: CC0-1.0 + +subdir('common') + +foreach pipeline : pipelines + pipeline = pipeline.split('/') + if pipeline.length() < 2 or pipeline[0] != 'simple' + continue + endif + + subdir(pipeline[1]) +endforeach From patchwork Mon Dec 4 00:10:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 19267 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 75BF1C32BC for ; Mon, 4 Dec 2023 00:10:50 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6969F629DB; Mon, 4 Dec 2023 01:10:49 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1701648649; bh=Kp5Os7saySrxWS5dhphg6mg+/T0P0d0hWZqdPlc1kVw=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=rE1oFNBITicOLqrPZaRF2GmaSJeCmLqbNYAFPIyVlNOE0V9EbT2H1eXCes6ZVY7kf 9T/YFjWwzMrDbRcRVMo+IGteINMjZTHiYLNqjd+8kTRLRrxC/4dbdpPTVTs4wRklVg A0oZ2bgjrDctIM9jziYuwuZbVPefinHCcvuUlQtBzmt1bK/kOhPyTMu/nWFOY1DNpm UEY7Ov5jTFcHIMl8cAKYao8DooATGbfFaVlPfZGacekOKe5BRP+ZqHT+UU2orriLos 6ip7oG/emVJn9OaVyk4ICdDu3Xy/keY3oaHL3052POks84zr+ZPzLm9HgUZ/du9ebJ B1aJE3zF+CKLA== Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6938C629CE for ; Mon, 4 Dec 2023 01:10:48 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="lCGkwUS4"; dkim-atps=neutral Received: by mail-wm1-x329.google.com with SMTP id 5b1f17b1804b1-40c07ed92fdso10505445e9.3 for ; Sun, 03 Dec 2023 16:10:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1701648648; x=1702253448; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=KQDxGYxBvF+f14bMPWP5VTUP7nmNuM1+782hhR4rrGo=; b=lCGkwUS4g9UfjuRb91eNgCgwlBodZj3ESQp9tBJeb/dD+gDlUdZ+cQzVEsr/sqTJRJ tdyJqjoZ29xGNRm40ANf2ku/YDR1iY1cjcAp9x9XujcQWchOI4QOeCMigRH9z/6XmDFs yJQa8bPGRWZOsEqqdC7N1T4trcMresdGIu3Ux3/dDTjgyK6uHR53+cLJ4qauaV3Ac2Ob szs+Vziy065QiY8gG5+RJGCiG7gwpcqV05WXHdxra2ucCV09w5HuRevLkEr8hzUBaYjv X1gz8qkuL2Wjm1Q98aoBNoFlBybgjK7N6OUiXhz/sQOQ/DDww9cZlYmuBBG6UoR79lkU AGfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701648648; x=1702253448; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KQDxGYxBvF+f14bMPWP5VTUP7nmNuM1+782hhR4rrGo=; b=t/gyMoW0MhcwmJWbsVrEHg1QIwJ6d3NU46j8st/SObAs+7fMr8sDSycDJi8IQUClGw Z0LmUqFdwNrOhcGqBRTm0vNHwawg65plCTVTxxVU5Zou9wmz2KJS275LPx9mAN07aE6H slDF7/rFXHtwvgQHlVyunE/HHrTXehYzYIWECKUHqy1cZ3VN/7AJCpmItN25HRsbzVMF +52Mbbia74zqMT515OPbZlJLejOEAPjhgCvYF1fnLig04k7l4wxiKra45STGbF3Zgb/g P71x1nP1UNaG2Be2QYCXHJYOLB352VXUcQc2K50/lEGMfnpYMjkuQ+yCEJ8r6oLpji42 hBtw== X-Gm-Message-State: AOJu0YwJTC7spNXGJhpA752tc219LblixtQquI4VgfvEb23LDJKkZ5BB 69mH8/zysj75znQTTiC7LSBUXgdmbjoRSnr6QMA= X-Google-Smtp-Source: AGHT+IFHqTGXO96FaS0N29uJYsWqohsKa0W83GkywXS5XrEtlhCM4F+meZ8CzbczgmotlPMgbJz6iw== X-Received: by 2002:a05:600c:3422:b0:40b:5e59:b7d5 with SMTP id y34-20020a05600c342200b0040b5e59b7d5mr2114673wmp.178.1701648648200; Sun, 03 Dec 2023 16:10:48 -0800 (PST) Received: from Lat-5310.. ([87.116.165.212]) by smtp.gmail.com with ESMTPSA id u29-20020adfa19d000000b003332db7d91dsm8835997wru.39.2023.12.03.16.10.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 03 Dec 2023 16:10:48 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Mon, 4 Dec 2023 03:10:11 +0300 Message-Id: <20231204001013.404720-6-andrey.konovalov@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231204001013.404720-1-andrey.konovalov@linaro.org> References: <20231204001013.404720-1-andrey.konovalov@linaro.org> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 5/7] libcamera: software_isp: introduce getStatsFD() X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Andrey Konovalov via libcamera-devel From: Andrey Konovalov Reply-To: Andrey Konovalov Cc: mripard@redhat.com, g.martti@gmail.com, t.langendam@gmail.com, srinivas.kandagatla@linaro.org, bryan.odonoghue@linaro.org, admin@dennisbonke.com Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The shared memory for passing the statistics to the Soft IPA is allocated inside the Soft ISP. The FD of this memory needs to be passed to the IPA. Signed-off-by: Andrey Konovalov --- include/libcamera/internal/software_isp.h | 4 ++++ include/libcamera/internal/software_isp/swisp_linaro.h | 2 ++ 2 files changed, 6 insertions(+) diff --git a/include/libcamera/internal/software_isp.h b/include/libcamera/internal/software_isp.h index 556203d9..2ce3c68a 100644 --- a/include/libcamera/internal/software_isp.h +++ b/include/libcamera/internal/software_isp.h @@ -16,6 +16,7 @@ #include #include +#include #include #include @@ -53,6 +54,9 @@ public: virtual int queueBuffers(FrameBuffer *input, const std::map &outputs) = 0; + /* A hack to pass the FD from Soft ISP to Soft IPA */ + virtual const SharedFD &getStatsFD() = 0; + Signal inputBufferReady; Signal outputBufferReady; diff --git a/include/libcamera/internal/software_isp/swisp_linaro.h b/include/libcamera/internal/software_isp/swisp_linaro.h index 0d8a31a6..157898a8 100644 --- a/include/libcamera/internal/software_isp/swisp_linaro.h +++ b/include/libcamera/internal/software_isp/swisp_linaro.h @@ -44,6 +44,8 @@ public: int queueBuffers(FrameBuffer *input, const std::map &outputs); + const SharedFD &getStatsFD() { return sharedStats_.fd(); } + void process(FrameBuffer *input, FrameBuffer *output); private: From patchwork Mon Dec 4 00:10:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 19268 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7AB20C3226 for ; Mon, 4 Dec 2023 00:10:52 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2910A629D2; Mon, 4 Dec 2023 01:10:52 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1701648652; bh=fjl8sMSFfkzTBOcgHmEeNYc0wONBoMHUPlxMl7V61oA=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=XEFp4p2jzwrSpSqycKLkUSAtPLfCLXMt+oLP3TzuHNXAJKLrDwLBRWTyro7B4e5Ku Y1B92qOhySjWVHizde4EAsGRxygrPjPp9l1rww1YDr9u5E1w2DyeZQxYXARD5DBCII d2+PbTss21XTeSbkIsOHDdOLa1pxX69CNAZmChZdYmcdn/tp9Es3askvqbThuG8e1x 7+7q8rbC6LUUCj+3QYjbVeZZffH+OsHX+EIzPnOhYKL7gE+8fm9cXTQeDv8dl0ntQC ZurnDW86J0xs6F+CW2wRWNTgN9d6JAS9cvumJCmpnv1MBQnGZ2rlfa2Wq/Q8IvMu1J VKjXC1kE9lWyQ== Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com [IPv6:2a00:1450:4864:20::32a]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 32040629D4 for ; Mon, 4 Dec 2023 01:10:50 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="rMyTDAN5"; dkim-atps=neutral Received: by mail-wm1-x32a.google.com with SMTP id 5b1f17b1804b1-40c0873cf84so9980935e9.1 for ; Sun, 03 Dec 2023 16:10:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1701648650; x=1702253450; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Ul3YjVRAwDCJplrzfkmo/zpiwDH9NzQCnUUAZU0wH4I=; b=rMyTDAN5Il4YulVA8EXKdIQogfBPkGA+DXLuyC2h0+ff4UCZb4stCX9lGK3/hsXI8m qClkd425EUBQa5/pwaV+bxc7VYoP8dibmPKL+m60aXNp6jrGgkV9w8opPSB6h/MAuTci x51CwOA599bla5xVsrH9Kx5s9yadbkTETWZ5lNj7BT9DWrAZU8LvHDluxiJEPkkxIShP s3B5wPcVzep78F3c2Gi8AYVDDU8270WWi1LKQ28LP4p5vHEIrO4Woci4zKK3Pu6klLAX zbPk6K8xkyB47cdDD7wNr4aD70YbEwIU6+iTZNu1ULdE8Ld0cj6wGErw2UXNELNTIUF+ Cuzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701648650; x=1702253450; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ul3YjVRAwDCJplrzfkmo/zpiwDH9NzQCnUUAZU0wH4I=; b=YgvETOd6IX69QYhXTjkIJxkKWHB7JGP/MvUzgCayNP/YybEKEtzGcFqg/yZHsgIYm9 IP9HxundOaM08znVxl382oBF7TCk6qfrh9w0xrH4xJkSib2tfBDUX8ve5txFcYwWmHGr hEJIzB7ni9vpatgLPqwRmEzq15rMb1jBgEmlAg1hKLlWDQeT3GRRaGs2vnvpYoPPHI9V LW4Ah1aTS8HHDeYyB6hHxlSl53r5wmBRvtk7F5P7dsiSmFJjIUbDO4o50K5wCp/eqLBq AkfKmZFUsNlkdoPETThue6+rzDetuRrSLhkgVSg2PEslI5SOF4VKl3vnko6uONvTZ9Gj VRvQ== X-Gm-Message-State: AOJu0Ywigf3m1aQmRaxxQOHojiYFeXoXzjDLm85x9cWZTvGUeqhP/rA9 thbUcRc5sGgoRqZJr0GhneMrG/rvPtGa94YmmgA= X-Google-Smtp-Source: AGHT+IG3XJdCtEoQWkAIW3GS7k8YeKBwDYHUpTbT4QwupC0WekEdIidnfCj/aEEl+j7Vg7eTvi4kEA== X-Received: by 2002:a05:600c:3d19:b0:40c:5a4:b0dc with SMTP id bh25-20020a05600c3d1900b0040c05a4b0dcmr1393199wmb.113.1701648649784; Sun, 03 Dec 2023 16:10:49 -0800 (PST) Received: from Lat-5310.. ([87.116.165.212]) by smtp.gmail.com with ESMTPSA id u29-20020adfa19d000000b003332db7d91dsm8835997wru.39.2023.12.03.16.10.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 03 Dec 2023 16:10:49 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Mon, 4 Dec 2023 03:10:12 +0300 Message-Id: <20231204001013.404720-7-andrey.konovalov@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231204001013.404720-1-andrey.konovalov@linaro.org> References: <20231204001013.404720-1-andrey.konovalov@linaro.org> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 6/7] libcamera: pipeline: simple: rename converterBuffers_ and related vars X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Andrey Konovalov via libcamera-devel From: Andrey Konovalov Reply-To: Andrey Konovalov Cc: mripard@redhat.com, g.martti@gmail.com, t.langendam@gmail.com, srinivas.kandagatla@linaro.org, bryan.odonoghue@linaro.org, admin@dennisbonke.com Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The converterBuffers_ and the converterQueue_ are not that specific to the Converter, and could be used by another entity doing the format conversion. Rename converterBuffers_, converterQueue_, and useConverter_ to conversionBuffers_, conversionQueue_ and useConversion_ to disassociate them from the Converter. Signed-off-by: Andrey Konovalov --- src/libcamera/pipeline/simple/simple.cpp | 63 ++++++++++++------------ 1 file changed, 32 insertions(+), 31 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 911051b2..878794f6 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -269,17 +269,18 @@ public: std::vector configs_; std::map> formats_; + std::vector> conversionBuffers_; + std::queue> conversionQueue_; + bool useConversion_; + std::unique_ptr converter_; - std::vector> converterBuffers_; - bool useConverter_; - std::queue> converterQueue_; private: void tryPipeline(unsigned int code, const Size &size); static std::vector routedSourcePads(MediaPad *sink); - void converterInputDone(FrameBuffer *buffer); - void converterOutputDone(FrameBuffer *buffer); + void conversionInputDone(FrameBuffer *buffer); + void conversionOutputDone(FrameBuffer *buffer); }; class SimpleCameraConfiguration : public CameraConfiguration @@ -503,8 +504,8 @@ int SimpleCameraData::init() << "Failed to create converter, disabling format conversion"; converter_.reset(); } else { - converter_->inputBufferReady.connect(this, &SimpleCameraData::converterInputDone); - converter_->outputBufferReady.connect(this, &SimpleCameraData::converterOutputDone); + converter_->inputBufferReady.connect(this, &SimpleCameraData::conversionInputDone); + converter_->outputBufferReady.connect(this, &SimpleCameraData::conversionOutputDone); } } @@ -740,7 +741,7 @@ void SimpleCameraData::bufferReady(FrameBuffer *buffer) * point converting an erroneous buffer. */ if (buffer->metadata().status != FrameMetadata::FrameSuccess) { - if (!useConverter_) { + if (!useConversion_) { /* No conversion, just complete the request. */ Request *request = buffer->request(); pipe->completeBuffer(request, buffer); @@ -756,16 +757,16 @@ void SimpleCameraData::bufferReady(FrameBuffer *buffer) if (buffer->metadata().status != FrameMetadata::FrameCancelled) video_->queueBuffer(buffer); - if (converterQueue_.empty()) + if (conversionQueue_.empty()) return; Request *request = nullptr; - for (auto &item : converterQueue_.front()) { + for (auto &item : conversionQueue_.front()) { FrameBuffer *outputBuffer = item.second; request = outputBuffer->request(); pipe->completeBuffer(request, outputBuffer); } - converterQueue_.pop(); + conversionQueue_.pop(); if (request) pipe->completeRequest(request); @@ -782,9 +783,9 @@ void SimpleCameraData::bufferReady(FrameBuffer *buffer) */ Request *request = buffer->request(); - if (useConverter_ && !converterQueue_.empty()) { + if (useConversion_ && !conversionQueue_.empty()) { const std::map &outputs = - converterQueue_.front(); + conversionQueue_.front(); if (!outputs.empty()) { FrameBuffer *outputBuffer = outputs.begin()->second; if (outputBuffer) @@ -801,14 +802,14 @@ void SimpleCameraData::bufferReady(FrameBuffer *buffer) * conversion is needed. If there's no queued request, just requeue the * captured buffer for capture. */ - if (useConverter_) { - if (converterQueue_.empty()) { + if (useConversion_) { + if (conversionQueue_.empty()) { video_->queueBuffer(buffer); return; } - converter_->queueBuffers(buffer, converterQueue_.front()); - converterQueue_.pop(); + converter_->queueBuffers(buffer, conversionQueue_.front()); + conversionQueue_.pop(); return; } @@ -817,13 +818,13 @@ void SimpleCameraData::bufferReady(FrameBuffer *buffer) pipe->completeRequest(request); } -void SimpleCameraData::converterInputDone(FrameBuffer *buffer) +void SimpleCameraData::conversionInputDone(FrameBuffer *buffer) { /* Queue the input buffer back for capture. */ video_->queueBuffer(buffer); } -void SimpleCameraData::converterOutputDone(FrameBuffer *buffer) +void SimpleCameraData::conversionOutputDone(FrameBuffer *buffer) { SimplePipelineHandler *pipe = SimpleCameraData::pipe(); @@ -1156,14 +1157,14 @@ int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c) /* Configure the converter if needed. */ std::vector> outputCfgs; - data->useConverter_ = config->needConversion(); + data->useConversion_ = config->needConversion(); for (unsigned int i = 0; i < config->size(); ++i) { StreamConfiguration &cfg = config->at(i); cfg.setStream(&data->streams_[i]); - if (data->useConverter_) + if (data->useConversion_) outputCfgs.push_back(cfg); } @@ -1189,7 +1190,7 @@ int SimplePipelineHandler::exportFrameBuffers(Camera *camera, Stream *stream, * Export buffers on the converter or capture video node, depending on * whether the converter is used or not. */ - if (data->useConverter_) + if (data->useConversion_) return data->converter_->exportBuffers(data->streamIndex(stream), count, buffers); else @@ -1210,13 +1211,13 @@ int SimplePipelineHandler::start(Camera *camera, [[maybe_unused]] const ControlL return -EBUSY; } - if (data->useConverter_) { + if (data->useConversion_) { /* * When using the converter allocate a fixed number of internal * buffers. */ ret = video->allocateBuffers(kNumInternalBuffers, - &data->converterBuffers_); + &data->conversionBuffers_); } else { /* Otherwise, prepare for using buffers from the only stream. */ Stream *stream = &data->streams_[0]; @@ -1235,7 +1236,7 @@ int SimplePipelineHandler::start(Camera *camera, [[maybe_unused]] const ControlL return ret; } - if (data->useConverter_) { + if (data->useConversion_) { ret = data->converter_->start(); if (ret < 0) { stop(camera); @@ -1243,7 +1244,7 @@ int SimplePipelineHandler::start(Camera *camera, [[maybe_unused]] const ControlL } /* Queue all internal buffers for capture. */ - for (std::unique_ptr &buffer : data->converterBuffers_) + for (std::unique_ptr &buffer : data->conversionBuffers_) video->queueBuffer(buffer.get()); } @@ -1255,7 +1256,7 @@ void SimplePipelineHandler::stopDevice(Camera *camera) SimpleCameraData *data = cameraData(camera); V4L2VideoDevice *video = data->video_; - if (data->useConverter_) + if (data->useConversion_) data->converter_->stop(); video->streamOff(); @@ -1263,7 +1264,7 @@ void SimplePipelineHandler::stopDevice(Camera *camera) video->bufferReady.disconnect(data, &SimpleCameraData::bufferReady); - data->converterBuffers_.clear(); + data->conversionBuffers_.clear(); releasePipeline(data); } @@ -1281,7 +1282,7 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request) * queue, it will be handed to the converter in the capture * completion handler. */ - if (data->useConverter_) { + if (data->useConversion_) { buffers.emplace(data->streamIndex(stream), buffer); } else { ret = data->video_->queueBuffer(buffer); @@ -1290,8 +1291,8 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request) } } - if (data->useConverter_) - data->converterQueue_.push(std::move(buffers)); + if (data->useConversion_) + data->conversionQueue_.push(std::move(buffers)); return 0; } From patchwork Mon Dec 4 00:10:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 19269 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 14353C322E for ; Mon, 4 Dec 2023 00:10:54 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A20D6629DB; Mon, 4 Dec 2023 01:10:53 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1701648653; bh=X2oPJy3GZr8rWVQz96jg/g4d2q0hf8JwiEbzhCfdJ2M=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=1LTKBiJQxrgY+1a3OWd8VsSTZ0zOiIozDMlxBsgUO0pXQ8ns8JMBCpSFEwVxKqsRC sU/DNa97+K3R6RLk41Lf17IdBja0+HGOpPOpivEUx/KAX+L9MRbNI6NLz6hitSGQNX 3tVDrHtO7LvolabtqHkCMW9+UvaajGzc+3/L/ZkTjqu17nd556pGf7RAOvlKLnlcgN 4T3+tSx3uCU4gR8WJ6QT9DPTbrcraYondw+ZFsMmljmMDCLajxXxgBH3kvKumuHQdm 6q9z5WamAye2Xk5wRvMtjQt+Uc6mt1+aMoAhhFmq7cy/oZUpWRgqIlFd+G/5ZOvt6L zFzJz1VfOUYKQ== Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B5B67629D6 for ; Mon, 4 Dec 2023 01:10:52 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="lU2VD5s0"; dkim-atps=neutral Received: by mail-wr1-x432.google.com with SMTP id ffacd0b85a97d-3316d09c645so3751104f8f.0 for ; Sun, 03 Dec 2023 16:10:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1701648652; x=1702253452; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=uWT6dMbaiuLENDzwFc4xPPz9KU9sfJhCzfIGiy01m4M=; b=lU2VD5s0eoTvLBuWNtBd/xhV3x1312d9e+V/YvJn7kQP62UCvnBNVWA2/89nmawLza RkovyqnwksnDu4q30D98mV2TH9zCtjEAVU7fvH2p1YfJakqURtak2z06SQ181nl0U32F 8UNLHAhABdgKuxAIbAdd8KlstK67zABHAXIAbaMvTspYfGFIsCV0GrqGQ+xi+Xmcy5KC eY9ms355JXFbDsdnQDB4DS3+pddZjCoG+WaN1BWrcYpYdwQ7PYeFg7Pz80xKssgDHsCx kd7WnRPV8r3mgmw/jO1wmtja/eS8fYOCN3aJ8jERvoEdqzHTH1qZ04UngsxBPWhxz7B8 Sb8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701648652; x=1702253452; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=uWT6dMbaiuLENDzwFc4xPPz9KU9sfJhCzfIGiy01m4M=; b=tTxCr539IBebJgHygsQKqIF54jKdA0Ggk+wJ9NeDuu6UIfRGhWWbeS+nkXG6UjqOUS +JIKQD/OeMiB2gZR1iAYkssAOKbxYb7DXqe8O4RZTm7UlwygZsyLDA1zbsIvoe9mFqVN /Ln/lhVmyLmOLLRGfhtb6xEz2YRpaswJTtNRgVPRQUhjgmjYTYhjGQjDviw/8VCsq5Af ge9lKvBhyhg6XnZ8r60qEE6rOQWUM/p6NTIcKP7YZp3mtCvqhnjcOAga51aUSKBcCqNI GBE1YJesdxFEqkn/VPOs+m0Jgp1YmxiecRnY1aj/5Vn4GQ+LuS9hp7MhOmvg4lFUhpCi qr8A== X-Gm-Message-State: AOJu0YwsGL/iLoU410JPnAWu57zsTP6+tIL8LgPLRKIkzCE1b2eQLVf8 txlhycslucYeNuWppdRpoj+Sm3xScMV11PbJPlc= X-Google-Smtp-Source: AGHT+IHBx/C1Kxp4bUv5MC/KL5sbrtJM9xvxvGO2tDRIUWyJiv8Tnv+6ckcymJb3yIONjJNX3uVVzA== X-Received: by 2002:adf:e5c9:0:b0:333:2fd2:4afb with SMTP id a9-20020adfe5c9000000b003332fd24afbmr2384787wrn.119.1701648652407; Sun, 03 Dec 2023 16:10:52 -0800 (PST) Received: from Lat-5310.. ([87.116.165.212]) by smtp.gmail.com with ESMTPSA id u29-20020adfa19d000000b003332db7d91dsm8835997wru.39.2023.12.03.16.10.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 03 Dec 2023 16:10:52 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Mon, 4 Dec 2023 03:10:13 +0300 Message-Id: <20231204001013.404720-8-andrey.konovalov@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231204001013.404720-1-andrey.konovalov@linaro.org> References: <20231204001013.404720-1-andrey.konovalov@linaro.org> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 7/7] [DNI] libcamera: pipeline: simple: enable use of SoftwareISP and Soft IPA X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Andrey Konovalov via libcamera-devel From: Andrey Konovalov Reply-To: Andrey Konovalov Cc: mripard@redhat.com, g.martti@gmail.com, t.langendam@gmail.com, srinivas.kandagatla@linaro.org, bryan.odonoghue@linaro.org, admin@dennisbonke.com Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Some quick changes to make SoftwareISP and Soft IPA usable in the cases when the pipeline doesn't use Converter. Signed-off-by: Andrey Konovalov --- src/libcamera/pipeline/simple/simple.cpp | 134 +++++++++++++++++++---- 1 file changed, 111 insertions(+), 23 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 878794f6..8c119a98 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -25,6 +25,8 @@ #include #include +#include +#include #include #include @@ -32,12 +34,14 @@ #include "libcamera/internal/camera_sensor.h" #include "libcamera/internal/converter.h" #include "libcamera/internal/device_enumerator.h" +#include "libcamera/internal/ipa_manager.h" #include "libcamera/internal/media_device.h" #include "libcamera/internal/pipeline_handler.h" +#include "libcamera/internal/software_isp.h" +#include "libcamera/internal/software_isp/swisp_linaro.h" #include "libcamera/internal/v4l2_subdevice.h" #include "libcamera/internal/v4l2_videodevice.h" - namespace libcamera { LOG_DEFINE_CATEGORY(SimplePipeline) @@ -274,6 +278,8 @@ public: bool useConversion_; std::unique_ptr converter_; + std::unique_ptr swIsp_; + std::unique_ptr ipa_; private: void tryPipeline(unsigned int code, const Size &size); @@ -281,6 +287,9 @@ private: void conversionInputDone(FrameBuffer *buffer); void conversionOutputDone(FrameBuffer *buffer); + + void ispStatsReady(int dummy); + void setSensorControls(const ControlList &sensorControls); }; class SimpleCameraConfiguration : public CameraConfiguration @@ -509,6 +518,38 @@ int SimpleCameraData::init() } } + /* + * Create Linaro's implementation of SoftwareIsp unconditionally if no converter is used + * - to be revisited + */ + if (!converter_) { + swIsp_ = std::make_unique("linaro"); + if (!swIsp_) { + LOG(SimplePipeline, Warning) + << "Failed to create software ISP, disabling software debayering"; + swIsp_.reset(); + } + swIsp_->inputBufferReady.connect(this, &SimpleCameraData::conversionInputDone); + swIsp_->outputBufferReady.connect(this, &SimpleCameraData::conversionOutputDone); + swIsp_->ispStatsReady.connect(this, &SimpleCameraData::ispStatsReady); + + /* Create Linaro's IPA for the SoftwareIsp - to be revisited */ + ipa_ = IPAManager::createIPA(pipe, 0, 0); + if (!ipa_) { + LOG(SimplePipeline, Warning) << "Creating IPA for software ISP failed"; + return -ENOENT; + } + ret = ipa_->init(IPASettings{ "No cfg file", "No sensor model" }, + swIsp_->getStatsFD(), + sensor_->controls()); + if (ret) { + LOG(SimplePipeline, Warning) << "IPA init failed"; + return ret; + } + ipa_->setSensorControls.connect(this, &SimpleCameraData::setSensorControls); + ipa_->configure(sensor_->controls()); + } + video_ = pipe->video(entities_.back().entity); ASSERT(video_); @@ -599,12 +640,15 @@ void SimpleCameraData::tryPipeline(unsigned int code, const Size &size) config.captureFormat = pixelFormat; config.captureSize = format.size; - if (!converter_) { - config.outputFormats = { pixelFormat }; - config.outputSizes = config.captureSize; - } else { + if (converter_) { config.outputFormats = converter_->formats(pixelFormat); config.outputSizes = converter_->sizes(format.size); + } else if (swIsp_) { + config.outputFormats = swIsp_->formats(pixelFormat); + config.outputSizes = swIsp_->sizes(pixelFormat, format.size); + } else { + config.outputFormats = { pixelFormat }; + config.outputSizes = config.captureSize; } configs_.push_back(config); @@ -750,9 +794,9 @@ void SimpleCameraData::bufferReady(FrameBuffer *buffer) } /* - * The converter is in use. Requeue the internal buffer for - * capture (unless the stream is being stopped), and complete - * the request with all the user-facing buffers. + * The converter or Software ISP is in use. Requeue the internal + * buffer for capture (unless the stream is being stopped), and + * complete the request with all the user-facing buffers. */ if (buffer->metadata().status != FrameMetadata::FrameCancelled) video_->queueBuffer(buffer); @@ -798,9 +842,9 @@ void SimpleCameraData::bufferReady(FrameBuffer *buffer) buffer->metadata().timestamp); /* - * Queue the captured and the request buffer to the converter if format - * conversion is needed. If there's no queued request, just requeue the - * captured buffer for capture. + * Queue the captured and the request buffer to the converter or Software + * ISP if format conversion is needed. If there's no queued request, just + * requeue the captured buffer for capture. */ if (useConversion_) { if (conversionQueue_.empty()) { @@ -808,7 +852,11 @@ void SimpleCameraData::bufferReady(FrameBuffer *buffer) return; } - converter_->queueBuffers(buffer, conversionQueue_.front()); + if (converter_) + converter_->queueBuffers(buffer, conversionQueue_.front()); + else + swIsp_->queueBuffers(buffer, conversionQueue_.front()); + conversionQueue_.pop(); return; } @@ -834,6 +882,21 @@ void SimpleCameraData::conversionOutputDone(FrameBuffer *buffer) pipe->completeRequest(request); } +void SimpleCameraData::ispStatsReady([[maybe_unused]] int dummy) +{ + if (!ipa_) + return; + + ipa_->processStats(sensor_->getControls({ V4L2_CID_ANALOGUE_GAIN, + V4L2_CID_EXPOSURE })); +} + +void SimpleCameraData::setSensorControls(const ControlList &sensorControls) +{ + ControlList ctrls(sensorControls); + sensor_->setControls(&ctrls); +} + /* Retrieve all source pads connected to a sink pad through active routes. */ std::vector SimpleCameraData::routedSourcePads(MediaPad *sink) { @@ -1013,8 +1076,10 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() /* Set the stride, frameSize and bufferCount. */ if (needConversion_) { std::tie(cfg.stride, cfg.frameSize) = - data_->converter_->strideAndFrameSize(cfg.pixelFormat, - cfg.size); + (data_->converter_) ? data_->converter_->strideAndFrameSize(cfg.pixelFormat, + cfg.size) + : data_->swIsp_->strideAndFrameSize(cfg.pixelFormat, + cfg.size); if (cfg.stride == 0) return Invalid; } else { @@ -1177,7 +1242,8 @@ int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c) inputCfg.stride = captureFormat.planes[0].bpl; inputCfg.bufferCount = kNumInternalBuffers; - return data->converter_->configure(inputCfg, outputCfgs); + return (data->converter_) ? data->converter_->configure(inputCfg, outputCfgs) + : data->swIsp_->configure(inputCfg, outputCfgs); } int SimplePipelineHandler::exportFrameBuffers(Camera *camera, Stream *stream, @@ -1191,8 +1257,10 @@ int SimplePipelineHandler::exportFrameBuffers(Camera *camera, Stream *stream, * whether the converter is used or not. */ if (data->useConversion_) - return data->converter_->exportBuffers(data->streamIndex(stream), - count, buffers); + return (data->converter_) ? data->converter_->exportBuffers(data->streamIndex(stream), + count, buffers) + : data->swIsp_->exportBuffers(data->streamIndex(stream), + count, buffers); else return data->video_->exportBuffers(count, buffers); } @@ -1237,10 +1305,24 @@ int SimplePipelineHandler::start(Camera *camera, [[maybe_unused]] const ControlL } if (data->useConversion_) { - ret = data->converter_->start(); - if (ret < 0) { - stop(camera); - return ret; + if (data->converter_) { + ret = data->converter_->start(); + if (ret < 0) { + stop(camera); + return ret; + } + } else if (data->swIsp_) { + ret = data->ipa_->start(); + if (ret < 0) { + stop(camera); + return ret; + } + ret = data->swIsp_->start(); + if (ret < 0) { + data->ipa_->stop(); + stop(camera); + return ret; + } } /* Queue all internal buffers for capture. */ @@ -1256,8 +1338,14 @@ void SimplePipelineHandler::stopDevice(Camera *camera) SimpleCameraData *data = cameraData(camera); V4L2VideoDevice *video = data->video_; - if (data->useConversion_) - data->converter_->stop(); + if (data->useConversion_) { + if (data->converter_) + data->converter_->stop(); + else if (data->swIsp_) { + data->swIsp_->stop(); + data->ipa_->stop(); + } + } video->streamOff(); video->releaseBuffers();