From patchwork Mon Dec 4 00:10:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 19262 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id DC740C3226 for ; Mon, 4 Dec 2023 00:10:38 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 30151629AD; Mon, 4 Dec 2023 01:10:37 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1701648638; bh=+e0PNE4utttoGieJ4PVEGh799g42QoXBhRXUSuJXb9U=; h=To:Date:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:Cc:From; b=sQ5QNTrLZjXFWn/pDK0JXze/64Ak9Ctzh/p68k/0selFVmqmbevgzSyMfNn/uPwte wi3I1dlSINssD9rxmixk140pS4JjWohc9Y18R+7qh1bH8mfwmiNzAjb1v9jvcmaR8p HfxeQS/RyYYCJ+sg6obND96nMZh6NiZaov8FaWcFU2h8xZG60O7lZ6Lj47NVv1pWt5 vB0W7UtkiArbmihO4ftX0jW/Xf0l0i0CEfKratyh3QbfDevNDnmqXM+PWdjymqgWHn MvNQedEBNiciuA+KXDBwtfyFFI6jH5vzkXsLpgYugjugQUI5nkPzLayeZ8y5M4dK2/ nZ6bUpAjtNYRg== Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CCE2D629AD for ; Mon, 4 Dec 2023 01:10:35 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="hpHMolsT"; dkim-atps=neutral Received: by mail-wr1-x42d.google.com with SMTP id ffacd0b85a97d-3316c6e299eso3371195f8f.1 for ; Sun, 03 Dec 2023 16:10:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1701648635; x=1702253435; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=H8DIcF0lhKcErB75vSHluuRs9rHPAfJJlhotNnBxcSk=; b=hpHMolsTxw3TDFlirRBRQYMUpyPmSbRfGlNjWUnfO537sc8zrN2koJVkUafOD8Kr1p nVBEGcZSUryDBFVerrB+335nHfHvP+srOA0Hp7uXC0Iry4LJAe+4EgvKvvUbRvvglDtE Qwxy3t7yzh+0WMTbHv2oU3Bi3TN/d3KK7fSbGt0JWKAWBjOfXLFct/owPqNsOVGBaNoU I8C4Q3aS1VAgql1Kf2EQJcZIJ4vPVMeM3zURcsFuzhxkl5qQoT5cdV7PiSgUT5YYaI/q Kd86563ikOYQQoJ/ek+VwENThNxlJVQi8ypLod35gT15n8YPHlPVzwanDl9kIG9mZCxZ GP1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701648635; x=1702253435; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=H8DIcF0lhKcErB75vSHluuRs9rHPAfJJlhotNnBxcSk=; b=T929aaIsGkRTeaX9LN8RVMqLvBpQTNagUUqGMUEuRxWIkQLNVARZ1CKpwWJ9wrZyWU 0SxnTtanrxqwXkvcwCkMEceUfrBRzmXkkKTc/IISttohq8UTOhebkMODFYB0uj9AxEWC VcTeEWksrmK8aNJk40UtTQpwWByhVv6FAfth5Hvti+CJNL+oadZSdK2IO7WhPKska8of CsQyj3FP41Ui/OJs3mh7+XklQAGMqSOOYCoKIrPu5tjxDKbhJsmcn5adOupWTIO5Bd03 QyGOJ1B1FmaWSwJqnvW9b/SuTP2SyaoUs8eVN3qd9mNgqA1nkbMCNbNKKz3diye4dkcX Tpkw== X-Gm-Message-State: AOJu0YzgofVtz/4Fv3p4bUm60GL8RKz8cf3kfuiLKgo4dHKR/s5JMhXi ++TRp+hGMYyM/bTaYhaM1adUtFrdK0hwDMEzwpM= X-Google-Smtp-Source: AGHT+IHqMiYdAb8jcKUiq3yIw5vaoZAlByQ7xo96J8VOyWVOGen8CcEQOa3IjJz/XxxAjBjND5SVFA== X-Received: by 2002:adf:f70d:0:b0:333:34c8:49f with SMTP id r13-20020adff70d000000b0033334c8049fmr1396870wrp.127.1701648635352; Sun, 03 Dec 2023 16:10:35 -0800 (PST) Received: from Lat-5310.. ([87.116.165.212]) by smtp.gmail.com with ESMTPSA id u29-20020adfa19d000000b003332db7d91dsm8835997wru.39.2023.12.03.16.10.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 03 Dec 2023 16:10:35 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Mon, 4 Dec 2023 03:10:06 +0300 Message-Id: <20231204001013.404720-1-andrey.konovalov@linaro.org> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 0/7] libcamera: introduce Software ISP and Software IPA X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Andrey Konovalov via libcamera-devel From: Andrey Konovalov Reply-To: Andrey Konovalov Cc: mripard@redhat.com, g.martti@gmail.com, t.langendam@gmail.com, srinivas.kandagatla@linaro.org, bryan.odonoghue@linaro.org, admin@dennisbonke.com Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Here is a draft implementation of Software ISP and Software IPA which provide debayering and implementation of image processing algorithms for systems without a hardware ISP, or in cases when there are no public drivers for the hardware ISP present in the system. The implementation of the Software ISP is a reference one. A naive AWB alorithm is implemented as part of a function which does debayering and statistics calculations - the algorithm part is to be moved to the IPA in the next version of the patch set. And for debayering itself there is already a more efficient implementation by Hans de Goede. This patch set is currently using the earlier debayering implementation as it is less lines of code and is OK for the initial discussion. Only RAW10P format from the sensor is currently supported, but other debayering functions can be easily added (which of them to call is decided based on the input format). The Software IPA has only auto exposure and AGC. For the AGC the analogue gain control of the camera sensor is used (if available). The algorithm is very much simplified, and is mostly included as a reference code. The 6th patch renames some variables in the simple pipeline handler for the Software ISP to use the same buffer handling code as the Converter currently does. This lets one to avoid adding extra code to the pipeline handler, but also makes the Software ISP and the Converter mutually exclusive. The Software ISP / IPA are intended to be used with the simple pipeline handler. The proper integration isn't done yet, and the last patch in this series unconditionally creates the Software ISP and Software IPA instances if the pipeline is not using a Converter. In the future the Software ISP / IPA would be created if the media pipeline can only produce raw Bayer frames. And a possibility to use different implementations of Software ISP and IPA will be added. This patch set uses SharedMemObject class used by the RPi pipeline handler - the second patch in the series moves the header file to a common directory. This patch set has been tested on Qualcomm RB5 board with a mezzanine board equipped with RPi camera v2 (not the standard RB5 camera mezzanine). Andrey Konovalov (7): libcamera: introduce SoftwareIsp class libcamera: internal: Move SharedMemObject class to a common directory libcamera: software_isp: add SwIspLinaro implementation of SoftwareIsp libcamera: ipa: add Soft IPA libcamera: software_isp: introduce getStatsFD() libcamera: pipeline: simple: rename converterBuffers_ and related vars [DNI] libcamera: pipeline: simple: enable use of SoftwareISP and Soft IPA include/libcamera/internal/meson.build | 3 + .../libcamera/internal}/shared_mem_object.h | 4 - include/libcamera/internal/software_isp.h | 71 +++ .../internal/software_isp/meson.build | 6 + .../internal/software_isp/statistics-linaro.h | 17 + .../internal/software_isp/swisp_linaro.h | 111 ++++ include/libcamera/ipa/meson.build | 1 + include/libcamera/ipa/soft.mojom | 27 + meson_options.txt | 3 +- src/ipa/simple/common/meson.build | 17 + src/ipa/simple/common/soft_base.cpp | 66 +++ src/ipa/simple/common/soft_base.h | 47 ++ src/ipa/simple/linaro/data/meson.build | 8 + src/ipa/simple/linaro/data/soft.conf | 3 + src/ipa/simple/linaro/meson.build | 26 + src/ipa/simple/linaro/soft_linaro.cpp | 210 +++++++ src/ipa/simple/meson.build | 12 + src/libcamera/meson.build | 2 + src/libcamera/pipeline/simple/simple.cpp | 193 +++++-- src/libcamera/software_isp.cpp | 25 + src/libcamera/software_isp/meson.build | 5 + src/libcamera/software_isp/swisp_linaro.cpp | 539 ++++++++++++++++++ 22 files changed, 1339 insertions(+), 57 deletions(-) rename {src/libcamera/pipeline/rpi/common => include/libcamera/internal}/shared_mem_object.h (98%) create mode 100644 include/libcamera/internal/software_isp.h create mode 100644 include/libcamera/internal/software_isp/meson.build create mode 100644 include/libcamera/internal/software_isp/statistics-linaro.h create mode 100644 include/libcamera/internal/software_isp/swisp_linaro.h create mode 100644 include/libcamera/ipa/soft.mojom create mode 100644 src/ipa/simple/common/meson.build create mode 100644 src/ipa/simple/common/soft_base.cpp create mode 100644 src/ipa/simple/common/soft_base.h create mode 100644 src/ipa/simple/linaro/data/meson.build create mode 100644 src/ipa/simple/linaro/data/soft.conf create mode 100644 src/ipa/simple/linaro/meson.build create mode 100644 src/ipa/simple/linaro/soft_linaro.cpp create mode 100644 src/ipa/simple/meson.build create mode 100644 src/libcamera/software_isp.cpp create mode 100644 src/libcamera/software_isp/meson.build create mode 100644 src/libcamera/software_isp/swisp_linaro.cpp