From patchwork Tue Dec 12 11:50:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 19310 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 884E0C3292 for ; Tue, 12 Dec 2023 11:51:28 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2A44562B49; Tue, 12 Dec 2023 12:51:28 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1702381888; bh=ZzgDbG5BAr3rpe3OXUbPePZWBKIIm0NVxT0vKz0m8uA=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=cEdDqZgMFMXGA5uKEPRYCeu0TnCNfwNorbSbu8C+FmDlLI7VVBktHB2mItkO0FODK 3ofBOWHwB/Gi32v+qWB2hgZ9FSK8tTd8Iil4tVHL7KPMQjxCxCwSufzqelUJH+UpqE VAyw4/hgMbzDOdXr0KWQgQX5y/gq1UaMwe7iiDQfaqVh2xh/o/fvGVmjDo4ZJh3rUh WjgLEkxqD+Lwc71dhYkq6bigI19T1Ced9mF7Fe7F4zRCuy8cUJxo7b4qCc1DgpN6gv M80E1N2X/UtyMtaa6P510iJ0tOoTRGWnZr0TtFJRAG86RjuTHEHWnAK84x439LjcOO cUz9iT4ndTr9w== Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com [IPv6:2a00:1450:4864:20::633]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BBDB062B3D for ; Tue, 12 Dec 2023 12:51:26 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="MQbcgskt"; dkim-atps=neutral Received: by mail-ej1-x633.google.com with SMTP id a640c23a62f3a-a1e2f34467aso520418066b.2 for ; Tue, 12 Dec 2023 03:51:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1702381886; x=1702986686; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hKQg9mAih69LodQyOzERLWYq7VTf2Z8iii77nqI92wQ=; b=MQbcgsktGHugdvuLnzatzQPDUpvY6YIc7U96EkUoW6jyl8zPg1VJaOsyfX7Cjhh3ML mG/Gd76rUSBhfohZiSYnFG7WYrsqmZHGT8JgGMshxduYmBKfUAY55n9GGkgOBb3lPdhi FOtVVaertFSTrrCr5ImRA2fZzEgso0Qwur2SgzurR1PoBrJCk5DGQ9qEh6Y477MWoI7V 8kkKeEJMkZjo6zq2Q5MqmZj3uSuvWt2FMvvXtwIZrD3Ig2YnAJC5opGUe1h3p5YGUuLa WoyvwqaX42UNgjltj165ZDAElpBJzdtvnD0+t7JLTYi5iYl0TLSCq+O0c1vfKjYV074N tYtg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702381886; x=1702986686; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hKQg9mAih69LodQyOzERLWYq7VTf2Z8iii77nqI92wQ=; b=tc1HGO5S9ZvOLumREAC5bUjh3iztsHBfY1jxJ/Upl2ghCGJ5ewsQ7I4bSpTKFODkJ9 Dbf86ChedALCGy/MErZd7DOvcWfA4UdXnDnX0zjSgBkLD6BL4GqOmkT8AY1UHje66qXI 2iVQPBPi8rLrLCQTiNavv90kXE2isT4z7OaRp5cpb+mYNuGD/FHcB5yUS7aX4Ao6Q0Lt wDxvtIM+tsZJv6h4YCcCXj7CtpKB7pKcRlWSH7D0UK3aljJw+r+3kGmyHwdluS5/igpJ MGASw6uD9GlPrZTlr9ExLYrJCt/uu8X/KKkMiWVuh6PTT5Abvo+yaPkt3mbpEMgfwZ8t cbrw== X-Gm-Message-State: AOJu0YwTiRd5NneqN5x58T0HEcBeb1Ng8Az4xXDubxXp50ahFIOCWNOB t92lbb8LAoC2jvCDJKjxlDsB0W1UMxvOBwsltLs= X-Google-Smtp-Source: AGHT+IGr398qcNcUU3H9InVX2CeGeOektYuF0P83RVZ5lKt68QTBifG8yfqWpP7dwY/fw8F1bLVOkw== X-Received: by 2002:a17:906:fcb5:b0:9be:2b53:ac4d with SMTP id qw21-20020a170906fcb500b009be2b53ac4dmr2976674ejb.74.1702381886220; Tue, 12 Dec 2023 03:51:26 -0800 (PST) Received: from Lat-5310.. ([87.116.161.153]) by smtp.gmail.com with ESMTPSA id tx17-20020a1709078e9100b00a1b75e0e061sm6213188ejc.130.2023.12.12.03.51.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Dec 2023 03:51:25 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Tue, 12 Dec 2023 14:50:44 +0300 Message-Id: <20231212115046.102726-6-andrey.konovalov@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231212115046.102726-1-andrey.konovalov@linaro.org> References: <20231212115046.102726-1-andrey.konovalov@linaro.org> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH v2 5/7] libcamera: software_isp: add SwIspLinaro implementation of SoftwareIsp X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Andrey Konovalov via libcamera-devel From: Andrey Konovalov Reply-To: Andrey Konovalov Cc: mripard@redhat.com, g.martti@gmail.com, t.langendam@gmail.com, srinivas.kandagatla@linaro.org, pavel@ucw.cz, bryan.odonoghue@linaro.org, admin@dennisbonke.com Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The implementation of SoftwareIsp handles creation of Soft IPA and interactions with it, so that the pipeline handler wouldn't need to care about the Soft IPA. Signed-off-by: Andrey Konovalov --- include/libcamera/internal/meson.build | 1 + .../internal/software_isp/meson.build | 1 + .../internal/software_isp/swisp_linaro.h | 117 ++++ src/libcamera/meson.build | 1 + src/libcamera/software_isp/meson.build | 20 + src/libcamera/software_isp/swisp_linaro.cpp | 589 ++++++++++++++++++ 6 files changed, 729 insertions(+) create mode 100644 include/libcamera/internal/software_isp/swisp_linaro.h create mode 100644 src/libcamera/software_isp/meson.build create mode 100644 src/libcamera/software_isp/swisp_linaro.cpp diff --git a/include/libcamera/internal/meson.build b/include/libcamera/internal/meson.build index b780777c..eeae801c 100644 --- a/include/libcamera/internal/meson.build +++ b/include/libcamera/internal/meson.build @@ -50,3 +50,4 @@ libcamera_internal_headers = files([ ]) subdir('converter') +subdir('software_isp') diff --git a/include/libcamera/internal/software_isp/meson.build b/include/libcamera/internal/software_isp/meson.build index a24fe7df..9f84f00e 100644 --- a/include/libcamera/internal/software_isp/meson.build +++ b/include/libcamera/internal/software_isp/meson.build @@ -2,4 +2,5 @@ libcamera_internal_headers += files([ 'statistics-linaro.h', + 'swisp_linaro.h', ]) diff --git a/include/libcamera/internal/software_isp/swisp_linaro.h b/include/libcamera/internal/software_isp/swisp_linaro.h new file mode 100644 index 00000000..c0df7863 --- /dev/null +++ b/include/libcamera/internal/software_isp/swisp_linaro.h @@ -0,0 +1,117 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2023, Linaro Ltd + * + * swisp_linaro.h - software ISP implementation by Linaro + */ + +#pragma once + +#include +#include + +#include + +#include +#include + +#include "libcamera/internal/shared_mem_object.h" +#include "libcamera/internal/software_isp.h" +#include "libcamera/internal/software_isp/statistics-linaro.h" + +namespace libcamera { + +class SwIspLinaro : public SoftwareIsp +{ +public: + SwIspLinaro(PipelineHandler *pipe, const ControlInfoMap &sensorControls); + ~SwIspLinaro() {} + + int loadConfiguration([[maybe_unused]] const std::string &filename) { return 0; } + bool isValid() const; + + std::vector formats(PixelFormat input); + SizeRange sizes(PixelFormat inputFormat, const Size &inputSize); + + std::tuple + strideAndFrameSize(const PixelFormat &outputFormat, const Size &size); + + int configure(const StreamConfiguration &inputCfg, + const std::vector> &outputCfgs); + int exportBuffers(unsigned int output, unsigned int count, + std::vector> *buffers); + void processStats(const ControlList &sensorControls); + + int start(); + void stop(); + + int queueBuffers(FrameBuffer *input, + const std::map &outputs); + + Signal &getSignalSetSensorControls(); + + void process(FrameBuffer *input, FrameBuffer *output); + +private: + SharedMemObject sharedStats_; + + class IspWorker : public Object + { + public: + IspWorker(SwIspLinaro *swIsp); + + std::vector formats(PixelFormat input); + SizeRange sizes(PixelFormat inputFormat, const Size &inputSize); + unsigned int outStride(const PixelFormat &outputFormat, + const Size &outSize); + + int configure(const StreamConfiguration &inputCfg, + const StreamConfiguration &outputCfg); + unsigned int outBufferSize(); + void process(FrameBuffer *input, FrameBuffer *output); + + private: + SwIspLinaro *swIsp_; + + typedef void (SwIspLinaro::IspWorker::*debayerFn)(uint8_t *dst, const uint8_t *src); + typedef SizeRange (*outSizesFn)(const Size &inSize); + typedef unsigned int (*outStrideFn)(const Size &outSize); + struct debayerInfo { + PixelFormat outPixelFmt; + debayerFn debayer; + outSizesFn getOutSizes; + outStrideFn getOutStride; + }; + // TODO: use inputFormat+outputFormat as the map key + // to enable multiple output formats + // TODO: use BayerFormat instead of PixelFormat as inputFormat + std::map debayerInfos_; + int setDebayerInfo(PixelFormat format); + debayerInfo *debayerInfo_; + + /* CSI-2 packed 10-bit raw bayer format (all the 4 orders) */ + void debayerRaw10P(uint8_t *dst, const uint8_t *src); + static SizeRange outSizesRaw10P(const Size &inSize); + static unsigned int outStrideRaw10P(const Size &outSize); + + unsigned int width_; + unsigned int height_; + unsigned int stride_; + Point redShift_; + unsigned int outHeight_; + unsigned int outStride_; + + unsigned long rNumerat_, rDenomin_; /* red gain for AWB */ + unsigned long bNumerat_, bDenomin_; /* blue gain for AWB */ + unsigned long gNumerat_, gDenomin_; /* green gain for AWB */ + + SwIspStats stats_; + }; + + std::unique_ptr ispWorker_; + Thread ispWorkerThread_; + + std::unique_ptr ipa_; +}; + +} /* namespace libcamera */ diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build index b3606969..e758ac9c 100644 --- a/src/libcamera/meson.build +++ b/src/libcamera/meson.build @@ -70,6 +70,7 @@ subdir('converter') subdir('ipa') subdir('pipeline') subdir('proxy') +subdir('software_isp') null_dep = dependency('', required : false) diff --git a/src/libcamera/software_isp/meson.build b/src/libcamera/software_isp/meson.build new file mode 100644 index 00000000..d4a8d499 --- /dev/null +++ b/src/libcamera/software_isp/meson.build @@ -0,0 +1,20 @@ +# SPDX-License-Identifier: CC0-1.0 + +# Software ISP is enabled for 'simple' pipeline handler. +# The 'pipelines' option value should be +# 'simple/' e.g.: +# -Dpipelines=simple/linaro +# The source file should be named swisp_.cpp, +# e.g. 'swisp_linaro.cpp'. + +foreach pipeline : pipelines + pipeline = pipeline.split('/') + if pipeline.length() == 2 and pipeline[0] == 'simple' + libcamera_sources += files([ + 'swisp_' + pipeline[1] + '.cpp', + ]) + # the 'break' below can be removed if/when multiple + # Software ISP implementations are allowed in single build + break + endif +endforeach diff --git a/src/libcamera/software_isp/swisp_linaro.cpp b/src/libcamera/software_isp/swisp_linaro.cpp new file mode 100644 index 00000000..b7f36db1 --- /dev/null +++ b/src/libcamera/software_isp/swisp_linaro.cpp @@ -0,0 +1,589 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2023, Linaro Ltd + * + * swisp_linaro.cpp - software ISP implementation by Linaro + */ + +#include "libcamera/internal/software_isp/swisp_linaro.h" + +#include +#include +#include + +#include +#include + +#include "libcamera/internal/bayer_format.h" +#include "libcamera/internal/framebuffer.h" +#include "libcamera/internal/ipa_manager.h" +#include "libcamera/internal/mapped_framebuffer.h" + +namespace libcamera { + +LOG_DECLARE_CATEGORY(SoftwareIsp) + +SwIspLinaro::SwIspLinaro(PipelineHandler *pipe, const ControlInfoMap &sensorControls) + : SoftwareIsp(pipe, sensorControls) +{ + ispWorker_ = std::make_unique(this); + if (!ispWorker_) { + LOG(SoftwareIsp, Error) + << "Failed to create ISP worker"; + return; + } + + sharedStats_ = SharedMemObject("softIsp_stats"); + if (!sharedStats_.fd().isValid()) { + LOG(SoftwareIsp, Error) + << "Failed to create shared memory for statistics"; + ispWorker_.reset(); + return; + } + + ipa_ = IPAManager::createIPA(pipe, 0, 0); + if (!ipa_) { + LOG(SoftwareIsp, Error) + << "Creating IPA for software ISP failed"; + ispWorker_.reset(); + return; + } + + int ret = ipa_->init(IPASettings{ "No cfg file", "No sensor model" }, + sharedStats_.fd(), + sensorControls); + if (ret) { + LOG(SoftwareIsp, Error) << "IPA init failed"; + ispWorker_.reset(); + return; + } + + ipa_->configure(sensorControls); + + ispWorker_->moveToThread(&ispWorkerThread_); +} + +void SwIspLinaro::processStats(const ControlList &sensorControls) +{ + ASSERT(ipa_); + ipa_->processStats(sensorControls); +} + +Signal &SwIspLinaro::getSignalSetSensorControls() +{ + ASSERT(ipa_); + return ipa_->setSensorControls; +} + +bool SwIspLinaro::isValid() const +{ + return !!ispWorker_; +} + +/* + * Demosaic Raw10P frame into RGB888 format. + * Two LS bits of the RAW10P's total 10 are ignored. + * Also this function performs the statistics calculations, and has a fairly + * naive grey world AWB algorithm implementation. + * \todo Split the stats calculations out of this function. + * \todo Move the AWB algorithm into the IPA module. + */ +void SwIspLinaro::IspWorker::debayerRaw10P(uint8_t *dst, const uint8_t *src) +{ + /* for brightness values in the 0 to 255 range: */ + static const unsigned int BRIGHT_LVL = 200U << 8; + static const unsigned int TOO_BRIGHT_LVL = 240U << 8; + + static const unsigned int RED_Y_MUL = 77; /* 0.30 * 256 */ + static const unsigned int GREEN_Y_MUL = 150; /* 0.59 * 256 */ + static const unsigned int BLUE_Y_MUL = 29; /* 0.11 * 256 */ + + int w_out = width_ - 2; + int h_out = height_ - 2; + + unsigned long sumR = 0; + unsigned long sumB = 0; + unsigned long sumG = 0; + + unsigned long bright_sum = 0; + unsigned long too_bright_sum = 0; + + for (int y = 0; y < h_out; y++) { + const uint8_t *pin_base = src + (y + 1) * stride_; + uint8_t *pout = dst + y * w_out * 3; + int phase_y = (y + redShift_.y) % 2; + + for (int x = 0; x < w_out; x++) { + int phase_x = (x + redShift_.x) % 2; + int phase = 2 * phase_y + phase_x; + + /* x part of the offset in the input buffer: */ + int x_m1 = x + x / 4; /* offset for (x-1) */ + int x_0 = x + 1 + (x + 1) / 4; /* offset for x */ + int x_p1 = x + 2 + (x + 2) / 4; /* offset for (x+1) */ + /* the colour component value to write to the output */ + unsigned val; + /* Y value times 256 */ + unsigned y_val; + + switch (phase) { + case 0: /* at R pixel */ + /* blue: ((-1,-1)+(1,-1)+(-1,1)+(1,1)) / 4 */ + val = ( *(pin_base + x_m1 - stride_) + + *(pin_base + x_p1 - stride_) + + *(pin_base + x_m1 + stride_) + + *(pin_base + x_p1 + stride_) ) >> 2; + y_val = BLUE_Y_MUL * val; + val = val * bNumerat_ / bDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* green: ((0,-1)+(-1,0)+(1,0)+(0,1)) / 4 */ + val = ( *(pin_base + x_0 - stride_) + + *(pin_base + x_p1) + + *(pin_base + x_m1) + + *(pin_base + x_0 + stride_) ) >> 2; + val = val * gNumerat_ / gDenomin_; + y_val += GREEN_Y_MUL * val; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* red: (0,0) */ + val = *(pin_base + x_0); + sumR += val; + y_val += RED_Y_MUL * val; + if (y_val > BRIGHT_LVL) ++bright_sum; + if (y_val > TOO_BRIGHT_LVL) ++too_bright_sum; + val = val * rNumerat_ / rDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + break; + case 1: /* at Gr pixel */ + /* blue: ((0,-1) + (0,1)) / 2 */ + val = ( *(pin_base + x_0 - stride_) + + *(pin_base + x_0 + stride_) ) >> 1; + y_val = BLUE_Y_MUL * val; + val = val * bNumerat_ / bDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* green: (0,0) */ + val = *(pin_base + x_0); + sumG += val; + y_val += GREEN_Y_MUL * val; + val = val * gNumerat_ / gDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* red: ((-1,0) + (1,0)) / 2 */ + val = ( *(pin_base + x_m1) + + *(pin_base + x_p1) ) >> 1; + y_val += RED_Y_MUL * val; + if (y_val > BRIGHT_LVL) ++bright_sum; + if (y_val > TOO_BRIGHT_LVL) ++too_bright_sum; + val = val * rNumerat_ / rDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + break; + case 2: /* at Gb pixel */ + /* blue: ((-1,0) + (1,0)) / 2 */ + val = ( *(pin_base + x_m1) + + *(pin_base + x_p1) ) >> 1; + y_val = BLUE_Y_MUL * val; + val = val * bNumerat_ / bDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* green: (0,0) */ + val = *(pin_base + x_0); + sumG += val; + y_val += GREEN_Y_MUL * val; + val = val * gNumerat_ / gDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* red: ((0,-1) + (0,1)) / 2 */ + val = ( *(pin_base + x_0 - stride_) + + *(pin_base + x_0 + stride_) ) >> 1; + y_val += RED_Y_MUL * val; + if (y_val > BRIGHT_LVL) ++bright_sum; + if (y_val > TOO_BRIGHT_LVL) ++too_bright_sum; + val = val * rNumerat_ / rDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + break; + default: /* at B pixel */ + /* blue: (0,0) */ + val = *(pin_base + x_0); + sumB += val; + y_val = BLUE_Y_MUL * val; + val = val * bNumerat_ / bDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* green: ((0,-1)+(-1,0)+(1,0)+(0,1)) / 4 */ + val = ( *(pin_base + x_0 - stride_) + + *(pin_base + x_p1) + + *(pin_base + x_m1) + + *(pin_base + x_0 + stride_) ) >> 2; + y_val += GREEN_Y_MUL * val; + val = val * gNumerat_ / gDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* red: ((-1,-1)+(1,-1)+(-1,1)+(1,1)) / 4 */ + val = ( *(pin_base + x_m1 - stride_) + + *(pin_base + x_p1 - stride_) + + *(pin_base + x_m1 + stride_) + + *(pin_base + x_p1 + stride_) ) >> 2; + y_val += RED_Y_MUL * val; + if (y_val > BRIGHT_LVL) ++bright_sum; + if (y_val > TOO_BRIGHT_LVL) ++too_bright_sum; + val = val * rNumerat_ / rDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + } + } + } + + /* calculate the fractions of "bright" and "too bright" pixels */ + stats_.bright_ratio = (float)bright_sum / (h_out * w_out); + stats_.too_bright_ratio = (float)too_bright_sum / (h_out * w_out); + + /* calculate red and blue gains for simple AWB */ + LOG(SoftwareIsp, Debug) + << "sumR = " << sumR << ", sumB = " << sumB << ", sumG = " << sumG; + + sumG /= 2; /* the number of G pixels is twice as big vs R and B ones */ + + /* normalize red, blue, and green sums to fit into 22-bit value */ + unsigned long fRed = sumR / 0x400000; + unsigned long fBlue = sumB / 0x400000; + unsigned long fGreen = sumG / 0x400000; + unsigned long fNorm = std::max({ 1UL, fRed, fBlue, fGreen }); + sumR /= fNorm; + sumB /= fNorm; + sumG /= fNorm; + + LOG(SoftwareIsp, Debug) << "fNorm = " << fNorm; + LOG(SoftwareIsp, Debug) + << "Normalized: sumR = " << sumR + << ", sumB= " << sumB << ", sumG = " << sumG; + + /* make sure red/blue gains never exceed approximately 256 */ + unsigned long minDenom; + rNumerat_ = (sumR + sumB + sumG) / 3; + minDenom = rNumerat_ / 0x100; + rDenomin_ = std::max(minDenom, sumR); + bNumerat_ = rNumerat_; + bDenomin_ = std::max(minDenom, sumB); + gNumerat_ = rNumerat_; + gDenomin_ = std::max(minDenom, sumG); + + LOG(SoftwareIsp, Debug) + << "rGain = [ " << rNumerat_ << " / " << rDenomin_ + << " ], bGain = [ " << bNumerat_ << " / " << bDenomin_ + << " ], gGain = [ " << gNumerat_ << " / " << gDenomin_ + << " (minDenom = " << minDenom << ")"; +} + +SizeRange SwIspLinaro::IspWorker::outSizesRaw10P(const Size &inSize) +{ + if (inSize.width < 2 || inSize.height < 2) { + LOG(SoftwareIsp, Error) + << "Input format size too small: " << inSize.toString(); + return {}; + } + + return SizeRange(Size(inSize.width - 2, inSize.height - 2)); +} + +unsigned int SwIspLinaro::IspWorker::outStrideRaw10P(const Size &outSize) +{ + return outSize.width * 3; +} + +SwIspLinaro::IspWorker::IspWorker(SwIspLinaro *swIsp) + : swIsp_(swIsp) +{ + debayerInfos_[formats::SBGGR10_CSI2P] = { formats::RGB888, + &SwIspLinaro::IspWorker::debayerRaw10P, + &SwIspLinaro::IspWorker::outSizesRaw10P, + &SwIspLinaro::IspWorker::outStrideRaw10P }; + debayerInfos_[formats::SGBRG10_CSI2P] = { formats::RGB888, + &SwIspLinaro::IspWorker::debayerRaw10P, + &SwIspLinaro::IspWorker::outSizesRaw10P, + &SwIspLinaro::IspWorker::outStrideRaw10P }; + debayerInfos_[formats::SGRBG10_CSI2P] = { formats::RGB888, + &SwIspLinaro::IspWorker::debayerRaw10P, + &SwIspLinaro::IspWorker::outSizesRaw10P, + &SwIspLinaro::IspWorker::outStrideRaw10P }; + debayerInfos_[formats::SRGGB10_CSI2P] = { formats::RGB888, + &SwIspLinaro::IspWorker::debayerRaw10P, + &SwIspLinaro::IspWorker::outSizesRaw10P, + &SwIspLinaro::IspWorker::outStrideRaw10P }; +} + +int SwIspLinaro::IspWorker::setDebayerInfo(PixelFormat format) +{ + const auto it = debayerInfos_.find(format); + if (it == debayerInfos_.end()) + return -1; + + debayerInfo_ = &it->second; + return 0; +} + +std::vector SwIspLinaro::IspWorker::formats(PixelFormat input) +{ + std::vector pixelFormats; + + const auto it = debayerInfos_.find(input); + if (it == debayerInfos_.end()) + LOG(SoftwareIsp, Info) + << "Unsupported input format " << input.toString(); + else + pixelFormats.push_back(it->second.outPixelFmt); + + return pixelFormats; +} + +SizeRange SwIspLinaro::IspWorker::sizes(PixelFormat inputFormat, + const Size &inputSize) +{ + const auto it = debayerInfos_.find(inputFormat); + if (it == debayerInfos_.end()) { + LOG(SoftwareIsp, Info) + << "Unsupported input format " << inputFormat.toString(); + return {}; + } + + return (*it->second.getOutSizes)(inputSize); +} + +unsigned int SwIspLinaro::IspWorker::outStride(const PixelFormat &outputFormat, + const Size &outSize) +{ + /* + * Assuming that the output stride depends only on the outputFormat, + * we use the first debayerInfos_ entry with the matching output format + */ + for (auto it = debayerInfos_.begin(); it != debayerInfos_.end(); it++) { + if (it->second.outPixelFmt == outputFormat) + return (*it->second.getOutStride)(outSize); + } + + return 0; +} + +int SwIspLinaro::IspWorker::configure(const StreamConfiguration &inputCfg, + const StreamConfiguration &outputCfg) +{ + if (setDebayerInfo(inputCfg.pixelFormat) != 0) { + LOG(SoftwareIsp, Error) + << "Input format " << inputCfg.pixelFormat + << "not supported"; + return -EINVAL; + } + + /* check that: + * - output format is valid + * - output size matches the input size and is valid */ + SizeRange outSizeRange = (*debayerInfo_->getOutSizes)(inputCfg.size); + if (debayerInfo_->outPixelFmt != outputCfg.pixelFormat || + outputCfg.size.isNull() || !outSizeRange.contains(outputCfg.size) || + (*debayerInfo_->getOutStride)(outputCfg.size) != outputCfg.stride) { + LOG(SoftwareIsp, Error) + << "Invalid output format/size/stride: " + << "\n " << outputCfg.pixelFormat << " (" + << debayerInfo_->outPixelFmt << ")" + << "\n " << outputCfg.size << " (" + << outSizeRange << ")" + << "\n " << outputCfg.stride << " (" + << (*debayerInfo_->getOutStride)(outputCfg.size) << ")"; + return -EINVAL; + } + + width_ = inputCfg.size.width; + height_ = inputCfg.size.height; + stride_ = inputCfg.stride; + + BayerFormat bayerFormat = + BayerFormat::fromPixelFormat(inputCfg.pixelFormat); + switch (bayerFormat.order) { + case BayerFormat::BGGR: + redShift_ = Point(0, 0); + break; + case BayerFormat::GBRG: + redShift_ = Point(1, 0); + break; + case BayerFormat::GRBG: + redShift_ = Point(0, 1); + break; + case BayerFormat::RGGB: + default: + redShift_ = Point(1, 1); + break; + } + + outStride_ = outputCfg.stride; + outHeight_ = outputCfg.size.height; + + LOG(SoftwareIsp, Info) + << "SoftwareISP configuration: " + << inputCfg.size << "-" << inputCfg.pixelFormat << " -> " + << outputCfg.size << "-" << outputCfg.pixelFormat; + + /* set r/g/b gains to 1.0 until frame data collected */ + rNumerat_ = rDenomin_ = 1; + bNumerat_ = bDenomin_ = 1; + gNumerat_ = gDenomin_ = 1; + + return 0; +} + +/* May not be called before SwIspLinaro::IspWorker::configure() */ +unsigned int SwIspLinaro::IspWorker::outBufferSize() +{ + return outHeight_ * outStride_; +} + +std::vector SwIspLinaro::formats(PixelFormat inputFormat) +{ + ASSERT(ispWorker_ != nullptr); + + return ispWorker_->formats(inputFormat); +} + +SizeRange SwIspLinaro::sizes(PixelFormat inputFormat, const Size &inputSize) +{ + ASSERT(ispWorker_ != nullptr); + + return ispWorker_->sizes(inputFormat, inputSize); +} + +std::tuple +SwIspLinaro::strideAndFrameSize(const PixelFormat &outputFormat, const Size &size) +{ + ASSERT(ispWorker_ != nullptr); + + unsigned int stride = ispWorker_->outStride(outputFormat, size); + + return std::make_tuple(stride, stride * size.height); +} + +int SwIspLinaro::configure(const StreamConfiguration &inputCfg, + const std::vector> &outputCfgs) +{ + ASSERT(ispWorker_ != nullptr); + + if (outputCfgs.size() != 1) { + LOG(SoftwareIsp, Error) + << "Unsupported number of output streams: " + << outputCfgs.size(); + return -EINVAL; + } + + return ispWorker_->configure(inputCfg, outputCfgs[0]); +} + +int SwIspLinaro::exportBuffers(unsigned int output, unsigned int count, + std::vector> *buffers) +{ + ASSERT(ispWorker_ != nullptr); + + /* single output for now */ + if (output >= 1) + return -EINVAL; + + unsigned int bufSize = ispWorker_->outBufferSize(); + + /* TODO: allocate from dma_heap; memfd buffs aren't allowed in FrameBuffer */ + for (unsigned int i = 0; i < count; i++) { + std::string name = "frame-" + std::to_string(i); + + const int ispFd = memfd_create(name.c_str(), 0); + int ret = ftruncate(ispFd, bufSize); + if (ret < 0) { + LOG(SoftwareIsp, Error) + << "ftruncate() for memfd failed " + << strerror(-ret); + return ret; + } + + FrameBuffer::Plane outPlane; + outPlane.fd = SharedFD(std::move(ispFd)); + outPlane.offset = 0; + outPlane.length = bufSize; + + std::vector planes{ outPlane }; + buffers->emplace_back(std::make_unique(std::move(planes))); + } + + return count; +} + +int SwIspLinaro::queueBuffers(FrameBuffer *input, + const std::map &outputs) +{ + unsigned int mask = 0; + + /* + * Validate the outputs as a sanity check: at least one output is + * required, all outputs must reference a valid stream and no two + * outputs can reference the same stream. + */ + if (outputs.empty()) + return -EINVAL; + + for (auto [index, buffer] : outputs) { + if (!buffer) + return -EINVAL; + if (index >= 1) /* only single stream atm */ + return -EINVAL; + if (mask & (1 << index)) + return -EINVAL; + + mask |= 1 << index; + } + + process(input, outputs.at(0)); + + return 0; +} + +int SwIspLinaro::start() +{ + int ret = ipa_->start(); + if (ret) + return ret; + + ispWorkerThread_.start(); + return 0; +} + +void SwIspLinaro::stop() +{ + ispWorkerThread_.exit(); + ispWorkerThread_.wait(); + + ipa_->stop(); +} + +void SwIspLinaro::IspWorker::process(FrameBuffer *input, FrameBuffer *output) +{ + /* Copy metadata from the input buffer */ + FrameMetadata &metadata = output->_d()->metadata(); + metadata.status = input->metadata().status; + metadata.sequence = input->metadata().sequence; + metadata.timestamp = input->metadata().timestamp; + + MappedFrameBuffer in(input, MappedFrameBuffer::MapFlag::Read); + MappedFrameBuffer out(output, MappedFrameBuffer::MapFlag::Write); + if (!in.isValid() || !out.isValid()) { + LOG(SoftwareIsp, Error) << "mmap-ing buffer(s) failed"; + metadata.status = FrameMetadata::FrameError; + swIsp_->outputBufferReady.emit(output); + swIsp_->inputBufferReady.emit(input); + return; + } + + (this->*debayerInfo_->debayer)(out.planes()[0].data(), in.planes()[0].data()); + metadata.planes()[0].bytesused = out.planes()[0].size(); + + *swIsp_->sharedStats_ = stats_; + swIsp_->ispStatsReady.emit(0); + + swIsp_->outputBufferReady.emit(output); + swIsp_->inputBufferReady.emit(input); +} + +void SwIspLinaro::process(FrameBuffer *input, FrameBuffer *output) +{ + ispWorker_->invokeMethod(&SwIspLinaro::IspWorker::process, + ConnectionTypeQueued, input, output); +} + +REGISTER_SOFTWAREISP(SwIspLinaro) + +} /* namespace libcamera */