From patchwork Sun Aug 6 18:01:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 18925 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9E058BDCBF for ; Sun, 6 Aug 2023 18:02:07 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 51039627EA; Sun, 6 Aug 2023 20:02:07 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1691344927; bh=hyKZ1FNjPHaejFx/L2CdUmflaw47Z8sL0Z6XmrV5sIA=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=BmfzV+3El69Zi/iyeRvPvqsdpknQHtg1LDG0dVgj/m1fPrskDCYF9bIEnagbqxOif NaQPpnfiiU2Baq7YCaZoiJqvd5XSckoMCZLNHH0gZE3kY5wlfEi38yWBmUg5KY9BnR tD2KJsdJDc7MIBghifJAdAKgxt41MsWk8bb6SY0UqhVqmYdJ/gi4IfgsbBRA+/mHUP knn8fL/8+oURQ4t7/Tdov+hGJmm6NJCcD1kviF7VmDbYAmyK/zNIrJfsgk5/SNSTzy /1w98zvxWXQGRx0mljjvYB+8Cf8BXtxMYpYwsqTZideCGYbcmrW1NlavB8txf32fyU 65x6tybgeFYPQ== Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com [IPv6:2a00:1450:4864:20::633]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 96A8B627E9 for ; Sun, 6 Aug 2023 20:02:05 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="S9y++uY2"; dkim-atps=neutral Received: by mail-ej1-x633.google.com with SMTP id a640c23a62f3a-99c93638322so528121366b.1 for ; Sun, 06 Aug 2023 11:02:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1691344925; x=1691949725; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MkCvqViWmK1XGUN8O+hfLZoKr0ZhnflyP+K3MZlgvTE=; b=S9y++uY2Au6Rda3f10zjrTD5Q200An8AgiSIOUKJ7/PKnHgac2SueoFA3jqQpyrD6+ uHTJTUYuovcim/M+BD+C6D9JF2LWJOZun7Z20H9e8zOddidMP2c6Y04p8SoNkDy9eSja MTi/+as5uGp0r3dD9dTzMD4ojEgAo7YlXu3ZFaRjZIMqeIlnB1RPHXBOnNBGpV6f+FBX ZyhEZgyLGhLIKGPX47VZewjCmyJ3lMhVuFxsM2Rqj1x60T/3BpQM1PgB7Nc9OY+dxuCf Ehxt4P53fZIruqatb2yA1wWu5Vhuy10oQgazpGxkcEBvqHg14PhkRsrwoP8CIfw16ryt f7UA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691344925; x=1691949725; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MkCvqViWmK1XGUN8O+hfLZoKr0ZhnflyP+K3MZlgvTE=; b=lNRsk2qi1v9jQjILy02VeAdxIAFqtUQiBprqFvYAwaAun1hS4qH504WhPDCQYWvkmj 4NCZc+L/VwatidpwhBaH2ZVX4+VlFp9I9HD673tWCbqHwS+dpXmLWp+3H9tKD3ve80Ak IH6izHgF44AjPGcMz3v4o32z5CrB1gVd+hp0cnIChgEiuxYhRRVQxrCRnY8wrENg1a35 06htGK1Ff7I2kZ/Ku55YSYa7rvzHmy904v4pQB9TDvPRPqrVuQg8bdjymSBrau7TZupP M/iG5BAezVmMc30Xcf2YdHkf25D+VWBY0MZUgZ4QMwRypyQhusifi43SnV9iYmgLn3Sl lgWw== X-Gm-Message-State: AOJu0Yx9XAK4p68ieTEo2pXbwJPlFCihCS4mxZv544jDAhYZZeMHGVNV 1HKU0cmikcCOojWZcHnz8gDiQWZGncmvs7RxYa0= X-Google-Smtp-Source: AGHT+IHaHiGEY/6Dk2sUtD2Fmw/bYtBRgwnDQT+uJJylH74r0OXV/gOPepdG+yBeUIgbnR9G/SzEfQ== X-Received: by 2002:a17:906:8a44:b0:99b:cdfd:fb44 with SMTP id gx4-20020a1709068a4400b0099bcdfdfb44mr5379077ejc.9.1691344925106; Sun, 06 Aug 2023 11:02:05 -0700 (PDT) Received: from Lat-5310.. ([87.116.165.37]) by smtp.gmail.com with ESMTPSA id j15-20020a170906430f00b0099b76c3041csm4197396ejm.7.2023.08.06.11.02.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 06 Aug 2023 11:02:04 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Sun, 6 Aug 2023 21:01:35 +0300 Message-Id: <20230806180136.304156-2-andrey.konovalov@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230806180136.304156-1-andrey.konovalov@linaro.org> References: <20230806180136.304156-1-andrey.konovalov@linaro.org> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 1/2] libcamera: converter: add software converter X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Andrey Konovalov via libcamera-devel From: Andrey Konovalov Reply-To: Andrey Konovalov Cc: bryan.odonoghue@linaro.org, srinivas.kandagatla@linaro.org Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Use the Converter interface to implement bilinear Bayer demosaic filtering and very simple AWB on CPU. Signed-off-by: Andrey Konovalov --- .../internal/converter/converter_softw.h | 90 ++++ src/libcamera/converter/converter_softw.cpp | 430 ++++++++++++++++++ src/libcamera/converter/meson.build | 3 +- 3 files changed, 522 insertions(+), 1 deletion(-) create mode 100644 include/libcamera/internal/converter/converter_softw.h create mode 100644 src/libcamera/converter/converter_softw.cpp diff --git a/include/libcamera/internal/converter/converter_softw.h b/include/libcamera/internal/converter/converter_softw.h new file mode 100644 index 00000000..73814c77 --- /dev/null +++ b/include/libcamera/internal/converter/converter_softw.h @@ -0,0 +1,90 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2023, Linaro Ltd + * + * converter_softw.h - interface of software converter (runs 100% on CPU) + */ + +#include +#include +#include + +#include +#include +#include + +#include + +#include "libcamera/internal/converter.h" + +namespace libcamera { + +class FrameBuffer; +class MediaDevice; +class Size; +class SizeRange; +struct StreamConfiguration; + +class SwConverter : public Converter +{ +public: + SwConverter(MediaDevice *media) + : Converter(media) {} + + int loadConfiguration([[maybe_unused]] const std::string &filename) { return 0; } + bool isValid() const { return true; } + + std::vector formats(PixelFormat input); + SizeRange sizes(const Size &input); + + std::tuple + strideAndFrameSize(const PixelFormat &pixelFormat, const Size &size); + + int configure(const StreamConfiguration &inputCfg, + const std::vector> &outputCfg); + int exportBuffers(unsigned int ouput, unsigned int count, + std::vector> *buffers); + + void process(FrameBuffer *input, FrameBuffer *output); + int start(); + void stop(); + + int queueBuffers(FrameBuffer *input, + const std::map &outputs); + +private: + class Isp : public Object + { + public: + Isp(SwConverter *converter) + : converter_(converter) {} + + int configure(const StreamConfiguration &inputCfg, + const StreamConfiguration &outputCfg); + int exportBuffers(unsigned int count, + std::vector> *buffers); + void process(FrameBuffer *input, FrameBuffer *output); + int start(); + void stop(); + + private: + void debayer(uint8_t *dst, const uint8_t *src); + + SwConverter *converter_; + + Thread thread_; + + unsigned int width_; + unsigned int height_; + unsigned int stride_; + Point red_shift_; + + unsigned long rNumerat_, rDenomin_; /* red gain for AWB */ + unsigned long bNumerat_, bDenomin_; /* blue gain for AWB */ + unsigned long gNumerat_, gDenomin_; /* green gain for AWB */ + }; + + std::unique_ptr isp_; +}; + +} /* namespace libcamera */ diff --git a/src/libcamera/converter/converter_softw.cpp b/src/libcamera/converter/converter_softw.cpp new file mode 100644 index 00000000..ab00113a --- /dev/null +++ b/src/libcamera/converter/converter_softw.cpp @@ -0,0 +1,430 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2023, Linaro Ltd + * + * converter_softw.h - interface of software converter (runs 100% on CPU) + */ + +#include "libcamera/internal/converter/converter_softw.h" + +#include +#include +#include +#include +#include + +#include +#include +#include + +#include +#include +#include +#include + +#include "libcamera/internal/bayer_format.h" +#include "libcamera/internal/framebuffer.h" + +namespace libcamera { + +LOG_DECLARE_CATEGORY(Converter) + +std::vector SwConverter::formats(PixelFormat input) +{ + std::vector pixelFormats; + BayerFormat inputFormat = BayerFormat::fromPixelFormat(input); + + /* Only RAW10P is currently supported */ + if (inputFormat.bitDepth == 10 && inputFormat.packing == BayerFormat::Packing::CSI2) + pixelFormats.push_back(formats::ARGB8888); + + if (pixelFormats.empty()) + LOG(Converter, Info) + << "Unsupported input format " << input.toString(); + + return pixelFormats; +} + +SizeRange SwConverter::sizes(const Size &input) +{ + if (input.width < 2 || input.height < 2) { + LOG(Converter, Error) + << "Input format size too small: " << input.toString(); + return {}; + } + + return SizeRange(Size(input.width - 2, input.height - 2)); +} + +std::tuple +SwConverter::strideAndFrameSize(const PixelFormat &pixelFormat, + const Size &size) +{ + /* Only ARGB8888 output is currently supported */ + if (pixelFormat == formats::ARGB8888) { + int stride = size.width * 4; + return std::make_tuple(stride, stride * (size.height)); + } + + return std::make_tuple(0, 0); +} + +int SwConverter::configure(const StreamConfiguration &inputCfg, + const std::vector> &outputCfgs) +{ + if (outputCfgs.size() != 1) { + LOG(Converter, Error) + << "Unsupported number of output streams: " + << outputCfgs.size(); + return -EINVAL; + } + + isp_ = std::make_unique(this); + + return isp_->configure(inputCfg, outputCfgs[0]); +} + +int SwConverter::Isp::configure(const StreamConfiguration &inputCfg, + const StreamConfiguration &outputCfg) +{ + BayerFormat bayerFormat = + BayerFormat::fromPixelFormat(inputCfg.pixelFormat); + width_ = inputCfg.size.width; + height_ = inputCfg.size.height; + stride_ = inputCfg.stride; + + if (bayerFormat.bitDepth != 10 || + bayerFormat.packing != BayerFormat::Packing::CSI2 || + width_ < 2 || height_ < 2) { + LOG(Converter, Error) << "Input format " + << inputCfg.size << "-" + << inputCfg.pixelFormat + << "not supported"; + return -EINVAL; + } + + switch (bayerFormat.order) { + case BayerFormat::BGGR: + red_shift_ = Point(0, 0); + break; + case BayerFormat::GBRG: + red_shift_ = Point(1, 0); + break; + case BayerFormat::GRBG: + red_shift_ = Point(0, 1); + break; + case BayerFormat::RGGB: + default: + red_shift_ = Point(1, 1); + break; + } + + if (outputCfg.size.width != width_ - 2 || + outputCfg.size.height != height_ - 2 || + outputCfg.stride != (width_ - 2) * 4 || + outputCfg.pixelFormat != formats::ARGB8888) { + LOG(Converter, Error) + << "Output format not supported"; + return -EINVAL; + } + + LOG(Converter, Info) << "SwConverter configuration: " + << inputCfg.size << "-" << inputCfg.pixelFormat + << " -> " + << outputCfg.size << "-" << outputCfg.pixelFormat; + + /* set r/g/b gains to 1.0 until frame data collected */ + rNumerat_ = rDenomin_ = 1; + bNumerat_ = bDenomin_ = 1; + gNumerat_ = gDenomin_ = 1; + + return 0; +} + +int SwConverter::exportBuffers(unsigned int output, unsigned int count, + std::vector> *buffers) +{ + /* single output for now */ + if (output >= 1) + return -EINVAL; + + return isp_->exportBuffers(count, buffers); +} + +int SwConverter::Isp::exportBuffers(unsigned int count, + std::vector> *buffers) +{ + /* V4L2_PIX_FMT_ABGR32 aka BGR32 for output: */ + unsigned int bufSize = (height_ - 2) * (width_ - 2) * 4; + + for (unsigned int i = 0; i < count; i++) { + std::string name = "frame-" + std::to_string(i); + + const int ispFd = memfd_create(name.c_str(), 0); + int ret = ftruncate(ispFd, bufSize); + if (ret < 0) { + LOG(Converter, Error) << "ftruncate() for memfd failed " + << strerror(-ret); + return ret; + } + + FrameBuffer::Plane outPlane; + outPlane.fd = SharedFD(std::move(ispFd)); + outPlane.offset = 0; + outPlane.length = bufSize; + + std::vector planes{ outPlane }; + buffers->emplace_back(std::make_unique(std::move(planes))); + } + + return count; +} + +int SwConverter::Isp::start() +{ + moveToThread(&thread_); + thread_.start(); + return 0; +} + +void SwConverter::Isp::stop() +{ + thread_.exit(); + thread_.wait(); +} + +int SwConverter::start() +{ + return isp_->start(); +} + +void SwConverter::stop() +{ + return isp_->stop(); +} + +int SwConverter::queueBuffers(FrameBuffer *input, + const std::map &outputs) +{ + unsigned int mask = 0; + + /* + * Validate the outputs as a sanity check: at least one output is + * required, all outputs must reference a valid stream and no two + * outputs can reference the same stream. + */ + if (outputs.empty()) + return -EINVAL; + + for (auto [index, buffer] : outputs) { + if (!buffer) + return -EINVAL; + if (index >= 1) /* only single stream atm */ + return -EINVAL; + if (mask & (1 << index)) + return -EINVAL; + + mask |= 1 << index; + } + + process(input, outputs.at(0)); + + return 0; +} + +void SwConverter::process(FrameBuffer *input, FrameBuffer *output) +{ + isp_->invokeMethod(&SwConverter::Isp::process, + ConnectionTypeQueued, input, output); +} + +void SwConverter::Isp::process(FrameBuffer *input, FrameBuffer *output) +{ + const uint8_t *rawData; + uint8_t *rgbData; + + /* Copy metadata from the input buffer */ + FrameMetadata &metadata = output->_d()->metadata(); + metadata.status = input->metadata().status; + metadata.sequence = input->metadata().sequence; + metadata.timestamp = input->metadata().timestamp; + + const FrameBuffer::Plane &plane = input->planes()[0]; + /* TODO: use MappedBuffer class for mapping the frame buffers */ + rawData = (const uint8_t *)mmap(NULL, plane.length, PROT_READ, + MAP_SHARED, plane.fd.get(), 0); + const FrameBuffer::Plane &rgbPlane = output->planes()[0]; + rgbData = (uint8_t *)mmap(NULL, rgbPlane.length, PROT_READ | PROT_WRITE, + MAP_SHARED, rgbPlane.fd.get(), 0); + LOG(Converter, Debug) << "raw length = " << plane.length + << ", rgb length = " << rgbPlane.length; + if (rawData == MAP_FAILED || rgbData == MAP_FAILED) { + LOG(Converter, Error) << "mmap-ing buffer(s) failed"; + metadata.status = FrameMetadata::FrameError; + converter_->outputBufferReady.emit(output); + converter_->inputBufferReady.emit(input); + return; + } + + debayer(rgbData, rawData); + + metadata.planes()[0].bytesused = rgbPlane.length; + + converter_->outputBufferReady.emit(output); + converter_->inputBufferReady.emit(input); +} + +void SwConverter::Isp::debayer(uint8_t *dst, const uint8_t *src) +{ + /* RAW10P input format is assumed */ + + /* output buffer is in BGR32 format and is of (width-2)*(height-2) */ + + int w_out = width_ - 2; + int h_out = height_ - 2; + + unsigned long sumR = 0; + unsigned long sumB = 0; + unsigned long sumG = 0; + + for (int y = 0; y < h_out; y++) { + const uint8_t *pin_base = src + (y + 1) * stride_; + uint8_t *pout = dst + y * w_out * 4; + int phase_y = (y + red_shift_.y) % 2; + + for (int x = 0; x < w_out; x++) { + int phase_x = (x + red_shift_.x) % 2; + int phase = 2 * phase_y + phase_x; + + /* x part of the offset in the input buffer: */ + int x_m1 = x + x / 4; /* offset for (x-1) */ + int x_0 = x + 1 + (x + 1) / 4; /* offset for x */ + int x_p1 = x + 2 + (x + 2) / 4; /* offset for (x+1) */ + /* the colour component value to write to the output */ + unsigned val; + + switch (phase) { + case 0: /* at R pixel */ + /* blue: ((-1,-1)+(1,-1)+(-1,1)+(1,1)) / 4 */ + val = ( *(pin_base + x_m1 - stride_) + + *(pin_base + x_p1 - stride_) + + *(pin_base + x_m1 + stride_) + + *(pin_base + x_p1 + stride_) ) >> 2; + val = val * bNumerat_ / bDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* green: ((0,-1)+(-1,0)+(1,0)+(0,1)) / 4 */ + val = ( *(pin_base + x_0 - stride_) + + *(pin_base + x_p1) + + *(pin_base + x_m1) + + *(pin_base + x_0 + stride_) ) >> 2; + val = val * gNumerat_ / gDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* red: (0,0) */ + val = *(pin_base + x_0); + sumR += val; + val = val * rNumerat_ / rDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* alpha */ + *pout++ = 0xff; + break; + case 1: /* at Gr pixel */ + /* blue: ((0,-1) + (0,1)) / 2 */ + val = ( *(pin_base + x_0 - stride_) + + *(pin_base + x_0 + stride_) ) >> 1; + val = val * bNumerat_ / bDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* green: (0,0) */ + val = *(pin_base + x_0); + sumG += val; + val = val * gNumerat_ / gDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* red: ((-1,0) + (1,0)) / 2 */ + val = ( *(pin_base + x_m1) + + *(pin_base + x_p1) ) >> 1; + val = val * rNumerat_ / rDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* alpha */ + *pout++ = 0xff; + break; + case 2: /* at Gb pixel */ + /* blue: ((-1,0) + (1,0)) / 2 */ + val = ( *(pin_base + x_m1) + + *(pin_base + x_p1) ) >> 1; + val = val * bNumerat_ / bDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* green: (0,0) */ + val = *(pin_base + x_0); + sumG += val; + val = val * gNumerat_ / gDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* red: ((0,-1) + (0,1)) / 2 */ + val = ( *(pin_base + x_0 - stride_) + + *(pin_base + x_0 + stride_) ) >> 1; + val = val * rNumerat_ / rDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* alpha */ + *pout++ = 0xff; + break; + default: /* at B pixel */ + /* blue: (0,0) */ + val = *(pin_base + x_0); + sumB += val; + val = val * bNumerat_ / bDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* green: ((0,-1)+(-1,0)+(1,0)+(0,1)) / 4 */ + val = ( *(pin_base + x_0 - stride_) + + *(pin_base + x_p1) + + *(pin_base + x_m1) + + *(pin_base + x_0 + stride_) ) >> 2; + val = val * gNumerat_ / gDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* red: ((-1,-1)+(1,-1)+(-1,1)+(1,1)) / 4 */ + val = ( *(pin_base + x_m1 - stride_) + + *(pin_base + x_p1 - stride_) + + *(pin_base + x_m1 + stride_) + + *(pin_base + x_p1 + stride_) ) >> 2; + val = val * rNumerat_ / rDenomin_; + *pout++ = (uint8_t)std::min(val, 0xffU); + /* alpha */ + *pout++ = 0xff; + } + } + } + + /* calculate red and blue gains for simple AWB */ + LOG(Converter, Debug) << "sumR = " << sumR + << ", sumB = " << sumB << ", sumG = " << sumG; + + sumG /= 2; /* the number of G pixels is twice as big vs R and B ones */ + + /* normalize red, blue, and green sums to fit into 22-bit value */ + unsigned long fRed = sumR / 0x400000; + unsigned long fBlue = sumB / 0x400000; + unsigned long fGreen = sumG / 0x400000; + unsigned long fNorm = std::max({ 1UL, fRed, fBlue, fGreen }); + sumR /= fNorm; + sumB /= fNorm; + sumG /= fNorm; + + LOG(Converter, Debug) << "fNorm = " << fNorm; + LOG(Converter, Debug) << "Normalized: sumR = " << sumR + << ", sumB= " << sumB << ", sumG = " << sumG; + + /* make sure red/blue gains never exceed approximately 256 */ + unsigned long minDenom; + rNumerat_ = (sumR + sumB + sumG) / 3; + minDenom = rNumerat_ / 0x100; + rDenomin_ = std::max(minDenom, sumR); + bNumerat_ = rNumerat_; + bDenomin_ = std::max(minDenom, sumB); + gNumerat_ = rNumerat_; + gDenomin_ = std::max(minDenom, sumG); + + LOG(Converter, Debug) << "rGain = [ " + << rNumerat_ << " / " << rDenomin_ + << " ], bGain = [ " << bNumerat_ << " / " << bDenomin_ + << " ], gGain = [ " << gNumerat_ << " / " << gDenomin_ + << " (minDenom = " << minDenom << ")"; +} + +} /* namespace libcamera */ diff --git a/src/libcamera/converter/meson.build b/src/libcamera/converter/meson.build index 2aa72fe4..1f1e0ea4 100644 --- a/src/libcamera/converter/meson.build +++ b/src/libcamera/converter/meson.build @@ -1,5 +1,6 @@ # SPDX-License-Identifier: CC0-1.0 libcamera_sources += files([ - 'converter_v4l2_m2m.cpp' + 'converter_softw.cpp', + 'converter_v4l2_m2m.cpp', ]) From patchwork Sun Aug 6 18:01:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 18926 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3C487C32A9 for ; Sun, 6 Aug 2023 18:02:09 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D4F75627ED; Sun, 6 Aug 2023 20:02:08 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1691344928; bh=DQrF5txzyFHQbU1zoqKHmA7Mq6yOXX+tka4gc5W3Oyo=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=Qt9WqEMMGQzh4I8UJWf6PRWOgfer320RUcKCBWR/nztGVud5adtSn7piDipa6Ug1e WzAdaODaCsqYylSHrmRGUHPP/yGIZ5GxYlqvER+4WkZHT6QKXLzv1RtCO2jauOP/VS eccZd3L4q6vfm9icurnPAKI/zUSXvkUbj9hc4G6gJbWcXKY81vb4/F9wc0t6XFDKi4 MEgPvscsZ6TWjnysXTNNNiJaAsgJ1NXJ23sSiaOF2Ehatv8nzDt6Jw1G7C9E7azv4e sznl/0C3psZrf69xiZV1ULabP4T1IgCtfJQoIAhHS/1Ut9XFzQAfJwOcMiMzWXO5Z+ SuYlC6G8G0lmA== Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com [IPv6:2a00:1450:4864:20::62a]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EC1A0627E9 for ; Sun, 6 Aug 2023 20:02:07 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="U0BVmJ9Z"; dkim-atps=neutral Received: by mail-ej1-x62a.google.com with SMTP id a640c23a62f3a-99bccc9ec02so580930466b.2 for ; Sun, 06 Aug 2023 11:02:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1691344927; x=1691949727; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/YB7of9WDJI9hMhptxdPm9+Lbuqc8NzAVcGxXJhM5bM=; b=U0BVmJ9ZP/NONAAccAjbbxelnMUeCICUt2MuQ72WpG1+0wiU5NXfehefYnXxgQBW5B 5OKGGjBpTULvGglqyHzmD97zhSd0nzhDPQ0sYLMUO3nbJEtNHPJDImDRj4RVRE4Vu8ek +su4BQzxFbF6aRBH/MqFOPEOblQat9NTccQSQd+c073j3TdnfnQTq4NEgANzzrYBLx6R 5Ci2CgAbTKIlWjQb0FP8iKIzyC+aFr9sfIUSOTfH3OsHcw8EPRYgVADpiYJevqa7h78G T5e1mMk9xIwZ7Z5fTKDZrpcsmLX9umt6fvAKwrglLpJw/8REULoCmYndzII3T7Qa3ii7 WuTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691344927; x=1691949727; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/YB7of9WDJI9hMhptxdPm9+Lbuqc8NzAVcGxXJhM5bM=; b=BjpTJ83RTq9ED2SYyLVDpM0ijHcpakND5apy4ejjFJee6hyN68qEwVVz7gm4L8C6tJ YwOMzo2a5tHJ2xiQ5a9GA3TiaZJAvIUANSEv6Op7TEPgpuyzf4stfaJklxJy7niABaoN K1g5UrvzxVDjNTqY4We6uH7nT0t2gxoarV0f7izYKZP+w1v+UD7hJKtBFf9p009qhvs1 LcOzzBNZlEITuk8xO81zdo/8q9mq0YadCG9Zx5dqgSRWxzWobQZ2HZWyRRBnmOlaMN0Z +ZXeBKK9Wqd0w7I/f3QKZ06VuHPksNkVw1iwC0I+AkEJcT8uzjTykClAJ6uZ5tWLOFdG t2bA== X-Gm-Message-State: AOJu0YzJWrnLp2cT2CfU4/tiIv/GIhFCueIJ1ndDZSprq66DY0oFOraB FWKrL2mstZFIshOUNJEtYwH/edggD1G63TLQf4Y= X-Google-Smtp-Source: AGHT+IGAN9g04jQ6FBC9L2lBHGmVsCPTx+6Kyh2b9XYFX4ZjVIP3JAb3U/5InYvgGS8psbI0j4hnQQ== X-Received: by 2002:a17:906:3196:b0:99c:c138:a694 with SMTP id 22-20020a170906319600b0099cc138a694mr4070551ejy.43.1691344927458; Sun, 06 Aug 2023 11:02:07 -0700 (PDT) Received: from Lat-5310.. ([87.116.165.37]) by smtp.gmail.com with ESMTPSA id j15-20020a170906430f00b0099b76c3041csm4197396ejm.7.2023.08.06.11.02.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 06 Aug 2023 11:02:07 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Sun, 6 Aug 2023 21:01:36 +0300 Message-Id: <20230806180136.304156-3-andrey.konovalov@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230806180136.304156-1-andrey.konovalov@linaro.org> References: <20230806180136.304156-1-andrey.konovalov@linaro.org> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 2/2] [DNI] libcamera: pipeline: simple: a hack to use sotware converter with qcom-camss X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Andrey Konovalov via libcamera-devel From: Andrey Konovalov Reply-To: Andrey Konovalov Cc: bryan.odonoghue@linaro.org, srinivas.kandagatla@linaro.org Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Quick and simple hack mostly to avoid calling acquireMediaDevice() for a converter without a media device underneath. ConverterFactoryBase::create() also relies on MediaDevice, so this hack doesn't use it with the software converter for now. Signed-off-by: Andrey Konovalov --- src/libcamera/pipeline/simple/simple.cpp | 22 ++++++++++++++++++++-- 1 file changed, 20 insertions(+), 2 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 05ba76bc..e5c53003 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -37,6 +37,7 @@ #include "libcamera/internal/v4l2_subdevice.h" #include "libcamera/internal/v4l2_videodevice.h" +#include "libcamera/internal/converter/converter_softw.h" namespace libcamera { @@ -194,7 +195,7 @@ static const SimplePipelineInfo supportedDevices[] = { { "imx7-csi", { { "pxp", 1 } } }, { "j721e-csi2rx", {} }, { "mxc-isi", {} }, - { "qcom-camss", {} }, + { "qcom-camss", { { "software", 1 } } }, { "sun6i-csi", {} }, }; @@ -359,6 +360,11 @@ private: std::map entities_; MediaDevice *converter_; + // dummy function to force Converter to work w/o media device: + MediaDevice *acquireSoftwareConverter(const char *name) + { + return (new MediaDevice(name)); + } }; /* ----------------------------------------------------------------------------- @@ -497,7 +503,11 @@ int SimpleCameraData::init() /* Open the converter, if any. */ MediaDevice *converter = pipe->converter(); if (converter) { - converter_ = ConverterFactoryBase::create(converter); + // hack to make Converter to work w/o media device underneath: + if (!converter->isValid() && converter->deviceNode() == "software") + converter_ = std::make_unique(converter); + else + converter_ = ConverterFactoryBase::create(converter); if (!converter_) { LOG(SimplePipeline, Warning) << "Failed to create converter, disabling format conversion"; @@ -1410,6 +1420,14 @@ bool SimplePipelineHandler::match(DeviceEnumerator *enumerator) return false; for (const auto &[name, streams] : info->converters) { + if (!strcmp(name, "software")) { + converter_ = acquireSoftwareConverter(name); + if (converter_) { + numStreams = streams; + break; + } + } + DeviceMatch converterMatch(name); converter_ = acquireMediaDevice(enumerator, converterMatch); if (converter_) {