From patchwork Tue Apr 22 21:59:12 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bryan O'Donoghue X-Patchwork-Id: 23232 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E4895C327D for ; Tue, 22 Apr 2025 21:59:57 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5FCE668AD9; Tue, 22 Apr 2025 23:59:57 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="mKkGIi1l"; dkim-atps=neutral Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3666368B36 for ; Tue, 22 Apr 2025 23:59:42 +0200 (CEST) Received: by mail-wm1-x331.google.com with SMTP id 5b1f17b1804b1-43d04dc73b7so52251125e9.3 for ; Tue, 22 Apr 2025 14:59:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745359181; x=1745963981; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rPosaOH1mtkxvmAKqwhdY3CYDHFWTfkbfbfqGtuksa4=; b=mKkGIi1lT7cMNuWX47AVqWn3elJoNBi2w/Ebg6DmkTawjN5nTvjqRobEeI89znvYNa WM2wdZeerhRlaBLf/8JyaHwrraeU63m2Ww8yMuPWzsJEIwLx7ptRkRrZme1D9UHcSfY0 HONXJlNsGS44gWAC+8Ae0CU2jmMcVv1TL6grLb1L19YFUp8W4dFpc4BK3OCZXNl5t5QI aMBrA0YNUTmGr2mnXmN+JGM/3ZWjcEvZApMy45pQBN5kmPprPIIGYX82NNCW3rCoiYkF zUCqI1rHw0UHp0anSabW60VvQBywC7tIqfr823WjxzabXWGAkdhGCXFmZGqDlv7nhALt 0RwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745359181; x=1745963981; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rPosaOH1mtkxvmAKqwhdY3CYDHFWTfkbfbfqGtuksa4=; b=oumPGtAJlvySoXewSOktjByr3IiJ4PMaZtjDPzamB20yRG6ony3b2HXNlPlA79FTf5 9AbXg4k3OGQhcYoP3G0L04fgqgsK2EIcX2S0SZ1a2SK/hshTgIJlkcP3I3rxVY1q8743 XIaCp+tscRxkTrGhdA1TkC3wQhe9Zo+CCciUHYdMTx/TFqL5+QwErPbPqve0G9ylXtkB fIg82h4cklBOSJmsJ1ABuW9x6BlBusduppuiCHReUU40qdZ1AguVFdhBzOQzjU/bdK7u 85w6iYBwVqDZr76Lfsf66lz4x6TloGUlXi+FPEW7kzOezi4oT9Mg8fh0AERI51GxdiWW q4CQ== X-Gm-Message-State: AOJu0Yx3zDz11kBOqAine/VmGtJL9ld61xDGGiaRMjnI+qv2+uwWEZzz ktZ171MKarIzWgEQafogJq7/PhB/wxh9zXoTMpn4vheUMIkgJm1hvn1/y6USX/33GRxolC5Mw93 u5VU= X-Gm-Gg: ASbGncv8qcG3c5Zm4JY4SN9fCWjvoGAWcuMc/zj6sFSGoVIy3tFKeb7Cke1LXzm1oKq cQtnVg4ADitwmF4RtUiJc/Nb/XA55aDtmp5sNO8fzz3qL+BfQ0/mHXr69Fd7fz4Cb0sydCSilI3 qmp1yR0yLBKHGnE6x6SzKpt9VboMUcXnxkzeJomid0RY7+j7xu0QH/TDu+tOSkGWAMsgI/+UCXC XD+n3uOJ58QGq71gcodKD6oTWb4388wfDIB7Fo93NIEdzIWEGqQDhiC9NgQOo8t1ilToUvCL7/g O8hWiZo8LE9HSW0Cq5EC6d7HBI+oG7zxqZ6H3nUjVf4DrA8e/ngSqIzKt61RdBiQJqeIrtp70w5 o2oCP30KfyGLYyv0pdOHAQe5JCbZMeh8= X-Google-Smtp-Source: AGHT+IHjuU3u6SF/3/4Ta9WlQP9bnpbH3jDhkuonLtp7sCPzymz6BASspe1bAuwohx5/+oOkubQ+TQ== X-Received: by 2002:a05:600c:1e0c:b0:43d:fa59:bcee with SMTP id 5b1f17b1804b1-4406ac1feebmr146436505e9.33.1745359181477; Tue, 22 Apr 2025 14:59:41 -0700 (PDT) Received: from inspiron14p-linux.ht.home (188-141-3-146.dynamic.upc.ie. [188.141.3.146]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-44092d2eccesm2726615e9.20.2025.04.22.14.59.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Apr 2025 14:59:40 -0700 (PDT) From: Bryan O'Donoghue To: libcamera-devel@lists.libcamera.org Cc: hdegoede@redhat.com, mzamazal@redhat.com, bryan.odonoghue@linaro.org, bod.linux@nxsw.ie Subject: [PATCH 19/27] libcamera: software_isp: debayer_egl: Add an eGL debayer class Date: Tue, 22 Apr 2025 22:59:12 +0100 Message-ID: <20250422215920.4297-20-bryan.odonoghue@linaro.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250422215920.4297-1-bryan.odonoghue@linaro.org> References: <20250422215920.4297-1-bryan.odonoghue@linaro.org> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a class to run the existing glsl debayer shaders on a GBM surface. Signed-off-by: Bryan O'Donoghue --- src/libcamera/software_isp/debayer_egl.cpp | 529 +++++++++++++++++++++ src/libcamera/software_isp/debayer_egl.h | 168 +++++++ src/libcamera/software_isp/meson.build | 30 ++ 3 files changed, 727 insertions(+) create mode 100644 src/libcamera/software_isp/debayer_egl.cpp create mode 100644 src/libcamera/software_isp/debayer_egl.h diff --git a/src/libcamera/software_isp/debayer_egl.cpp b/src/libcamera/software_isp/debayer_egl.cpp new file mode 100644 index 00000000..008938f8 --- /dev/null +++ b/src/libcamera/software_isp/debayer_egl.cpp @@ -0,0 +1,529 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Linaro Ltd. + * + * Authors: + * Bryan O'Donoghue + * + * debayer_cpu.cpp - EGL based debayering class + */ + +#include +#include +#include + +#include + +#include "libcamera/internal/glsl_shaders.h" +#include "debayer_egl.h" + +namespace libcamera { + +DebayerEGL::DebayerEGL(std::unique_ptr stats) + : stats_(std::move(stats)) +{ +} + +DebayerEGL::~DebayerEGL() +{ + if (eglImageBayerIn_) + delete eglImageBayerIn_; +} + +// 0xB0D make into public in the base +static bool isStandardBayerOrder(BayerFormat::Order order) +{ + return order == BayerFormat::BGGR || order == BayerFormat::GBRG || + order == BayerFormat::GRBG || order == BayerFormat::RGGB; +} + +// 0xB0D make into public in the base +int DebayerEGL::getInputConfig(PixelFormat inputFormat, DebayerInputConfig &config) +{ + BayerFormat bayerFormat = + BayerFormat::fromPixelFormat(inputFormat); + + if ((bayerFormat.bitDepth == 8 || bayerFormat.bitDepth == 10) && + bayerFormat.packing == BayerFormat::Packing::None && + isStandardBayerOrder(bayerFormat.order)) { + config.bpp = (bayerFormat.bitDepth + 7) & ~7; + config.patternSize.width = 2; + config.patternSize.height = 2; + config.outputFormats = std::vector({ formats::ARGB8888 }); + return 0; + } + + if (bayerFormat.bitDepth == 10 && + bayerFormat.packing == BayerFormat::Packing::CSI2 && + isStandardBayerOrder(bayerFormat.order)) { + config.bpp = 10; + config.patternSize.width = 4; /* 5 bytes per *4* pixels */ + config.patternSize.height = 2; + config.outputFormats = std::vector({ formats::ARGB8888 }); + return 0; + } + + LOG(Debayer, Info) + << "Unsupported input format " << inputFormat.toString(); + return -EINVAL; +} + +// 0xB0D make into public in the base +int DebayerEGL::getOutputConfig(PixelFormat outputFormat, DebayerOutputConfig &config) +{ + if (outputFormat == formats::ARGB8888) { + config.bpp = 32; + return 0; + } + + LOG(Debayer, Error) + << "Unsupported output format " << outputFormat.toString(); + + return -EINVAL; +} + +int DebayerEGL::getShaderVariableLocations(void) +{ + attributeVertex_ = glGetAttribLocation(programId_, "vertexIn"); + attributeTexture_ = glGetAttribLocation(programId_, "textureIn"); + + textureUniformY_ = glGetUniformLocation(programId_, "tex_y"); + textureUniformU_ = glGetUniformLocation(programId_, "tex_u"); + textureUniformV_ = glGetUniformLocation(programId_, "tex_v"); + textureUniformStep_ = glGetUniformLocation(programId_, "tex_step"); + textureUniformSize_ = glGetUniformLocation(programId_, "tex_size"); + textureUniformStrideFactor_ = glGetUniformLocation(programId_, "stride_factor"); + textureUniformBayerFirstRed_ = glGetUniformLocation(programId_, "tex_bayer_first_red"); + textureUniformProjMatrix_ = glGetUniformLocation(programId_, "proj_matrix"); + + LOG(Debayer, Info) << "vertexIn " << attributeVertex_ << " textureIn " << attributeTexture_ + << " tex_y " << textureUniformY_ + << " tex_u " << textureUniformU_ + << " tex_v " << textureUniformV_ + << " tex_step " << textureUniformStep_ + << " tex_size " << textureUniformSize_ + << " stride_factor " << textureUniformStrideFactor_ + << " tex_bayer_first_red " << textureUniformBayerFirstRed_ + << " proj_matrix " << textureUniformProjMatrix_; + return 0; +} + +int DebayerEGL::initBayerShaders(PixelFormat inputFormat, PixelFormat outputFormat) +{ + std::vector shaderEnv; + unsigned int fragmentShaderDataLen; + unsigned char *fragmentShaderData; + unsigned int vertexShaderDataLen; + unsigned char *vertexShaderData; + GLenum err; + + swapRedBlueGains_ = false; + + // Target gles 100 glsl requires "#version x" as first directive in shader + egl_.pushEnv(shaderEnv, "#version 100"); + + // Specify GL_OES_EGL_image_external + egl_.pushEnv(shaderEnv, "#extension GL_OES_EGL_image_external: enable"); + + switch (outputFormat) { + case formats::ARGB8888: + break; + case formats::BGRA8888: + swapRedBlueGains_ = true; + break; + default: + goto invalid_fmt; + } + + // Pixel location parameters + switch (inputFormat) { + case libcamera::formats::SBGGR8: + case libcamera::formats::SBGGR10_CSI2P: + case libcamera::formats::SBGGR12_CSI2P: + firstRed_x_ = 1.0; + firstRed_y_ = 1.0; + break; + case libcamera::formats::SGBRG8: + case libcamera::formats::SGBRG10_CSI2P: + case libcamera::formats::SGBRG12_CSI2P: + firstRed_x_ = 0.0; + firstRed_y_ = 1.0; + break; + case libcamera::formats::SGRBG8: + case libcamera::formats::SGRBG10_CSI2P: + case libcamera::formats::SGRBG12_CSI2P: + firstRed_x_ = 1.0; + firstRed_y_ = 0.0; + break; + case libcamera::formats::SRGGB8: + case libcamera::formats::SRGGB10_CSI2P: + case libcamera::formats::SRGGB12_CSI2P: + firstRed_x_ = 0.0; + firstRed_y_ = 0.0; + break; + default: + goto invalid_fmt; + break; + }; + + // Shader selection + switch (inputFormat) { + case libcamera::formats::SBGGR8: + case libcamera::formats::SGBRG8: + case libcamera::formats::SGRBG8: + case libcamera::formats::SRGGB8: + fragmentShaderData = bayer_8_frag; + fragmentShaderDataLen = bayer_8_frag_len; + vertexShaderData = bayer_8_vert; + vertexShaderDataLen = bayer_8_vert_len; + break; + case libcamera::formats::SBGGR10_CSI2P: + case libcamera::formats::SGBRG10_CSI2P: + case libcamera::formats::SGRBG10_CSI2P: + case libcamera::formats::SRGGB10_CSI2P: + egl_.pushEnv(shaderEnv, "#define RAW10P"); + fragmentShaderData = bayer_1x_packed_frag; + fragmentShaderDataLen = bayer_1x_packed_frag_len; + vertexShaderData = identity_vert; + vertexShaderDataLen = identity_vert_len; + break; + case libcamera::formats::SBGGR12_CSI2P: + case libcamera::formats::SGBRG12_CSI2P: + case libcamera::formats::SGRBG12_CSI2P: + case libcamera::formats::SRGGB12_CSI2P: + egl_.pushEnv(shaderEnv, "#define RAW12P"); + fragmentShaderData = bayer_1x_packed_frag; + fragmentShaderDataLen = bayer_1x_packed_frag_len; + vertexShaderData = identity_vert; + vertexShaderDataLen = identity_vert_len; + break; + default: + goto invalid_fmt; + break; + }; + + if (egl_.compileVertexShader(vertexShaderId_, vertexShaderData, vertexShaderDataLen, shaderEnv)) + goto compile_fail; + + if (egl_.compileFragmentShader(fragmentShaderId_, fragmentShaderData, fragmentShaderDataLen, shaderEnv)) + goto compile_fail; + + if (egl_.linkProgram(programId_, vertexShaderId_, fragmentShaderId_)) + goto link_fail; + + egl_.dumpShaderSource(vertexShaderId_); + egl_.dumpShaderSource(fragmentShaderId_); + + /* Ensure we set the programId_ */ + egl_.useProgram(programId_); + err = glGetError(); + if (err != GL_NO_ERROR) + goto program_fail; + + if (getShaderVariableLocations()) + goto parameters_fail; + + return 0; + +parameters_fail: + LOG(Debayer, Error) << "Program parameters fail"; + return -ENODEV; + +program_fail: + LOG(Debayer, Error) << "Use program error " << err; + return -ENODEV; + +link_fail: + LOG(Debayer, Error) << "Linking program fail"; + return -ENODEV; + + +compile_fail: + LOG(Debayer, Error) << "Compile debayer shaders fail"; + return -ENODEV; + +invalid_fmt: + LOG(Debayer, Error) << "Unsupported input output format combination"; + return -EINVAL; +} + +int DebayerEGL::configure(const StreamConfiguration &inputCfg, + const std::vector> &outputCfgs, + bool ccmEnabled) +{ + if (getInputConfig(inputCfg.pixelFormat, inputConfig_) != 0) + return -EINVAL; + + if (stats_->configure(inputCfg) != 0) + return -EINVAL; + + const Size &stats_pattern_size = stats_->patternSize(); + if (inputConfig_.patternSize.width != stats_pattern_size.width || + inputConfig_.patternSize.height != stats_pattern_size.height) { + LOG(Debayer, Error) + << "mismatching stats and debayer pattern sizes for " + << inputCfg.pixelFormat.toString(); + return -EINVAL; + } +(void)ccmEnabled; + inputConfig_.stride = inputCfg.stride; + width_ = inputCfg.size.width; + height_ = inputCfg.size.height; + + if (outputCfgs.size() != 1) { + LOG(Debayer, Error) + << "Unsupported number of output streams: " + << outputCfgs.size(); + return -EINVAL; + } + + LOG(Debayer, Info) << "Input size " << inputCfg.size << " stride " << inputCfg.stride; + + if (gbmSurface_.initSurface(inputCfg.size.width, inputCfg.size.height)) + return -ENODEV; + + if (egl_.initEGLContext(&gbmSurface_)) + return -ENODEV; + + eglImageBayerIn_ = new eGLImage(width_, height_, 32); + + // Create a single BO (calling gbm_surface_lock_front_buffer() again before gbm_surface_release_buffer() would create another BO) + if (gbmSurface_.mapSurface()) + return -ENODEV; + + StreamConfiguration &outputCfg = outputCfgs[0]; + SizeRange outSizeRange = sizes(inputCfg.pixelFormat, inputCfg.size); + + outputConfig_.stride = gbmSurface_.getStride(); + outputConfig_.frameSize = gbmSurface_.getFrameSize(); + + LOG(Debayer, Debug) << "Overriding stream config stride " + << outputCfg.stride << " with GBM surface stride " + << outputConfig_.stride; + outputCfg.stride = outputConfig_.stride; + + if (!outSizeRange.contains(outputCfg.size) || outputConfig_.stride != outputCfg.stride) { + LOG(Debayer, Error) + << "Invalid output size/stride: " + << "\n " << outputCfg.size << " (" << outSizeRange << ")" + << "\n " << outputCfg.stride << " (" << outputConfig_.stride << ")"; + return -EINVAL; + } + + window_.x = ((inputCfg.size.width - outputCfg.size.width) / 2) & + ~(inputConfig_.patternSize.width - 1); + window_.y = ((inputCfg.size.height - outputCfg.size.height) / 2) & + ~(inputConfig_.patternSize.height - 1); + window_.width = outputCfg.size.width; + window_.height = outputCfg.size.height; + + /* Don't pass x,y since process() already adjusts src before passing it */ + stats_->setWindow(Rectangle(window_.size())); + + LOG(Debayer, Info) << "Input width " << inputCfg.size.width << " height " << inputCfg.size.height; + LOG(Debayer, Info) << "Output width " << outputCfg.size.width << " height " << outputCfg.size.height; + LOG(Debayer, Info) << "Output stride " << outputCfg.size.width << " height " << outputCfg.size.height; + + if (initBayerShaders(inputCfg.pixelFormat, outputCfg.pixelFormat)) + return -EINVAL; + + return 0; +} + +Size DebayerEGL::patternSize(PixelFormat inputFormat) +{ + DebayerEGL::DebayerInputConfig config; + + if (getInputConfig(inputFormat, config) != 0) + return {}; + + return config.patternSize; +} + +std::vector DebayerEGL::formats(PixelFormat inputFormat) +{ + DebayerEGL::DebayerInputConfig config; + + if (getInputConfig(inputFormat, config) != 0) + return std::vector(); + + return config.outputFormats; +} + +std::tuple +DebayerEGL::strideAndFrameSize(const PixelFormat &outputFormat, const Size &size) +{ + DebayerEGL::DebayerOutputConfig config; + + if (getOutputConfig(outputFormat, config) != 0) + return std::make_tuple(0, 0); + + /* round up to multiple of 8 for 64 bits alignment */ + unsigned int stride = (size.width * config.bpp / 8 + 7) & ~7; + + return std::make_tuple(stride, stride * size.height); +} + +void DebayerEGL::setShaderVariableValues(void) +{ + /* + * Raw Bayer 8-bit, and packed raw Bayer 10-bit/12-bit formats + * are stored in a GL_LUMINANCE texture. The texture width is + * equal to the stride. + */ + GLfloat firstRed[] = {firstRed_x_, firstRed_y_}; + GLfloat imgSize[] = {(GLfloat)width_, + (GLfloat)height_}; + GLfloat Step[] = {1.0f / (inputConfig_.stride - 1), + 1.0f / (height_ - 1)}; + GLfloat Stride = 1.0f; + GLfloat projIdentityMatrix[] = { + 1, 0, 0, 0, + 0, 1, 0, 0, + 0, 0, 1, 0, + 0, 0, 0, 1 + }; + + // vertexIn - bayer_8.vert + glEnableVertexAttribArray(attributeVertex_); + glVertexAttribPointer(attributeVertex_, 2, GL_FLOAT, GL_TRUE, + 2 * sizeof(GLfloat), vcoordinates); + + // textureIn - bayer_8.vert + glEnableVertexAttribArray(attributeTexture_); + glVertexAttribPointer(attributeTexture_, 2, GL_FLOAT, GL_TRUE, + 2 * sizeof(GLfloat), tcoordinates); + + glUniform1i(textureUniformY_, 0); // tex_y - bayer_8.vert - set for no reason + glUniform2fv(textureUniformBayerFirstRed_, 1, firstRed); // tex_bayer_first_red - bayer_8.vert + glUniform2fv(textureUniformSize_, 1, imgSize); // tex_size - bayer_8.vert + glUniform2fv(textureUniformStep_, 1, Step); // tex_step - bayer_8.vert + glUniform1f(textureUniformStrideFactor_, Stride); // stride_factor - identity.vert + glUniformMatrix4fv(textureUniformProjMatrix_, 1, + GL_FALSE, projIdentityMatrix); // No scaling + + LOG(Debayer, Debug) << "vertexIn " << attributeVertex_ << " textureIn " << attributeTexture_ + << " tex_y " << textureUniformY_ + << " tex_u " << textureUniformU_ + << " tex_v " << textureUniformV_ + << " tex_step " << textureUniformStep_ + << " tex_size " << textureUniformSize_ + << " stride_factor " << textureUniformStrideFactor_ + << " tex_bayer_first_red " << textureUniformBayerFirstRed_; + + LOG (Debayer, Debug) << "textureUniformY_ = 0 " << + " firstRed.x " << firstRed[0] << + " firstRed.y " << firstRed[1] << + " textureUniformSize_.width " << imgSize[0] << " " + " textureUniformSize_.height " << imgSize[1] << + " textureUniformStep_.x " << Step[0] << + " textureUniformStep_.y " << Step[1] << + " textureUniformStrideFactor_ " << Stride << + " textureUniformProjMatrix_ " << textureUniformProjMatrix_; + return; +} + +void DebayerEGL::debayerGPU(MappedFrameBuffer &in, MappedFrameBuffer &out) +{ + LOG(Debayer, Debug) + << "Input height " << height_ + << " width " << width_ + << " fd " << in.fds()[0]; + + // eGL context switch + egl_.makeCurrent(); + + // make texture unit 0 explicit this doesn't really matter probably remove ? + glActiveTexture(GL_TEXTURE0); + + // Greate a standard texture + // we will replace this with the DMA version at some point + egl_.createTexture2D(eglImageBayerIn_, inputConfig_.stride, height_, in.planes()[0].data()); + + // Setup the scene + setShaderVariableValues(); + glViewport(0, 0, width_, height_); + glClear(GL_COLOR_BUFFER_BIT); + glDisable(GL_BLEND); + + // Draw the scene + glDrawArrays(GL_TRIANGLE_FAN, 0, DEBAYER_OPENGL_COORDS); + + // eglclientWaitScynKhr / eglwaitsynckr ? + egl_.swapBuffers(); + + // Copy from the output GBM buffer to our output plane + // once we get render to texture working the + // explicit lock ioctl, memcpy and unlock ioctl won't be required + gbmSurface_.getFrameBufferData(out.planes()[0].data(), out.planes()[0].size()); +} + +void DebayerEGL::process(uint32_t frame, FrameBuffer *input, FrameBuffer *output, DebayerParams params) +{ + bench_.startFrame(); + + std::vector dmaSyncers; + + dmaSyncBegin(dmaSyncers, input, output); + + setParams(params); + + /* Copy metadata from the input buffer */ + FrameMetadata &metadata = output->_d()->metadata(); + metadata.status = input->metadata().status; + metadata.sequence = input->metadata().sequence; + metadata.timestamp = input->metadata().timestamp; + + MappedFrameBuffer in(input, MappedFrameBuffer::MapFlag::Read); + MappedFrameBuffer out(output, MappedFrameBuffer::MapFlag::Write); + if (!in.isValid() || !out.isValid()) { + LOG(Debayer, Error) << "mmap-ing buffer(s) failed"; + metadata.status = FrameMetadata::FrameError; + return; + } + + debayerGPU(in, out); + + dmaSyncers.clear(); + + bench_.finishFrame(); + + metadata.planes()[0].bytesused = out.planes()[0].size(); + + // Calculate stats for the whole frame + stats_->processFrame(frame, 0, input); + + outputBufferReady.emit(output); + inputBufferReady.emit(input); +} + +SizeRange DebayerEGL::sizes(PixelFormat inputFormat, const Size &inputSize) +{ + Size patternSize = this->patternSize(inputFormat); + unsigned int borderHeight = patternSize.height; + + if (patternSize.isNull()) + return {}; + + /* No need for top/bottom border with a pattern height of 2 */ + if (patternSize.height == 2) + borderHeight = 0; + + /* + * For debayer interpolation a border is kept around the entire image + * and the minimum output size is pattern-height x pattern-width. + */ + if (inputSize.width < (3 * patternSize.width) || + inputSize.height < (2 * borderHeight + patternSize.height)) { + LOG(Debayer, Warning) + << "Input format size too small: " << inputSize.toString(); + return {}; + } + + return SizeRange(Size(patternSize.width, patternSize.height), + Size((inputSize.width - 2 * patternSize.width) & ~(patternSize.width - 1), + (inputSize.height - 2 * borderHeight) & ~(patternSize.height - 1)), + patternSize.width, patternSize.height); +} + +} /* namespace libcamera */ diff --git a/src/libcamera/software_isp/debayer_egl.h b/src/libcamera/software_isp/debayer_egl.h new file mode 100644 index 00000000..14f6997a --- /dev/null +++ b/src/libcamera/software_isp/debayer_egl.h @@ -0,0 +1,168 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2025, Bryan O'Donoghue. + * + * Authors: + * Bryan O'Donoghue + * + * debayer_opengl.h - EGL debayer header + */ + +#pragma once + +#include +#include +#include + +#define GL_GLEXT_PROTOTYPES +#include +#define EGL_EGLEXT_PROTOTYPES +#include +#include +#include + +#include + +#include "debayer.h" +#include "egl.h" + +#include "libcamera/internal/bayer_format.h" +#include "libcamera/internal/framebuffer.h" +#include "libcamera/internal/mapped_framebuffer.h" +#include "libcamera/internal/software_isp/benchmark.h" +#include "libcamera/internal/software_isp/swstats_cpu.h" + +namespace libcamera { + +/** + * \class DebayerEGL + * \brief Class for debayering using an EGL Shader + * + * Implements an EGL shader based debayering solution. + */ +class DebayerEGL : public Debayer +{ +public: + /** + * \brief Constructs a DebayerEGL object. + * \param[in] stats Pointer to the stats object to use. + */ + DebayerEGL(std::unique_ptr stats); + ~DebayerEGL(); + + /* + * Setup the Debayer object according to the passed in parameters. + * Return 0 on success, a negative errno value on failure + * (unsupported parameters). + */ + int configure(const StreamConfiguration &inputCfg, + const std::vector> &outputCfgs, + bool ccmEnabled); + + /* + * Get width and height at which the bayer-pattern repeats. + * Return pattern-size or an empty Size for an unsupported inputFormat. + */ + Size patternSize(PixelFormat inputFormat); + + std::vector formats(PixelFormat input); + std::tuple + strideAndFrameSize(const PixelFormat &outputFormat, const Size &size); + + void process(uint32_t frame, FrameBuffer *input, FrameBuffer *output, DebayerParams params); + + /** + * \brief Get the file descriptor for the statistics. + * + * \return the file descriptor pointing to the statistics. + */ + const SharedFD &getStatsFD() { return stats_->getStatsFD(); } + + /** + * \brief Get the output frame size. + * + * \return The output frame size. + */ + unsigned int frameSize() { return outputConfig_.frameSize; } + + SizeRange sizes(PixelFormat inputFormat, const Size &inputSize); + + /** + * \brief EGL context is associated with a PID so don't moveToThread + * + * \return nothing + */ + void moveToThread(Thread *thread) { (void)thread; return; } + +private: + + int getInputConfig(PixelFormat inputFormat, DebayerInputConfig &config); + int getOutputConfig(PixelFormat outputFormat, DebayerOutputConfig &config); + int setupStandardBayerOrder(BayerFormat::Order order); + void pushEnv(std::vector &shaderEnv, const char *str); + int initBayerShaders(PixelFormat inputFormat, PixelFormat outputFormat); + int initEGLContext(); + int generateTextures(); + int compileShaderProgram(GLuint &shaderId, GLenum shaderType, + unsigned char *shaderData, int shaderDataLen, + std::vector shaderEnv); + int linkShaderProgram(void); + int getShaderVariableLocations(); + void setShaderVariableValues(void); + void configureTexture(GLuint &texture); + void debayerGPU(MappedFrameBuffer &in, MappedFrameBuffer &out); + + // Shader program identifiers + GLuint vertexShaderId_; + GLuint fragmentShaderId_; + GLuint programId_; + enum { + BAYER_INPUT_INDEX = 0, + BAYER_OUTPUT_INDEX, + BAYER_BUF_NUM, + }; + + // Pointer to object representing input texture + eGLImage *eglImageBayerIn_; + + // Shader parameters + float firstRed_x_; + float firstRed_y_; + GLint attributeVertex_; + GLint attributeTexture_; + GLint textureUniformY_; + GLint textureUniformU_; + GLint textureUniformV_; + GLint textureUniformStep_; + GLint textureUniformSize_; + GLint textureUniformStrideFactor_; + GLint textureUniformBayerFirstRed_; + GLint textureUniformProjMatrix_; + + Rectangle window_; + std::unique_ptr stats_; + eGL egl_; + GBM gbmSurface_; + uint32_t width_; + uint32_t height_; + + #define DEBAYER_OPENGL_COORDS 4 + + GLfloat vcoordinates[DEBAYER_OPENGL_COORDS][2] = { + /* Vertex coordinates */ + -1.0f, -1.0f, + -1.0f, +1.0f, + +1.0f, +1.0f, + +1.0f, -1.0f, + }; + + GLfloat tcoordinates[DEBAYER_OPENGL_COORDS][2] = { + /* Texture coordinates */ + 0.0f, 1.0f, + 0.0f, 0.0f, + 1.0f, 0.0f, + 1.0f, 1.0f, + }; +}; + +} /* namespace libcamera */ diff --git a/src/libcamera/software_isp/meson.build b/src/libcamera/software_isp/meson.build index 59fa5f02..9a7ac455 100644 --- a/src/libcamera/software_isp/meson.build +++ b/src/libcamera/software_isp/meson.build @@ -14,3 +14,33 @@ libcamera_internal_sources += files([ 'software_isp.cpp', 'swstats_cpu.cpp', ]) + +libegl = cc.find_library('EGL', required : false) +libgbm = cc.find_library('gbm', required: false) +libglesv2 = cc.find_library('GLESv2', required : false) + +if libegl.found() + config_h.set('HAVE_LIBEGL', 1) +endif + +if libgbm.found() + config_h.set('HAVE_GBM', 1) +endif + +if libglesv2.found() + config_h.set('HAVE_GLESV2', 1) +endif + +if libegl.found() and libgbm.found() and libglesv2.found() +libcamera_deps += [ + libegl, + libgbm, + libglesv2, +] + +libcamera_internal_sources += files([ + 'debayer_egl.cpp', + 'egl.cpp', + 'gbm.cpp', +]) +endif