From patchwork Tue Apr 22 21:59:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bryan O'Donoghue X-Patchwork-Id: 23233 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2BCF5C331E for ; Tue, 22 Apr 2025 22:00:00 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6532668AD9; Tue, 22 Apr 2025 23:59:59 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="TjMPDZ5P"; dkim-atps=neutral Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 51F6068B3E for ; Tue, 22 Apr 2025 23:59:43 +0200 (CEST) Received: by mail-wm1-x333.google.com with SMTP id 5b1f17b1804b1-43cebe06e9eso47639735e9.3 for ; Tue, 22 Apr 2025 14:59:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745359182; x=1745963982; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1+V+tT+5bE8QKuoSfA3FLpmNoP41vVB3yUTW2/ZNDdc=; b=TjMPDZ5PXaCXvZstXbX3j1gZVqW9Uu0PiT5TA9E3MGQDPI8K7Sp+Ekg8NZASAq0Qrt N5qiX0hnsk6NZ/fTZ87rwzr4fIQ1RN42dkGhpYlBH9aXYNyv3xZszWTN51rZNhMzhAJ1 7RmQN8OjULYhEP35EEmj0M22E3xqunYKeOiJZ0Np7rCmE3Ld9AAptdovEbQfupL/U2mr +zl60DS3eJZHKFetzEF5VFi5BE6IExMxGX9LkUOn3dDQcjIfMEw1gZSfG+nndY1EFU83 gBGkRtDORj3dDEo6HnJCcfv1GA5JTjHORKYP8jHkmmRqO/abNB59JeZvid7/P2eYCaje v4ew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745359182; x=1745963982; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1+V+tT+5bE8QKuoSfA3FLpmNoP41vVB3yUTW2/ZNDdc=; b=ND+I9R8bIv0dUef6uGa9GB80t4l8wQQNpE23DqOEbj0Y21Kez9NO50nXA7uxiHF45D 8smK6ZK/Sh8kxcPk0IfBTP8Wc5L3bHR10fJewHZDwEwADQSkxiJV6Sq0K4I+xzLjXLBm j9//o703mD/15ncewsAosBxvwt9mILypYww7sVVkDIhL6c9fsNMrjKMaL0Uj3EJVI33g MFQd7GunVx1khXrl9T8Uja4BEPvPaku58zUpc7GDuL1GIAkc2q9boKoQiE4J5EpJStvf Vs1pdJA/5RlXzzj9MYk1sW0EanI/govfbPxAAGkQJdRyZxR88im53gRJlLTARe56/BHA /ptw== X-Gm-Message-State: AOJu0Yz1QH5TvyELKgTyIl4rfJIc4EZQ/bXiLcqAyNrKYdn8Xbyzi+FQ KLBiXhLEMUZPieNlwxc/dXPcCWUXfyznOB81egliwg/KfXNbmI9NzfRl2eePx5YXS0vU1h9NDc6 /LsI= X-Gm-Gg: ASbGncvsAoHZHb8y4RFVFuhS4AQj15g0R9DPT9rFTEXkltLusy4dmW6NjuQ2tCBJ+uP g75QBP1LvI/YyCBJPsNcatNzbO1K18I4VShiFcbQTNF18HF8ev8OO0jRWCao5lZZ/Jc0xfbeB1+ g0vZTMDyts1D8E4MSRpezc+vnJl187pW/Ld7NC1kwYz2Z/RZnKdQohU3F8JIf1iX5P1Q5/D3fVa iVbp/ParPbbHO2U6/7IkBOkANwMHv3vjWOEPW9hb4gzNJE/1cT5yEluJ3u8nKQ3d/D4iPtjlqxc gpANtYpJkpqXyWLxoWeuaGkJeLNLP4HCCr4nS+0Oh9LJQgevb0foU/WIWCIKxYI4lQaOV4KgR/V zI05yJHZOUFP0v10ySo/s X-Google-Smtp-Source: AGHT+IEOxUWQw6/5ixhOH5WdHssUArvknGU+ztgqIBZDDfyZizCqWDwi+O3kWCLH2Lv2XrJdMg9HIw== X-Received: by 2002:a05:600c:5395:b0:43d:db5:7b1a with SMTP id 5b1f17b1804b1-4406ab96f63mr164481765e9.12.1745359182425; Tue, 22 Apr 2025 14:59:42 -0700 (PDT) Received: from inspiron14p-linux.ht.home (188-141-3-146.dynamic.upc.ie. [188.141.3.146]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-44092d2eccesm2726615e9.20.2025.04.22.14.59.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Apr 2025 14:59:41 -0700 (PDT) From: Bryan O'Donoghue To: libcamera-devel@lists.libcamera.org Cc: hdegoede@redhat.com, mzamazal@redhat.com, bryan.odonoghue@linaro.org, bod.linux@nxsw.ie Subject: [PATCH 20/27] libcamera: software_isp: debayer_egl: Extend logic to enable application of softISP RGB debayer params Date: Tue, 22 Apr 2025 22:59:13 +0100 Message-ID: <20250422215920.4297-21-bryan.odonoghue@linaro.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250422215920.4297-1-bryan.odonoghue@linaro.org> References: <20250422215920.4297-1-bryan.odonoghue@linaro.org> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The existing SoftISP calculates RGB gain values as a lookup table of 256 values which shifts for each frame depending on the required correction. We can pass the required tables into the debayer shaders as textures, one texture for R, G and B respectively. The debayer shader will do its debayer interpolation and then if the appropriate define is specified use the calculated R, G and B values as indexes into our bayer colour gain table. TODO: - Validate the number of available texture units and bail out if < 4 In reality ~ all hardware has at least 16 units - Rename the debayer class to something like eglDebayer to reflect the fact its eGL based. - Add in the CCM tables too. Signed-off-by: Bryan O'Donoghue --- src/libcamera/software_isp/debayer_egl.cpp | 74 +++++++++++++++++----- src/libcamera/software_isp/debayer_egl.h | 15 +++-- 2 files changed, 69 insertions(+), 20 deletions(-) diff --git a/src/libcamera/software_isp/debayer_egl.cpp b/src/libcamera/software_isp/debayer_egl.cpp index 008938f8..f02ec8b2 100644 --- a/src/libcamera/software_isp/debayer_egl.cpp +++ b/src/libcamera/software_isp/debayer_egl.cpp @@ -26,6 +26,15 @@ DebayerEGL::DebayerEGL(std::unique_ptr stats) DebayerEGL::~DebayerEGL() { + if (eglImageBlueLookup_) + delete eglImageBlueLookup_; + + if (eglImageGreenLookup_) + delete eglImageGreenLookup_; + + if (eglImageRedLookup_) + delete eglImageRedLookup_; + if (eglImageBayerIn_) delete eglImageBayerIn_; } @@ -87,9 +96,11 @@ int DebayerEGL::getShaderVariableLocations(void) attributeVertex_ = glGetAttribLocation(programId_, "vertexIn"); attributeTexture_ = glGetAttribLocation(programId_, "textureIn"); - textureUniformY_ = glGetUniformLocation(programId_, "tex_y"); - textureUniformU_ = glGetUniformLocation(programId_, "tex_u"); - textureUniformV_ = glGetUniformLocation(programId_, "tex_v"); + textureUniformBayerDataIn_ = glGetUniformLocation(programId_, "tex_y"); + textureUniformRedLookupDataIn_ = glGetUniformLocation(programId_, "red_param"); + textureUniformGreenLookupDataIn_ = glGetUniformLocation(programId_, "green_param"); + textureUniformBlueLookupDataIn_ = glGetUniformLocation(programId_, "blue_param"); + textureUniformStep_ = glGetUniformLocation(programId_, "tex_step"); textureUniformSize_ = glGetUniformLocation(programId_, "tex_size"); textureUniformStrideFactor_ = glGetUniformLocation(programId_, "stride_factor"); @@ -97,9 +108,10 @@ int DebayerEGL::getShaderVariableLocations(void) textureUniformProjMatrix_ = glGetUniformLocation(programId_, "proj_matrix"); LOG(Debayer, Info) << "vertexIn " << attributeVertex_ << " textureIn " << attributeTexture_ - << " tex_y " << textureUniformY_ - << " tex_u " << textureUniformU_ - << " tex_v " << textureUniformV_ + << " tex_y " << textureUniformBayerDataIn_ + << " red_param " << textureUniformRedLookupDataIn_ + << " red_param " << textureUniformGreenLookupDataIn_ + << " red_param " << textureUniformBlueLookupDataIn_ << " tex_step " << textureUniformStep_ << " tex_size " << textureUniformSize_ << " stride_factor " << textureUniformStrideFactor_ @@ -202,6 +214,9 @@ int DebayerEGL::initBayerShaders(PixelFormat inputFormat, PixelFormat outputForm break; }; + // Flag to shaders that we have parameter gain tables + egl_.pushEnv(shaderEnv, "#define APPLY_BAYER_PARAMETERS"); + if (egl_.compileVertexShader(vertexShaderId_, vertexShaderData, vertexShaderDataLen, shaderEnv)) goto compile_fail; @@ -285,7 +300,24 @@ int DebayerEGL::configure(const StreamConfiguration &inputCfg, if (egl_.initEGLContext(&gbmSurface_)) return -ENODEV; - eglImageBayerIn_ = new eGLImage(width_, height_, 32); + // Raw bayer input as texture + eglImageBayerIn_ = new eGLImage(width_, height_, 32, GL_TEXTURE0, 0); + if (!eglImageBayerIn_) + return -ENOMEM; + + /// RGB correction tables as 2d textures + // eGL doesn't support glTexImage2D so we do a little hack with 2D to compensate + eglImageRedLookup_ = new eGLImage(DebayerParams::kRGBLookupSize, 1, 32, GL_TEXTURE1, 1); + if (!eglImageRedLookup_) + return -ENOMEM; + + eglImageGreenLookup_ = new eGLImage(DebayerParams::kRGBLookupSize, 1, 32, GL_TEXTURE2, 2); + if (!eglImageGreenLookup_) + return -ENOMEM; + + eglImageBlueLookup_ = new eGLImage(DebayerParams::kRGBLookupSize, 1, 32, GL_TEXTURE3, 3); + if (!eglImageBlueLookup_) + return -ENOMEM; // Create a single BO (calling gbm_surface_lock_front_buffer() again before gbm_surface_release_buffer() would create another BO) if (gbmSurface_.mapSurface()) @@ -394,7 +426,14 @@ void DebayerEGL::setShaderVariableValues(void) glVertexAttribPointer(attributeTexture_, 2, GL_FLOAT, GL_TRUE, 2 * sizeof(GLfloat), tcoordinates); - glUniform1i(textureUniformY_, 0); // tex_y - bayer_8.vert - set for no reason + // Set the sampler2D to the respective texture unit for each texutre + // To simultaneously sample multiple textures we need to use multiple + // texture units + glUniform1i(textureUniformBayerDataIn_, eglImageBayerIn_->texture_unit_uniform_id_); + glUniform1i(textureUniformRedLookupDataIn_, eglImageRedLookup_->texture_unit_uniform_id_); + glUniform1i(textureUniformGreenLookupDataIn_, eglImageGreenLookup_->texture_unit_uniform_id_); + glUniform1i(textureUniformBlueLookupDataIn_, eglImageBlueLookup_->texture_unit_uniform_id_); + glUniform2fv(textureUniformBayerFirstRed_, 1, firstRed); // tex_bayer_first_red - bayer_8.vert glUniform2fv(textureUniformSize_, 1, imgSize); // tex_size - bayer_8.vert glUniform2fv(textureUniformStep_, 1, Step); // tex_step - bayer_8.vert @@ -403,9 +442,10 @@ void DebayerEGL::setShaderVariableValues(void) GL_FALSE, projIdentityMatrix); // No scaling LOG(Debayer, Debug) << "vertexIn " << attributeVertex_ << " textureIn " << attributeTexture_ - << " tex_y " << textureUniformY_ - << " tex_u " << textureUniformU_ - << " tex_v " << textureUniformV_ + << " tex_y " << textureUniformBayerDataIn_ + << " red_param " << textureUniformRedLookupDataIn_ + << " red_param " << textureUniformGreenLookupDataIn_ + << " red_param " << textureUniformBlueLookupDataIn_ << " tex_step " << textureUniformStep_ << " tex_size " << textureUniformSize_ << " stride_factor " << textureUniformStrideFactor_ @@ -423,7 +463,7 @@ void DebayerEGL::setShaderVariableValues(void) return; } -void DebayerEGL::debayerGPU(MappedFrameBuffer &in, MappedFrameBuffer &out) +void DebayerEGL::debayerGPU(MappedFrameBuffer &in, MappedFrameBuffer &out, DebayerParams ¶ms) { LOG(Debayer, Debug) << "Input height " << height_ @@ -433,13 +473,15 @@ void DebayerEGL::debayerGPU(MappedFrameBuffer &in, MappedFrameBuffer &out) // eGL context switch egl_.makeCurrent(); - // make texture unit 0 explicit this doesn't really matter probably remove ? - glActiveTexture(GL_TEXTURE0); - // Greate a standard texture // we will replace this with the DMA version at some point egl_.createTexture2D(eglImageBayerIn_, inputConfig_.stride, height_, in.planes()[0].data()); + // Populate bayer parameters + egl_.createTexture2D(eglImageRedLookup_, DebayerParams::kRGBLookupSize, 1, ¶ms.red); + egl_.createTexture2D(eglImageGreenLookup_, DebayerParams::kRGBLookupSize, 1, ¶ms.green); + egl_.createTexture2D(eglImageBlueLookup_, DebayerParams::kRGBLookupSize, 1, ¶ms.blue); + // Setup the scene setShaderVariableValues(); glViewport(0, 0, width_, height_); @@ -482,7 +524,7 @@ void DebayerEGL::process(uint32_t frame, FrameBuffer *input, FrameBuffer *output return; } - debayerGPU(in, out); + debayerGPU(in, out, params); dmaSyncers.clear(); diff --git a/src/libcamera/software_isp/debayer_egl.h b/src/libcamera/software_isp/debayer_egl.h index 14f6997a..c8a19c3c 100644 --- a/src/libcamera/software_isp/debayer_egl.h +++ b/src/libcamera/software_isp/debayer_egl.h @@ -110,7 +110,7 @@ private: int getShaderVariableLocations(); void setShaderVariableValues(void); void configureTexture(GLuint &texture); - void debayerGPU(MappedFrameBuffer &in, MappedFrameBuffer &out); + void debayerGPU(MappedFrameBuffer &in, MappedFrameBuffer &out, DebayerParams ¶ms); // Shader program identifiers GLuint vertexShaderId_; @@ -125,20 +125,27 @@ private: // Pointer to object representing input texture eGLImage *eglImageBayerIn_; + eGLImage *eglImageRedLookup_; + eGLImage *eglImageGreenLookup_; + eGLImage *eglImageBlueLookup_; + // Shader parameters float firstRed_x_; float firstRed_y_; GLint attributeVertex_; GLint attributeTexture_; - GLint textureUniformY_; - GLint textureUniformU_; - GLint textureUniformV_; GLint textureUniformStep_; GLint textureUniformSize_; GLint textureUniformStrideFactor_; GLint textureUniformBayerFirstRed_; GLint textureUniformProjMatrix_; + #define DEBAYER_EGL_MIN_TEXTURE_UNITS 4 + GLint textureUniformBayerDataIn_; + GLint textureUniformRedLookupDataIn_; + GLint textureUniformGreenLookupDataIn_; + GLint textureUniformBlueLookupDataIn_; + Rectangle window_; std::unique_ptr stats_; eGL egl_;