From patchwork Mon Jul 6 11:39:00 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bryan O'Donoghue X-Patchwork-Id: 27252 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5DEF1C330A for ; Mon, 6 Jul 2026 11:39:22 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3E4DF66045; Mon, 6 Jul 2026 13:39:21 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="HmFJWVuh"; dkim-atps=neutral Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 96D7A6604A for ; Mon, 6 Jul 2026 13:39:17 +0200 (CEST) Received: by mail-wr1-x432.google.com with SMTP id ffacd0b85a97d-4720d22c94aso2906527f8f.1 for ; Mon, 06 Jul 2026 04:39:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1783337957; x=1783942757; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=iBp5CTAibDRDjJr1C9jbaznxXxt4zBdSgPR4F1nm75g=; b=HmFJWVuhbGyJquuQK/joVOlkIPjew1VpWH6+JUqGSdF6EgsOXwsT8419HeUYqryWkw GdmlxoPqXHEmrhdDZLWsLaDdLsWYWUqwwzyaruL8mjFP22oZYa6UnSdG870SoSqemngq 5v/Hf7aAMib1q/fLJnz0cBxWOIxSK816lNSuaQTeVrgwp04p4sFqvgy+8Mk/qwN2kAM2 6X95egt6CI2x8JrPs6fbqnRkuJmYySLIX4Ao9109xAN6HlN1m3Giov1LNtJc88yQPG8L X5lX7bvdPQFrpPYYTsQjRQDxuPYL5KQS5lrdQKXnVhk3lV7M/jPc5K3x71kscUQc2/Ig Bgfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1783337957; x=1783942757; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=iBp5CTAibDRDjJr1C9jbaznxXxt4zBdSgPR4F1nm75g=; b=Mu7lYbT4TeLK2IsZUktBMmthlD/JpWke7ciI4wedYEplz/Wjpv+031jSSC9k2SUIIu X+GFzOb54mtAMWsDHqJu2SRDnc+WZxq2NoSh7QzF7jhZwNtRc4NAlQkxmaCBfihQUulg myJm10UYCO/D5IUOZpJ9DHu3y4AHKy9ExhAG5AeJbC1KMcYGlZJu+cXg0GGOJiV4mOy0 hsfXM4nwHKukbdx2XFyCMUZ1CE3i9dDA78qHiiG+F9FA/j7cuhLHZM81voSnPJjvjO5J +8xDixUhCKVvoGfSDJ9Xf6M2n8Z5wpWHwZjPWsw2IhjQ5BhSzVDdn0wutNQwuyFamcnD TFXg== X-Gm-Message-State: AOJu0YyPylEy9Rm+mulX3YbxDyEbBU0JsQdwoyBlcilsX7GaskfBx1KW 2gLp2nT+NONsfs3JfFNgktB1PEa5tf/yY36XOgjTjWwFkLDg2tUxdAfaChDgVqlMk/OgeEzHgDv WfOew X-Gm-Gg: AfdE7cl8uacFh14r0yBhOEi7IL/w4cT1S+DFnTvs8lBh6vcj6X0UGnejdopMolRCcxy haZpck6GVJ6RyC7Z3XqYfgmJzxgv6ngGZeICBf2//AhGBYseB0oWxYYwH5KDNcv9bqBD5tjb9dF KJ+KOkOdFw3P+TpiY+Rn3kUCnNbRMX882yH4BkHmT6cVCsjaX2033RkakN6SBE/XTovTBTud4Of 3szPUDCIedksLou4I6StX3a2QLFte/X4UKK6+smxn69uHNk+kspHlzGUVppbaNsSqg2u8CNhKAG idxre5Rd46gln+wRj4YgDVWzbW+kaeqaLd7LpubxTLM0rtrxBI7711RXfhk72/lPECuneE473Q7 8vfHMW+PIXPKZGZtZKGTYhId54M6XDguizt6kmr7y3P1SNiHUtYnJLqkpkO9MChVC8VAt635HBs 2LQfDqZqeDm05pOIbLjuGRFWyQ4e12MFRfllHHFfI= X-Received: by 2002:a05:6000:188a:b0:467:f84c:4ca0 with SMTP id ffacd0b85a97d-47aaaada7d2mr10692693f8f.22.1783337957027; Mon, 06 Jul 2026 04:39:17 -0700 (PDT) Received: from inspiron14p-linux ([109.77.44.220]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-47a9de1e6ccsm24796084f8f.5.2026.07.06.04.39.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Jul 2026 04:39:16 -0700 (PDT) From: Bryan O'Donoghue To: libcamera-devel@lists.libcamera.org Cc: bryan.odonoghue@linaro.org, pavel@ucw.cz Subject: [PATCH v5 7/7] libcamera: software_isp: debayer_egl: Implement input/output frame caching mechanism Date: Mon, 6 Jul 2026 12:39:00 +0100 Message-ID: <20260706113900.2530549-8-bryan.odonoghue@linaro.org> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260706113900.2530549-1-bryan.odonoghue@linaro.org> References: <20260706113900.2530549-1-bryan.odonoghue@linaro.org> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement a texture caching mechanism for both input and output frames and for both types of input frame. The before/after on a Qualcomm x1e is: 9.737ms per frame 5.691ms per frame The before/after on a Qualcomm sm8250 is: 21.710ms per frame 17.336ms per frame for i in {1..20} do cam -c /base/soc@0/cci@ac16000/i2c-bus@1/camera@10 -s width=1920,height=1080 --capture=60 Interestingly there appears to be an absolute ~ 4.x ms per frame uplift as opposed to what intuition might suggest a proportional. Signed-off-by: Bryan O'Donoghue --- src/libcamera/software_isp/debayer_egl.cpp | 112 ++++++++++++++++----- src/libcamera/software_isp/debayer_egl.h | 13 ++- 2 files changed, 96 insertions(+), 29 deletions(-) diff --git a/src/libcamera/software_isp/debayer_egl.cpp b/src/libcamera/software_isp/debayer_egl.cpp index c3a7b7952..ca3353d9f 100644 --- a/src/libcamera/software_isp/debayer_egl.cpp +++ b/src/libcamera/software_isp/debayer_egl.cpp @@ -345,6 +345,9 @@ int DebayerEGL::configure(const StreamConfiguration &inputCfg, */ stats_->setWindow(Rectangle(window_.size())); + inputBufferCount_ = inputCfg.bufferCount; + outputBufferCount_ = outputCfg.bufferCount; + return 0; } @@ -504,36 +507,95 @@ void DebayerEGL::setShaderVariableValues(eGLImage &eglImageIn, const DebayerPara return; } -int DebayerEGL::debayerGPU(FrameBuffer *input, FrameBuffer *output, const DebayerParams ¶ms, std::optional *inMapped, std::optional *inDmaSyncer) +eGLImage *DebayerEGL::getCachedInputFrameBuffer(FrameBuffer *input, std::optional *inMapped, std::optional *inDmaSyncer) { - bool dmabuf_import_succeeded = false; - - /* eGL context switch */ - egl_.makeCurrent(); - - /* Try to create texture for input buffer via dmabuf import */ - if (!eglImageBayerIn_->dmabuf_import_failed_) { - if (egl_.createInputDMABufTexture2D(*eglImageBayerIn_, input->planes()[0].fd.get()) == 0) - dmabuf_import_succeeded = true; - else - LOG(Debayer, Info) << "Importing input buffer with DMABuf import failed, falling back to upload"; + const SharedFD &fd = input->planes()[0].fd; + bool use_dmabuf = false; + + auto [input_cache, cache_miss] = eglImageBayerIn_.try_emplace(fd); + if (cache_miss) { + if (eglImageBayerIn_.size() > inputBufferCount_) { + eglImageBayerIn_.erase(inputRing_.front()); + inputRing_.pop_front(); + LOG(Debayer, Error) << "Input cache " << inputBufferCount_ << " exceeded evicted entry"; + } + input_cache->second = std::make_unique(glFormat_, inputConfig_.stride / bytesPerPixel_, + height_, inputConfig_.stride, GL_TEXTURE0, 0); + inputRing_.push_back(fd); + } + eGLImage *eglImageIn = input_cache->second.get(); + + /* Try to create texture for input buffer via dmabuf import only on cache miss */ + if (cache_miss) { + if (egl_.createInputDMABufTexture2D(*eglImageIn, input->planes()[0].fd.get()) == 0) + use_dmabuf = true; + } else if (!eglImageIn->dmabuf_import_failed_) { + use_dmabuf = true; + egl_.activateBindTexture(*eglImageIn); } - /* Otherwise create texture for input buffer via upload from CPU */ - if (!dmabuf_import_succeeded) { + if (!use_dmabuf) { inDmaSyncer->emplace(input->planes()[0].fd, DmaSyncer::SyncType::Read); inMapped->emplace(input, MappedFrameBuffer::MapFlag::Read); if (!inMapped->value().isValid()) { LOG(Debayer, Error) << "mmap-ing buffer(s) failed"; - return -ENODEV; + if (cache_miss) { + eglImageBayerIn_.erase(input_cache); + inputRing_.pop_back(); + } + return nullptr; + } + if (cache_miss) + egl_.createTexture2D(*eglImageIn, inMapped->value().planes()[0].data()); + else + egl_.updateTexture2D(*eglImageIn, inMapped->value().planes()[0].data()); + } + + return eglImageIn; +} + +eGLImage *DebayerEGL::getCachedOutputFrameBuffer(FrameBuffer *output) +{ + const SharedFD &fd = output->planes()[0].fd; + + auto [output_cache, cache_miss] = eglImageBayerOut_.try_emplace(fd); + if (cache_miss) { + if (eglImageBayerOut_.size() > outputBufferCount_) { + eglImageBayerOut_.erase(outputRing_.front()); + outputRing_.pop_front(); + LOG(Debayer, Error) << "Output cache " << outputBufferCount_ << " exceeded evicted entry"; } - egl_.createTexture2D(*eglImageBayerIn_, inMapped->value().planes()[0].data()); + output_cache->second = std::make_unique(GL_RGBA, outputSize_.width, + outputSize_.height, outputConfig_.stride, GL_TEXTURE1, 1); + if (egl_.createOutputDMABufTexture2D(*output_cache->second, output->planes()[0].fd.get())) { + eglImageBayerOut_.erase(output_cache); + return nullptr; + } + outputRing_.push_back(fd); } + eGLImage *eglImageOut = output_cache->second.get(); + + return eglImageOut; +} + +int DebayerEGL::debayerGPU(FrameBuffer *input, FrameBuffer *output, const DebayerParams ¶ms, std::optional *inMapped, std::optional *inDmaSyncer) +{ + eGLImage *eglImageIn; + eGLImage *eglImageOut; + + /* eGL context switch */ + egl_.makeCurrent(); - /* Generate the output render framebuffer as render to texture */ - egl_.createOutputDMABufTexture2D(*eglImageBayerOut_, output->planes()[0].fd.get()); + eglImageIn = getCachedInputFrameBuffer(input, inMapped, inDmaSyncer); + if (!eglImageIn) + return -ENOMEM; + eglImageOut = getCachedOutputFrameBuffer(output); + if (!eglImageOut) + return -ENOMEM; + + egl_.attachTextureToFBO(*eglImageOut); + setShaderVariableValues(*eglImageIn, params); - setShaderVariableValues(*eglImageBayerIn_, params); glViewport(0, 0, width_, height_); glClear(GL_COLOR_BUFFER_BIT); glDrawArrays(GL_TRIANGLE_FAN, 0, DEBAYER_OPENGL_COORDS); @@ -615,19 +677,15 @@ int DebayerEGL::start() if (initBayerShaders(inputPixelFormat_, outputPixelFormat_)) return -EINVAL; - /* Raw bayer input as texture */ - eglImageBayerIn_ = std::make_unique(glFormat_, inputConfig_.stride / bytesPerPixel_, height_, inputConfig_.stride, GL_TEXTURE0, 0); - - /* Texture we will render to */ - eglImageBayerOut_ = std::make_unique(GL_RGBA, outputSize_.width, outputSize_.height, outputConfig_.stride, GL_TEXTURE1, 1); - return 0; } void DebayerEGL::stop() { - eglImageBayerOut_.reset(); - eglImageBayerIn_.reset(); + eglImageBayerOut_.clear(); + eglImageBayerIn_.clear(); + outputRing_.clear(); + inputRing_.clear(); if (programId_) glDeleteProgram(programId_); diff --git a/src/libcamera/software_isp/debayer_egl.h b/src/libcamera/software_isp/debayer_egl.h index 348d7305b..dde6ba9bf 100644 --- a/src/libcamera/software_isp/debayer_egl.h +++ b/src/libcamera/software_isp/debayer_egl.h @@ -9,9 +9,11 @@ #pragma once +#include #include #include #include +#include #include #define GL_GLEXT_PROTOTYPES @@ -68,14 +70,21 @@ private: void setShaderVariableValues(eGLImage &eGLImageIn, const DebayerParams ¶ms); int debayerGPU(FrameBuffer *input, FrameBuffer *output, const DebayerParams ¶ms, std::optional *mappedInputBuffer, std::optional *inputBufferDmaSyncer); + eGLImage *getCachedInputFrameBuffer(FrameBuffer *input, std::optional *inMapped, std::optional *inDmaSyncer); + eGLImage *getCachedOutputFrameBuffer(FrameBuffer *output); + /* Shader program identifiers */ GLuint vertexShaderId_ = 0; GLuint fragmentShaderId_ = 0; GLuint programId_ = 0; /* Pointer to object representing input texture */ - std::unique_ptr eglImageBayerIn_; - std::unique_ptr eglImageBayerOut_; + std::unordered_map> eglImageBayerIn_; + std::unordered_map> eglImageBayerOut_; + std::deque inputRing_; + std::deque outputRing_; + unsigned int inputBufferCount_; + unsigned int outputBufferCount_; /* Shader parameters */ float firstRed_x_;