From patchwork Fri Jun 26 11:33:25 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bryan O'Donoghue X-Patchwork-Id: 27055 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id ACDB8C3307 for ; Fri, 26 Jun 2026 11:33:52 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id DF18F6590C; Fri, 26 Jun 2026 13:33:51 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="AdSWOPZV"; dkim-atps=neutral Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2CF2C65902 for ; Fri, 26 Jun 2026 13:33:47 +0200 (CEST) Received: by mail-wm1-x329.google.com with SMTP id 5b1f17b1804b1-49222fb062bso8934565e9.1 for ; Fri, 26 Jun 2026 04:33:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1782473627; x=1783078427; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ggDf+Tx2g2b9HHWJEEpcyZgRvHrHYXQMzOnA7eG6sds=; b=AdSWOPZVjYCx6HLEjAtqHBFCp0OqCRZzotQt5LulnARU9E4o3eSU/CCA9IIWtzQ9/n fjQTi+Li/3PBk7QYIXRNEMm2+2a/7Ozx2wBUtA0d4ELHdjcmvDaUaqsii81IUSD85oe+ Hyqb1pqANGLBoCVg2CW1T8UT1ylP40LahKZf8w+whiCEosamchJ6woxhrT6apOskGJWV YD5b6hl17+FtpSO06L3xogJ3Ir/+qAF16EKK0yBAddQVeL+gu5P/q5bhsJkr/2TpPAIb ypn7+qS2+W7+5XumDQHgRYSAXbyKPu9R957R/2p7hAKdOlgIiKZRkjSvpR8ghwtcqnOD B98g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1782473627; x=1783078427; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=ggDf+Tx2g2b9HHWJEEpcyZgRvHrHYXQMzOnA7eG6sds=; b=D4mtCSuxiEBDmPHM6IKojdgUDIKjkufZl50ez1HdBNfB6Cx2WBwYn14gY6qsFlrF1w 25mvzu6Ent6Vl6VdFh+2r8nQRKrnbuBcqQ8X66v1rJCGzYq0QfsBA+60169L+6x+Vw+v nH93gnP2UseAak25HArAREunBGz/VY5B0nBFwSEv4uBzFJpyylFnYpDfIUg/2saf+Q0m get5Tu0s1qKO/yKu/eRhDp90tazRv0YcpMlCNngEmbCsdNcXeHZI5zqx7+BpagTYEHnB uIe2rKbmki1hUcfNzs3xeMGPc8UdmIoBarC+uY0I3M8v9Znj1eDj2Z37w66R3S2Ickt6 3vlA== X-Gm-Message-State: AOJu0YxWhHVMhIHrfMgoKe6UdTHlvUA6ej9a3B5tgy7Tm+HAmZfu3SN+ 53Dsws4YiFMkk/SA3C10f72VbHw9iwjaMBkHst9bXUouhyeiLyuRNAYqt2zG0ToI5GXY9hAe3b8 uT1MT4WU= X-Gm-Gg: AfdE7clJ78tIoAQmbo+mBbAVTSAQQnH20bCkgf1sf6SuoAqHiIb+ETDw1CBrAnsrAue PYbznnFGwAm8yrRQ7go0Xv+wLBubO3aQB8H16cwCgzzJyStaeVnuU4uVTK4u8nY8MWeEBAnF+aW fxVA/fVT0SIo2hbr2fG9BXhV2SE0gCa04j3H+I/YOR6vjkKURHzdi6UhWfN6gi4Yyf7M9FuxXpr 6EICUz+Qs/c7hgpG/KzUb0BvhMOJ484S0oLrqvFKIHz6Ar8t21uY7H4Nn5g4dI2VApi+RoQP9yw lRjFTC3zv15MziMrq5uanFQHr8echX2Wabxg5eKhedblRl2IGQ4WLOBREo+4hj5hmk0zWqzcEBo xyH4xZ6mEVHEo2YA/YgjKXUoD2v2gAehHEGhUydJchqv/ECsdrQ9NQRU2gpEHNoWnqTe+yRl0zd fTK7d1XJKCSGb5zGagoS7mZ27NK8OYiaYxpTk9 X-Received: by 2002:a05:600c:4e87:b0:492:25a0:1730 with SMTP id 5b1f17b1804b1-49266893494mr90648115e9.32.1782473626579; Fri, 26 Jun 2026 04:33:46 -0700 (PDT) Received: from inspiron14p-linux ([109.76.78.98]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49268fe31b0sm79265985e9.4.2026.06.26.04.33.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 Jun 2026 04:33:45 -0700 (PDT) From: Bryan O'Donoghue To: libcamera-devel@lists.libcamera.org Cc: bryan.odonoghue@linaro.org, pavel@ucw.cz Subject: [PATCH v3 8/8] libcamera: software_isp: debayer_egl: Implement input/output frame caching mechanism Date: Fri, 26 Jun 2026 12:33:25 +0100 Message-ID: <20260626113325.3218045-9-bryan.odonoghue@linaro.org> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260626113325.3218045-1-bryan.odonoghue@linaro.org> References: <20260626113325.3218045-1-bryan.odonoghue@linaro.org> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement a texture caching mechanism for both input and output frames and for both types of input frame. The before/after on a Qualcomm x1e is: 9.737ms per frame 5.691ms per frame The before/after on a Qualcomm sm8250 is: 21.710ms per frame 17.336ms per frame for i in {1..20} do cam -c /base/soc@0/cci@ac16000/i2c-bus@1/camera@10 -s width=1920,height=1080 --capture=60 Interestingly there appears to be an absolute ~ 4.x ms per frame uplift as opposed to what intuition might suggest a proportional. Signed-off-by: Bryan O'Donoghue Reviewed-by: Robert Mader --- src/libcamera/software_isp/debayer_egl.cpp | 87 +++++++++++++++++----- src/libcamera/software_isp/debayer_egl.h | 10 ++- 2 files changed, 75 insertions(+), 22 deletions(-) diff --git a/src/libcamera/software_isp/debayer_egl.cpp b/src/libcamera/software_isp/debayer_egl.cpp index 53bb67c17..fc37f0b75 100644 --- a/src/libcamera/software_isp/debayer_egl.cpp +++ b/src/libcamera/software_isp/debayer_egl.cpp @@ -355,6 +355,9 @@ int DebayerEGL::configure(const StreamConfiguration &inputCfg, */ stats_->setWindow(Rectangle(window_.size())); + inputBufferCount_ = inputCfg.bufferCount; + outputBufferCount_ = outputCfg.bufferCount; + return 0; } @@ -514,34 +517,84 @@ void DebayerEGL::setShaderVariableValues(eGLImage &eglImageIn, const DebayerPara return; } -int DebayerEGL::debayerGPU(FrameBuffer *input, FrameBuffer *output, const DebayerParams ¶ms, std::optional *inMapped, std::optional *inDmaSyncer) +eGLImage *DebayerEGL::getCachedInputFrameBuffer(FrameBuffer *input, std::optional *inMapped, std::optional *inDmaSyncer) { - /* eGL context switch */ - egl_.makeCurrent(); + auto [input_cache, cache_miss] = eglImageBayerIn_.try_emplace(input->planes()[0].fd.get()); + if (cache_miss) { + if (eglImageBayerIn_.size() > inputBufferCount_) { + LOG(Debayer, Error) << "Input count " << inputBufferCount_ << " exhausted"; + return nullptr; + } + input_cache->second = std::make_unique(glFormat_, inputConfig_.stride / bytesPerPixel_, + height_, inputConfig_.stride, GL_TEXTURE0, 0); + } + eGLImage *eglImageIn = input_cache->second.get(); /* Try to create texture for input buffer via dmabuf import */ - if (use_dmabuf_) { - if (egl_.createInputDMABufTexture2D(*eglImageBayerIn_, input->planes()[0].fd.get()) != 0) { + if (use_dmabuf_ && cache_miss) { + if (egl_.createInputDMABufTexture2D(*eglImageIn, input->planes()[0].fd.get()) != 0) { use_dmabuf_ = false; LOG(Debayer, Info) << "Importing input buffer with DMABuf import failed, falling back to upload"; } } - /* Otherwise create texture for input buffer via upload from CPU */ - if (!use_dmabuf_) { + if (use_dmabuf_) { + /* Cache hit using dmabuf activate and bind */ + if (!cache_miss) + egl_.activateBindTexture(*eglImageIn); + } else { + /* Otherwise create texture for input buffer via upload from CPU */ inDmaSyncer->emplace(input->planes()[0].fd, DmaSyncer::SyncType::Read); inMapped->emplace(input, MappedFrameBuffer::MapFlag::Read); if (!inMapped->value().isValid()) { LOG(Debayer, Error) << "mmap-ing buffer(s) failed"; - return -ENODEV; + return nullptr; } - egl_.createTexture2D(*eglImageBayerIn_, inMapped->value().planes()[0].data()); + if (cache_miss) + egl_.createTexture2D(*eglImageIn, inMapped->value().planes()[0].data()); + else + egl_.updateTexture2D(*eglImageIn, inMapped->value().planes()[0].data()); } - /* Generate the output render framebuffer as render to texture */ - egl_.createOutputDMABufTexture2D(*eglImageBayerOut_, output->planes()[0].fd.get()); + return eglImageIn; +} + +eGLImage *DebayerEGL::getCachedOutputFrameBuffer(FrameBuffer *output) +{ + auto [output_cache, cache_miss] = eglImageBayerOut_.try_emplace(output->planes()[0].fd.get()); + if (cache_miss) { + if (eglImageBayerOut_.size() > outputBufferCount_) { + LOG(Debayer, Error) << "Output buffer count " << outputBufferCount_ << " exhaustion"; + return nullptr; + } + output_cache->second = std::make_unique(GL_RGBA, outputSize_.width, + outputSize_.height, outputConfig_.stride, GL_TEXTURE1, 1); + egl_.createOutputDMABufTexture2D(*output_cache->second, output->planes()[0].fd.get()); + } + eGLImage *eglImageOut = output_cache->second.get(); + + return eglImageOut; +} + +int DebayerEGL::debayerGPU(FrameBuffer *input, FrameBuffer *output, const DebayerParams ¶ms, std::optional *inMapped, std::optional *inDmaSyncer) +{ + eGLImage *eglImageIn; + eGLImage *eglImageOut; + + /* eGL context switch */ + egl_.makeCurrent(); + + eglImageIn = getCachedInputFrameBuffer(input, inMapped, inDmaSyncer); + if (!eglImageIn) + return -ENOMEM; + + eglImageOut = getCachedOutputFrameBuffer(output); + if (!eglImageOut) + return -ENOMEM; + + egl_.attachTextureToFBO(*eglImageOut); + setShaderVariableValues(*eglImageIn, params); - setShaderVariableValues(*eglImageBayerIn_, params); glViewport(0, 0, width_, height_); glClear(GL_COLOR_BUFFER_BIT); glDrawArrays(GL_TRIANGLE_FAN, 0, DEBAYER_OPENGL_COORDS); @@ -623,19 +676,13 @@ int DebayerEGL::start() if (initBayerShaders(inputPixelFormat_, outputPixelFormat_)) return -EINVAL; - /* Raw bayer input as texture */ - eglImageBayerIn_ = std::make_unique(glFormat_, inputConfig_.stride / bytesPerPixel_, height_, inputConfig_.stride, GL_TEXTURE0, 0); - - /* Texture we will render to */ - eglImageBayerOut_ = std::make_unique(GL_RGBA, outputSize_.width, outputSize_.height, outputConfig_.stride, GL_TEXTURE1, 1); - return 0; } void DebayerEGL::stop() { - eglImageBayerOut_.reset(); - eglImageBayerIn_.reset(); + eglImageBayerOut_.clear(); + eglImageBayerIn_.clear(); if (programId_) glDeleteProgram(programId_); diff --git a/src/libcamera/software_isp/debayer_egl.h b/src/libcamera/software_isp/debayer_egl.h index d8509e9f2..ddb3ef378 100644 --- a/src/libcamera/software_isp/debayer_egl.h +++ b/src/libcamera/software_isp/debayer_egl.h @@ -22,6 +22,7 @@ #include "libcamera/internal/mapped_framebuffer.h" #include "libcamera/internal/software_isp/benchmark.h" #include "libcamera/internal/software_isp/swstats_cpu.h" +#include "libcamera/internal/v4l2_videodevice.h" #include #include @@ -70,14 +71,19 @@ private: bool use_dmabuf_; + eGLImage *getCachedInputFrameBuffer(FrameBuffer *input, std::optional *inMapped, std::optional *inDmaSyncer); + eGLImage *getCachedOutputFrameBuffer(FrameBuffer *output); + /* Shader program identifiers */ GLuint vertexShaderId_ = 0; GLuint fragmentShaderId_ = 0; GLuint programId_ = 0; /* Pointer to object representing input texture */ - std::unique_ptr eglImageBayerIn_; - std::unique_ptr eglImageBayerOut_; + std::unordered_map> eglImageBayerIn_; + std::unordered_map> eglImageBayerOut_; + unsigned int inputBufferCount_; + unsigned int outputBufferCount_; /* Shader parameters */ float firstRed_x_;