{"id":27259,"url":"https://patchwork.libcamera.org/api/patches/27259/?format=json","web_url":"https://patchwork.libcamera.org/patch/27259/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20260706222245.247396-7-bryan.odonoghue@linaro.org>","date":"2026-07-06T22:22:45","name":"[v6,6/6] libcamera: software_isp: debayer_egl: Implement input/output frame caching mechanism","commit_ref":null,"pull_url":null,"state":"accepted","archived":false,"hash":"87b289d66ce6ce65770be92d9bc1669fadb79e67","submitter":{"id":175,"url":"https://patchwork.libcamera.org/api/people/175/?format=json","name":"Bryan O'Donoghue","email":"bryan.odonoghue@linaro.org"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/27259/mbox/","series":[{"id":6042,"url":"https://patchwork.libcamera.org/api/series/6042/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=6042","date":"2026-07-06T22:22:39","name":"libcamera: software_isp: gpu: Add go faster stripes","version":6,"mbox":"https://patchwork.libcamera.org/series/6042/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/27259/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/27259/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 83314C3308\n\tfor ;\n\tMon, 6 Jul 2026 22:23:09 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 91B0666063;\n\tTue, 7 Jul 2026 00:23:08 +0200 (CEST)","from mail-wm1-x336.google.com (mail-wm1-x336.google.com\n\t[IPv6:2a00:1450:4864:20::336])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 5D12E6605B\n\tfor ;\n\tTue, 7 Jul 2026 00:23:03 +0200 (CEST)","by mail-wm1-x336.google.com with SMTP id\n\t5b1f17b1804b1-493bab44440so207565e9.0\n\tfor ;\n\tMon, 06 Jul 2026 15:23:03 -0700 (PDT)","from inspiron14p-linux ([109.77.44.220])\n\tby smtp.gmail.com with ESMTPSA id\n\tffacd0b85a97d-47aa039ad21sm27916413f8f.20.2026.07.06.15.23.02\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tMon, 06 Jul 2026 15:23:02 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"iPF+/l4T\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=linaro.org; s=google; t=1783376583; x=1783981383;\n\tdarn=lists.libcamera.org; \n\th=content-transfer-encoding:mime-version:references:in-reply-to\n\t:message-id:date:subject:cc:to:from:from:to:cc:subject:date\n\t:message-id:reply-to;\n\tbh=SslCStkiHkFC7e1FQGIriVH2RTlbB3ja2yOPPwrhQ6k=;\n\tb=iPF+/l4TGt/PV5R9y09GPcKjeTDVNbNNfIz2CaSPPI3KU7nT4u1+CprXsuaZo/fN34\n\t5vonCZWKpbn5LO7L7KzbFKVjdIEB7CwktNFegV/r7WKXyHcIVapMfZyC4/K7F/agPSFM\n\tzjfW7Nlc82N66YwmD/nsqQfYhNI4/aClgCVv08G7l+6HESSt8jMoLw5/T6Ke9eKxdCAr\n\t6bzHXyEOK8T/LXJDsN/UMPl1TxNvjIcw13r8hK3d8k0n7lJBWIv02u0VPlJSnO6B55wT\n\t+axHadLN5ejfAl9aRMeEL/Tbq8Kcetf0Doo2C8Y6Zyz8eX6tyALibGLS1OIUVuPLhdJ9\n\tNvAA==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20251104; t=1783376583; x=1783981383;\n\th=content-transfer-encoding:mime-version:references:in-reply-to\n\t:message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from\n\t:to:cc:subject:date:message-id:reply-to;\n\tbh=SslCStkiHkFC7e1FQGIriVH2RTlbB3ja2yOPPwrhQ6k=;\n\tb=Ka9Xl2DXpLQFv8XH3MXxwngpXeAibRzpQ86EmSCSTMmjaOIDgJMotlInwXkySDzKfu\n\ttY0sZsC9lKIuIO1mSpPZ5TspOEbWLCla6jFEzxGXgRYpYu+AhOdvTTypgYOdFr6oDYx+\n\tP0tJqzValw1Rs5trAQX27BzEy1uPCfmYCchX/80YQgb76/bw4MLNaoeWT65qPxfzNHTL\n\tl/h9eTbjKIaXDfXieXV788NjNE01cBNfTqRVRAMZTrMP5F0GcMMeKsqoitWt1oKua8GU\n\t7OtIqTfCzbSkRiTym58k2S7b/lHxSrnzEGhCBpl7VM8opg4yGpMK5d0vOeXjL5ggjvWC\n\tqicg==","X-Gm-Message-State":"AOJu0Yz+RrfK/CMYAwOAXY3kPS69B1Q/AGAjU0IXDZtwN9ZbIj7xX9u6\n\tJdlOr/OBzfZpAd0s/8R0gNmGjg3vSoHx4QysyrubqAhRZmvEFF39BqcPSyDIN16tzvs+9XcykNt\n\t5Ixtg","X-Gm-Gg":"AfdE7cnr/N7hwd/Nq6Zo+bH8U/8cfpPLETdkhwuG196TWyhxEfH1s8vFaDtp+2z8jos\n\twcp5Qyl7852QKXexVuq6g/dI2EGWUTLgPMn9G3vyW3Oi2wM4Q8W+inCqkZpKXvT2vIgIFLk/xYz\n\tPQuLozjEjR1Ys4SI/uCIdjfc1DYSvV42meeiJ/j/5RDWi5AZl8Q1B6O5MU+FS1UoiChDbesn4YA\n\te1QJbo9saXIM2wLNLtJBJcE0zxky9jGL8tCIZKAwJUrqpjOuGLpFx1NYSAp748/ZR9NYu2Q3Ilf\n\tJ5JAZ5e5EF57JET1+f0X7l4GGOieJ8kWfQN0i1ooINXN8ibjZH8JMTi1bw/Dmg/mx8vwOnq2DOE\n\tu//fcwu6d3+XnYHg6U9XmlVDxqg4D1Hu4A+8Ltjjk+EvLt9jxeygQXCHNeLDEwBZdOeI295TFHT\n\ttSSUGhYPQZFFPTO6RfEtXxLIq3+a+N","X-Received":"by 2002:a05:600c:1f84:b0:492:7025:13fd with SMTP id\n\t5b1f17b1804b1-493e0f7a2b7mr1946195e9.0.1783376582805; \n\tMon, 06 Jul 2026 15:23:02 -0700 (PDT)","From":"Bryan O'Donoghue ","To":"libcamera-devel@lists.libcamera.org","Cc":"bryan.odonoghue@linaro.org,\n\tpavel@ucw.cz","Subject":"[PATCH v6 6/6] libcamera: software_isp: debayer_egl: Implement\n\tinput/output frame caching mechanism","Date":"Mon, 6 Jul 2026 23:22:45 +0100","Message-ID":"<20260706222245.247396-7-bryan.odonoghue@linaro.org>","X-Mailer":"git-send-email 2.54.0","In-Reply-To":"<20260706222245.247396-1-bryan.odonoghue@linaro.org>","References":"<20260706222245.247396-1-bryan.odonoghue@linaro.org>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Implement a texture caching mechanism for both input and output frames and\nfor both types of input frame.\n\nThe before/after on a Qualcomm x1e is:\n\n9.737ms per frame\n5.691ms per frame\n\nThe before/after on a Qualcomm sm8250 is:\n\n21.710ms per frame\n17.336ms per frame\n\nfor i in {1..20} do\ncam -c /base/soc@0/cci@ac16000/i2c-bus@1/camera@10 -s width=1920,height=1080 --capture=60\n\nInterestingly there appears to be an absolute ~ 4.x ms per frame uplift as\nopposed to what intuition might suggest a proportional.\n\nSigned-off-by: Bryan O'Donoghue \n---\n src/libcamera/software_isp/debayer_egl.cpp | 125 ++++++++++++++++-----\n src/libcamera/software_isp/debayer_egl.h | 10 +-\n 2 files changed, 104 insertions(+), 31 deletions(-)","diff":"diff --git a/src/libcamera/software_isp/debayer_egl.cpp b/src/libcamera/software_isp/debayer_egl.cpp\nindex c3a7b7952..a7cca5dde 100644\n--- a/src/libcamera/software_isp/debayer_egl.cpp\n+++ b/src/libcamera/software_isp/debayer_egl.cpp\n@@ -345,6 +345,9 @@ int DebayerEGL::configure(const StreamConfiguration &inputCfg,\n \t */\n \tstats_->setWindow(Rectangle(window_.size()));\n \n+\tinputBufferCount_ = inputCfg.bufferCount;\n+\toutputBufferCount_ = outputCfg.bufferCount;\n+\n \treturn 0;\n }\n \n@@ -504,36 +507,106 @@ void DebayerEGL::setShaderVariableValues(eGLImage &eglImageIn, const DebayerPara\n \treturn;\n }\n \n-int DebayerEGL::debayerGPU(FrameBuffer *input, FrameBuffer *output, const DebayerParams ¶ms, std::optional *inMapped, std::optional *inDmaSyncer)\n+eGLImage *DebayerEGL::getCachedInputFrameBuffer(FrameBuffer *input, std::optional *inMapped, std::optional *inDmaSyncer)\n {\n-\tbool dmabuf_import_succeeded = false;\n+\tconst SharedFD &fd = input->planes()[0].fd;\n+\tbool cache_miss = true;\n+\teGLImage *eglImageIn;\n+\n+\tfor (auto &[ifd, img] : eglImageInCache_) {\n+\t\tif (ifd == fd) {\n+\t\t\teglImageIn = img.get();\n+\t\t\tcache_miss = false;\n+\t\t\tbreak;\n+\t\t}\n+\t}\n \n-\t/* eGL context switch */\n-\tegl_.makeCurrent();\n+\tif (cache_miss) {\n+\t\tif (eglImageInCache_.size() >= inputBufferCount_) {\n+\t\t\teglImageInCache_.pop_front();\n+\t\t\tLOG(Debayer, Debug) << \"Input cache \" << inputBufferCount_ << \" exceeded evicted entry\";\n+\t\t}\n+\n+\t\teglImageInCache_.emplace_back(fd, std::make_unique(glFormat_, inputConfig_.stride / bytesPerPixel_, height_, inputConfig_.stride, GL_TEXTURE0, 0));\n+\t\teglImageIn = eglImageInCache_.back().second.get();\n+\n+\t\tif (egl_.createInputDMABufTexture2D(*eglImageIn, input->planes()[0].fd.get()) == 0)\n+\t\t\treturn eglImageIn;\n+\n+\t} else if (!eglImageIn->dmabuf_import_failed_) {\n+\t\tegl_.activateBindTexture(*eglImageIn);\n+\t\treturn eglImageIn;\n+\t}\n+\n+\t/* DMA mode fail create/update an existing texture using the slow path */\n+\tinDmaSyncer->emplace(input->planes()[0].fd, DmaSyncer::SyncType::Read);\n+\tinMapped->emplace(input, MappedFrameBuffer::MapFlag::Read);\n+\tif (!inMapped->value().isValid()) {\n+\t\tLOG(Debayer, Error) << \"mmap-ing buffer(s) failed\";\n+\t\tif (cache_miss) {\n+\t\t\teglImageInCache_.pop_back();\n+\t\t}\n+\t\treturn nullptr;\n+\t}\n+\tif (cache_miss)\n+\t\tegl_.createTexture2D(*eglImageIn, inMapped->value().planes()[0].data());\n+\telse\n+\t\tegl_.updateTexture2D(*eglImageIn, inMapped->value().planes()[0].data());\n+\n+\treturn eglImageIn;\n+}\n \n-\t/* Try to create texture for input buffer via dmabuf import */\n-\tif (!eglImageBayerIn_->dmabuf_import_failed_) {\n-\t\tif (egl_.createInputDMABufTexture2D(*eglImageBayerIn_, input->planes()[0].fd.get()) == 0)\n-\t\t\tdmabuf_import_succeeded = true;\n-\t\telse\n-\t\t\tLOG(Debayer, Info) << \"Importing input buffer with DMABuf import failed, falling back to upload\";\n+eGLImage *DebayerEGL::getCachedOutputFrameBuffer(FrameBuffer *output)\n+{\n+\tconst SharedFD &fd = output->planes()[0].fd;\n+\tbool cache_miss = true;\n+\teGLImage *eglImageOut;\n+\n+\tfor (auto &[ifd, img] : eglImageOutCache_) {\n+\t\tif (ifd == fd) {\n+\t\t\teglImageOut = img.get();\n+\t\t\tcache_miss = false;\n+\t\t\tbreak;\n+\t\t}\n \t}\n \n-\t/* Otherwise create texture for input buffer via upload from CPU */\n-\tif (!dmabuf_import_succeeded) {\n-\t\tinDmaSyncer->emplace(input->planes()[0].fd, DmaSyncer::SyncType::Read);\n-\t\tinMapped->emplace(input, MappedFrameBuffer::MapFlag::Read);\n-\t\tif (!inMapped->value().isValid()) {\n-\t\t\tLOG(Debayer, Error) << \"mmap-ing buffer(s) failed\";\n-\t\t\treturn -ENODEV;\n+\tif (cache_miss) {\n+\t\tif (eglImageOutCache_.size() >= outputBufferCount_) {\n+\t\t\teglImageOutCache_.pop_front();\n+\t\t\tLOG(Debayer, Debug) << \"Output cache \" << outputBufferCount_ << \" exceeded evicted entry\";\n+\t\t}\n+\n+\t\teglImageOutCache_.emplace_back(fd, std::make_unique(GL_RGBA, outputSize_.width,\n+\t\t\t\t\t outputSize_.height, outputConfig_.stride, GL_TEXTURE1, 1));\n+\t\teglImageOut = eglImageOutCache_.back().second.get();\n+\n+\t\tif (egl_.createOutputDMABufTexture2D(*eglImageOut, output->planes()[0].fd.get())) {\n+\t\t\teglImageOutCache_.pop_back();\n+\t\t\treturn nullptr;\n \t\t}\n-\t\tegl_.createTexture2D(*eglImageBayerIn_, inMapped->value().planes()[0].data());\n \t}\n \n-\t/* Generate the output render framebuffer as render to texture */\n-\tegl_.createOutputDMABufTexture2D(*eglImageBayerOut_, output->planes()[0].fd.get());\n+\treturn eglImageOut;\n+}\n+\n+int DebayerEGL::debayerGPU(FrameBuffer *input, FrameBuffer *output, const DebayerParams ¶ms, std::optional *inMapped, std::optional *inDmaSyncer)\n+{\n+\teGLImage *eglImageIn;\n+\teGLImage *eglImageOut;\n+\n+\t/* eGL context switch */\n+\tegl_.makeCurrent();\n+\n+\teglImageIn = getCachedInputFrameBuffer(input, inMapped, inDmaSyncer);\n+\tif (!eglImageIn)\n+\t\treturn -ENOMEM;\n+\teglImageOut = getCachedOutputFrameBuffer(output);\n+\tif (!eglImageOut)\n+\t\treturn -ENOMEM;\n+\n+\tegl_.attachTextureToFBO(*eglImageOut);\n+\tsetShaderVariableValues(*eglImageIn, params);\n \n-\tsetShaderVariableValues(*eglImageBayerIn_, params);\n \tglViewport(0, 0, width_, height_);\n \tglClear(GL_COLOR_BUFFER_BIT);\n \tglDrawArrays(GL_TRIANGLE_FAN, 0, DEBAYER_OPENGL_COORDS);\n@@ -615,19 +688,13 @@ int DebayerEGL::start()\n \tif (initBayerShaders(inputPixelFormat_, outputPixelFormat_))\n \t\treturn -EINVAL;\n \n-\t/* Raw bayer input as texture */\n-\teglImageBayerIn_ = std::make_unique(glFormat_, inputConfig_.stride / bytesPerPixel_, height_, inputConfig_.stride, GL_TEXTURE0, 0);\n-\n-\t/* Texture we will render to */\n-\teglImageBayerOut_ = std::make_unique(GL_RGBA, outputSize_.width, outputSize_.height, outputConfig_.stride, GL_TEXTURE1, 1);\n-\n \treturn 0;\n }\n \n void DebayerEGL::stop()\n {\n-\teglImageBayerOut_.reset();\n-\teglImageBayerIn_.reset();\n+\teglImageOutCache_.clear();\n+\teglImageInCache_.clear();\n \n \tif (programId_)\n \t\tglDeleteProgram(programId_);\ndiff --git a/src/libcamera/software_isp/debayer_egl.h b/src/libcamera/software_isp/debayer_egl.h\nindex 348d7305b..d6223f3a9 100644\n--- a/src/libcamera/software_isp/debayer_egl.h\n+++ b/src/libcamera/software_isp/debayer_egl.h\n@@ -9,6 +9,7 @@\n \n #pragma once\n \n+#include \n #include \n #include \n #include \n@@ -68,14 +69,19 @@ private:\n \tvoid setShaderVariableValues(eGLImage &eGLImageIn, const DebayerParams ¶ms);\n \tint debayerGPU(FrameBuffer *input, FrameBuffer *output, const DebayerParams ¶ms, std::optional *mappedInputBuffer, std::optional *inputBufferDmaSyncer);\n \n+\teGLImage *getCachedInputFrameBuffer(FrameBuffer *input, std::optional *inMapped, std::optional *inDmaSyncer);\n+\teGLImage *getCachedOutputFrameBuffer(FrameBuffer *output);\n+\n \t/* Shader program identifiers */\n \tGLuint vertexShaderId_ = 0;\n \tGLuint fragmentShaderId_ = 0;\n \tGLuint programId_ = 0;\n \n \t/* Pointer to object representing input texture */\n-\tstd::unique_ptr eglImageBayerIn_;\n-\tstd::unique_ptr eglImageBayerOut_;\n+\tstd::deque>> eglImageInCache_;\n+\tstd::deque>> eglImageOutCache_;\n+\tunsigned int inputBufferCount_;\n+\tunsigned int outputBufferCount_;\n \n \t/* Shader parameters */\n \tfloat firstRed_x_;\n","prefixes":["v6","6/6"]}