From patchwork Wed Jun 24 08:58:49 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bryan O'Donoghue X-Patchwork-Id: 27036 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EE8B4C3308 for ; Wed, 24 Jun 2026 08:59:21 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 79F4C65895; Wed, 24 Jun 2026 10:59:21 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="jGh1QZqe"; dkim-atps=neutral Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8BBC265891 for ; Wed, 24 Jun 2026 10:59:08 +0200 (CEST) Received: by mail-wm1-x334.google.com with SMTP id 5b1f17b1804b1-4924593f45dso8073385e9.1 for ; Wed, 24 Jun 2026 01:59:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1782291548; x=1782896348; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LvpHhq7SDZeODR/gVrmxjhKv6G3W7LqpHWaMD33/8yc=; b=jGh1QZqe0JkL8Bn4+Xlq1G/PlJi+XewVawIdb3Rq/1OM51XLJN0Z1BDmJY1D+6sgBa +AKLAwYNKFBvc9yZL6M6wEqLf+FSh9uqCzy0boiPT1KZbL+rH9ndgyYd+HIRIbsgoPpC mX4mteY9QeCB5ZHtMFIDsPvq9WG2tndDEflPGYgO1Upfgmf1eRWUezunkIPA/+ZpTtqY sVa3uUpyk9sxRjQdsHJs/rBiTiTQ32IPaHH+mcP5i/O7cMKPYWV8w4WVYT2uZXshvUkL /0aKvPuI0Pu6Ad9tUHclB3/WUGavhRxIw2/RAegQkIS0UfxYPeKMbhoFWwIA7/iZLAX1 6qEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1782291548; x=1782896348; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=LvpHhq7SDZeODR/gVrmxjhKv6G3W7LqpHWaMD33/8yc=; b=A05gQToU1ecLr+3q+hj8Xt+Wd9wAZHc4x7chdkPdm7Co/HIRr2KzQtnjyLt2zZrSiU 9aKsR7dXzeN9yRRCUuQ7YsgxKMaBguWmWyBSp/G70WviUBFLzaq/VXNOlfT8PxKNIQgk BnSaTlSzZdhajY7vNk6evA59GHdnosSOJp1NKyktPZzbYFjGWw3yRuzRPnpJlHumt8Qv 5GLu0MODOkxwJ5la2g+qpaUSt61c1HC+ji7XJ/i61zn6QaqnyGxrkOXdae2HBrMc2umC 241820C4y6G8tgIwHm1ai6SLHdeVMRi6GA2DmoAhHooIQD4nLeEQi1ppwKF7BucxrWXs XfZQ== X-Gm-Message-State: AOJu0YwXtVIeaitRPO1qX2fvZTPL9VCD7Umm40fDhXewmgswNQYVHP14 u9aIbMeIWvC87q/uBC3JMOW4ykPDKJn8eFR/XtDDDvLPc+gCTHgs0E4ZS+aukaFhs9/pF0LgJ8t xCGOwusw= X-Gm-Gg: AfdE7cmOSP+WslFDzqAijgZiaBlT+7jfs56F0vJvAPz27a2nhRc9tr2AcXKMBDqYMlT 1AETPwR0hAaeTm6li0lc1kw3Sajq8xYEY47vHk6R7ZABJl8v7WoVyjWsRzdxHmv4S5pdxBA+kaz tKbgD9CJkc5/ox6kIfoVbPn2R2g+qSmXVyvAJW0NPTlBOjMPP9XNkZfx1+o4jKtFvqFjwqFb18H N23s291AApPw/aOfTd9/CfyGiVjZWkq0/uhaNeZvOBq+Xbwr0SRmh85Ixt7XJolutlgyq8O+ROG HPRDIQN6r/zALRakV99XNClfAGWuPzay2m0wJCvf2HnT7xC/oIpzrtl9eylsXEqlJvqXcfgIhfq avjamXXoss1knZws1WYcQOsnq9ZJPrV4BTL0xdPgVjaQiTvORLSR7Mk8xLx8VAio4rhEa84+D7u pr9GKbwdeflpH4Zfqm7xIAD76whRon X-Received: by 2002:a05:600c:a012:b0:492:2e1c:1d19 with SMTP id 5b1f17b1804b1-49260875bbamr32747295e9.31.1782291547966; Wed, 24 Jun 2026 01:59:07 -0700 (PDT) Received: from inspiron14p-linux ([109.76.100.231]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49261063d6esm25128375e9.2.2026.06.24.01.59.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2026 01:59:06 -0700 (PDT) From: Bryan O'Donoghue To: libcamera-devel@lists.libcamera.org Cc: bryan.odonoghue@linaro.org, pavel@ucw.cz Subject: [PATCH 10/10] libcamera: software_isp: debayer_egl: Implement input/output frame caching mechanism Date: Wed, 24 Jun 2026 09:58:49 +0100 Message-ID: <20260624085849.873784-11-bryan.odonoghue@linaro.org> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260624085849.873784-1-bryan.odonoghue@linaro.org> References: <20260624085849.873784-1-bryan.odonoghue@linaro.org> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement a texture caching mechanism for both input and output frames and for both types of input frame. The before/after on a Qualcomm x1e is: 9.737ms per frame 5.691ms per frame The before/after on a Qualcomm sm8250 is: 21.710ms per frame 17.336ms per frame for i in {1..20} do cam -c /base/soc@0/cci@ac16000/i2c-bus@1/camera@10 -s width=1920,height=1080 --capture=60 Interestingly there appears to be an absolute ~ 4.x ms per frame uplift as opposed to what intuition might suggest a proportional. Signed-off-by: Bryan O'Donoghue Tested-by: Robert Mader Reviewed-by: Robert Mader --- src/libcamera/software_isp/debayer_egl.cpp | 108 +++++++++++++++++---- src/libcamera/software_isp/debayer_egl.h | 12 ++- 2 files changed, 100 insertions(+), 20 deletions(-) diff --git a/src/libcamera/software_isp/debayer_egl.cpp b/src/libcamera/software_isp/debayer_egl.cpp index 0568c413b..8ac5cb76f 100644 --- a/src/libcamera/software_isp/debayer_egl.cpp +++ b/src/libcamera/software_isp/debayer_egl.cpp @@ -355,6 +355,12 @@ int DebayerEGL::configure(const StreamConfiguration &inputCfg, */ stats_->setWindow(Rectangle(window_.size())); + inputBufferCache_ = std::make_unique(inputCfg.bufferCount); + outputBufferCache_ = std::make_unique(outputCfg.bufferCount); + + eglImageBayerIn_.resize(inputCfg.bufferCount); + eglImageBayerOut_.resize(outputCfg.bufferCount); + return 0; } @@ -514,34 +520,106 @@ void DebayerEGL::setShaderVariableValues(eGLImage &eglImageIn, const DebayerPara return; } -int DebayerEGL::debayerGPU(FrameBuffer *input, FrameBuffer *output, const DebayerParams ¶ms, std::optional *inMapped, std::optional *inDmaSyncer) +int DebayerEGL::getBufferCache(V4L2BufferCache &cache, FrameBuffer *framebuffer, bool &cache_hit) { - /* eGL context switch */ - egl_.makeCurrent(); + int cache_idx; + + cache_idx = cache.get(*framebuffer, cache_hit); + if (cache_idx < 0) { + LOG(Debayer, Error) << "buffer exceeds configured cache size"; + return -ENODEV; + } + cache.put(cache_idx); + + return cache_idx; +} + +eGLImage *DebayerEGL::getCachedInputFrameBuffer(FrameBuffer *input, std::optional *inMapped, std::optional *inDmaSyncer) +{ + eGLImage *eglImageIn; + bool cache_hit; + int cache_idx; + + cache_idx = getBufferCache(*inputBufferCache_, input, cache_hit); + if (cache_idx < 0) + return nullptr; + + if (!cache_hit) { + eglImageBayerIn_[cache_idx] = std::make_unique(glFormat_, inputConfig_.stride / bytesPerPixel_, + height_, inputConfig_.stride, GL_TEXTURE0, 0); + } + + eglImageIn = eglImageBayerIn_[cache_idx].get(); /* Try to create texture for input buffer via dmabuf import */ - if (use_dmabuf_) { - if (egl_.createInputDMABufTexture2D(*eglImageBayerIn_, input->planes()[0].fd.get()) != 0) { + if (use_dmabuf_ && !cache_hit) { + if (egl_.createInputDMABufTexture2D(*eglImageIn, input->planes()[0].fd.get()) != 0) { use_dmabuf_ = false; LOG(Debayer, Info) << "Importing input buffer with DMABuf import failed, falling back to upload"; } } + /* Cache hit using dmabuf activate and bind */ + if (use_dmabuf_ && cache_hit) { + egl_.activateBindTexture(*eglImageIn); + } + /* Otherwise create texture for input buffer via upload from CPU */ if (!use_dmabuf_) { inDmaSyncer->emplace(input->planes()[0].fd, DmaSyncer::SyncType::Read); inMapped->emplace(input, MappedFrameBuffer::MapFlag::Read); if (!inMapped->value().isValid()) { LOG(Debayer, Error) << "mmap-ing buffer(s) failed"; - return -ENODEV; + return nullptr; } - egl_.createInputTexture2D(*eglImageBayerIn_, inMapped->value().planes()[0].data()); + if (cache_hit) + egl_.updateInputTexture2D(*eglImageIn, inMapped->value().planes()[0].data()); + else + egl_.createInputTexture2D(*eglImageIn, inMapped->value().planes()[0].data()); } - /* Generate the output render framebuffer as render to texture */ - egl_.createOutputDMABufTexture2D(*eglImageBayerOut_, output->planes()[0].fd.get()); + return eglImageIn; +} + +eGLImage *DebayerEGL::getCachedOutputFrameBuffer(FrameBuffer *output) +{ + eGLImage *eglImageOut; + bool cache_hit; + int cache_idx; + + cache_idx = getBufferCache(*outputBufferCache_, output, cache_hit); + if (cache_idx < 0) + return nullptr; + + if (!cache_hit) { + eglImageBayerOut_[cache_idx] = std::make_unique(GL_RGBA, outputSize_.width, + outputSize_.height, outputConfig_.stride, GL_TEXTURE1, 1); + egl_.createOutputDMABufTexture2D(*eglImageBayerOut_[cache_idx], output->planes()[0].fd.get()); + } + eglImageOut = eglImageBayerOut_[cache_idx].get(); + + return eglImageOut; +} + +int DebayerEGL::debayerGPU(FrameBuffer *input, FrameBuffer *output, const DebayerParams ¶ms, std::optional *inMapped, std::optional *inDmaSyncer) +{ + eGLImage *eglImageIn; + eGLImage *eglImageOut; + + /* eGL context switch */ + egl_.makeCurrent(); + + eglImageIn = getCachedInputFrameBuffer(input, inMapped, inDmaSyncer); + if (!eglImageIn) + return -ENOMEM; + + eglImageOut = getCachedOutputFrameBuffer(output); + if (!eglImageOut) + return -ENOMEM; + + egl_.attachTextureToFBO(*eglImageOut); + setShaderVariableValues(*eglImageIn, params); - setShaderVariableValues(*eglImageBayerIn_, params); glViewport(0, 0, width_, height_); glClear(GL_COLOR_BUFFER_BIT); glDrawArrays(GL_TRIANGLE_FAN, 0, DEBAYER_OPENGL_COORDS); @@ -623,19 +701,13 @@ int DebayerEGL::start() if (initBayerShaders(inputPixelFormat_, outputPixelFormat_)) return -EINVAL; - /* Raw bayer input as texture */ - eglImageBayerIn_ = std::make_unique(glFormat_, inputConfig_.stride / bytesPerPixel_, height_, inputConfig_.stride, GL_TEXTURE0, 0); - - /* Texture we will render to */ - eglImageBayerOut_ = std::make_unique(GL_RGBA, outputSize_.width, outputSize_.height, outputConfig_.stride, GL_TEXTURE1, 1); - return 0; } void DebayerEGL::stop() { - eglImageBayerOut_.reset(); - eglImageBayerIn_.reset(); + eglImageBayerOut_.clear(); + eglImageBayerIn_.clear(); if (programId_) glDeleteProgram(programId_); diff --git a/src/libcamera/software_isp/debayer_egl.h b/src/libcamera/software_isp/debayer_egl.h index d8509e9f2..238fe7345 100644 --- a/src/libcamera/software_isp/debayer_egl.h +++ b/src/libcamera/software_isp/debayer_egl.h @@ -22,6 +22,7 @@ #include "libcamera/internal/mapped_framebuffer.h" #include "libcamera/internal/software_isp/benchmark.h" #include "libcamera/internal/software_isp/swstats_cpu.h" +#include "libcamera/internal/v4l2_videodevice.h" #include #include @@ -70,14 +71,21 @@ private: bool use_dmabuf_; + int getBufferCache(V4L2BufferCache &buffercache, FrameBuffer *framebuffer, bool &hit); + eGLImage *getCachedInputFrameBuffer(FrameBuffer *input, std::optional *inMapped, std::optional *inDmaSyncer); + eGLImage *getCachedOutputFrameBuffer(FrameBuffer *output); + + std::unique_ptr inputBufferCache_; + std::unique_ptr outputBufferCache_; + /* Shader program identifiers */ GLuint vertexShaderId_ = 0; GLuint fragmentShaderId_ = 0; GLuint programId_ = 0; /* Pointer to object representing input texture */ - std::unique_ptr eglImageBayerIn_; - std::unique_ptr eglImageBayerOut_; + std::vector> eglImageBayerIn_; + std::vector> eglImageBayerOut_; /* Shader parameters */ float firstRed_x_;