From patchwork Sun Aug 24 00:48:48 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bryan O'Donoghue X-Patchwork-Id: 24218 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BE968BEFBE for ; Sun, 24 Aug 2025 00:49:46 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2D9CB69342; Sun, 24 Aug 2025 02:49:46 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="XZkdN03+"; dkim-atps=neutral Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2646069329 for ; Sun, 24 Aug 2025 02:49:10 +0200 (CEST) Received: by mail-wm1-x333.google.com with SMTP id 5b1f17b1804b1-45a1b0c82eeso27029845e9.3 for ; Sat, 23 Aug 2025 17:49:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1755996549; x=1756601349; darn=lists.libcamera.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=DTTcwJfG7ak8JDbsC+gvp8t3BtyZQS47SSilc/BHVNE=; b=XZkdN03+Qy7eNNNE2y0OgCOvuFCRkJFKhlD90YkwW4KuJt02eZ94BzYh518sdjWVIo BhYU3A6H+cAH3PQZS2WqzH4WmZo8Usr+k0wQJObkri48UemwxhYP4wDa0s7pcRWuc2PN 88U2zF4DZIbOIuVSnTgUODaO+8YJuTDQRhxWGRpGZw9+BIeki4nNddEYsz4rd5QCj0S4 U/RbwBAuaID4OXtHgHvbA4NA9aR+8gXcErWRnxTuocHN/ltnJAdgFIlPgmJswxGpOLaH jfTs7itXmf/DJNQkS7/UdfQcNz6HNoc96c72m1E14yrh60c5dE6fbRHPzhQ16OezcmyA Gyhg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1755996549; x=1756601349; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DTTcwJfG7ak8JDbsC+gvp8t3BtyZQS47SSilc/BHVNE=; b=tQA1DoCWQNfxP1n24/3QCGGZn6VkKDPw4u4qpzGLkcosgsE9mH9BfWs9zqf9uQ10pX i4vg9/GpE1Rpcml4wL+1Of1WMJOC/kU/QwGHp1xXeZ+E6/X2tyeXe15/E4vAef+WizuD 0i1lWUAc3p30xmTpv0xoBojxXAC6UodtVlviGmoqHfRIQnR/IX2/PYOrfVJM6L+FlCrc 2NWL+njAd3ewIEg7G6ko09oFNvm/sg+HPHvaWkoqMJBJ+wCH+wue4bZ/4v9GW4M0bJ0Z 7S6YhPl27t6eiweN3WFS1RZ6zJ6vRoWdRomQ0EKAOVOQ1pc4/GagZ6hjbc6lhT08rgRp CwpQ== X-Gm-Message-State: AOJu0Ywr3h1XiGRX4IVblsTEA3oPoZ/9ao2tvEQRzqLPt22Q0zStEDoP 4zWR3ajWFq+LmjzBPSRAChfOn+z02DCa1UKtwrCAveCAc48j5TlHq1LM3qRK0+hIm3pvXLp3lCo w0MtQMnA= X-Gm-Gg: ASbGncubyNDk4EKbPcgs9Ew8fGGlS4p5rNUd3x2tiAHN1h5ePB57Pu06i9NhBldNjEm BW9KHzwNbYtb8y3uvw90Wwgb/F+MsawRzU+SGFSve9rr/JQk6Xa95HutdvnayD/Ac7LOfP7+1hm AZMoLInCjVvK0zokwqsIQ6+cEynT+IOkWFJwNntBD0Y8j1z+xYUDkVzUmqvaqBCQyo4xDO81JSq 5KS28BnVsGmsjRTDCenuLu6BcGtx3M8AcOTK2FUkxsu85ucJkpAWkZM9/pFvrgEaA/B1msCpxpK CxRloavGRA892efnyLm8bx/mMk6q14TFnM7zKLKwaj2XS0P7bx1U69eDk9nY+8q16nuWnqaKjYQ 6IRS5GT5vtTphqY8f/BxgAO+anBFp916VmiqnqwThz+dnDiXOjDtb+ut3hkNuqryPw/RayXvpoO 9AcSHb2N04wlv9eLbsYMiv X-Google-Smtp-Source: AGHT+IHxExngAvTts5eFr72F2OqlFav1cKsrzg4+O3442fpQkH5wQYkFeve3xd3u2gXQf1X/k4qs+w== X-Received: by 2002:a05:600c:3b2a:b0:453:2066:4a26 with SMTP id 5b1f17b1804b1-45b5179f3d1mr90594065e9.16.1755996549319; Sat, 23 Aug 2025 17:49:09 -0700 (PDT) Received: from [192.168.0.13] (188-141-3-146.dynamic.upc.ie. [188.141.3.146]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-45b4e1d530esm69347225e9.0.2025.08.23.17.49.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 23 Aug 2025 17:49:08 -0700 (PDT) From: Bryan O'Donoghue Date: Sun, 24 Aug 2025 01:48:48 +0100 Subject: [PATCH v2 36/37] libcamera: software_isp: Add a gpuisp todo list MIME-Version: 1.0 Message-Id: <20250824-b4-v0-5-2-gpuisp-v2-a-v2-36-96f4576c814e@linaro.org> References: <20250824-b4-v0-5-2-gpuisp-v2-a-v2-0-96f4576c814e@linaro.org> In-Reply-To: <20250824-b4-v0-5-2-gpuisp-v2-a-v2-0-96f4576c814e@linaro.org> To: libcamera-devel@lists.libcamera.org Cc: Bryan O'Donoghue X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=3052; i=bryan.odonoghue@linaro.org; h=from:subject:message-id; bh=fkjJdxpwJCUqkX+i6uEWQGqq/pTEpLO+hSigKW03abk=; b=owEBbQKS/ZANAwAKASJxO7Ohjcg6AcsmYgBoqmFifMwU/iPZVmSeEQPpeXThnFXdg8dMtnCoN oL0+CMes5GJAjMEAAEKAB0WIQTmk/sqq6Nt4Rerb7QicTuzoY3IOgUCaKphYgAKCRAicTuzoY3I OkhFEACT/4I0D0wz+IL3JLsvDp88MQU2iTV8CpoqWWLi2fHPnZSPlwm+7FaAtYjELHFvw8BnUuI B1GUTenXy4GzWB5D3h/kcVIqQ70qyASAdvf8O7EAQAyc9roHtnoiwERo9HKxFruwnYE9+sxqgoA oAF3/VMzYtNbFqT2+jFDpSlzhmfFlblDLEmg7QGCIroJBRLGck0+RAiom284erBdApIRKMhKu1A cO0ix4kphZRsZ2ETN29F+WKca02r2ndqYFfryj4nnkyl2zUHFoFYaCqGIIKKhqDVxKG/5PdwImA NYu5IRwbcyDut3vB0eKg9XAzAkwKvAM68RKYRCZ6SdTXfSYyxdBBUb6KXXy2n7bTMrL26DJ2HHo sEyyChkV/F3ik+UVhmU5A8f6/UrTdGwJoWU6gZO6LrxelGGHHtA05sR1URXBe0PL+GhHgpYPhos wT1pUUpGd8/fF1mOMPyzlAag+RXhJEIHlqHId7nNKX9DMRUrcrSxC4t5z5WY7v1tLcIQs9c68T8 HQRXFWB3cTTna5f6zxGFCkOruSkeRpZv0Z2VralO8Ml35iFsgnKP2tTbYow3EHwRe7drShKe+ri Z1Eicw+GL4hIKIglTRVTuIHKMYe9YdlEVQmIWyzoYizCNod1KvDdE5UA/Vm4pXQO0/qvcjtNoFf rCQRtmRV5I5u8tg== X-Developer-Key: i=bryan.odonoghue@linaro.org; a=openpgp; fpr=E693FB2AABA36DE117AB6FB422713BB3A18DC83A X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" List the series of things to do in GPU ISP in perceived order of difficulty. Signed-off-by: Bryan O'Donoghue --- src/libcamera/software_isp/gpuisp-todo.txt | 61 ++++++++++++++++++++++++++++++ 1 file changed, 61 insertions(+) diff --git a/src/libcamera/software_isp/gpuisp-todo.txt b/src/libcamera/software_isp/gpuisp-todo.txt new file mode 100644 index 0000000000000000000000000000000000000000..d283959c3ee2864f8ec3534107938505ae858c61 --- /dev/null +++ b/src/libcamera/software_isp/gpuisp-todo.txt @@ -0,0 +1,61 @@ +List the TODOs in perceived order of ease. + +Version 2: +Make GPUISP default: + - Right now the environment variable allows over-riding to swtich + from CPU to GPU. + - Once we support 24 BPP output on GPUISP we will have the same + pixel format support as CPU and can set the default to GPU without + regressing functionality + +glTexture1D: + - Initial code was developed for < GLES 2.O but since we have fixed + on GLES >= 2.0 this means we can use glTexture1D + - Provided this is so amend the shaders to do val = texture(x, y, 0); + not texture(x, y, 0.5) the 0.5 is because of using glTexture2D + +Denoising: + - Run a denoise algorithm in the shaders + - Supply a control to influence the noise-floor ? + +Dead pixel correction: + - Add logic to correct dead pixels in the fragment shaders + +Version 1: +24 bit output support: + - Take the BPP we already capture and get a 24 bit GBM surface + - Pass a compile-time parameter to the shaders to tell them to do + gl_FragColor = rgb not gl_FragColor = rgba + - Version 2: + This is not possible. + gl_FragColor expects vec4 not vec3 on the output. + If you really want RGB888 run cpuisp. + +Surfaceless GBM: + - We get a GBM surface and then have to swap buffers + If we rework for surfaceless GBM and EGL then the swap buffer can + be dropped. + - Version 2: + Complete GBM surface removed, memcpy() phase removed also + +dma-buf texture upload: + - Currently we pass the input buffer to glCreateTexture2D. + We should be able to make the upload of the input buffer go faster + by using eglCreateImageKHR and enumerated the dma-buf contents. + - Version 2: + Complete sm8250 test platform shows 20x performance increase + with CCM. + +Render-to-texture: + - Right now we render to the GBM provided surface framebuffer + and then memcpy from that buffer to the target output buffer. + This necessitates flushing the cache on the target buffer in + addition to the memcpy(). + - Render-to-texture where we generate the target framebuffer + directly from a dma-buf handle will mitigate the memcpy() phase. + - It should be the case then that the consumer of the output buffer + i.e. the thing that's not libcamera is responsible to flush the cache + if-and-only-if that user writes to the buffer. + - We need to flush the cache on the buffer because we are memcpying() to it. + - Version 2: + Done in version 2