{"id":24249,"url":"https://patchwork.libcamera.org/api/1.1/patches/24249/?format=json","web_url":"https://patchwork.libcamera.org/patch/24249/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20250827090739.86955-13-david.plowman@raspberrypi.com>","date":"2025-08-27T09:07:39","name":"[RFC,12/12] pipline: rpi: Support memory cameras processing requests","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"4928ba855768376db6d3ba2f05fb6fdab07cfb41","submitter":{"id":42,"url":"https://patchwork.libcamera.org/api/1.1/people/42/?format=json","name":"David Plowman","email":"david.plowman@raspberrypi.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/24249/mbox/","series":[{"id":5407,"url":"https://patchwork.libcamera.org/api/1.1/series/5407/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=5407","date":"2025-08-27T09:07:27","name":"Bayer Re-Processing","version":1,"mbox":"https://patchwork.libcamera.org/series/5407/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/24249/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/24249/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 2D6F9C332E\n\tfor ;\n\tWed, 27 Aug 2025 09:08:13 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 6210569309;\n\tWed, 27 Aug 2025 11:08:12 +0200 (CEST)","from mail-wm1-x335.google.com (mail-wm1-x335.google.com\n\t[IPv6:2a00:1450:4864:20::335])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 65E2469303\n\tfor ;\n\tWed, 27 Aug 2025 11:07:53 +0200 (CEST)","by mail-wm1-x335.google.com with SMTP id\n\t5b1f17b1804b1-45a1b00f23eso38669175e9.0\n\tfor ;\n\tWed, 27 Aug 2025 02:07:53 -0700 (PDT)","from raspberrypi.pitowers.org\n\t([2a00:1098:3142:1f:ffc9:aff6:7f7f:893b])\n\tby smtp.gmail.com with ESMTPSA id\n\t5b1f17b1804b1-45b6f30fe02sm21498675e9.18.2025.08.27.02.07.51\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tWed, 27 Aug 2025 02:07:51 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key;\n\tunprotected) header.d=raspberrypi.com header.i=@raspberrypi.com\n\theader.b=\"g52tKWAV\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=raspberrypi.com; s=google; t=1756285673; x=1756890473;\n\tdarn=lists.libcamera.org; \n\th=content-transfer-encoding:mime-version:references:in-reply-to\n\t:message-id:date:subject:cc:to:from:from:to:cc:subject:date\n\t:message-id:reply-to;\n\tbh=9qoMSMv7GS7D3NfmPMIenbPQUZV27K9S7JmNlJHn57g=;\n\tb=g52tKWAV21Fj400oQStiwLpZrfo31zogq7am39woGNkN8STtgBXAkh+NQZn2nCKJLx\n\tf408ckGkoKIfEKQttJ+q+UpibAjjCaBxR7xRucAdxOrq1WDWIXKz30f0nRw4GhI1BceT\n\tdO2hX2gzpCbUcZx3lfJBFGW3o53OxtgdSymSBh9LTdPSVswjezsTAT8hWcL915YRd1UT\n\tb62utf+a9lmQcn5WpW1/gJ9f/uM7+MZ0AGavys3NIuFYztullkfhrhzggy0LdC6huoFW\n\tVmIbvJj/vlH4A9VsRlRCgce2bVVZnU/6suU1R9rX+LP3YojaKtEwigAkkHpjJxCdZxVz\n\tdVmQ==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20230601; t=1756285673; x=1756890473;\n\th=content-transfer-encoding:mime-version:references:in-reply-to\n\t:message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc\n\t:subject:date:message-id:reply-to;\n\tbh=9qoMSMv7GS7D3NfmPMIenbPQUZV27K9S7JmNlJHn57g=;\n\tb=EtH0mnnszfyeqE//Bf9kURke4yd8FD7izCmAqIXCdv3JT09pU7b/51IPZ0CTz9MjLE\n\tSTLwvtc6p44PykQI+TUjePklhhsVbdk9XTj9gXuWkx5mL/hj3MkBhhwBrTbjGfLd7KZp\n\tzuyhB4fFsbCs7/kqmZvolPMJqB4SG5SmXc9CyAOnhAPMJlvxvPnAVMv5uhZteo+nZ1j2\n\t28/mPdLDHI9n1wZxG1bqTZYU73f9+ST21IJCjTAINgdc2SjoLHKL35sqezsAesUbsf+C\n\ttcSSuxiE/aJwdmhCNy7W0pCfDBB7KH7m1PWG9GHzOE/3YpqNQD9cXAzmePvXGJKduDOr\n\tSStg==","X-Gm-Message-State":"AOJu0YxV7ODGlmdZZNZzr/hJyatZFhWDnIveVBML9rsARM7DMIDj1ttF\n\t/UD5DTbfxbn2eaVbpAnZ74civoaUNssS4qmC/4BXu1XsO15QTCi2vSNhynxpxcmUkVRg42v8nF3\n\teYRc+","X-Gm-Gg":"ASbGncu+a2ymcCkro56Pdkn8Ag5aELV+hq4on1gMI7WmQOelpfPHDGKb0NhhwwrCU65\n\t/gQcubdNr1GbJSHiCefOO+F3akD6+ifTc7AoQNFlG6SwQvOZJ7Z4qz3YL36A8r/01w07dKlk79T\n\txN69ANTaJUKbvp8QglDIZVbAHKMZ+PP9/Ya29uIhYeHJ/mvFcRIIwIHKQ43KsPAzQVvuxWyI3k3\n\tlJJUr9q1dntpDZKmZ0oKEn9fu38XmweEaMF3sxpXZyv4teb8xvPUUg73Rea3gAlt/1KetU0Nslq\n\tLlmUdU/rj9OmVsRZQGiN8D78Tx9zUASnoxpBq4a9U5ZW29QqtjpCu9jN7/eptYpjH8K5mFte0Ia\n\tz9rCVipZXd+B73hYHEmm/I3iQcY41+hxeRkuXmE4PcnBe4cG9F2+CSHa0dSlYTJiiLao46Ovo+r\n\tAj03rGQ6rvBfKqp+v4nDAlEbjhFUpg6A8OM5T9Nr8pkeZ7z6//ow==","X-Google-Smtp-Source":"AGHT+IFsOsFg4QSpn/ny3ZFkYfdpQTWfNuDMnx5KiLyPLNutkN645L9sQgGUcUKtgPIPqG5zucklyw==","X-Received":"by 2002:a05:600c:b95:b0:459:d645:bff7 with SMTP id\n\t5b1f17b1804b1-45b5179cf7dmr161674605e9.12.1756285672467; \n\tWed, 27 Aug 2025 02:07:52 -0700 (PDT)","From":"David Plowman ","To":"libcamera-devel@lists.libcamera.org","Cc":"David Plowman ","Subject":"[RFC PATCH 12/12] pipline: rpi: Support memory cameras processing\n\trequests","Date":"Wed, 27 Aug 2025 10:07:39 +0100","Message-Id":"<20250827090739.86955-13-david.plowman@raspberrypi.com>","X-Mailer":"git-send-email 2.39.5","In-Reply-To":"<20250827090739.86955-1-david.plowman@raspberrypi.com>","References":"<20250827090739.86955-1-david.plowman@raspberrypi.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"This is the final commit in the set to support memory cameras (Bayer\nreprocessing).\n\nThere are some small fix-ups to allow the camera system to start\nwithout the CFE being involved, and we add runMemoryCamera() which\npushs a buffer from the request into the Back End ISP.\n\nSigned-off-by: David Plowman \n---\n .../pipeline/rpi/common/pipeline_base.cpp | 23 ++++----\n src/libcamera/pipeline/rpi/pisp/pisp.cpp | 53 ++++++++++++++++---\n 2 files changed, 61 insertions(+), 15 deletions(-)","diff":"diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp\nindex 31bacc7c..8ca1e41f 100644\n--- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp\n+++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp\n@@ -727,15 +727,19 @@ int PipelineHandlerBase::start(Camera *camera, const ControlList *controls)\n \t/* A good moment to add an initial clock sample. */\n \tdata->wallClockRecovery_.addSample();\n \n-\t/*\n-\t * Reset the delayed controls with the gain and exposure values set by\n-\t * the IPA.\n-\t */\n-\tdata->delayedCtrls_->reset(0);\n-\tdata->state_ = CameraData::State::Idle;\n+\t/* A memory camera wouldn't have a front end device, so you would skip this. */\n+\tif (data->frontendDevice()) {\n+\t\t/*\n+\t\t * Reset the delayed controls with the gain and exposure values set by\n+\t\t * the IPA.\n+\t\t */\n+\t\tdata->delayedCtrls_->reset(0);\n+\n+\t\t/* Enable SOF event generation. */\n+\t\tdata->frontendDevice()->setFrameStartEnabled(true);\n+\t}\n \n-\t/* Enable SOF event generation. */\n-\tdata->frontendDevice()->setFrameStartEnabled(true);\n+\tdata->state_ = CameraData::State::Idle;\n \n \tdata->platformStart();\n \n@@ -762,7 +766,8 @@ void PipelineHandlerBase::stopDevice(Camera *camera)\n \t\tstream->dev()->streamOff();\n \n \t/* Disable SOF event generation. */\n-\tdata->frontendDevice()->setFrameStartEnabled(false);\n+\tif (data->frontendDevice())\n+\t\tdata->frontendDevice()->setFrameStartEnabled(false);\n \n \tdata->clearIncompleteRequests();\n \ndiff --git a/src/libcamera/pipeline/rpi/pisp/pisp.cpp b/src/libcamera/pipeline/rpi/pisp/pisp.cpp\nindex bb22e6d2..a637c6be 100644\n--- a/src/libcamera/pipeline/rpi/pisp/pisp.cpp\n+++ b/src/libcamera/pipeline/rpi/pisp/pisp.cpp\n@@ -780,6 +780,7 @@ private:\n \tvoid prepareCfe();\n \tvoid prepareBe(uint32_t bufferId, bool stitchSwapBuffers);\n \n+\tvoid runMemoryCamera();\n \tvoid tryRunPipeline() override;\n \n \tstruct CfeJob {\n@@ -1243,7 +1244,7 @@ PipelineHandlerPiSP::platformCreateCamera(std::unique_ptr &came\n \n \t/* Wire up all the IPA connections. */\n \tdata->ipa_->prepareIspComplete.connect(data, &PiSPCameraData::prepareIspComplete);\n-\tdata->ipa_->processStatsComplete.connect(data, &PiSPCameraData::processStatsComplete);\n+\t/* We don't want the processStatsComplete callback, though, for memory cams. */\n \n \t/* Create and register the camera. */\n \tconst std::string &id = cfe ? data->sensor_->id() : \"memory\";\n@@ -1785,8 +1786,11 @@ void PiSPCameraData::platformStart()\n \n \tcfeJobQueue_ = {};\n \n-\tfor (unsigned int i = 0; i < config_.numCfeConfigQueue; i++)\n-\t\tprepareCfe();\n+\t/* A memory camera wouldn't be using the CFE. */\n+\tif (cfe_[Cfe::Output0].dev()) {\n+\t\tfor (unsigned int i = 0; i < config_.numCfeConfigQueue; i++)\n+\t\t\tprepareCfe();\n+\t}\n \n \t/* Clear the debug dump file history. */\n \tlast_dump_file_.clear();\n@@ -1890,8 +1894,12 @@ void PiSPCameraData::beInputDequeue(FrameBuffer *buffer)\n \t\t\t<< \", buffer id \" << cfe_[Cfe::Output0].getBufferId(buffer)\n \t\t\t<< \", timestamp: \" << buffer->metadata().timestamp;\n \n-\t/* The ISP input buffer gets re-queued into CFE. */\n-\thandleStreamBuffer(buffer, &cfe_[Cfe::Output0]);\n+\t/*\n+\t * The ISP input buffer gets re-queued into CFE (but not for memory cameras\n+\t * that aren't using the CFE).\n+\t */\n+\tif (cfe_[Cfe::Output0].dev())\n+\t\thandleStreamBuffer(buffer, &cfe_[Cfe::Output0]);\n \thandleState();\n }\n \n@@ -2361,7 +2369,15 @@ void PiSPCameraData::prepareCfe()\n \n void PiSPCameraData::prepareBe(uint32_t bufferId, bool stitchSwapBuffers)\n {\n-\tFrameBuffer *buffer = cfe_[Cfe::Output0].getBuffers().at(bufferId).buffer;\n+\tFrameBuffer *buffer;\n+\tif (!sensor_->isMemory())\n+\t\tbuffer = cfe_[Cfe::Output0].getBuffers().at(bufferId).buffer;\n+\telse {\n+\t\t/* Memory camera. Back end input buffer is in the request. */\n+\t\tRequest *request = requestQueue_.front();\n+\t\tconst libcamera::Stream *rawStream = &isp_[Isp::Input];\n+\t\tbuffer = request->findBuffer(rawStream);\n+\t}\n \n \tLOG(RPI, Debug) << \"Input re-queue to ISP, buffer id \" << bufferId\n \t\t\t<< \", timestamp: \" << buffer->metadata().timestamp;\n@@ -2412,8 +2428,33 @@ void PiSPCameraData::prepareBe(uint32_t bufferId, bool stitchSwapBuffers)\n \tisp_[Isp::Config].queueBuffer(config.buffer);\n }\n \n+void PiSPCameraData::runMemoryCamera()\n+{\n+\tRequest *request = requestQueue_.front();\n+\n+\tapplyScalerCrop(request->controls());\n+\n+\trequest->metadata().clear();\n+\n+\tstate_ = State::Busy;\n+\n+\tipa::RPi::PrepareParams params;\n+\tparams.buffers.embedded = 0;\n+\tparams.ipaContext = requestQueue_.front()->sequence();\n+\tparams.requestControls = request->controls();\n+\n+\tLOG(RPI, Debug) << \"runMemoryCamera: call prepareIsp\";\n+\tipa_->prepareIsp(params);\n+}\n+\n void PiSPCameraData::tryRunPipeline()\n {\n+\tif (state_ == State::Idle && !requestQueue_.empty() && sensor_->isMemory()) {\n+\t\trunMemoryCamera();\n+\n+\t\treturn;\n+\t}\n+\n \t/* If any of our request or buffer queues are empty, we cannot proceed. */\n \tif (state_ != State::Idle || requestQueue_.empty() || !cfeJobComplete())\n \t\treturn;\n","prefixes":["RFC","12/12"]}