{"id":25476,"url":"https://patchwork.libcamera.org/api/patches/25476/?format=json","web_url":"https://patchwork.libcamera.org/patch/25476/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20251210164055.17856-12-david.plowman@raspberrypi.com>","date":"2025-12-10T16:15:26","name":"[11/11] pipline: rpi: Support memory cameras processing requests","commit_ref":null,"pull_url":null,"state":"new","archived":false,"hash":"57f41f8db4be8ba2fb43703872f7ca5d7c6a1381","submitter":{"id":42,"url":"https://patchwork.libcamera.org/api/people/42/?format=json","name":"David Plowman","email":"david.plowman@raspberrypi.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/25476/mbox/","series":[{"id":5650,"url":"https://patchwork.libcamera.org/api/series/5650/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=5650","date":"2025-12-10T16:15:15","name":"Bayer re-processing","version":1,"mbox":"https://patchwork.libcamera.org/series/5650/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/25476/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/25476/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 96AEDC326C\n\tfor ;\n\tWed, 10 Dec 2025 16:41:15 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 45967614EE;\n\tWed, 10 Dec 2025 17:41:15 +0100 (CET)","from mail-wm1-x329.google.com (mail-wm1-x329.google.com\n\t[IPv6:2a00:1450:4864:20::329])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id B9A8D614B2\n\tfor ;\n\tWed, 10 Dec 2025 17:41:08 +0100 (CET)","by mail-wm1-x329.google.com with SMTP id\n\t5b1f17b1804b1-47796a837c7so69085e9.0\n\tfor ;\n\tWed, 10 Dec 2025 08:41:08 -0800 (PST)","from davidp-pi5.pitowers.org\n\t([2a00:1098:3142:1f:88ea:c658:5b20:5e46])\n\tby smtp.gmail.com with ESMTPSA id\n\t5b1f17b1804b1-47a88371e13sm1270415e9.11.2025.12.10.08.41.07\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tWed, 10 Dec 2025 08:41:07 -0800 (PST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key;\n\tunprotected) header.d=raspberrypi.com header.i=@raspberrypi.com\n\theader.b=\"QICpIUTS\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=raspberrypi.com; s=google; t=1765384868; x=1765989668;\n\tdarn=lists.libcamera.org; \n\th=content-transfer-encoding:mime-version:references:in-reply-to\n\t:message-id:date:subject:cc:to:from:from:to:cc:subject:date\n\t:message-id:reply-to;\n\tbh=Z2u6QWJXLLkERKnj/8SmMWdGzZUbKd7GPA9TOeKwsPo=;\n\tb=QICpIUTSVpG58ybyWpey7plogzGABP9dgjoL2sKfBwIYOeCvI2+Y8UyFfYVl6DSI6P\n\tIt9YxRHGEmau1oUNkJa6c4Rb8nbEE8LHSExSDdib5r+gJdLybzT+ALNG9lnY49F+4Ici\n\tKDxHT7UlcVYHw8dtvE8U/go9fiyyeALdiMYuGoQ9jQkxyLapMXHgbQ1xakukxnu5vum8\n\tJ5T6ByRKnMN5IEkZbWNqPMvyKf9PsNlSfLQAi2vzBfvsIx2RJPMGtZJnGSGjXkPgUhD8\n\tfjf3oGWdDL1TaaYcWTNwU0tK0g+bvnJRmgtKw0AoBFzBejoTB2V+gtpJKWkz5KACll8B\n\tZ6JQ==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20230601; t=1765384868; x=1765989668;\n\th=content-transfer-encoding:mime-version:references:in-reply-to\n\t:message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from\n\t:to:cc:subject:date:message-id:reply-to;\n\tbh=Z2u6QWJXLLkERKnj/8SmMWdGzZUbKd7GPA9TOeKwsPo=;\n\tb=BFxkte2pamTLPIQe4/vAUyyLAgvmYz56KqMFDokKxQ/AguD0mQv5hEdQmXCRvC81oY\n\tUyL2wbw8j/oLsH+RDohIMlioqBjDVNvFeTd+KEzUagF+k4LinCSH0IiHg5lsIQt1hqoU\n\tVdTOGqR5dLdpi2mVNEKo7DDcCw8Kim9kSrYoR8m8je3M5CM8Ch3KZ0T1VPDHs1HOo2Iv\n\tcjar1FH+DAHxl2oggvxyorDc1rCf7z/xe7YG6WBGpfUpI1ZLAaASu+82FtO1holB3mqk\n\t09VFcDlh5n6EoLzeLJAKzVUnDT4qomMh3UodiiNwXLbuhhO3FTf4nPcVoONbStZl83I5\n\tp2+A==","X-Gm-Message-State":"AOJu0YxJvuZquxJ6FKFhb9IlPKyPVnzs2OmRQjJJnfxMMmk4efyt0qWN\n\tMP66NmwJ5U60OY+/pqFNBoyDysbP/Mq71WlB42ihPx7Ip/sBb79kxmWZf+9oX5uOAKgx4mej+XJ\n\tcWvJB","X-Gm-Gg":"ASbGncvUZcFiKqysy86qrKeVj3JNpZpyMUlZrn9GQT3WaEsD+HT5o332PMeEw5qTuYm\n\tVCotfmEjMRWKsuL8eIEa7WlR1wCwhaNC2Fmt6DSZwJD8wtUHv0Hyg0yG0YVZIHH755YUpw1pRwx\n\t/WJMEB0QCocWfAnpbl2+SS+4F1daqLF5bi0Etfll9MFlZaBfwyJ2HOCuF6HplZ8K4WIiE2jvZgG\n\tcA6t0b8mj2KPIyVmfDx1aCYL2ejKjrAxHgYaxRr+trA1uTjfzG8VtL+GAd7gu6WWbWRoI2/qEGw\n\tnsSslQcAFAN9KHAscpiSgwVq/q0IQDUwFRYCKh113T1Xfoss3fMOWh/9xnoTO9KfVFsCI4Qz8Bb\n\t+ta3TN+lITCOnGkxJFE/jTbi8a5CPHyXiZFIjaexpzxtxnWuKPIe4WgrAa1R/iuMULmbbtOHzu6\n\t7Y5flyQihe05KPOODTZwCk7HBtzUERmM/89Gv2fCy3X71gv8k9ssAId/HVLUko/of1nnzQAagzK\n\teCRm0uNt1RoVBO7OWT1+mZRK1e4mQ==","X-Google-Smtp-Source":"AGHT+IEBIlga7joPo2AUOCNwHCI7eLccqhlH1/s15HBJdBzNhBAkHnrsTbLp9eFhTv4xbM08/GHnZA==","X-Received":"by 2002:a05:600c:3487:b0:46f:b32e:5094 with SMTP id\n\t5b1f17b1804b1-47a83811c87mr28065745e9.32.1765384867858; \n\tWed, 10 Dec 2025 08:41:07 -0800 (PST)","From":"David Plowman ","To":"libcamera-devel@lists.libcamera.org","Cc":"David Plowman ","Subject":"[PATCH 11/11] pipline: rpi: Support memory cameras processing\n\trequests","Date":"Wed, 10 Dec 2025 16:15:26 +0000","Message-ID":"<20251210164055.17856-12-david.plowman@raspberrypi.com>","X-Mailer":"git-send-email 2.47.3","In-Reply-To":"<20251210164055.17856-1-david.plowman@raspberrypi.com>","References":"<20251210164055.17856-1-david.plowman@raspberrypi.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"This is the final commit in the set to support memory cameras (Bayer\nreprocessing).\n\nThere are some small fix-ups to allow the camera system to start\nwithout the CFE being involved, and we add runMemoryCamera() which\npushes a buffer from the request into the Back End ISP.\n\nSigned-off-by: David Plowman \n---\n .../pipeline/rpi/common/pipeline_base.cpp | 23 ++++---\n src/libcamera/pipeline/rpi/pisp/pisp.cpp | 63 +++++++++++++++++--\n 2 files changed, 71 insertions(+), 15 deletions(-)","diff":"diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp\nindex 2f02ec4c..48dbfaa5 100644\n--- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp\n+++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp\n@@ -772,15 +772,19 @@ int PipelineHandlerBase::start(Camera *camera, const ControlList *controls)\n \t/* A good moment to add an initial clock sample. */\n \tdata->wallClockRecovery_.addSample();\n \n-\t/*\n-\t * Reset the delayed controls with the gain and exposure values set by\n-\t * the IPA.\n-\t */\n-\tdata->delayedCtrls_->reset(0);\n-\tdata->state_ = CameraData::State::Idle;\n+\t/* A memory camera wouldn't have a front end device, so you would skip this. */\n+\tif (data->frontendDevice()) {\n+\t\t/*\n+\t\t * Reset the delayed controls with the gain and exposure values set by\n+\t\t * the IPA.\n+\t\t */\n+\t\tdata->delayedCtrls_->reset(0);\n+\n+\t\t/* Enable SOF event generation. */\n+\t\tdata->frontendDevice()->setFrameStartEnabled(true);\n+\t}\n \n-\t/* Enable SOF event generation. */\n-\tdata->frontendDevice()->setFrameStartEnabled(true);\n+\tdata->state_ = CameraData::State::Idle;\n \n \tdata->platformStart();\n \n@@ -807,7 +811,8 @@ void PipelineHandlerBase::stopDevice(Camera *camera)\n \t\tstream->dev()->streamOff();\n \n \t/* Disable SOF event generation. */\n-\tdata->frontendDevice()->setFrameStartEnabled(false);\n+\tif (data->frontendDevice())\n+\t\tdata->frontendDevice()->setFrameStartEnabled(false);\n \n \tdata->clearIncompleteRequests();\n \ndiff --git a/src/libcamera/pipeline/rpi/pisp/pisp.cpp b/src/libcamera/pipeline/rpi/pisp/pisp.cpp\nindex da3f5f40..ce350378 100644\n--- a/src/libcamera/pipeline/rpi/pisp/pisp.cpp\n+++ b/src/libcamera/pipeline/rpi/pisp/pisp.cpp\n@@ -780,6 +780,7 @@ private:\n \tvoid prepareCfe();\n \tvoid prepareBe(uint32_t bufferId, bool stitchSwapBuffers);\n \n+\tvoid runMemoryCamera();\n \tvoid tryRunPipeline() override;\n \n \tstruct CfeJob {\n@@ -1244,7 +1245,9 @@ PipelineHandlerPiSP::platformCreateCamera(std::unique_ptr &came\n \n \t/* Wire up all the IPA connections. */\n \tdata->ipa_->prepareIspComplete.connect(data, &PiSPCameraData::prepareIspComplete);\n-\tdata->ipa_->processStatsComplete.connect(data, &PiSPCameraData::processStatsComplete);\n+\t/* Connect IPA stats for hardware cameras; memory cams don't need it. */\n+\tif (cfe)\n+\t\tdata->ipa_->processStatsComplete.connect(data, &PiSPCameraData::processStatsComplete);\n \n \t/* Create and register the camera. */\n \tconst std::string &id = cfe ? data->sensor_->id() : \"memory\";\n@@ -1787,8 +1790,11 @@ void PiSPCameraData::platformStart()\n \n \tcfeJobQueue_ = {};\n \n-\tfor (unsigned int i = 0; i < config_.numCfeConfigQueue; i++)\n-\t\tprepareCfe();\n+\t/* A memory camera wouldn't be using the CFE. */\n+\tif (cfe_[Cfe::Output0].dev()) {\n+\t\tfor (unsigned int i = 0; i < config_.numCfeConfigQueue; i++)\n+\t\t\tprepareCfe();\n+\t}\n \n \t/* Clear the debug dump file history. */\n \tlast_dump_file_.clear();\n@@ -1892,8 +1898,12 @@ void PiSPCameraData::beInputDequeue(FrameBuffer *buffer)\n \t\t\t<< \", buffer id \" << cfe_[Cfe::Output0].getBufferId(buffer)\n \t\t\t<< \", timestamp: \" << buffer->metadata().timestamp;\n \n-\t/* The ISP input buffer gets re-queued into CFE. */\n-\thandleStreamBuffer(buffer, &cfe_[Cfe::Output0]);\n+\t/*\n+\t * The ISP input buffer gets re-queued into CFE (but not for memory cameras\n+\t * that aren't using the CFE).\n+\t */\n+\tif (cfe_[Cfe::Output0].dev())\n+\t\thandleStreamBuffer(buffer, &cfe_[Cfe::Output0]);\n \thandleState();\n }\n \n@@ -2363,7 +2373,23 @@ void PiSPCameraData::prepareCfe()\n \n void PiSPCameraData::prepareBe(uint32_t bufferId, bool stitchSwapBuffers)\n {\n-\tFrameBuffer *buffer = cfe_[Cfe::Output0].getBuffers().at(bufferId).buffer;\n+\tFrameBuffer *buffer;\n+\tif (!sensor_->isMemory())\n+\t\tbuffer = cfe_[Cfe::Output0].getBuffers().at(bufferId).buffer;\n+\telse {\n+\t\t/* Memory camera. Back end input buffer is in the request. */\n+\t\tRequest *request = requestQueue_.front();\n+\t\tconst libcamera::Stream *rawStream = &isp_[Isp::Input];\n+\t\tbuffer = request->findBuffer(rawStream);\n+\t\t/* If there's no buffer to process, there's no recovering from that. */\n+\t\tif (!buffer)\n+\t\t\tLOG(RPI, Fatal) << \"No input buffer supplied for re-processing\";\n+\t\t/*\n+\t\t * Can't see how an application would set bytesused if it were different\n+\t\t * from the allocated size (length), so assume that's the number we need.\n+\t\t */\n+\t\tbuffer->_d()->metadata().planes()[0].bytesused = buffer->planes()[0].length;\n+\t}\n \n \tLOG(RPI, Debug) << \"Input re-queue to ISP, buffer id \" << bufferId\n \t\t\t<< \", timestamp: \" << buffer->metadata().timestamp;\n@@ -2414,8 +2440,33 @@ void PiSPCameraData::prepareBe(uint32_t bufferId, bool stitchSwapBuffers)\n \tisp_[Isp::Config].queueBuffer(config.buffer);\n }\n \n+void PiSPCameraData::runMemoryCamera()\n+{\n+\tRequest *request = requestQueue_.front();\n+\n+\tapplyScalerCrop(request->controls());\n+\n+\trequest->metadata().clear();\n+\n+\tstate_ = State::Busy;\n+\n+\tipa::RPi::PrepareParams params;\n+\tparams.buffers.embedded = 0;\n+\tparams.ipaContext = requestQueue_.front()->sequence();\n+\tparams.requestControls = request->controls();\n+\n+\tLOG(RPI, Debug) << \"runMemoryCamera: call prepareIsp\";\n+\tipa_->prepareIsp(params);\n+}\n+\n void PiSPCameraData::tryRunPipeline()\n {\n+\tif (state_ == State::Idle && !requestQueue_.empty() && sensor_->isMemory()) {\n+\t\trunMemoryCamera();\n+\n+\t\treturn;\n+\t}\n+\n \t/* If any of our request or buffer queues are empty, we cannot proceed. */\n \tif (state_ != State::Idle || requestQueue_.empty() || !cfeJobComplete())\n \t\treturn;\n","prefixes":["11/11"]}