From patchwork Wed Dec 10 16:15:26 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 25476 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 96AEDC326C for ; Wed, 10 Dec 2025 16:41:15 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 45967614EE; Wed, 10 Dec 2025 17:41:15 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="QICpIUTS"; dkim-atps=neutral Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B9A8D614B2 for ; Wed, 10 Dec 2025 17:41:08 +0100 (CET) Received: by mail-wm1-x329.google.com with SMTP id 5b1f17b1804b1-47796a837c7so69085e9.0 for ; Wed, 10 Dec 2025 08:41:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1765384868; x=1765989668; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Z2u6QWJXLLkERKnj/8SmMWdGzZUbKd7GPA9TOeKwsPo=; b=QICpIUTSVpG58ybyWpey7plogzGABP9dgjoL2sKfBwIYOeCvI2+Y8UyFfYVl6DSI6P It9YxRHGEmau1oUNkJa6c4Rb8nbEE8LHSExSDdib5r+gJdLybzT+ALNG9lnY49F+4Ici KDxHT7UlcVYHw8dtvE8U/go9fiyyeALdiMYuGoQ9jQkxyLapMXHgbQ1xakukxnu5vum8 J5T6ByRKnMN5IEkZbWNqPMvyKf9PsNlSfLQAi2vzBfvsIx2RJPMGtZJnGSGjXkPgUhD8 fjf3oGWdDL1TaaYcWTNwU0tK0g+bvnJRmgtKw0AoBFzBejoTB2V+gtpJKWkz5KACll8B Z6JQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1765384868; x=1765989668; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=Z2u6QWJXLLkERKnj/8SmMWdGzZUbKd7GPA9TOeKwsPo=; b=BFxkte2pamTLPIQe4/vAUyyLAgvmYz56KqMFDokKxQ/AguD0mQv5hEdQmXCRvC81oY UyL2wbw8j/oLsH+RDohIMlioqBjDVNvFeTd+KEzUagF+k4LinCSH0IiHg5lsIQt1hqoU VdTOGqR5dLdpi2mVNEKo7DDcCw8Kim9kSrYoR8m8je3M5CM8Ch3KZ0T1VPDHs1HOo2Iv cjar1FH+DAHxl2oggvxyorDc1rCf7z/xe7YG6WBGpfUpI1ZLAaASu+82FtO1holB3mqk 09VFcDlh5n6EoLzeLJAKzVUnDT4qomMh3UodiiNwXLbuhhO3FTf4nPcVoONbStZl83I5 p2+A== X-Gm-Message-State: AOJu0YxJvuZquxJ6FKFhb9IlPKyPVnzs2OmRQjJJnfxMMmk4efyt0qWN MP66NmwJ5U60OY+/pqFNBoyDysbP/Mq71WlB42ihPx7Ip/sBb79kxmWZf+9oX5uOAKgx4mej+XJ cWvJB X-Gm-Gg: ASbGncvUZcFiKqysy86qrKeVj3JNpZpyMUlZrn9GQT3WaEsD+HT5o332PMeEw5qTuYm VCotfmEjMRWKsuL8eIEa7WlR1wCwhaNC2Fmt6DSZwJD8wtUHv0Hyg0yG0YVZIHH755YUpw1pRwx /WJMEB0QCocWfAnpbl2+SS+4F1daqLF5bi0Etfll9MFlZaBfwyJ2HOCuF6HplZ8K4WIiE2jvZgG cA6t0b8mj2KPIyVmfDx1aCYL2ejKjrAxHgYaxRr+trA1uTjfzG8VtL+GAd7gu6WWbWRoI2/qEGw nsSslQcAFAN9KHAscpiSgwVq/q0IQDUwFRYCKh113T1Xfoss3fMOWh/9xnoTO9KfVFsCI4Qz8Bb +ta3TN+lITCOnGkxJFE/jTbi8a5CPHyXiZFIjaexpzxtxnWuKPIe4WgrAa1R/iuMULmbbtOHzu6 7Y5flyQihe05KPOODTZwCk7HBtzUERmM/89Gv2fCy3X71gv8k9ssAId/HVLUko/of1nnzQAagzK eCRm0uNt1RoVBO7OWT1+mZRK1e4mQ== X-Google-Smtp-Source: AGHT+IEBIlga7joPo2AUOCNwHCI7eLccqhlH1/s15HBJdBzNhBAkHnrsTbLp9eFhTv4xbM08/GHnZA== X-Received: by 2002:a05:600c:3487:b0:46f:b32e:5094 with SMTP id 5b1f17b1804b1-47a83811c87mr28065745e9.32.1765384867858; Wed, 10 Dec 2025 08:41:07 -0800 (PST) Received: from davidp-pi5.pitowers.org ([2a00:1098:3142:1f:88ea:c658:5b20:5e46]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-47a88371e13sm1270415e9.11.2025.12.10.08.41.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Dec 2025 08:41:07 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman Subject: [PATCH 11/11] pipline: rpi: Support memory cameras processing requests Date: Wed, 10 Dec 2025 16:15:26 +0000 Message-ID: <20251210164055.17856-12-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20251210164055.17856-1-david.plowman@raspberrypi.com> References: <20251210164055.17856-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This is the final commit in the set to support memory cameras (Bayer reprocessing). There are some small fix-ups to allow the camera system to start without the CFE being involved, and we add runMemoryCamera() which pushes a buffer from the request into the Back End ISP. Signed-off-by: David Plowman --- .../pipeline/rpi/common/pipeline_base.cpp | 23 ++++--- src/libcamera/pipeline/rpi/pisp/pisp.cpp | 63 +++++++++++++++++-- 2 files changed, 71 insertions(+), 15 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 2f02ec4c..48dbfaa5 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -772,15 +772,19 @@ int PipelineHandlerBase::start(Camera *camera, const ControlList *controls) /* A good moment to add an initial clock sample. */ data->wallClockRecovery_.addSample(); - /* - * Reset the delayed controls with the gain and exposure values set by - * the IPA. - */ - data->delayedCtrls_->reset(0); - data->state_ = CameraData::State::Idle; + /* A memory camera wouldn't have a front end device, so you would skip this. */ + if (data->frontendDevice()) { + /* + * Reset the delayed controls with the gain and exposure values set by + * the IPA. + */ + data->delayedCtrls_->reset(0); + + /* Enable SOF event generation. */ + data->frontendDevice()->setFrameStartEnabled(true); + } - /* Enable SOF event generation. */ - data->frontendDevice()->setFrameStartEnabled(true); + data->state_ = CameraData::State::Idle; data->platformStart(); @@ -807,7 +811,8 @@ void PipelineHandlerBase::stopDevice(Camera *camera) stream->dev()->streamOff(); /* Disable SOF event generation. */ - data->frontendDevice()->setFrameStartEnabled(false); + if (data->frontendDevice()) + data->frontendDevice()->setFrameStartEnabled(false); data->clearIncompleteRequests(); diff --git a/src/libcamera/pipeline/rpi/pisp/pisp.cpp b/src/libcamera/pipeline/rpi/pisp/pisp.cpp index da3f5f40..ce350378 100644 --- a/src/libcamera/pipeline/rpi/pisp/pisp.cpp +++ b/src/libcamera/pipeline/rpi/pisp/pisp.cpp @@ -780,6 +780,7 @@ private: void prepareCfe(); void prepareBe(uint32_t bufferId, bool stitchSwapBuffers); + void runMemoryCamera(); void tryRunPipeline() override; struct CfeJob { @@ -1244,7 +1245,9 @@ PipelineHandlerPiSP::platformCreateCamera(std::unique_ptr &came /* Wire up all the IPA connections. */ data->ipa_->prepareIspComplete.connect(data, &PiSPCameraData::prepareIspComplete); - data->ipa_->processStatsComplete.connect(data, &PiSPCameraData::processStatsComplete); + /* Connect IPA stats for hardware cameras; memory cams don't need it. */ + if (cfe) + data->ipa_->processStatsComplete.connect(data, &PiSPCameraData::processStatsComplete); /* Create and register the camera. */ const std::string &id = cfe ? data->sensor_->id() : "memory"; @@ -1787,8 +1790,11 @@ void PiSPCameraData::platformStart() cfeJobQueue_ = {}; - for (unsigned int i = 0; i < config_.numCfeConfigQueue; i++) - prepareCfe(); + /* A memory camera wouldn't be using the CFE. */ + if (cfe_[Cfe::Output0].dev()) { + for (unsigned int i = 0; i < config_.numCfeConfigQueue; i++) + prepareCfe(); + } /* Clear the debug dump file history. */ last_dump_file_.clear(); @@ -1892,8 +1898,12 @@ void PiSPCameraData::beInputDequeue(FrameBuffer *buffer) << ", buffer id " << cfe_[Cfe::Output0].getBufferId(buffer) << ", timestamp: " << buffer->metadata().timestamp; - /* The ISP input buffer gets re-queued into CFE. */ - handleStreamBuffer(buffer, &cfe_[Cfe::Output0]); + /* + * The ISP input buffer gets re-queued into CFE (but not for memory cameras + * that aren't using the CFE). + */ + if (cfe_[Cfe::Output0].dev()) + handleStreamBuffer(buffer, &cfe_[Cfe::Output0]); handleState(); } @@ -2363,7 +2373,23 @@ void PiSPCameraData::prepareCfe() void PiSPCameraData::prepareBe(uint32_t bufferId, bool stitchSwapBuffers) { - FrameBuffer *buffer = cfe_[Cfe::Output0].getBuffers().at(bufferId).buffer; + FrameBuffer *buffer; + if (!sensor_->isMemory()) + buffer = cfe_[Cfe::Output0].getBuffers().at(bufferId).buffer; + else { + /* Memory camera. Back end input buffer is in the request. */ + Request *request = requestQueue_.front(); + const libcamera::Stream *rawStream = &isp_[Isp::Input]; + buffer = request->findBuffer(rawStream); + /* If there's no buffer to process, there's no recovering from that. */ + if (!buffer) + LOG(RPI, Fatal) << "No input buffer supplied for re-processing"; + /* + * Can't see how an application would set bytesused if it were different + * from the allocated size (length), so assume that's the number we need. + */ + buffer->_d()->metadata().planes()[0].bytesused = buffer->planes()[0].length; + } LOG(RPI, Debug) << "Input re-queue to ISP, buffer id " << bufferId << ", timestamp: " << buffer->metadata().timestamp; @@ -2414,8 +2440,33 @@ void PiSPCameraData::prepareBe(uint32_t bufferId, bool stitchSwapBuffers) isp_[Isp::Config].queueBuffer(config.buffer); } +void PiSPCameraData::runMemoryCamera() +{ + Request *request = requestQueue_.front(); + + applyScalerCrop(request->controls()); + + request->metadata().clear(); + + state_ = State::Busy; + + ipa::RPi::PrepareParams params; + params.buffers.embedded = 0; + params.ipaContext = requestQueue_.front()->sequence(); + params.requestControls = request->controls(); + + LOG(RPI, Debug) << "runMemoryCamera: call prepareIsp"; + ipa_->prepareIsp(params); +} + void PiSPCameraData::tryRunPipeline() { + if (state_ == State::Idle && !requestQueue_.empty() && sensor_->isMemory()) { + runMemoryCamera(); + + return; + } + /* If any of our request or buffer queues are empty, we cannot proceed. */ if (state_ != State::Idle || requestQueue_.empty() || !cfeJobComplete()) return;