From patchwork Wed Aug 27 09:07:39 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 24249 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2D6F9C332E for ; Wed, 27 Aug 2025 09:08:13 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6210569309; Wed, 27 Aug 2025 11:08:12 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="g52tKWAV"; dkim-atps=neutral Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 65E2469303 for ; Wed, 27 Aug 2025 11:07:53 +0200 (CEST) Received: by mail-wm1-x335.google.com with SMTP id 5b1f17b1804b1-45a1b00f23eso38669175e9.0 for ; Wed, 27 Aug 2025 02:07:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1756285673; x=1756890473; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9qoMSMv7GS7D3NfmPMIenbPQUZV27K9S7JmNlJHn57g=; b=g52tKWAV21Fj400oQStiwLpZrfo31zogq7am39woGNkN8STtgBXAkh+NQZn2nCKJLx f408ckGkoKIfEKQttJ+q+UpibAjjCaBxR7xRucAdxOrq1WDWIXKz30f0nRw4GhI1BceT dO2hX2gzpCbUcZx3lfJBFGW3o53OxtgdSymSBh9LTdPSVswjezsTAT8hWcL915YRd1UT b62utf+a9lmQcn5WpW1/gJ9f/uM7+MZ0AGavys3NIuFYztullkfhrhzggy0LdC6huoFW VmIbvJj/vlH4A9VsRlRCgce2bVVZnU/6suU1R9rX+LP3YojaKtEwigAkkHpjJxCdZxVz dVmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1756285673; x=1756890473; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9qoMSMv7GS7D3NfmPMIenbPQUZV27K9S7JmNlJHn57g=; b=EtH0mnnszfyeqE//Bf9kURke4yd8FD7izCmAqIXCdv3JT09pU7b/51IPZ0CTz9MjLE STLwvtc6p44PykQI+TUjePklhhsVbdk9XTj9gXuWkx5mL/hj3MkBhhwBrTbjGfLd7KZp zuyhB4fFsbCs7/kqmZvolPMJqB4SG5SmXc9CyAOnhAPMJlvxvPnAVMv5uhZteo+nZ1j2 28/mPdLDHI9n1wZxG1bqTZYU73f9+ST21IJCjTAINgdc2SjoLHKL35sqezsAesUbsf+C tcSSuxiE/aJwdmhCNy7W0pCfDBB7KH7m1PWG9GHzOE/3YpqNQD9cXAzmePvXGJKduDOr SStg== X-Gm-Message-State: AOJu0YxV7ODGlmdZZNZzr/hJyatZFhWDnIveVBML9rsARM7DMIDj1ttF /UD5DTbfxbn2eaVbpAnZ74civoaUNssS4qmC/4BXu1XsO15QTCi2vSNhynxpxcmUkVRg42v8nF3 eYRc+ X-Gm-Gg: ASbGncu+a2ymcCkro56Pdkn8Ag5aELV+hq4on1gMI7WmQOelpfPHDGKb0NhhwwrCU65 /gQcubdNr1GbJSHiCefOO+F3akD6+ifTc7AoQNFlG6SwQvOZJ7Z4qz3YL36A8r/01w07dKlk79T xN69ANTaJUKbvp8QglDIZVbAHKMZ+PP9/Ya29uIhYeHJ/mvFcRIIwIHKQ43KsPAzQVvuxWyI3k3 lJJUr9q1dntpDZKmZ0oKEn9fu38XmweEaMF3sxpXZyv4teb8xvPUUg73Rea3gAlt/1KetU0Nslq LlmUdU/rj9OmVsRZQGiN8D78Tx9zUASnoxpBq4a9U5ZW29QqtjpCu9jN7/eptYpjH8K5mFte0Ia z9rCVipZXd+B73hYHEmm/I3iQcY41+hxeRkuXmE4PcnBe4cG9F2+CSHa0dSlYTJiiLao46Ovo+r Aj03rGQ6rvBfKqp+v4nDAlEbjhFUpg6A8OM5T9Nr8pkeZ7z6//ow== X-Google-Smtp-Source: AGHT+IFsOsFg4QSpn/ny3ZFkYfdpQTWfNuDMnx5KiLyPLNutkN645L9sQgGUcUKtgPIPqG5zucklyw== X-Received: by 2002:a05:600c:b95:b0:459:d645:bff7 with SMTP id 5b1f17b1804b1-45b5179cf7dmr161674605e9.12.1756285672467; Wed, 27 Aug 2025 02:07:52 -0700 (PDT) Received: from raspberrypi.pitowers.org ([2a00:1098:3142:1f:ffc9:aff6:7f7f:893b]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-45b6f30fe02sm21498675e9.18.2025.08.27.02.07.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Aug 2025 02:07:51 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman Subject: [RFC PATCH 12/12] pipline: rpi: Support memory cameras processing requests Date: Wed, 27 Aug 2025 10:07:39 +0100 Message-Id: <20250827090739.86955-13-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250827090739.86955-1-david.plowman@raspberrypi.com> References: <20250827090739.86955-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This is the final commit in the set to support memory cameras (Bayer reprocessing). There are some small fix-ups to allow the camera system to start without the CFE being involved, and we add runMemoryCamera() which pushs a buffer from the request into the Back End ISP. Signed-off-by: David Plowman --- .../pipeline/rpi/common/pipeline_base.cpp | 23 ++++---- src/libcamera/pipeline/rpi/pisp/pisp.cpp | 53 ++++++++++++++++--- 2 files changed, 61 insertions(+), 15 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 31bacc7c..8ca1e41f 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -727,15 +727,19 @@ int PipelineHandlerBase::start(Camera *camera, const ControlList *controls) /* A good moment to add an initial clock sample. */ data->wallClockRecovery_.addSample(); - /* - * Reset the delayed controls with the gain and exposure values set by - * the IPA. - */ - data->delayedCtrls_->reset(0); - data->state_ = CameraData::State::Idle; + /* A memory camera wouldn't have a front end device, so you would skip this. */ + if (data->frontendDevice()) { + /* + * Reset the delayed controls with the gain and exposure values set by + * the IPA. + */ + data->delayedCtrls_->reset(0); + + /* Enable SOF event generation. */ + data->frontendDevice()->setFrameStartEnabled(true); + } - /* Enable SOF event generation. */ - data->frontendDevice()->setFrameStartEnabled(true); + data->state_ = CameraData::State::Idle; data->platformStart(); @@ -762,7 +766,8 @@ void PipelineHandlerBase::stopDevice(Camera *camera) stream->dev()->streamOff(); /* Disable SOF event generation. */ - data->frontendDevice()->setFrameStartEnabled(false); + if (data->frontendDevice()) + data->frontendDevice()->setFrameStartEnabled(false); data->clearIncompleteRequests(); diff --git a/src/libcamera/pipeline/rpi/pisp/pisp.cpp b/src/libcamera/pipeline/rpi/pisp/pisp.cpp index bb22e6d2..a637c6be 100644 --- a/src/libcamera/pipeline/rpi/pisp/pisp.cpp +++ b/src/libcamera/pipeline/rpi/pisp/pisp.cpp @@ -780,6 +780,7 @@ private: void prepareCfe(); void prepareBe(uint32_t bufferId, bool stitchSwapBuffers); + void runMemoryCamera(); void tryRunPipeline() override; struct CfeJob { @@ -1243,7 +1244,7 @@ PipelineHandlerPiSP::platformCreateCamera(std::unique_ptr &came /* Wire up all the IPA connections. */ data->ipa_->prepareIspComplete.connect(data, &PiSPCameraData::prepareIspComplete); - data->ipa_->processStatsComplete.connect(data, &PiSPCameraData::processStatsComplete); + /* We don't want the processStatsComplete callback, though, for memory cams. */ /* Create and register the camera. */ const std::string &id = cfe ? data->sensor_->id() : "memory"; @@ -1785,8 +1786,11 @@ void PiSPCameraData::platformStart() cfeJobQueue_ = {}; - for (unsigned int i = 0; i < config_.numCfeConfigQueue; i++) - prepareCfe(); + /* A memory camera wouldn't be using the CFE. */ + if (cfe_[Cfe::Output0].dev()) { + for (unsigned int i = 0; i < config_.numCfeConfigQueue; i++) + prepareCfe(); + } /* Clear the debug dump file history. */ last_dump_file_.clear(); @@ -1890,8 +1894,12 @@ void PiSPCameraData::beInputDequeue(FrameBuffer *buffer) << ", buffer id " << cfe_[Cfe::Output0].getBufferId(buffer) << ", timestamp: " << buffer->metadata().timestamp; - /* The ISP input buffer gets re-queued into CFE. */ - handleStreamBuffer(buffer, &cfe_[Cfe::Output0]); + /* + * The ISP input buffer gets re-queued into CFE (but not for memory cameras + * that aren't using the CFE). + */ + if (cfe_[Cfe::Output0].dev()) + handleStreamBuffer(buffer, &cfe_[Cfe::Output0]); handleState(); } @@ -2361,7 +2369,15 @@ void PiSPCameraData::prepareCfe() void PiSPCameraData::prepareBe(uint32_t bufferId, bool stitchSwapBuffers) { - FrameBuffer *buffer = cfe_[Cfe::Output0].getBuffers().at(bufferId).buffer; + FrameBuffer *buffer; + if (!sensor_->isMemory()) + buffer = cfe_[Cfe::Output0].getBuffers().at(bufferId).buffer; + else { + /* Memory camera. Back end input buffer is in the request. */ + Request *request = requestQueue_.front(); + const libcamera::Stream *rawStream = &isp_[Isp::Input]; + buffer = request->findBuffer(rawStream); + } LOG(RPI, Debug) << "Input re-queue to ISP, buffer id " << bufferId << ", timestamp: " << buffer->metadata().timestamp; @@ -2412,8 +2428,33 @@ void PiSPCameraData::prepareBe(uint32_t bufferId, bool stitchSwapBuffers) isp_[Isp::Config].queueBuffer(config.buffer); } +void PiSPCameraData::runMemoryCamera() +{ + Request *request = requestQueue_.front(); + + applyScalerCrop(request->controls()); + + request->metadata().clear(); + + state_ = State::Busy; + + ipa::RPi::PrepareParams params; + params.buffers.embedded = 0; + params.ipaContext = requestQueue_.front()->sequence(); + params.requestControls = request->controls(); + + LOG(RPI, Debug) << "runMemoryCamera: call prepareIsp"; + ipa_->prepareIsp(params); +} + void PiSPCameraData::tryRunPipeline() { + if (state_ == State::Idle && !requestQueue_.empty() && sensor_->isMemory()) { + runMemoryCamera(); + + return; + } + /* If any of our request or buffer queues are empty, we cannot proceed. */ if (state_ != State::Idle || requestQueue_.empty() || !cfeJobComplete()) return;