From patchwork Wed Dec 10 16:15:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 25471 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9AEBEC326C for ; Wed, 10 Dec 2025 16:41:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 65EEF614BA; Wed, 10 Dec 2025 17:41:10 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="d+RxSkA+"; dkim-atps=neutral Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8CCB8614AB for ; Wed, 10 Dec 2025 17:41:04 +0100 (CET) Received: by mail-wm1-x334.google.com with SMTP id 5b1f17b1804b1-477632d9326so47941455e9.1 for ; Wed, 10 Dec 2025 08:41:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1765384864; x=1765989664; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xHq6lLERxDdLmM67XaICA48L1mdiYZYJFWVPoXLhIsw=; b=d+RxSkA+wLq68Uc2Rq3rqDU3u28gw+ehaGXVnjpTbmnYDBvCYqME96d8D4wHG2aC7F Gu2MjbHblpYEi7HVzfdtqIlFUz+Xv7kyGT4VdxU9lOdV397DXvAH0cUwDSHKQCvLlUR/ KtAZJ/DpiOHXZyyc3sf489ZWiGMXZT+asWL8hl78RjcsCFguRyD47sfy0tk9P4LZFA0w Wy8T4fBoFUJKA4QGSOCnihpj+Q1kdIEPK3Y5CI0yN9yXObmo2+O2HKuCOwpWF32qljLQ z2I5SvXsxNRQTCbk/m3LqAEQXZJ3rrSfLPsyqqdf9wmb3uAlLNLDMdyrg4VE3DMCDLlj rkog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1765384864; x=1765989664; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=xHq6lLERxDdLmM67XaICA48L1mdiYZYJFWVPoXLhIsw=; b=M08qjaDDg3ThUhmoehEoD6XXQLhnw58oknZ0491QtgdvE2ZkW8v9Cxb+GBeXI7OrX7 g0N7QAEwo4NqiX1S5DLwugpne6LUjL6XU3YMzAxznZoI6xPiaxcugYLX/Rn4c1an7cbE q/kzn9StaL98Ztme8Zfs7Zc+H6b6BEsXerw5jRDXSznyXD4SSVNb4c4qjx6vSGhrMiQ7 cgdlVziwSbJHw1NlFk1AT0TEhclnoRVXaWGVn3rnkyG2WjPnYANKFe1AOQI+VQ5aj+m7 ceuZ2Jea+tnhBOMXPjovhh5v5l7AksAgaifpaMmBkXtZLB3OgBQs6yeeoE50QnvWRQyD 9ZUQ== X-Gm-Message-State: AOJu0YzqbymgBslI06S6y9u/NRCksqZh3f7UtDW4kvEBS9EDHgKmHyQy ZdRpXfA8Yz9v6BG/mkj64SBHgnDAUMdhsdi4Gw2tqnyeabNd0sC2GYFuDdLxINcB95uCLaepxMJ UTztp X-Gm-Gg: ASbGncubbbXkzF4L5MojJ/+NQUtRFdJGbaj3D4sfSQQvX9YeJtI+2bkw9uUV3toy+iZ LfswFoyA4lLr6hq1e9HOfDSPjRLHuw9Wiq4tfwPTp8mBRIFw1pJ+s+Duw/9yzpBIi+JVrgYHEq4 5kIByBL1j63N4m+v4042AvRL/xQJsNxa5Z8Jq6EdtHHQMbDU1pk3CDLafzte1MSDOZIPBO4Qiv0 ojT/O/FqS2TFjLWOTKp+ihpUnLkc+MClm8YYdMgj4NGJz6kVLhGjBKlpIEM2VrO4umL40yHDVgP Bc2Z658U/CfQY7pFumKa6N2SuBEcRFCceqn/RfqgvATC1rRV3qQa9xUe0Vp/HmfZJEi0hZLq/pR P36gpnf6gaMkMyWlY1aKus9E+2DOpiIwSmmD2mjgMn+upHA+8jtiymhRtrrS4a2gVj15v2eJsxs V5vf3lUaVLa8COUMNiO7gfk5c6Fp+oRtX0amjFkCvNRz8AGNmAJeOBGeGRLKNRHuO3VB6neXi2t Ovl+fYLgkwGwze2+4Mxg5FA6YRReQ== X-Google-Smtp-Source: AGHT+IGWCUCABkuOVZ1Ydh+rnYbbB5LdkrfVDglFf9NQJ1dKY6iVjW18ECqHtnKtVWgR/E66qfLXhw== X-Received: by 2002:a05:600c:820c:b0:477:a54a:acba with SMTP id 5b1f17b1804b1-47a8384c38bmr34671815e9.17.1765384863555; Wed, 10 Dec 2025 08:41:03 -0800 (PST) Received: from davidp-pi5.pitowers.org ([2a00:1098:3142:1f:88ea:c658:5b20:5e46]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-47a88371e13sm1270415e9.11.2025.12.10.08.41.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Dec 2025 08:41:03 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman Subject: [PATCH 06/11] libcamera: pipeline: rpi: Allow creation of the first "memory" camera Date: Wed, 10 Dec 2025 16:15:21 +0000 Message-ID: <20251210164055.17856-7-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20251210164055.17856-1-david.plowman@raspberrypi.com> References: <20251210164055.17856-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The Raspberry Pi PiSP pipeline handler is updated to indicate that it supports the creation of "memory" cameras, and extended actually to do so. Memory cameras are created on demand when applications request them. Subsequent commits will support the correct handling of camera configurations with "input" streams, that will require the use of these memory cameras. Signed-off-by: David Plowman --- .../pipeline/rpi/common/pipeline_base.cpp | 19 ++ .../pipeline/rpi/common/pipeline_base.h | 1 + src/libcamera/pipeline/rpi/pisp/pisp.cpp | 220 +++++++++++++----- 3 files changed, 188 insertions(+), 52 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 00b088fc..6878bdee 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -1217,6 +1217,25 @@ int CameraData::loadIPA(ipa::RPi::InitResult *result) return ipa_->init(settings, params, result); } +int CameraData::loadNamedIPA(std::string const &tuningFile, ipa::RPi::InitResult *result) +{ + int ret; + + ipa_ = IPAManager::createIPA(pipe(), 1, 1); + + if (!ipa_) + return -ENOENT; + + IPASettings settings(tuningFile, "default"); + ipa::RPi::InitParams params; + + ret = platformInitIpa(params); + if (ret) + return ret; + + return ipa_->init(settings, params, result); +} + int CameraData::configureIPA(const CameraConfiguration *config, ipa::RPi::ConfigResult *result) { ipa::RPi::ConfigParams params; diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index 1f174473..9453ae7e 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -72,6 +72,7 @@ public: int loadPipelineConfiguration(); int loadIPA(ipa::RPi::InitResult *result); + int loadNamedIPA(std::string const &settings, ipa::RPi::InitResult *result); int configureIPA(const CameraConfiguration *config, ipa::RPi::ConfigResult *result); virtual int platformInitIpa(ipa::RPi::InitParams ¶ms) = 0; virtual int platformConfigureIpa(ipa::RPi::ConfigParams ¶ms) = 0; diff --git a/src/libcamera/pipeline/rpi/pisp/pisp.cpp b/src/libcamera/pipeline/rpi/pisp/pisp.cpp index c08210b4..726ab063 100644 --- a/src/libcamera/pipeline/rpi/pisp/pisp.cpp +++ b/src/libcamera/pipeline/rpi/pisp/pisp.cpp @@ -815,6 +815,11 @@ public: bool match(DeviceEnumerator *enumerator) override; + bool supportsMemoryCamera() override; + + std::shared_ptr createMemoryCamera(DeviceEnumerator *enumerator, + std::string_view settings) override; + private: PiSPCameraData *cameraData(Camera *camera) { @@ -822,6 +827,8 @@ private: } int prepareBuffers(Camera *camera) override; + std::shared_ptr platformCreateCamera(std::unique_ptr &cameraData, + MediaDevice *cfe, MediaDevice *isp); int platformRegister(std::unique_ptr &cameraData, std::shared_ptr cfe, std::shared_ptr isp) override; @@ -889,10 +896,8 @@ bool PipelineHandlerPiSP::match(DeviceEnumerator *enumerator) PiSPCameraData *pisp = static_cast(cameraData.get()); - pisp->fe_ = SharedMemObject - ("pisp_frontend", true, pisp->pispVariant_); - pisp->be_ = SharedMemObject - ("pisp_backend", BackEnd::Config({}), pisp->pispVariant_); + pisp->fe_ = SharedMemObject("pisp_frontend", true, pisp->pispVariant_); + pisp->be_ = SharedMemObject("pisp_backend", BackEnd::Config({}), pisp->pispVariant_); if (!pisp->fe_.fd().isValid() || !pisp->be_.fd().isValid()) { LOG(RPI, Error) << "Failed to create ISP shared objects"; @@ -915,6 +920,84 @@ bool PipelineHandlerPiSP::match(DeviceEnumerator *enumerator) return false; } +bool PipelineHandlerPiSP::supportsMemoryCamera() +{ + return true; +} + +static bool get_variant(unsigned int be_version, const libpisp::PiSPVariant **variant) +{ + for (const auto &hw : libpisp::get_variants()) { + if (hw.BackEndVersion() == be_version) { + *variant = &hw; + return true; + } + } + + return false; +} + +std::shared_ptr +PipelineHandlerPiSP::createMemoryCamera(DeviceEnumerator *enumerator, + std::string_view settings) +{ + DeviceMatch isp("pispbe"); + isp.add("pispbe-input"); + isp.add("pispbe-config"); + isp.add("pispbe-output0"); + isp.add("pispbe-output1"); + isp.add("pispbe-tdn_output"); + isp.add("pispbe-tdn_input"); + isp.add("pispbe-stitch_output"); + isp.add("pispbe-stitch_input"); + std::shared_ptr ispDevice = acquireMediaDevice(enumerator, isp); + + if (!ispDevice) { + LOG(RPI, Warning) << "Unable to acquire ISP instance"; + return nullptr; + } + + const libpisp::PiSPVariant *variant = nullptr; + if (!get_variant(ispDevice->hwRevision(), &variant)) { + LOG(RPI, Warning) << "Failed to find variant"; + return nullptr; + } + + std::unique_ptr cameraData = + std::make_unique(this, *variant); + PiSPCameraData *pisp = + static_cast(cameraData.get()); + + /* Only need the back end, but creating the front end makes things easier */ + pisp->fe_ = SharedMemObject("pisp_frontend", true, pisp->pispVariant_); + pisp->be_ = SharedMemObject("pisp_backend", BackEnd::Config({}), pisp->pispVariant_); + + if (!pisp->fe_.fd().isValid() || !pisp->be_.fd().isValid()) { + LOG(RPI, Error) << "Failed to create ISP shared objects"; + return nullptr; + } + + /* + * Next, we want to do PipelineHandlerBase::registerCamera, including: + * loadIPA + * platformReigster + * loadPipelineConfiguration + */ + + /* loadIPA is a bit different for us as we have a filename */ + ipa::RPi::InitResult result; + std::string configFile = std::string(settings); + if (pisp->loadNamedIPA(configFile, &result)) { + LOG(RPI, Error) << "Failed to load a suitable IPA library"; + return nullptr; + } + LOG(RPI, Info) << "Loaded IPA library " << configFile << " for memory camera"; + + std::shared_ptr camera = platformCreateCamera(cameraData, nullptr, ispDevice.get()); + + return camera; +} + int PipelineHandlerPiSP::prepareBuffers(Camera *camera) { PiSPCameraData *data = cameraData(camera); @@ -1022,17 +1105,51 @@ int PipelineHandlerPiSP::prepareBuffers(Camera *camera) return 0; } -int PipelineHandlerPiSP::platformRegister(std::unique_ptr &cameraData, - std::shared_ptr cfe, - std::shared_ptr isp) +static int createCameraCfe(PiSPCameraData *data, MediaDevice *cfe, std::set &streams) { - PiSPCameraData *data = static_cast(cameraData.get()); - int ret; - MediaEntity *cfeImage = cfe->getEntityByName("rp1-cfe-fe_image0"); MediaEntity *cfeEmbedded = cfe->getEntityByName("rp1-cfe-embedded"); MediaEntity *cfeStats = cfe->getEntityByName("rp1-cfe-fe_stats"); MediaEntity *cfeConfig = cfe->getEntityByName("rp1-cfe-fe_config"); + + /* Locate and open the cfe video streams. */ + data->cfe_[Cfe::Output0] = RPi::Stream("CFE Image", cfeImage, StreamFlag::RequiresMmap); + data->cfe_[Cfe::Embedded] = RPi::Stream("CFE Embedded", cfeEmbedded); + data->cfe_[Cfe::Stats] = RPi::Stream("CFE Stats", cfeStats); + data->cfe_[Cfe::Config] = RPi::Stream("CFE Config", cfeConfig, + StreamFlag::Recurrent | StreamFlag::RequiresMmap); + + /* Wire up all the buffer connections. */ + data->cfe_[Cfe::Output0].dev()->bufferReady.connect(data, &PiSPCameraData::cfeBufferDequeue); + data->cfe_[Cfe::Stats].dev()->bufferReady.connect(data, &PiSPCameraData::cfeBufferDequeue); + data->cfe_[Cfe::Config].dev()->bufferReady.connect(data, &PiSPCameraData::cfeBufferDequeue); + data->cfe_[Cfe::Embedded].dev()->bufferReady.connect(data, &PiSPCameraData::cfeBufferDequeue); + + data->csi2Subdev_ = std::make_unique(cfe->getEntityByName("csi2")); + data->feSubdev_ = std::make_unique(cfe->getEntityByName("pisp-fe")); + data->csi2Subdev_->open(); + data->feSubdev_->open(); + + /* + * The below grouping is just for convenience so that we can easily + * iterate over all streams in one go. + */ + data->streams_.push_back(&data->cfe_[Cfe::Output0]); + data->streams_.push_back(&data->cfe_[Cfe::Config]); + data->streams_.push_back(&data->cfe_[Cfe::Stats]); + if (data->sensorMetadata_) + data->streams_.push_back(&data->cfe_[Cfe::Embedded]); + + data->ipa_->setCameraTimeout.connect(data, &PiSPCameraData::setCameraTimeout); + + /* Applications may ask for this stream. */ + streams.insert(&data->cfe_[Cfe::Output0]); + + return 0; +} + +static int createCameraBe(PiSPCameraData *data, MediaDevice *isp, std::set &streams) +{ MediaEntity *ispInput = isp->getEntityByName("pispbe-input"); MediaEntity *IpaPrepare = isp->getEntityByName("pispbe-config"); MediaEntity *ispOutput0 = isp->getEntityByName("pispbe-output0"); @@ -1042,13 +1159,6 @@ int PipelineHandlerPiSP::platformRegister(std::unique_ptr &came MediaEntity *ispStitchOutput = isp->getEntityByName("pispbe-stitch_output"); MediaEntity *ispStitchInput = isp->getEntityByName("pispbe-stitch_input"); - /* Locate and open the cfe video streams. */ - data->cfe_[Cfe::Output0] = RPi::Stream("CFE Image", cfeImage, StreamFlag::RequiresMmap); - data->cfe_[Cfe::Embedded] = RPi::Stream("CFE Embedded", cfeEmbedded); - data->cfe_[Cfe::Stats] = RPi::Stream("CFE Stats", cfeStats); - data->cfe_[Cfe::Config] = RPi::Stream("CFE Config", cfeConfig, - StreamFlag::Recurrent | StreamFlag::RequiresMmap); - /* Tag the ISP input stream as an import stream. */ data->isp_[Isp::Input] = RPi::Stream("ISP Input", ispInput, StreamFlag::ImportOnly); @@ -1071,34 +1181,15 @@ int PipelineHandlerPiSP::platformRegister(std::unique_ptr &came StreamFlag::Recurrent); /* Wire up all the buffer connections. */ - data->cfe_[Cfe::Output0].dev()->bufferReady.connect(data, &PiSPCameraData::cfeBufferDequeue); - data->cfe_[Cfe::Stats].dev()->bufferReady.connect(data, &PiSPCameraData::cfeBufferDequeue); - data->cfe_[Cfe::Config].dev()->bufferReady.connect(data, &PiSPCameraData::cfeBufferDequeue); data->isp_[Isp::Input].dev()->bufferReady.connect(data, &PiSPCameraData::beInputDequeue); data->isp_[Isp::Config].dev()->bufferReady.connect(data, &PiSPCameraData::beOutputDequeue); data->isp_[Isp::Output0].dev()->bufferReady.connect(data, &PiSPCameraData::beOutputDequeue); data->isp_[Isp::Output1].dev()->bufferReady.connect(data, &PiSPCameraData::beOutputDequeue); - data->cfe_[Cfe::Embedded].dev()->bufferReady.connect(data, &PiSPCameraData::cfeBufferDequeue); - - data->csi2Subdev_ = std::make_unique(cfe->getEntityByName("csi2")); - data->feSubdev_ = std::make_unique(cfe->getEntityByName("pisp-fe")); - data->csi2Subdev_->open(); - data->feSubdev_->open(); /* - * Open all CFE and ISP streams. The exception is the embedded data - * stream, which only gets opened below if the IPA reports that the sensor - * supports embedded data. - * * The below grouping is just for convenience so that we can easily * iterate over all streams in one go. */ - data->streams_.push_back(&data->cfe_[Cfe::Output0]); - data->streams_.push_back(&data->cfe_[Cfe::Config]); - data->streams_.push_back(&data->cfe_[Cfe::Stats]); - if (data->sensorMetadata_) - data->streams_.push_back(&data->cfe_[Cfe::Embedded]); - data->streams_.push_back(&data->isp_[Isp::Input]); data->streams_.push_back(&data->isp_[Isp::Output0]); data->streams_.push_back(&data->isp_[Isp::Output1]); @@ -1108,38 +1199,63 @@ int PipelineHandlerPiSP::platformRegister(std::unique_ptr &came data->streams_.push_back(&data->isp_[Isp::StitchInput]); data->streams_.push_back(&data->isp_[Isp::StitchOutput]); + /* Applications may ask for these stream. */ + streams.insert(&data->isp_[Isp::Output0]); + streams.insert(&data->isp_[Isp::Output1]); + + return 0; +} + +std::shared_ptr +PipelineHandlerPiSP::platformCreateCamera(std::unique_ptr &cameraData, + MediaDevice *cfe, MediaDevice *isp) +{ + PiSPCameraData *data = static_cast(cameraData.get()); + int ret; + std::set streams; + + /* cfe is a null pointer when making a memory camera that doesn't use the front end. */ + if (cfe) { + ret = createCameraCfe(data, cfe, streams); + if (ret) + return nullptr; + } + + ret = createCameraBe(data, isp, streams); + if (ret) + return nullptr; + + /* All the streams must open successfully. */ for (auto stream : data->streams_) { ret = stream->dev()->open(); if (ret) - return ret; + return nullptr; } - /* Write up all the IPA connections. */ + /* Wire up all the IPA connections. */ data->ipa_->prepareIspComplete.connect(data, &PiSPCameraData::prepareIspComplete); data->ipa_->processStatsComplete.connect(data, &PiSPCameraData::processStatsComplete); - data->ipa_->setCameraTimeout.connect(data, &PiSPCameraData::setCameraTimeout); - - /* - * List the available streams an application may request. At present, we - * do not advertise CFE Embedded and ISP Statistics streams, as there - * is no mechanism for the application to request non-image buffer formats. - */ - std::set streams; - streams.insert(&data->cfe_[Cfe::Output0]); - streams.insert(&data->isp_[Isp::Output0]); - streams.insert(&data->isp_[Isp::Output1]); /* Create and register the camera. */ - const std::string &id = data->sensor_->id(); + const std::string &id = cfe ? data->sensor_->id() : "memory"; std::shared_ptr camera = Camera::create(std::move(cameraData), id, streams); - PipelineHandler::registerCamera(std::move(camera)); LOG(RPI, Info) << "Registered camera " << id - << " to CFE device " << cfe->deviceNode() + << (cfe ? " to CFE device " + cfe->deviceNode() : "") << " and ISP device " << isp->deviceNode() << " using PiSP variant " << data->pispVariant_.Name(); + return camera; +} + +int PipelineHandlerPiSP::platformRegister(std::unique_ptr &cameraData, + std::shared_ptr cfe, + std::shared_ptr isp) +{ + std::shared_ptr camera = platformCreateCamera(cameraData, cfe.get(), isp.get()); + PipelineHandler::registerCamera(std::move(camera)); + return 0; }