From patchwork Wed Aug 27 09:07:34 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 24243 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5CFB4C332B for ; Wed, 27 Aug 2025 09:08:03 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 25883692F7; Wed, 27 Aug 2025 11:08:01 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="BfmQoCW1"; dkim-atps=neutral Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 87B3A692E9 for ; Wed, 27 Aug 2025 11:07:49 +0200 (CEST) Received: by mail-wm1-x333.google.com with SMTP id 5b1f17b1804b1-45a1b0c52f3so40411945e9.3 for ; Wed, 27 Aug 2025 02:07:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1756285669; x=1756890469; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=15ju4tj93kD+K5/upT09hykhZm9/EYdCq0k49NZ7haY=; b=BfmQoCW1pGP9hwNAmcFRaCB6YrvMxcENv8qYDYPyGxiqWYYYrYqJeoPl7tHOZ230Sm Ro7RdRz6NoWtC8CohJWmhUgy4IhXLKIC9NI01vK+DvZfoH4c0yVACz7HH85V9PGteRar ByvucFj0hJQQlYLkz4y0Ab6fD0AuCVBgUHVM7kph/nVOOzBs65bfTo30reGZbVpRGwKR kV+HGRVQbaKzXAWdf6tXj1t2AgrX596hs/kRB6l2MxXB7Qq3vwa1MvK/P3+g9TVa1I+d UEaKDXy8+bbm633oR2OkkEfRmn1FVZU1xLRDYjCYvAcoS3qJCStlX0HlAvVIJldjyaYw 6RNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1756285669; x=1756890469; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=15ju4tj93kD+K5/upT09hykhZm9/EYdCq0k49NZ7haY=; b=fgicWQLarH7GOzX0LDomTTwSy+AKDsBAo//pGdLOhp79bAxy5Q6Zphx719j6xn+7CZ 0tBeeTbyO5nkMCS0kxRpDz/+RcbGmOYFNgekhPQclQjvQPIeffHCqBcyiX1KxXXbjt8X i0DEt0sIzwr3hvXbUhrnZ6pmBNnZIjjqXS+uiFzAsvCEAPOxZGQt0sR7Bgqg+AOtBFWc NqNJ1eWaZwFOWcSYNvlvlI3jGUd17+qfSOq8UQYJ3uyHOsJuJlX2Zs3rOG4d0pvd7DL6 5CcWWD+JYARb5kmfxoHSJBRGBclvYEo6VNSsfe+whaZxdfbsVQMpXA/GvM083b5hHvFn gdAw== X-Gm-Message-State: AOJu0YwZroHKwMIOwFUgyvuDqZitwlfjEvpT2XVGf0Dam5ViSjWZMnow fRZ4W/3f2Xs18tvZ/VFJ/6FCeuYesHw+bQ/xzz/YJTS+WiH7k1DYrv6UsaiusM0G/5If4s+8WwL 3D9io X-Gm-Gg: ASbGncsQhu0DMcRNTDqxoKRN/XiOov2P8UI0WzjH7PHlb8fBxvlaaVKecpq0gHwQzQ2 fgfZuwvBHRQMgH4QrWps/vmom6sB0vMRZ/5bCyM5v5khBVk371DVpT8DWPbjomSZow8Ehz0qp4X 0ANdsd0XwGjv9EQ/mcFVQ2ESWWChg8VeLKB+Pa5bOo4H/0ZikavEdoPgcoRGEN6O4dR9MHTEvGX E/5FwL5IuCj/LK5xXaX/+n5LNpwfF1+7CBtfXQXnpYPgPJYn6Af554tzcrpDZQ0bylHUvWOIAuK ElFcfxZJljdNh5ugoqFuMMjTTici8BFyDFjpQ+mFrLtio5klxab0FIcl1oL6+/+j+ArnRZNy/Yt 9v3pFzDSJpyKsQ1axMqI3aDcGZwdQLOeWpPniz1E5JgQESPwFYS9rg3eCdsRlmfSfh+MyPBW5ga qQNw/EyL+ddfC/leCZ/rggl4s7+stY1ImynXMZQAkjSaWQRqML8Q== X-Google-Smtp-Source: AGHT+IFZErb7y1K1DBt+l1CjR7+seOHhh31OzJWDN4JYPLdyjqr1j63cYEicoT3yt+XAkJNzo/QcYw== X-Received: by 2002:a05:600c:a47:b0:456:18cf:66b5 with SMTP id 5b1f17b1804b1-45b517caa5emr140404875e9.22.1756285668398; Wed, 27 Aug 2025 02:07:48 -0700 (PDT) Received: from raspberrypi.pitowers.org ([2a00:1098:3142:1f:ffc9:aff6:7f7f:893b]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-45b6f30fe02sm21498675e9.18.2025.08.27.02.07.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Aug 2025 02:07:47 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman Subject: [RFC PATCH 07/12] libcamera: pipeline: rpi: Allow creation of the first "memory" camera Date: Wed, 27 Aug 2025 10:07:34 +0100 Message-Id: <20250827090739.86955-8-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250827090739.86955-1-david.plowman@raspberrypi.com> References: <20250827090739.86955-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Signed-off-by: David Plowman --- .../pipeline/rpi/common/pipeline_base.cpp | 19 ++ .../pipeline/rpi/common/pipeline_base.h | 1 + src/libcamera/pipeline/rpi/pisp/pisp.cpp | 218 ++++++++++++++---- 3 files changed, 187 insertions(+), 51 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 563df198..1d4d6088 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -1170,6 +1170,25 @@ int CameraData::loadIPA(ipa::RPi::InitResult *result) return ipa_->init(settings, params, result); } +int CameraData::loadNamedIPA(std::string const &tuningFile, ipa::RPi::InitResult *result) +{ + int ret; + + ipa_ = IPAManager::createIPA(pipe(), 1, 1); + + if (!ipa_) + return -ENOENT; + + IPASettings settings(tuningFile, "default"); + ipa::RPi::InitParams params; + + ret = platformInitIpa(params); + if (ret) + return ret; + + return ipa_->init(settings, params, result); +} + int CameraData::configureIPA(const CameraConfiguration *config, ipa::RPi::ConfigResult *result) { ipa::RPi::ConfigParams params; diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index 4bce4ec4..397ad6f8 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -72,6 +72,7 @@ public: int loadPipelineConfiguration(); int loadIPA(ipa::RPi::InitResult *result); + int loadNamedIPA(std::string const &settings, ipa::RPi::InitResult *result); int configureIPA(const CameraConfiguration *config, ipa::RPi::ConfigResult *result); virtual int platformInitIpa(ipa::RPi::InitParams ¶ms) = 0; virtual int platformConfigureIpa(ipa::RPi::ConfigParams ¶ms) = 0; diff --git a/src/libcamera/pipeline/rpi/pisp/pisp.cpp b/src/libcamera/pipeline/rpi/pisp/pisp.cpp index 745cb546..d18035ec 100644 --- a/src/libcamera/pipeline/rpi/pisp/pisp.cpp +++ b/src/libcamera/pipeline/rpi/pisp/pisp.cpp @@ -815,6 +815,11 @@ public: bool match(DeviceEnumerator *enumerator) override; + bool supportsMemoryCamera() override; + + std::shared_ptr createMemoryCamera(DeviceEnumerator *enumerator, + std::string_view settings) override; + private: PiSPCameraData *cameraData(Camera *camera) { @@ -822,6 +827,8 @@ private: } int prepareBuffers(Camera *camera) override; + std::shared_ptr platformCreateCamera(std::unique_ptr &cameraData, + MediaDevice *cfe, MediaDevice *isp); int platformRegister(std::unique_ptr &cameraData, MediaDevice *cfe, MediaDevice *isp) override; }; @@ -888,10 +895,8 @@ bool PipelineHandlerPiSP::match(DeviceEnumerator *enumerator) PiSPCameraData *pisp = static_cast(cameraData.get()); - pisp->fe_ = SharedMemObject - ("pisp_frontend", true, pisp->pispVariant_); - pisp->be_ = SharedMemObject - ("pisp_backend", BackEnd::Config({}), pisp->pispVariant_); + pisp->fe_ = SharedMemObject("pisp_frontend", true, pisp->pispVariant_); + pisp->be_ = SharedMemObject("pisp_backend", BackEnd::Config({}), pisp->pispVariant_); if (!pisp->fe_.fd().isValid() || !pisp->be_.fd().isValid()) { LOG(RPI, Error) << "Failed to create ISP shared objects"; @@ -914,6 +919,84 @@ bool PipelineHandlerPiSP::match(DeviceEnumerator *enumerator) return false; } +bool PipelineHandlerPiSP::supportsMemoryCamera() +{ + return true; +} + +static bool get_variant(unsigned int be_version, const libpisp::PiSPVariant **variant) +{ + for (const auto &hw : libpisp::get_variants()) { + if (hw.BackEndVersion() == be_version) { + *variant = &hw; + return true; + } + } + + return false; +} + +std::shared_ptr +PipelineHandlerPiSP::createMemoryCamera(DeviceEnumerator *enumerator, + std::string_view settings) +{ + DeviceMatch isp("pispbe"); + isp.add("pispbe-input"); + isp.add("pispbe-config"); + isp.add("pispbe-output0"); + isp.add("pispbe-output1"); + isp.add("pispbe-tdn_output"); + isp.add("pispbe-tdn_input"); + isp.add("pispbe-stitch_output"); + isp.add("pispbe-stitch_input"); + MediaDevice *ispDevice = acquireMediaDevice(enumerator, isp); + + if (!ispDevice) { + LOG(RPI, Warning) << "Unable to acquire ISP instance"; + return nullptr; + } + + const libpisp::PiSPVariant *variant = nullptr; + if (!get_variant(ispDevice->hwRevision(), &variant)) { + LOG(RPI, Warning) << "Failed to find variant"; + return nullptr; + } + + std::unique_ptr cameraData = + std::make_unique(this, *variant); + PiSPCameraData *pisp = + static_cast(cameraData.get()); + + /* Only need the back end, but creating the front end makes things easier */ + pisp->fe_ = SharedMemObject("pisp_frontend", true, pisp->pispVariant_); + pisp->be_ = SharedMemObject("pisp_backend", BackEnd::Config({}), pisp->pispVariant_); + + if (!pisp->fe_.fd().isValid() || !pisp->be_.fd().isValid()) { + LOG(RPI, Error) << "Failed to create ISP shared objects"; + return nullptr; + } + + /* + * Next, we want to do PipelineHandlerBase::registerCamera, including: + * loadIPA + * platformReigster + * loadPipelineConfiguration + */ + + /* loadIPA is a bit different for us as we have a filename */ + ipa::RPi::InitResult result; + std::string configFile = std::string(settings); + if (pisp->loadNamedIPA(configFile, &result)) { + LOG(RPI, Error) << "Failed to load a suitable IPA library"; + return nullptr; + } + LOG(RPI, Info) << "Loaded IPA library " << configFile << " for memory camera"; + + std::shared_ptr camera = platformCreateCamera(cameraData, nullptr, ispDevice); + + return camera; +} + int PipelineHandlerPiSP::prepareBuffers(Camera *camera) { PiSPCameraData *data = cameraData(camera); @@ -1021,16 +1104,51 @@ int PipelineHandlerPiSP::prepareBuffers(Camera *camera) return 0; } -int PipelineHandlerPiSP::platformRegister(std::unique_ptr &cameraData, - MediaDevice *cfe, MediaDevice *isp) +static int createCameraCfe(PiSPCameraData *data, MediaDevice *cfe, std::set &streams) { - PiSPCameraData *data = static_cast(cameraData.get()); - int ret; - MediaEntity *cfeImage = cfe->getEntityByName("rp1-cfe-fe_image0"); MediaEntity *cfeEmbedded = cfe->getEntityByName("rp1-cfe-embedded"); MediaEntity *cfeStats = cfe->getEntityByName("rp1-cfe-fe_stats"); MediaEntity *cfeConfig = cfe->getEntityByName("rp1-cfe-fe_config"); + + /* Locate and open the cfe video streams. */ + data->cfe_[Cfe::Output0] = RPi::Stream("CFE Image", cfeImage, StreamFlag::RequiresMmap); + data->cfe_[Cfe::Embedded] = RPi::Stream("CFE Embedded", cfeEmbedded); + data->cfe_[Cfe::Stats] = RPi::Stream("CFE Stats", cfeStats); + data->cfe_[Cfe::Config] = RPi::Stream("CFE Config", cfeConfig, + StreamFlag::Recurrent | StreamFlag::RequiresMmap); + + /* Wire up all the buffer connections. */ + data->cfe_[Cfe::Output0].dev()->bufferReady.connect(data, &PiSPCameraData::cfeBufferDequeue); + data->cfe_[Cfe::Stats].dev()->bufferReady.connect(data, &PiSPCameraData::cfeBufferDequeue); + data->cfe_[Cfe::Config].dev()->bufferReady.connect(data, &PiSPCameraData::cfeBufferDequeue); + data->cfe_[Cfe::Embedded].dev()->bufferReady.connect(data, &PiSPCameraData::cfeBufferDequeue); + + data->csi2Subdev_ = std::make_unique(cfe->getEntityByName("csi2")); + data->feSubdev_ = std::make_unique(cfe->getEntityByName("pisp-fe")); + data->csi2Subdev_->open(); + data->feSubdev_->open(); + + /* + * The below grouping is just for convenience so that we can easily + * iterate over all streams in one go. + */ + data->streams_.push_back(&data->cfe_[Cfe::Output0]); + data->streams_.push_back(&data->cfe_[Cfe::Config]); + data->streams_.push_back(&data->cfe_[Cfe::Stats]); + if (data->sensorMetadata_) + data->streams_.push_back(&data->cfe_[Cfe::Embedded]); + + data->ipa_->setCameraTimeout.connect(data, &PiSPCameraData::setCameraTimeout); + + /* Applications may ask for this stream. */ + streams.insert(&data->cfe_[Cfe::Output0]); + + return 0; +} + +static int createCameraBe(PiSPCameraData *data, MediaDevice *isp, std::set &streams) +{ MediaEntity *ispInput = isp->getEntityByName("pispbe-input"); MediaEntity *IpaPrepare = isp->getEntityByName("pispbe-config"); MediaEntity *ispOutput0 = isp->getEntityByName("pispbe-output0"); @@ -1040,13 +1158,6 @@ int PipelineHandlerPiSP::platformRegister(std::unique_ptr &came MediaEntity *ispStitchOutput = isp->getEntityByName("pispbe-stitch_output"); MediaEntity *ispStitchInput = isp->getEntityByName("pispbe-stitch_input"); - /* Locate and open the cfe video streams. */ - data->cfe_[Cfe::Output0] = RPi::Stream("CFE Image", cfeImage, StreamFlag::RequiresMmap); - data->cfe_[Cfe::Embedded] = RPi::Stream("CFE Embedded", cfeEmbedded); - data->cfe_[Cfe::Stats] = RPi::Stream("CFE Stats", cfeStats); - data->cfe_[Cfe::Config] = RPi::Stream("CFE Config", cfeConfig, - StreamFlag::Recurrent | StreamFlag::RequiresMmap); - /* Tag the ISP input stream as an import stream. */ data->isp_[Isp::Input] = RPi::Stream("ISP Input", ispInput, StreamFlag::ImportOnly); @@ -1069,34 +1180,15 @@ int PipelineHandlerPiSP::platformRegister(std::unique_ptr &came StreamFlag::Recurrent); /* Wire up all the buffer connections. */ - data->cfe_[Cfe::Output0].dev()->bufferReady.connect(data, &PiSPCameraData::cfeBufferDequeue); - data->cfe_[Cfe::Stats].dev()->bufferReady.connect(data, &PiSPCameraData::cfeBufferDequeue); - data->cfe_[Cfe::Config].dev()->bufferReady.connect(data, &PiSPCameraData::cfeBufferDequeue); data->isp_[Isp::Input].dev()->bufferReady.connect(data, &PiSPCameraData::beInputDequeue); data->isp_[Isp::Config].dev()->bufferReady.connect(data, &PiSPCameraData::beOutputDequeue); data->isp_[Isp::Output0].dev()->bufferReady.connect(data, &PiSPCameraData::beOutputDequeue); data->isp_[Isp::Output1].dev()->bufferReady.connect(data, &PiSPCameraData::beOutputDequeue); - data->cfe_[Cfe::Embedded].dev()->bufferReady.connect(data, &PiSPCameraData::cfeBufferDequeue); - - data->csi2Subdev_ = std::make_unique(cfe->getEntityByName("csi2")); - data->feSubdev_ = std::make_unique(cfe->getEntityByName("pisp-fe")); - data->csi2Subdev_->open(); - data->feSubdev_->open(); /* - * Open all CFE and ISP streams. The exception is the embedded data - * stream, which only gets opened below if the IPA reports that the sensor - * supports embedded data. - * * The below grouping is just for convenience so that we can easily * iterate over all streams in one go. */ - data->streams_.push_back(&data->cfe_[Cfe::Output0]); - data->streams_.push_back(&data->cfe_[Cfe::Config]); - data->streams_.push_back(&data->cfe_[Cfe::Stats]); - if (data->sensorMetadata_) - data->streams_.push_back(&data->cfe_[Cfe::Embedded]); - data->streams_.push_back(&data->isp_[Isp::Input]); data->streams_.push_back(&data->isp_[Isp::Output0]); data->streams_.push_back(&data->isp_[Isp::Output1]); @@ -1106,38 +1198,62 @@ int PipelineHandlerPiSP::platformRegister(std::unique_ptr &came data->streams_.push_back(&data->isp_[Isp::StitchInput]); data->streams_.push_back(&data->isp_[Isp::StitchOutput]); + /* Applications may ask for these stream. */ + streams.insert(&data->isp_[Isp::Output0]); + streams.insert(&data->isp_[Isp::Output1]); + + return 0; +} + +std::shared_ptr +PipelineHandlerPiSP::platformCreateCamera(std::unique_ptr &cameraData, + MediaDevice *cfe, MediaDevice *isp) +{ + PiSPCameraData *data = static_cast(cameraData.get()); + int ret; + std::set streams; + + /* cfe is a null pointer when making a memory camera that doesn't use the front end. */ + if (cfe) { + ret = createCameraCfe(data, cfe, streams); + if (ret) + return nullptr; + } + + ret = createCameraBe(data, isp, streams); + if (ret) + return nullptr; + + /* All the streams must open successfully. */ for (auto stream : data->streams_) { ret = stream->dev()->open(); if (ret) - return ret; + return nullptr; } - /* Write up all the IPA connections. */ + /* Wire up all the IPA connections. */ data->ipa_->prepareIspComplete.connect(data, &PiSPCameraData::prepareIspComplete); data->ipa_->processStatsComplete.connect(data, &PiSPCameraData::processStatsComplete); - data->ipa_->setCameraTimeout.connect(data, &PiSPCameraData::setCameraTimeout); - - /* - * List the available streams an application may request. At present, we - * do not advertise CFE Embedded and ISP Statistics streams, as there - * is no mechanism for the application to request non-image buffer formats. - */ - std::set streams; - streams.insert(&data->cfe_[Cfe::Output0]); - streams.insert(&data->isp_[Isp::Output0]); - streams.insert(&data->isp_[Isp::Output1]); /* Create and register the camera. */ - const std::string &id = data->sensor_->id(); + const std::string &id = cfe ? data->sensor_->id() : "memory"; std::shared_ptr camera = Camera::create(std::move(cameraData), id, streams); - PipelineHandler::registerCamera(std::move(camera)); LOG(RPI, Info) << "Registered camera " << id - << " to CFE device " << cfe->deviceNode() + << (cfe ? " to CFE device " + cfe->deviceNode() : "") << " and ISP device " << isp->deviceNode() << " using PiSP variant " << data->pispVariant_.Name(); + return camera; +} + +int PipelineHandlerPiSP::platformRegister(std::unique_ptr &cameraData, + MediaDevice *cfe, MediaDevice *isp) +{ + std::shared_ptr camera = platformCreateCamera(cameraData, cfe, isp); + PipelineHandler::registerCamera(std::move(camera)); + return 0; }