From patchwork Tue Sep 30 12:26:22 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24498 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0681CC324C for ; Tue, 30 Sep 2025 12:32:08 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B26296B5F3; Tue, 30 Sep 2025 14:32:06 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="oD87gvvk"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A73BA6936E for ; Tue, 30 Sep 2025 14:32:04 +0200 (CEST) Received: from ideasonboard.com (unknown [94.31.94.171]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 61B06220; Tue, 30 Sep 2025 14:30:36 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1759235436; bh=BtyttId+RSHnpn6/RIQh9Tpcn88MdDojifzZ4WPkTLo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=oD87gvvkVSfDExpjOp6+64NaB8u6D/EDnMTlf4p+zcvOfX/uU34IBo6cqb9dbXxPf zMdfDjjXxg01f73ItxYajvG7KGaVqZFWVvoC0FeEoPf8SEmJQYCl/StiRuz/zta4nr berJ5ULn2VZCoIP1BGr1j5FhtaLiy4l2S3gqdjsk= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Kieran Bingham Subject: [PATCH v1 01/33] libcamera: rkisp1: Only connect delayed controls at start/stop Date: Tue, 30 Sep 2025 14:26:22 +0200 Message-ID: <20250930122726.1837524-2-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> References: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham The RKISP1 path may potentially have multiple cameras connected through complex pipelines such as video multiplexors or multiple FPGA paths. The RKISP1 pipeline handler notifies DelayedControls that a new frame is commencing by using the frameStart event on the ISP and using that to signal to DelayedControls that it is time to process the controls for the next frame. When more than one camera is connected to an ISP it is important not to signal events to an inactive Camera. Move the frameStart signal connection from CreateCamera() to start() and introduce a corresponding disconnect at stop(). Signed-off-by: Kieran Bingham --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index ecd13831539f..605a0724615d 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -1097,6 +1097,11 @@ int PipelineHandlerRkISP1::start(Camera *camera, [[maybe_unused]] const ControlL utils::ScopeExitActions actions; int ret; + isp_->frameStart.connect(data->delayedCtrls_.get(), + &DelayedControls::applyControls); + + actions += [&]() { isp_->frameStart.disconnect(data->delayedCtrls_.get()); }; + /* Allocate buffers for internal pipeline usage. */ ret = allocateBuffers(camera); if (ret) @@ -1168,6 +1173,8 @@ void PipelineHandlerRkISP1::stopDevice(Camera *camera) isp_->setFrameStartEnabled(false); + isp_->frameStart.disconnect(data->delayedCtrls_.get()); + data->ipa_->stop(); if (hasSelfPath_) @@ -1354,8 +1361,6 @@ int PipelineHandlerRkISP1::createCamera(MediaEntity *sensor) data->delayedCtrls_ = std::make_unique(data->sensor_->device(), params); - isp_->frameStart.connect(data->delayedCtrls_.get(), - &DelayedControls::applyControls); uint32_t supportedBlocks = kDefaultExtParamsBlocks; From patchwork Tue Sep 30 12:26:23 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24499 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9BB6BC324C for ; Tue, 30 Sep 2025 12:34:26 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 527556B5F3; Tue, 30 Sep 2025 14:34:25 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="C1R8xuMn"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4679E62C35 for ; Tue, 30 Sep 2025 14:34:22 +0200 (CEST) Received: from ideasonboard.com (unknown [94.31.94.171]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id E158F220; Tue, 30 Sep 2025 14:32:53 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1759235574; bh=il1MPl9gegwLqBk2WXmpryeIZ0Js/Yid7F7UPXStGWk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=C1R8xuMnRuxo6hBbCSIQaHDVb+7VeXpn/adQqxv7aBNcOwl1IhoX639zDdlNxdUVp kXet4DPLlva3UKz2xBD4j04pTpeMIr2sS2LWMqEkhcoEqIbycqBnHid7qeY1Zg6RBQ KUA65etvLFsftcgZbyMgr13QWqXq7SRe/v2cu5P4= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Kieran Bingham , Paul Elder Subject: [PATCH v1 02/33] libcamera: v4l2: Support fromEntityName with shared_ptr Date: Tue, 30 Sep 2025 14:26:23 +0200 Message-ID: <20250930122726.1837524-3-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> References: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham To facilitate expanding the use of a shared_ptr in preference of a raw MediaDevice* pointer, add an additional implementation for fromEntityName for both of v4l2_videodevice and v4l2_subdevice to support this type. Signed-off-by: Kieran Bingham Signed-off-by: Paul Elder --- include/libcamera/internal/v4l2_subdevice.h | 2 ++ include/libcamera/internal/v4l2_videodevice.h | 2 ++ src/libcamera/v4l2_subdevice.cpp | 15 +++++++++++++++ src/libcamera/v4l2_videodevice.cpp | 15 +++++++++++++++ 4 files changed, 34 insertions(+) diff --git a/include/libcamera/internal/v4l2_subdevice.h b/include/libcamera/internal/v4l2_subdevice.h index c1cde1df2e36..d1c8b838d49b 100644 --- a/include/libcamera/internal/v4l2_subdevice.h +++ b/include/libcamera/internal/v4l2_subdevice.h @@ -163,6 +163,8 @@ public: static std::unique_ptr fromEntityName(const MediaDevice *media, const std::string &entity); + static std::unique_ptr + fromEntityName(std::shared_ptr, const std::string &entity); protected: std::string logPrefix() const override; diff --git a/include/libcamera/internal/v4l2_videodevice.h b/include/libcamera/internal/v4l2_videodevice.h index 6caafc4dcf08..4f80f7176174 100644 --- a/include/libcamera/internal/v4l2_videodevice.h +++ b/include/libcamera/internal/v4l2_videodevice.h @@ -228,6 +228,8 @@ public: static std::unique_ptr fromEntityName(const MediaDevice *media, const std::string &entity); + static std::unique_ptr + fromEntityName(std::shared_ptr, const std::string &entity); V4L2PixelFormat toV4L2PixelFormat(const PixelFormat &pixelFormat) const; diff --git a/src/libcamera/v4l2_subdevice.cpp b/src/libcamera/v4l2_subdevice.cpp index 31a2ac72a7ee..1825e9e55ca4 100644 --- a/src/libcamera/v4l2_subdevice.cpp +++ b/src/libcamera/v4l2_subdevice.cpp @@ -1768,6 +1768,21 @@ V4L2Subdevice::fromEntityName(const MediaDevice *media, return std::make_unique(mediaEntity); } +/** + * \brief Create a new video subdevice instance from \a entity in media device + * \a media + * \param[in] media The media device where the entity is registered + * \param[in] entity The media entity name + * + * \return A newly created V4L2Subdevice on success, nullptr otherwise + */ +std::unique_ptr +V4L2Subdevice::fromEntityName(std::shared_ptr media, + const std::string &entity) +{ + return fromEntityName(media.get(), entity); +} + std::string V4L2Subdevice::logPrefix() const { return "'" + entity_->name() + "'"; diff --git a/src/libcamera/v4l2_videodevice.cpp b/src/libcamera/v4l2_videodevice.cpp index 7b48d911db73..9be5cfbc0395 100644 --- a/src/libcamera/v4l2_videodevice.cpp +++ b/src/libcamera/v4l2_videodevice.cpp @@ -2114,6 +2114,21 @@ V4L2VideoDevice::fromEntityName(const MediaDevice *media, return std::make_unique(mediaEntity); } +/** + * \brief Create a new video device instance from \a entity in media device + * \a media + * \param[in] media The media device where the entity is registered + * \param[in] entity The media entity name + * + * \return A newly created V4L2VideoDevice on success, nullptr otherwise + */ +std::unique_ptr +V4L2VideoDevice::fromEntityName(std::shared_ptr media, + const std::string &entity) +{ + return fromEntityName(media.get(), entity); +} + /** * \brief Convert \a PixelFormat to a V4L2PixelFormat supported by the device * \param[in] pixelFormat The PixelFormat to convert From patchwork Tue Sep 30 12:26:24 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24500 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 882CAC328C for ; Tue, 30 Sep 2025 12:36:41 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9AF1E6B5F3; Tue, 30 Sep 2025 14:36:40 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="nBERiahr"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5F80362C35 for ; Tue, 30 Sep 2025 14:36:38 +0200 (CEST) Received: from ideasonboard.com (unknown [94.31.94.171]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id E9D57220; Tue, 30 Sep 2025 14:35:09 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1759235710; bh=1paAzkLHJJSAYTPeqrTHPk7Qouev7jOwD5sy7W2K7zU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nBERiahr65xHcCVALGNiieB8g3XkRaeke/8a4ZY7uvMQh/WerWoQNXrzZeFCeI+Ov tVIyktetfeNuXmYfUEhfP5Ei0CmgQhgnCYIo6JS4f1WErp6w2JhHW7eCb627J1E+Cx ulj9ct7Ubk+i8BVWT3zN828S0gHbxE/5ATdlQWCo= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Kieran Bingham , Paul Elder Subject: [PATCH v1 03/33] libcamera: pipeline: utilise shared MediaDevice pointers Date: Tue, 30 Sep 2025 14:26:24 +0200 Message-ID: <20250930122726.1837524-4-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> References: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham Adapt the PipelineHandler::acquireMediaDevice() support function to return a shared pointer instead of the underlying raw pointer. Propagate this update to all pipeline handlers that use the MediaDevice and store a std::shared_ptr accordingly. Signed-off-by: Kieran Bingham Signed-off-by: Paul Elder --- include/libcamera/internal/pipeline_handler.h | 8 ++++---- src/libcamera/pipeline/imx8-isi/imx8-isi.cpp | 2 +- src/libcamera/pipeline/ipu3/cio2.cpp | 2 +- src/libcamera/pipeline/ipu3/cio2.h | 2 +- src/libcamera/pipeline/ipu3/imgu.cpp | 3 ++- src/libcamera/pipeline/ipu3/imgu.h | 4 ++-- src/libcamera/pipeline/ipu3/ipu3.cpp | 4 ++-- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 2 +- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 2 +- src/libcamera/pipeline/rkisp1/rkisp1_path.cpp | 2 +- src/libcamera/pipeline/rkisp1/rkisp1_path.h | 2 +- .../pipeline/rpi/common/pipeline_base.cpp | 6 ++++-- src/libcamera/pipeline/rpi/common/pipeline_base.h | 9 ++++++--- src/libcamera/pipeline/rpi/pisp/pisp.cpp | 10 ++++++---- src/libcamera/pipeline/rpi/vc4/vc4.cpp | 13 +++++++++---- src/libcamera/pipeline/simple/simple.cpp | 13 +++++++------ src/libcamera/pipeline/uvcvideo/uvcvideo.cpp | 6 +++--- src/libcamera/pipeline/vimc/vimc.cpp | 6 +++--- src/libcamera/pipeline_handler.cpp | 14 ++++++++------ test/delayed_controls.cpp | 2 +- test/libtest/buffer_source.cpp | 2 +- 21 files changed, 65 insertions(+), 49 deletions(-) diff --git a/include/libcamera/internal/pipeline_handler.h b/include/libcamera/internal/pipeline_handler.h index e89d6a33e398..0b25cffb44e5 100644 --- a/include/libcamera/internal/pipeline_handler.h +++ b/include/libcamera/internal/pipeline_handler.h @@ -38,8 +38,8 @@ public: virtual ~PipelineHandler(); virtual bool match(DeviceEnumerator *enumerator) = 0; - MediaDevice *acquireMediaDevice(DeviceEnumerator *enumerator, - const DeviceMatch &dm); + std::shared_ptr acquireMediaDevice(DeviceEnumerator *enumerator, + const DeviceMatch &dm); bool acquire(Camera *camera); void release(Camera *camera); @@ -72,7 +72,7 @@ public: protected: void registerCamera(std::shared_ptr camera); - void hotplugMediaDevice(MediaDevice *media); + void hotplugMediaDevice(std::shared_ptr media); virtual int queueRequestDevice(Camera *camera, Request *request) = 0; virtual void stopDevice(Camera *camera) = 0; @@ -86,7 +86,7 @@ protected: private: void unlockMediaDevices(); - void mediaDeviceDisconnected(MediaDevice *media); + void mediaDeviceDisconnected(std::shared_ptr media); virtual void disconnect(); void doQueueRequest(Request *request); diff --git a/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp b/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp index de09431cb9b9..3f414676ec28 100644 --- a/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp +++ b/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp @@ -139,7 +139,7 @@ private: void bufferReady(FrameBuffer *buffer); - MediaDevice *isiDev_; + std::shared_ptr isiDev_; std::unique_ptr crossbar_; std::vector pipes_; diff --git a/src/libcamera/pipeline/ipu3/cio2.cpp b/src/libcamera/pipeline/ipu3/cio2.cpp index aa544d7b0303..ebbd424c56ea 100644 --- a/src/libcamera/pipeline/ipu3/cio2.cpp +++ b/src/libcamera/pipeline/ipu3/cio2.cpp @@ -112,7 +112,7 @@ std::vector CIO2Device::sizes(const PixelFormat &format) const * \return 0 on success or a negative error code otherwise * \retval -ENODEV No supported image sensor is connected to this CIO2 instance */ -int CIO2Device::init(const MediaDevice *media, unsigned int index) +int CIO2Device::init(std::shared_ptr media, unsigned int index) { int ret; diff --git a/src/libcamera/pipeline/ipu3/cio2.h b/src/libcamera/pipeline/ipu3/cio2.h index 3aa3a1ca8ef8..d844cb7ae831 100644 --- a/src/libcamera/pipeline/ipu3/cio2.h +++ b/src/libcamera/pipeline/ipu3/cio2.h @@ -38,7 +38,7 @@ public: std::vector formats() const; std::vector sizes(const PixelFormat &format) const; - int init(const MediaDevice *media, unsigned int index); + int init(std::shared_ptr media, unsigned int index); int configure(const Size &size, const Transform &transform, V4L2DeviceFormat *outputFormat); diff --git a/src/libcamera/pipeline/ipu3/imgu.cpp b/src/libcamera/pipeline/ipu3/imgu.cpp index 7be780913fae..391e000f6e12 100644 --- a/src/libcamera/pipeline/ipu3/imgu.cpp +++ b/src/libcamera/pipeline/ipu3/imgu.cpp @@ -330,7 +330,8 @@ FOV calcFOV(const Size &in, const ImgUDevice::PipeConfig &pipe) * * \return 0 on success or a negative error code otherwise */ -int ImgUDevice::init(MediaDevice *media, unsigned int index) +int ImgUDevice::init(std::shared_ptr media, + unsigned int index) { int ret; diff --git a/src/libcamera/pipeline/ipu3/imgu.h b/src/libcamera/pipeline/ipu3/imgu.h index fa508316b301..272f861f98c4 100644 --- a/src/libcamera/pipeline/ipu3/imgu.h +++ b/src/libcamera/pipeline/ipu3/imgu.h @@ -64,7 +64,7 @@ public: Size viewfinder; }; - int init(MediaDevice *media, unsigned int index); + int init(std::shared_ptr media, unsigned int index); PipeConfig calculatePipeConfig(Pipe *pipe); @@ -118,7 +118,7 @@ private: V4L2DeviceFormat *outputFormat); std::string name_; - MediaDevice *media_; + std::shared_ptr media_; }; } /* namespace libcamera */ diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index d6b7edcb5a7f..695762c750f8 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -164,8 +164,8 @@ private: ImgUDevice imgu0_; ImgUDevice imgu1_; - MediaDevice *cio2MediaDev_; - MediaDevice *imguMediaDev_; + std::shared_ptr cio2MediaDev_; + std::shared_ptr imguMediaDev_; std::vector ipaBuffers_; }; diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 38bdc6138ed1..434b2d5e8a5f 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -682,7 +682,7 @@ private: bool registerTPGCamera(MediaLink *link); bool registerSensorCamera(MediaLink *link); - MediaDevice *media_; + std::shared_ptr media_; std::unique_ptr isp_; std::unique_ptr stats_; std::unique_ptr params_; diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 605a0724615d..c3c5b23d13f1 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -218,7 +218,7 @@ private: int updateControls(RkISP1CameraData *data); - MediaDevice *media_; + std::shared_ptr media_; std::unique_ptr isp_; std::unique_ptr param_; std::unique_ptr stat_; diff --git a/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp b/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp index d318bea92446..8d5298c04833 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp @@ -64,7 +64,7 @@ RkISP1Path::RkISP1Path(const char *name, const Span &formats, { } -bool RkISP1Path::init(MediaDevice *media) +bool RkISP1Path::init(std::shared_ptr media) { std::string resizer = std::string("rkisp1_resizer_") + name_ + "path"; std::string video = std::string("rkisp1_") + name_ + "path"; diff --git a/src/libcamera/pipeline/rkisp1/rkisp1_path.h b/src/libcamera/pipeline/rkisp1/rkisp1_path.h index 0b60c499ac64..0c68e9eb99af 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1_path.h +++ b/src/libcamera/pipeline/rkisp1/rkisp1_path.h @@ -37,7 +37,7 @@ public: RkISP1Path(const char *name, const Span &formats, const Size &minResolution, const Size &maxResolution); - bool init(MediaDevice *media); + bool init(std::shared_ptr media); int setEnabled(bool enable) { return link_->setEnabled(enable); } bool isEnabled() const { return link_->flags() & MEDIA_LNK_FL_ENABLED; } diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index c209aa596311..9d65dc83573b 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -786,8 +786,10 @@ int PipelineHandlerBase::queueRequestDevice(Camera *camera, Request *request) } int PipelineHandlerBase::registerCamera(std::unique_ptr &cameraData, - MediaDevice *frontend, const std::string &frontendName, - MediaDevice *backend, MediaEntity *sensorEntity) + std::shared_ptr frontend, + const std::string &frontendName, + std::shared_ptr backend, + MediaEntity *sensorEntity) { CameraData *data = cameraData.get(); int ret; diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index 4bce4ec4f978..15628259afc6 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -224,13 +224,16 @@ public: protected: int registerCamera(std::unique_ptr &cameraData, - MediaDevice *frontent, const std::string &frontendName, - MediaDevice *backend, MediaEntity *sensorEntity); + std::shared_ptr frontend, + const std::string &frontendName, + std::shared_ptr backend, + MediaEntity *sensorEntity); void mapBuffers(Camera *camera, const BufferMap &buffers, unsigned int mask); virtual int platformRegister(std::unique_ptr &cameraData, - MediaDevice *unicam, MediaDevice *isp) = 0; + std::shared_ptr unicam, + std::shared_ptr isp) = 0; private: CameraData *cameraData(Camera *camera) diff --git a/src/libcamera/pipeline/rpi/pisp/pisp.cpp b/src/libcamera/pipeline/rpi/pisp/pisp.cpp index 082724c5a0fb..77acd2f64092 100644 --- a/src/libcamera/pipeline/rpi/pisp/pisp.cpp +++ b/src/libcamera/pipeline/rpi/pisp/pisp.cpp @@ -866,7 +866,8 @@ private: int prepareBuffers(Camera *camera) override; int platformRegister(std::unique_ptr &cameraData, - MediaDevice *cfe, MediaDevice *isp) override; + std::shared_ptr cfe, + std::shared_ptr isp) override; }; bool PipelineHandlerPiSP::match(DeviceEnumerator *enumerator) @@ -884,7 +885,7 @@ bool PipelineHandlerPiSP::match(DeviceEnumerator *enumerator) cfe.add("rp1-cfe-fe-image0"); cfe.add("rp1-cfe-fe-stats"); cfe.add("rp1-cfe-fe-config"); - MediaDevice *cfeDevice = acquireMediaDevice(enumerator, cfe); + std::shared_ptr cfeDevice = acquireMediaDevice(enumerator, cfe); if (!cfeDevice) { LOG(RPI, Debug) << "Unable to acquire a CFE instance"; @@ -900,7 +901,7 @@ bool PipelineHandlerPiSP::match(DeviceEnumerator *enumerator) isp.add("pispbe-tdn_input"); isp.add("pispbe-stitch_output"); isp.add("pispbe-stitch_input"); - MediaDevice *ispDevice = acquireMediaDevice(enumerator, isp); + std::shared_ptr ispDevice = acquireMediaDevice(enumerator, isp); if (!ispDevice) { LOG(RPI, Debug) << "Unable to acquire ISP instance"; @@ -1065,7 +1066,8 @@ int PipelineHandlerPiSP::prepareBuffers(Camera *camera) } int PipelineHandlerPiSP::platformRegister(std::unique_ptr &cameraData, - MediaDevice *cfe, MediaDevice *isp) + std::shared_ptr cfe, + std::shared_ptr isp) { PiSPCameraData *data = static_cast(cameraData.get()); int ret; diff --git a/src/libcamera/pipeline/rpi/vc4/vc4.cpp b/src/libcamera/pipeline/rpi/vc4/vc4.cpp index 99d43bd0a2f5..f0cb99d59e74 100644 --- a/src/libcamera/pipeline/rpi/vc4/vc4.cpp +++ b/src/libcamera/pipeline/rpi/vc4/vc4.cpp @@ -5,6 +5,8 @@ * Pipeline handler for VC4-based Raspberry Pi devices */ +#include + #include #include #include @@ -158,7 +160,8 @@ private: int prepareBuffers(Camera *camera) override; int platformRegister(std::unique_ptr &cameraData, - MediaDevice *unicam, MediaDevice *isp) override; + std::shared_ptr unicam, + std::shared_ptr isp) override; }; bool PipelineHandlerVc4::match(DeviceEnumerator *enumerator) @@ -173,7 +176,7 @@ bool PipelineHandlerVc4::match(DeviceEnumerator *enumerator) */ for (unsigned int i = 0; i < numUnicamDevices; i++) { DeviceMatch unicam("unicam"); - MediaDevice *unicamDevice = acquireMediaDevice(enumerator, unicam); + std::shared_ptr unicamDevice = acquireMediaDevice(enumerator, unicam); if (!unicamDevice) { LOG(RPI, Debug) << "Unable to acquire a Unicam instance"; @@ -181,7 +184,7 @@ bool PipelineHandlerVc4::match(DeviceEnumerator *enumerator) } DeviceMatch isp("bcm2835-isp"); - MediaDevice *ispDevice = acquireMediaDevice(enumerator, isp); + std::shared_ptr ispDevice = acquireMediaDevice(enumerator, isp); if (!ispDevice) { LOG(RPI, Debug) << "Unable to acquire ISP instance"; @@ -303,7 +306,9 @@ int PipelineHandlerVc4::prepareBuffers(Camera *camera) return 0; } -int PipelineHandlerVc4::platformRegister(std::unique_ptr &cameraData, MediaDevice *unicam, MediaDevice *isp) +int PipelineHandlerVc4::platformRegister(std::unique_ptr &cameraData, + std::shared_ptr unicam, + std::shared_ptr isp) { Vc4CameraData *data = static_cast(cameraData.get()); diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index c816cffc9e6a..5d4ce61ed5be 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -410,7 +410,7 @@ public: V4L2VideoDevice *video(const MediaEntity *entity); V4L2Subdevice *subdev(const MediaEntity *entity); - MediaDevice *converter() { return converter_; } + MediaDevice *converter() { return converter_.get(); } bool swIspEnabled() const { return swIspEnabled_; } protected: @@ -430,7 +430,8 @@ private: return static_cast(camera->_d()); } - bool matchDevice(MediaDevice *media, const SimplePipelineInfo &info, + bool matchDevice(std::shared_ptr media, + const SimplePipelineInfo &info, DeviceEnumerator *enumerator); std::vector locateSensors(MediaDevice *media); @@ -441,7 +442,7 @@ private: std::map entities_; - MediaDevice *converter_; + std::shared_ptr converter_; bool swIspEnabled_; }; @@ -1668,7 +1669,7 @@ int SimplePipelineHandler::resetRoutingTable(V4L2Subdevice *subdev) return 0; } -bool SimplePipelineHandler::matchDevice(MediaDevice *media, +bool SimplePipelineHandler::matchDevice(std::shared_ptr media, const SimplePipelineInfo &info, DeviceEnumerator *enumerator) { @@ -1699,7 +1700,7 @@ bool SimplePipelineHandler::matchDevice(MediaDevice *media, } /* Locate the sensors. */ - std::vector sensors = locateSensors(media); + std::vector sensors = locateSensors(media.get()); if (sensors.empty()) { LOG(SimplePipeline, Info) << "No sensor found for " << media->deviceNode(); return false; @@ -1817,7 +1818,7 @@ bool SimplePipelineHandler::matchDevice(MediaDevice *media, bool SimplePipelineHandler::match(DeviceEnumerator *enumerator) { - MediaDevice *media; + std::shared_ptr media; for (const SimplePipelineInfo &inf : supportedDevices) { DeviceMatch dm(inf.driver); diff --git a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp index 4b5816dfdde0..034f9c39d3e9 100644 --- a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp +++ b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp @@ -46,7 +46,7 @@ public: { } - int init(MediaDevice *media); + int init(std::shared_ptr media); void addControl(uint32_t cid, const ControlInfo &v4l2info, ControlInfoMap::Map *ctrls); void imageBufferReady(FrameBuffer *buffer); @@ -466,7 +466,7 @@ int PipelineHandlerUVC::queueRequestDevice(Camera *camera, Request *request) bool PipelineHandlerUVC::match(DeviceEnumerator *enumerator) { - MediaDevice *media; + std::shared_ptr media; DeviceMatch dm("uvcvideo"); media = acquireMediaDevice(enumerator, dm); @@ -508,7 +508,7 @@ void PipelineHandlerUVC::releaseDevice(Camera *camera) data->video_->close(); } -int UVCCameraData::init(MediaDevice *media) +int UVCCameraData::init(std::shared_ptr media) { int ret; diff --git a/src/libcamera/pipeline/vimc/vimc.cpp b/src/libcamera/pipeline/vimc/vimc.cpp index 5022101505a1..fdb92b4a82d9 100644 --- a/src/libcamera/pipeline/vimc/vimc.cpp +++ b/src/libcamera/pipeline/vimc/vimc.cpp @@ -49,7 +49,7 @@ LOG_DEFINE_CATEGORY(VIMC) class VimcCameraData : public Camera::Private { public: - VimcCameraData(PipelineHandler *pipe, MediaDevice *media) + VimcCameraData(PipelineHandler *pipe, std::shared_ptr media) : Camera::Private(pipe), media_(media) { } @@ -59,7 +59,7 @@ public: void imageBufferReady(FrameBuffer *buffer); void paramsComputed(unsigned int id, const Flags flags); - MediaDevice *media_; + std::shared_ptr media_; std::unique_ptr sensor_; std::unique_ptr debayer_; std::unique_ptr scaler_; @@ -479,7 +479,7 @@ bool PipelineHandlerVimc::match(DeviceEnumerator *enumerator) dm.add("RGB/YUV Input"); dm.add("Scaler"); - MediaDevice *media = acquireMediaDevice(enumerator, dm); + std::shared_ptr media = acquireMediaDevice(enumerator, dm); if (!media) return false; diff --git a/src/libcamera/pipeline_handler.cpp b/src/libcamera/pipeline_handler.cpp index e5f9e55c9783..825b8767fe3b 100644 --- a/src/libcamera/pipeline_handler.cpp +++ b/src/libcamera/pipeline_handler.cpp @@ -129,10 +129,12 @@ PipelineHandler::~PipelineHandler() * * \context This function shall be called from the CameraManager thread. * - * \return A pointer to the matching MediaDevice, or nullptr if no match is found + * \return A shared pointer to the matching MediaDevice, or nullptr if no match + * is found */ -MediaDevice *PipelineHandler::acquireMediaDevice(DeviceEnumerator *enumerator, - const DeviceMatch &dm) +std::shared_ptr +PipelineHandler::acquireMediaDevice(DeviceEnumerator *enumerator, + const DeviceMatch &dm) { std::shared_ptr media = enumerator->search(dm); if (!media) @@ -143,7 +145,7 @@ MediaDevice *PipelineHandler::acquireMediaDevice(DeviceEnumerator *enumerator, mediaDevices_.push_back(media); - return media.get(); + return media; } /** @@ -725,7 +727,7 @@ void PipelineHandler::registerCamera(std::shared_ptr camera) * handler gets notified and automatically disconnects all the cameras it has * registered without requiring any manual intervention. */ -void PipelineHandler::hotplugMediaDevice(MediaDevice *media) +void PipelineHandler::hotplugMediaDevice(std::shared_ptr media) { media->disconnected.connect(this, [this, media] { mediaDeviceDisconnected(media); }); } @@ -733,7 +735,7 @@ void PipelineHandler::hotplugMediaDevice(MediaDevice *media) /** * \brief Slot for the MediaDevice disconnected signal */ -void PipelineHandler::mediaDeviceDisconnected(MediaDevice *media) +void PipelineHandler::mediaDeviceDisconnected(std::shared_ptr media) { media->disconnected.disconnect(this); diff --git a/test/delayed_controls.cpp b/test/delayed_controls.cpp index 7bd30e7aead8..b305be48aafc 100644 --- a/test/delayed_controls.cpp +++ b/test/delayed_controls.cpp @@ -47,7 +47,7 @@ protected: return TestSkip; } - dev_ = V4L2VideoDevice::fromEntityName(media_.get(), "vivid-000-vid-cap"); + dev_ = V4L2VideoDevice::fromEntityName(media_, "vivid-000-vid-cap"); if (dev_->open()) { cerr << "Failed to open video device" << endl; return TestFail; diff --git a/test/libtest/buffer_source.cpp b/test/libtest/buffer_source.cpp index dde11f365e43..19b25fed50a0 100644 --- a/test/libtest/buffer_source.cpp +++ b/test/libtest/buffer_source.cpp @@ -52,7 +52,7 @@ int BufferSource::allocate(const StreamConfiguration &config) return TestSkip; } - std::unique_ptr video = V4L2VideoDevice::fromEntityName(media_.get(), videoDeviceName); + std::unique_ptr video = V4L2VideoDevice::fromEntityName(media_, videoDeviceName); if (!video) { std::cout << "Failed to get video device from entity " << videoDeviceName << std::endl; From patchwork Tue Sep 30 12:26:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24501 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 00DD5C328C for ; Tue, 30 Sep 2025 12:39:00 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 094DC62C35; Tue, 30 Sep 2025 14:39:00 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="wjboq9wn"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EDA8862C35 for ; Tue, 30 Sep 2025 14:38:57 +0200 (CEST) Received: from ideasonboard.com (unknown [94.31.94.171]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 7B6ED220; Tue, 30 Sep 2025 14:37:29 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1759235849; bh=H0nNmAan3FzGqsERM4FnQaO71k5GY1eNHpOgMviD1Wk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=wjboq9wnOo2ITZCJhRhqcebs8CN4cX9Z1cNqykaerDo4Lcn8IJTJPzFyVh3TyWAE6 MzmG/zFd3+/vg467xKsiECSYZMCYn0ej1Y5ldwZxWJmrX9K4iEbRwf0EPG9Ctxi0Zk aniPIRcWEOG/+z0sbEnlym2gwFXobIAwRBTXj+vo= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Kieran Bingham , Paul Elder Subject: [PATCH v1 04/33] libcamera: v4l2: Remove fromEntityName(MediaDevice*) Date: Tue, 30 Sep 2025 14:26:25 +0200 Message-ID: <20250930122726.1837524-5-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> References: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham Now that all users of the fromEntityName() helpers use the shared_ptr variant, remove the original functions that are no longer used. Signed-off-by: Kieran Bingham Signed-off-by: Paul Elder --- include/libcamera/internal/v4l2_subdevice.h | 2 -- include/libcamera/internal/v4l2_videodevice.h | 2 -- src/libcamera/v4l2_subdevice.cpp | 17 +---------------- src/libcamera/v4l2_videodevice.cpp | 17 +---------------- 4 files changed, 2 insertions(+), 36 deletions(-) diff --git a/include/libcamera/internal/v4l2_subdevice.h b/include/libcamera/internal/v4l2_subdevice.h index d1c8b838d49b..b4b41806638c 100644 --- a/include/libcamera/internal/v4l2_subdevice.h +++ b/include/libcamera/internal/v4l2_subdevice.h @@ -161,8 +161,6 @@ public: const std::string &model(); const V4L2SubdeviceCapability &caps() const { return caps_; } - static std::unique_ptr - fromEntityName(const MediaDevice *media, const std::string &entity); static std::unique_ptr fromEntityName(std::shared_ptr, const std::string &entity); diff --git a/include/libcamera/internal/v4l2_videodevice.h b/include/libcamera/internal/v4l2_videodevice.h index 4f80f7176174..5a7dcfdda118 100644 --- a/include/libcamera/internal/v4l2_videodevice.h +++ b/include/libcamera/internal/v4l2_videodevice.h @@ -226,8 +226,6 @@ public: void setDequeueTimeout(utils::Duration timeout); Signal<> dequeueTimeout; - static std::unique_ptr - fromEntityName(const MediaDevice *media, const std::string &entity); static std::unique_ptr fromEntityName(std::shared_ptr, const std::string &entity); diff --git a/src/libcamera/v4l2_subdevice.cpp b/src/libcamera/v4l2_subdevice.cpp index 1825e9e55ca4..643d63ea18bd 100644 --- a/src/libcamera/v4l2_subdevice.cpp +++ b/src/libcamera/v4l2_subdevice.cpp @@ -1758,7 +1758,7 @@ const std::string &V4L2Subdevice::model() * \return A newly created V4L2Subdevice on success, nullptr otherwise */ std::unique_ptr -V4L2Subdevice::fromEntityName(const MediaDevice *media, +V4L2Subdevice::fromEntityName(std::shared_ptr media, const std::string &entity) { MediaEntity *mediaEntity = media->getEntityByName(entity); @@ -1768,21 +1768,6 @@ V4L2Subdevice::fromEntityName(const MediaDevice *media, return std::make_unique(mediaEntity); } -/** - * \brief Create a new video subdevice instance from \a entity in media device - * \a media - * \param[in] media The media device where the entity is registered - * \param[in] entity The media entity name - * - * \return A newly created V4L2Subdevice on success, nullptr otherwise - */ -std::unique_ptr -V4L2Subdevice::fromEntityName(std::shared_ptr media, - const std::string &entity) -{ - return fromEntityName(media.get(), entity); -} - std::string V4L2Subdevice::logPrefix() const { return "'" + entity_->name() + "'"; diff --git a/src/libcamera/v4l2_videodevice.cpp b/src/libcamera/v4l2_videodevice.cpp index 9be5cfbc0395..bb57c1b76a5b 100644 --- a/src/libcamera/v4l2_videodevice.cpp +++ b/src/libcamera/v4l2_videodevice.cpp @@ -2104,7 +2104,7 @@ void V4L2VideoDevice::watchdogExpired() * \return A newly created V4L2VideoDevice on success, nullptr otherwise */ std::unique_ptr -V4L2VideoDevice::fromEntityName(const MediaDevice *media, +V4L2VideoDevice::fromEntityName(std::shared_ptr media, const std::string &entity) { MediaEntity *mediaEntity = media->getEntityByName(entity); @@ -2114,21 +2114,6 @@ V4L2VideoDevice::fromEntityName(const MediaDevice *media, return std::make_unique(mediaEntity); } -/** - * \brief Create a new video device instance from \a entity in media device - * \a media - * \param[in] media The media device where the entity is registered - * \param[in] entity The media entity name - * - * \return A newly created V4L2VideoDevice on success, nullptr otherwise - */ -std::unique_ptr -V4L2VideoDevice::fromEntityName(std::shared_ptr media, - const std::string &entity) -{ - return fromEntityName(media.get(), entity); -} - /** * \brief Convert \a PixelFormat to a V4L2PixelFormat supported by the device * \param[in] pixelFormat The PixelFormat to convert From patchwork Tue Sep 30 12:26:26 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24502 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id F01B3C328C for ; Tue, 30 Sep 2025 12:41:18 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id F1E596B5F3; Tue, 30 Sep 2025 14:41:17 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="PHsQEIOn"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0259762C35 for ; Tue, 30 Sep 2025 14:41:16 +0200 (CEST) Received: from ideasonboard.com (unknown [94.31.94.171]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id ABADE220; Tue, 30 Sep 2025 14:39:47 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1759235987; bh=NpJwAIW/h86p8McA0VZd6AtC/f4coDXaMOOB+wgZoJY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PHsQEIOnoGzoUkmiRftUHqVIDaMBSeKLWjx+lYaGyplzZKxR/xaZsUillnlegA+Vx SVjcmf67G+70Y7SfxOETRi282w8TUtc6CfJf12rztCP8L84K4KYJ0zUeU4gygdHfho Z+fYd8i8ahtijbcjsjxcx5TlwTLfkucDPyLDe8s8= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 05/33] libcamera: converter: Utilise shared MediaDevice pointers Date: Tue, 30 Sep 2025 14:26:26 +0200 Message-ID: <20250930122726.1837524-6-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> References: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" With the upcoming addition of V4L2 requests support, the converters need to keep a handle to the corresponding media device. Prepare for that by changing the constructor parameter from a raw pointer to a shared pointer. Signed-off-by: Stefan Klug --- include/libcamera/internal/converter.h | 9 +++++---- .../libcamera/internal/converter/converter_v4l2_m2m.h | 2 +- src/libcamera/converter.cpp | 5 +++-- src/libcamera/converter/converter_v4l2_m2m.cpp | 4 ++-- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 2 +- src/libcamera/pipeline/simple/simple.cpp | 4 ++-- 6 files changed, 14 insertions(+), 12 deletions(-) diff --git a/include/libcamera/internal/converter.h b/include/libcamera/internal/converter.h index 644ec429bb25..4915af7ac5de 100644 --- a/include/libcamera/internal/converter.h +++ b/include/libcamera/internal/converter.h @@ -46,7 +46,7 @@ public: Up, }; - Converter(MediaDevice *media, Features features = Feature::None); + Converter(std::shared_ptr media, Features features = Feature::None); virtual ~Converter(); virtual int loadConfiguration(const std::string &filename) = 0; @@ -107,7 +107,7 @@ public: const std::vector &compatibles() const { return compatibles_; } - static std::unique_ptr create(MediaDevice *media); + static std::unique_ptr create(std::shared_ptr media); static std::vector &factories(); static std::vector names(); @@ -116,7 +116,8 @@ private: static void registerType(ConverterFactoryBase *factory); - virtual std::unique_ptr createInstance(MediaDevice *media) const = 0; + virtual std::unique_ptr + createInstance(std::shared_ptr media) const = 0; std::string name_; std::vector compatibles_; @@ -131,7 +132,7 @@ public: { } - std::unique_ptr createInstance(MediaDevice *media) const override + std::unique_ptr createInstance(std::shared_ptr media) const override { return std::make_unique<_Converter>(media); } diff --git a/include/libcamera/internal/converter/converter_v4l2_m2m.h b/include/libcamera/internal/converter/converter_v4l2_m2m.h index 0ad7bf7fdbe2..d316754040dd 100644 --- a/include/libcamera/internal/converter/converter_v4l2_m2m.h +++ b/include/libcamera/internal/converter/converter_v4l2_m2m.h @@ -36,7 +36,7 @@ class V4L2M2MDevice; class V4L2M2MConverter : public Converter { public: - V4L2M2MConverter(MediaDevice *media); + V4L2M2MConverter(std::shared_ptr media); int loadConfiguration([[maybe_unused]] const std::string &filename) override { return 0; } bool isValid() const override { return m2m_ != nullptr; } diff --git a/src/libcamera/converter.cpp b/src/libcamera/converter.cpp index d551b908d523..142fb29a1272 100644 --- a/src/libcamera/converter.cpp +++ b/src/libcamera/converter.cpp @@ -8,6 +8,7 @@ #include "libcamera/internal/converter.h" #include +#include #include @@ -68,7 +69,7 @@ LOG_DEFINE_CATEGORY(Converter) * This searches for the entity implementing the data streaming function in the * media graph entities and use its device node as the converter device node. */ -Converter::Converter(MediaDevice *media, Features features) +Converter::Converter(std::shared_ptr media, Features features) { const std::vector &entities = media->entities(); auto it = std::find_if(entities.begin(), entities.end(), @@ -332,7 +333,7 @@ ConverterFactoryBase::ConverterFactoryBase(const std::string name, std::initiali * \return A new instance of the converter subclass corresponding to the media * device, or null if the media device driver name doesn't match anything */ -std::unique_ptr ConverterFactoryBase::create(MediaDevice *media) +std::unique_ptr ConverterFactoryBase::create(std::shared_ptr media) { const std::vector &factories = ConverterFactoryBase::factories(); diff --git a/src/libcamera/converter/converter_v4l2_m2m.cpp b/src/libcamera/converter/converter_v4l2_m2m.cpp index 0ce7a7a67f11..b2bd54f368d8 100644 --- a/src/libcamera/converter/converter_v4l2_m2m.cpp +++ b/src/libcamera/converter/converter_v4l2_m2m.cpp @@ -10,6 +10,7 @@ #include #include +#include #include #include @@ -261,8 +262,7 @@ void V4L2M2MConverter::V4L2M2MStream::captureBufferReady(FrameBuffer *buffer) * \brief Construct a V4L2M2MConverter instance * \param[in] media The media device implementing the converter */ - -V4L2M2MConverter::V4L2M2MConverter(MediaDevice *media) +V4L2M2MConverter::V4L2M2MConverter(std::shared_ptr media) : Converter(media) { if (deviceNode().empty()) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index c3c5b23d13f1..b52d1ff736cd 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -1450,7 +1450,7 @@ bool PipelineHandlerRkISP1::match(DeviceEnumerator *enumerator) std::shared_ptr dwpMediaDevice = enumerator->search(dwp); if (dwpMediaDevice) { - dewarper_ = std::make_unique(dwpMediaDevice.get()); + dewarper_ = std::make_unique(dwpMediaDevice); if (dewarper_->isValid()) { dewarper_->outputBufferReady.connect( this, &PipelineHandlerRkISP1::dewarpBufferReady); diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 5d4ce61ed5be..a6ea1dd05c82 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -410,7 +410,7 @@ public: V4L2VideoDevice *video(const MediaEntity *entity); V4L2Subdevice *subdev(const MediaEntity *entity); - MediaDevice *converter() { return converter_.get(); } + std::shared_ptr converter() { return converter_; } bool swIspEnabled() const { return swIspEnabled_; } protected: @@ -582,7 +582,7 @@ int SimpleCameraData::init() int ret; /* Open the converter, if any. */ - MediaDevice *converter = pipe->converter(); + std::shared_ptr converter = pipe->converter(); if (converter) { converter_ = ConverterFactoryBase::create(converter); if (!converter_) { From patchwork Tue Sep 30 12:26:27 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24505 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7CB34C324C for ; Tue, 30 Sep 2025 12:43:33 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 358256B5FB; Tue, 30 Sep 2025 14:43:33 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ThpcbrDH"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CC44362C35 for ; Tue, 30 Sep 2025 14:43:31 +0200 (CEST) Received: from ideasonboard.com (unknown [94.31.94.171]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 6686E220; Tue, 30 Sep 2025 14:42:03 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1759236123; bh=oOkmDPZr8uyu26EjbRBgIQZd5obUIF8cP7kLqdyUf48=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ThpcbrDHtYH6pFgMHwlfQ6+Bfu2dSdIFGFcitB8CugAEOCZnYRtJ3Pll/PzzBgwNS tz7t5lyYZUE+HKEo0hNSrLxNnFvTnBndj3URy+fOISKNfC9SRXAMWbeDE4HreWlQS1 fgNO1fmhIurwTYJP1JxqtroO9cu/cyyiltLcqWNQ= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 06/33] libcamera: Add support for V4L2 requests Date: Tue, 30 Sep 2025 14:26:27 +0200 Message-ID: <20250930122726.1837524-7-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> References: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The V4L2 requests API provides support to atomically tie controls to a set of buffers. This is especially common for m2m devices. Such a request is represented by a fd that is allocated vi MEDIA_IOC_REQUEST_ALLOC and then passed to various V4L2 function. Implement a V4L2Request class to wrap such an fd and add the corresponding utility functions. Signed-off-by: Stefan Klug --- include/libcamera/internal/media_device.h | 7 ++ include/libcamera/internal/meson.build | 1 + include/libcamera/internal/v4l2_device.h | 5 +- include/libcamera/internal/v4l2_request.h | 49 ++++++++ include/libcamera/internal/v4l2_videodevice.h | 3 +- src/libcamera/media_device.cpp | 47 ++++++++ src/libcamera/meson.build | 1 + src/libcamera/v4l2_device.cpp | 28 ++++- src/libcamera/v4l2_request.cpp | 107 ++++++++++++++++++ src/libcamera/v4l2_videodevice.cpp | 10 +- 10 files changed, 250 insertions(+), 8 deletions(-) create mode 100644 include/libcamera/internal/v4l2_request.h create mode 100644 src/libcamera/v4l2_request.cpp diff --git a/include/libcamera/internal/media_device.h b/include/libcamera/internal/media_device.h index b3a48b98d64b..74cb9ba1542d 100644 --- a/include/libcamera/internal/media_device.h +++ b/include/libcamera/internal/media_device.h @@ -18,6 +18,7 @@ #include #include "libcamera/internal/media_object.h" +#include "libcamera/internal/v4l2_request.h" namespace libcamera { @@ -57,6 +58,11 @@ public: std::vector locateEntities(unsigned int function); + int allocateRequests(unsigned int count, + std::vector> *requests); + + bool supportsRequests(); + protected: std::string logPrefix() const override; @@ -87,6 +93,7 @@ private: UniqueFD fd_; bool valid_; bool acquired_; + std::optional supportsRequests_; std::map objects_; std::vector entities_; diff --git a/include/libcamera/internal/meson.build b/include/libcamera/internal/meson.build index 45c299f6a332..e9540a2f734f 100644 --- a/include/libcamera/internal/meson.build +++ b/include/libcamera/internal/meson.build @@ -44,6 +44,7 @@ libcamera_internal_headers = files([ 'sysfs.h', 'v4l2_device.h', 'v4l2_pixelformat.h', + 'v4l2_request.h', 'v4l2_subdevice.h', 'v4l2_videodevice.h', 'vector.h', diff --git a/include/libcamera/internal/v4l2_device.h b/include/libcamera/internal/v4l2_device.h index 5bc9da96677d..dbbd118abd00 100644 --- a/include/libcamera/internal/v4l2_device.h +++ b/include/libcamera/internal/v4l2_device.h @@ -24,6 +24,7 @@ #include #include "libcamera/internal/formats.h" +#include "libcamera/internal/v4l2_request.h" namespace libcamera { @@ -37,8 +38,8 @@ public: const ControlInfoMap &controls() const { return controls_; } - ControlList getControls(Span ids); - int setControls(ControlList *ctrls); + ControlList getControls(Span ids, const V4L2Request *request = nullptr); + int setControls(ControlList *ctrls, const V4L2Request *request = nullptr); const struct v4l2_query_ext_ctrl *controlInfo(uint32_t id) const; diff --git a/include/libcamera/internal/v4l2_request.h b/include/libcamera/internal/v4l2_request.h new file mode 100644 index 000000000000..bf1bea3261af --- /dev/null +++ b/include/libcamera/internal/v4l2_request.h @@ -0,0 +1,49 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2025, Ideas On Board + * + * V4L2 requests + */ + +#pragma once + +#include + +#include + +#include +#include +#include +#include +#include + +#include +#include + +namespace libcamera { + +class V4L2Request : protected Loggable +{ +public: + bool isValid() const { return fd_.isValid(); } + int fd() const { return fd_.get(); } + + int reinit(); + int queue(); + + V4L2Request(int fd = -1); + ~V4L2Request() = default; + + Signal requestDone; + +private: + LIBCAMERA_DISABLE_COPY_AND_MOVE(V4L2Request) + + void requestReady(); + std::string logPrefix() const override; + + UniqueFD fd_; + EventNotifier fdNotifier_; +}; + +} /* namespace libcamera */ diff --git a/include/libcamera/internal/v4l2_videodevice.h b/include/libcamera/internal/v4l2_videodevice.h index 5a7dcfdda118..2d290971a0ee 100644 --- a/include/libcamera/internal/v4l2_videodevice.h +++ b/include/libcamera/internal/v4l2_videodevice.h @@ -33,6 +33,7 @@ #include "libcamera/internal/formats.h" #include "libcamera/internal/v4l2_device.h" #include "libcamera/internal/v4l2_pixelformat.h" +#include "libcamera/internal/v4l2_request.h" namespace libcamera { @@ -217,7 +218,7 @@ public: int importBuffers(unsigned int count); int releaseBuffers(); - int queueBuffer(FrameBuffer *buffer); + int queueBuffer(FrameBuffer *buffer, const V4L2Request *request = nullptr); Signal bufferReady; int streamOn(); diff --git a/src/libcamera/media_device.cpp b/src/libcamera/media_device.cpp index 353f34a81eca..673c53fefdd7 100644 --- a/src/libcamera/media_device.cpp +++ b/src/libcamera/media_device.cpp @@ -20,6 +20,7 @@ #include #include +#include "libcamera/internal/v4l2_request.h" /** * \file media_device.h @@ -851,4 +852,50 @@ std::vector MediaDevice::locateEntities(unsigned int function) return found; } +/** + * \brief Allocate requests + * \param[in] count Number of requests to allocate + * \param[out] requests Vector to store allocated requests + * + * Allocates and stores \a count requests in \a requests. If allocation fails, + * and error is returned and \a requests is cleared. + * + * \return 0 on success or a negative error code otherwise + */ +int MediaDevice::allocateRequests(unsigned int count, + std::vector> *requests) +{ + requests->resize(count); + for (unsigned int i = 0; i < count; i++) { + int requestFd; + int ret = ::ioctl(fd_.get(), MEDIA_IOC_REQUEST_ALLOC, &requestFd); + if (ret < 0) { + requests->clear(); + return -errno; + } + (*requests)[i] = std::make_unique(requestFd); + } + + return 0; +} + +/** + * \brief Check if requests are supported + * + * Checks if the device supports V4L2 requests by trying to allocate a single + * request. The result is cached, so the allocation is only tried once. + * + * \return True if the device supports requests, false otherwise + */ +bool MediaDevice::supportsRequests() +{ + if (supportsRequests_.has_value()) + return supportsRequests_.value(); + + std::vector> requests; + supportsRequests_ = (allocateRequests(1, &requests) == 0); + + return supportsRequests_.value(); +} + } /* namespace libcamera */ diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build index 5b9b86f211f1..34e20f557514 100644 --- a/src/libcamera/meson.build +++ b/src/libcamera/meson.build @@ -54,6 +54,7 @@ libcamera_internal_sources = files([ 'sysfs.cpp', 'v4l2_device.cpp', 'v4l2_pixelformat.cpp', + 'v4l2_request.cpp', 'v4l2_subdevice.cpp', 'v4l2_videodevice.cpp', 'vector.cpp', diff --git a/src/libcamera/v4l2_device.cpp b/src/libcamera/v4l2_device.cpp index 8c78b8c424e5..7a669a0303c1 100644 --- a/src/libcamera/v4l2_device.cpp +++ b/src/libcamera/v4l2_device.cpp @@ -162,6 +162,7 @@ void V4L2Device::close() /** * \brief Read controls from the device * \param[in] ids The list of controls to read, specified by their ID + * \param[in] request An optional request * * This function reads the value of all controls contained in \a ids, and * returns their values as a ControlList. @@ -171,10 +172,12 @@ void V4L2Device::close() * during validation of the requested controls, no control is read and this * function returns an empty control list. * + * If \a request is specified the controls tied to that request are read. + * * \return The control values in a ControlList on success, or an empty list on * error */ -ControlList V4L2Device::getControls(Span ids) +ControlList V4L2Device::getControls(Span ids, const V4L2Request *request) { if (ids.empty()) return {}; @@ -242,10 +245,16 @@ ControlList V4L2Device::getControls(Span ids) } struct v4l2_ext_controls v4l2ExtCtrls = {}; - v4l2ExtCtrls.which = V4L2_CTRL_WHICH_CUR_VAL; v4l2ExtCtrls.controls = v4l2Ctrls.data(); v4l2ExtCtrls.count = v4l2Ctrls.size(); + if (request) { + v4l2ExtCtrls.which = V4L2_CTRL_WHICH_REQUEST_VAL; + v4l2ExtCtrls.request_fd = request->fd(); + } else { + v4l2ExtCtrls.which = V4L2_CTRL_WHICH_CUR_VAL; + } + int ret = ioctl(VIDIOC_G_EXT_CTRLS, &v4l2ExtCtrls); if (ret) { unsigned int errorIdx = v4l2ExtCtrls.error_idx; @@ -273,6 +282,7 @@ ControlList V4L2Device::getControls(Span ids) /** * \brief Write controls to the device * \param[in] ctrls The list of controls to write + * \param[in] request And optional request * * This function writes the value of all controls contained in \a ctrls, and * stores the values actually applied to the device in the corresponding @@ -288,11 +298,15 @@ ControlList V4L2Device::getControls(Span ids) * are written and their values are updated in \a ctrls, while all other * controls are not written and their values are not changed. * + * If \a request is set, the controls will be applied to that request. If the + * device doesn't support requests, -EACCESS will be returned. If \a request is + * invalid, -EINVAL will be returned. + * * \return 0 on success or an error code otherwise * \retval -EINVAL One of the control is not supported or not accessible * \retval i The index of the control that failed */ -int V4L2Device::setControls(ControlList *ctrls) +int V4L2Device::setControls(ControlList *ctrls, const V4L2Request *request) { if (ctrls->empty()) return 0; @@ -377,10 +391,16 @@ int V4L2Device::setControls(ControlList *ctrls) } struct v4l2_ext_controls v4l2ExtCtrls = {}; - v4l2ExtCtrls.which = V4L2_CTRL_WHICH_CUR_VAL; v4l2ExtCtrls.controls = v4l2Ctrls.data(); v4l2ExtCtrls.count = v4l2Ctrls.size(); + if (request) { + v4l2ExtCtrls.which = V4L2_CTRL_WHICH_REQUEST_VAL; + v4l2ExtCtrls.request_fd = request->fd(); + } else { + v4l2ExtCtrls.which = V4L2_CTRL_WHICH_CUR_VAL; + } + int ret = ioctl(VIDIOC_S_EXT_CTRLS, &v4l2ExtCtrls); if (ret) { unsigned int errorIdx = v4l2ExtCtrls.error_idx; diff --git a/src/libcamera/v4l2_request.cpp b/src/libcamera/v4l2_request.cpp new file mode 100644 index 000000000000..47a397ac47be --- /dev/null +++ b/src/libcamera/v4l2_request.cpp @@ -0,0 +1,107 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2025, Ideas On Board + * + * V4L2 Request API + */ + +#include "libcamera/internal/v4l2_request.h" + +#include +#include +#include +#include +#include + +#include + +#include +#include + +/** + * \file v4l2_request.h + * \brief V4L2 Request + */ + +namespace libcamera { + +LOG_DECLARE_CATEGORY(V4L2) + +/** + * \class V4L2Request + * \brief V4L2Request object and API + * + * The V4L2Request class wraps a V4L2 request fd and provides some convenience + * functions to handle request. + * + * It is usually constructed by calling \a MediaDevice::allocateRequests(). + * + * A request can then be passed to the V4L2Device::setControls(), + * V4L2Device::getControls() and V4L2VideoDevice::queueBuffer(). + */ + +/** + * \brief Construct a V4L2Request + * \param[in] fd The request fd + */ +V4L2Request::V4L2Request(int fd) + : fd_(fd), fdNotifier_(fd, EventNotifier::Exception) +{ + if (!fd_.isValid()) + return; + + fdNotifier_.activated.connect(this, &V4L2Request::requestReady); + fdNotifier_.setEnabled(false); +} + +/** + * \brief Reinit the request + * + * Calls MEDIA_REQUEST_IOC_REINIT om the request fd. + * + * \return 0 on success or a negative error code otherwise + */ +int V4L2Request::reinit() +{ + fdNotifier_.setEnabled(false); + + if (::ioctl(fd_.get(), MEDIA_REQUEST_IOC_REINIT) < 0) + return -errno; + + return 0; +} + +/** + * \brief Reinit the request + * + * Calls MEDIA_REQUEST_IOC_QUEUE om the request fd. + * + * \return 0 on success or a negative error code otherwise + */ +int V4L2Request::queue() +{ + if (::ioctl(fd_.get(), MEDIA_REQUEST_IOC_QUEUE) < 0) + return -errno; + + fdNotifier_.setEnabled(true); + + return 0; +} + +std::string V4L2Request::logPrefix() const +{ + return "Request [" + std::to_string(fd()) + "]"; +} + +/** + * \brief Slot to handle request done events + * + * When this slot is called, the request is done and the requestDone will be + * emitted. + */ +void V4L2Request::requestReady() +{ + requestDone.emit(this); +} + +} /* namespace libcamera */ diff --git a/src/libcamera/v4l2_videodevice.cpp b/src/libcamera/v4l2_videodevice.cpp index bb57c1b76a5b..8ce739f4bc65 100644 --- a/src/libcamera/v4l2_videodevice.cpp +++ b/src/libcamera/v4l2_videodevice.cpp @@ -30,6 +30,7 @@ #include "libcamera/internal/framebuffer.h" #include "libcamera/internal/media_device.h" #include "libcamera/internal/media_object.h" +#include "libcamera/internal/v4l2_request.h" /** * \file v4l2_videodevice.h @@ -1629,6 +1630,7 @@ int V4L2VideoDevice::releaseBuffers() /** * \brief Queue a buffer to the video device * \param[in] buffer The buffer to be queued + * \param[in] request An optional request * * For capture video devices the \a buffer will be filled with data by the * device. For output video devices the \a buffer shall contain valid data and @@ -1641,9 +1643,11 @@ int V4L2VideoDevice::releaseBuffers() * Note that queueBuffer() will fail if the device is in the process of being * stopped from a streaming state through streamOff(). * + * If \a request is specified, the buffer will be tied to that request. + * * \return 0 on success or a negative error code otherwise */ -int V4L2VideoDevice::queueBuffer(FrameBuffer *buffer) +int V4L2VideoDevice::queueBuffer(FrameBuffer *buffer, const V4L2Request *request) { struct v4l2_plane v4l2Planes[VIDEO_MAX_PLANES] = {}; struct v4l2_buffer buf = {}; @@ -1674,6 +1678,10 @@ int V4L2VideoDevice::queueBuffer(FrameBuffer *buffer) buf.type = bufferType_; buf.memory = memoryType_; buf.field = V4L2_FIELD_NONE; + if (request) { + buf.flags = V4L2_BUF_FLAG_REQUEST_FD; + buf.request_fd = request->fd(); + } bool multiPlanar = V4L2_TYPE_IS_MULTIPLANAR(buf.type); Span planes = buffer->planes(); From patchwork Tue Sep 30 12:26:28 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24506 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 84C27C324C for ; Tue, 30 Sep 2025 12:45:49 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 52F556B599; Tue, 30 Sep 2025 14:45:48 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="pkDQ2dtK"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 937C462C35 for ; Tue, 30 Sep 2025 14:45:46 +0200 (CEST) Received: from ideasonboard.com (unknown [94.31.94.171]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 72B5B220; Tue, 30 Sep 2025 14:44:18 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1759236258; bh=BVbXjsMWssaQrvYxbc4fY/xNVBD4x0df8BkWn+0hXRY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=pkDQ2dtKZY7jA0jZLNOpm5yWT08JK2XxfX7JnZBF/rkkHE4hmFcScermzJ/mDOZS6 t7aO4Z8b2zkuohK46fERQs+cOHtZePodX1OWoOchXm1TBBd68PV5xi3E1xSN3KuhKs tlt/aeQW2HAhwGx1hJaH3z2MFJFvqym80w2S+lsA= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 07/33] libcamera: converter: Add V4L2 request support Date: Tue, 30 Sep 2025 14:26:28 +0200 Message-ID: <20250930122726.1837524-8-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> References: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add V4L2 request support to the V4L2M2MConverter class. Signed-off-by: Stefan Klug --- include/libcamera/internal/converter.h | 4 +++- .../internal/converter/converter_v4l2_m2m.h | 6 ++++-- src/libcamera/converter.cpp | 3 +++ src/libcamera/converter/converter_v4l2_m2m.cpp | 12 ++++++++---- 4 files changed, 18 insertions(+), 7 deletions(-) diff --git a/include/libcamera/internal/converter.h b/include/libcamera/internal/converter.h index 4915af7ac5de..4b811686fcf6 100644 --- a/include/libcamera/internal/converter.h +++ b/include/libcamera/internal/converter.h @@ -22,6 +22,7 @@ #include #include +#include "libcamera/internal/v4l2_request.h" namespace libcamera { @@ -79,7 +80,8 @@ public: virtual void stop() = 0; virtual int queueBuffers(FrameBuffer *input, - const std::map &outputs) = 0; + const std::map &outputs, + const V4L2Request *request = nullptr) = 0; virtual int setInputCrop(const Stream *stream, Rectangle *rect) = 0; virtual std::pair inputCropBounds() = 0; diff --git a/include/libcamera/internal/converter/converter_v4l2_m2m.h b/include/libcamera/internal/converter/converter_v4l2_m2m.h index d316754040dd..1b2a88c4a608 100644 --- a/include/libcamera/internal/converter/converter_v4l2_m2m.h +++ b/include/libcamera/internal/converter/converter_v4l2_m2m.h @@ -66,7 +66,8 @@ public: Alignment align = Alignment::Down) override; int queueBuffers(FrameBuffer *input, - const std::map &outputs) override; + const std::map &outputs, + const V4L2Request *request = nullptr) override; int setInputCrop(const Stream *stream, Rectangle *rect) override; std::pair inputCropBounds() override { return inputCropBounds_; } @@ -88,7 +89,8 @@ private: int start(); void stop(); - int queueBuffers(FrameBuffer *input, FrameBuffer *output); + int queueBuffers(FrameBuffer *input, FrameBuffer *output, + const V4L2Request *request = nullptr); int setInputSelection(unsigned int target, Rectangle *rect); int getInputSelection(unsigned int target, Rectangle *rect); diff --git a/src/libcamera/converter.cpp b/src/libcamera/converter.cpp index 142fb29a1272..ec0a6db6c035 100644 --- a/src/libcamera/converter.cpp +++ b/src/libcamera/converter.cpp @@ -205,11 +205,14 @@ Converter::~Converter() * \param[in] input The frame buffer to apply the conversion * \param[out] outputs The container holding the output stream pointers and * their respective frame buffer outputs. + * \param[in] request An optional request * * This function queues the \a input frame buffer on the output streams of the * \a outputs map key and retrieve the output frame buffer indicated by the * buffer map value. * + * If \a request is provided the buffers are tied to that request. + * * \return 0 on success or a negative error code otherwise */ diff --git a/src/libcamera/converter/converter_v4l2_m2m.cpp b/src/libcamera/converter/converter_v4l2_m2m.cpp index b2bd54f368d8..ff11a9735db7 100644 --- a/src/libcamera/converter/converter_v4l2_m2m.cpp +++ b/src/libcamera/converter/converter_v4l2_m2m.cpp @@ -22,6 +22,7 @@ #include #include "libcamera/internal/media_device.h" +#include "libcamera/internal/v4l2_request.h" #include "libcamera/internal/v4l2_videodevice.h" /** @@ -197,9 +198,11 @@ void V4L2M2MConverter::V4L2M2MStream::stop() m2m_->output()->releaseBuffers(); } -int V4L2M2MConverter::V4L2M2MStream::queueBuffers(FrameBuffer *input, FrameBuffer *output) +int V4L2M2MConverter::V4L2M2MStream::queueBuffers(FrameBuffer *input, + FrameBuffer *output, + const V4L2Request *request) { - int ret = m2m_->output()->queueBuffer(input); + int ret = m2m_->output()->queueBuffer(input, request); if (ret < 0) return ret; @@ -696,7 +699,8 @@ int V4L2M2MConverter::validateOutput(StreamConfiguration *cfg, bool *adjusted, * \copydoc libcamera::Converter::queueBuffers */ int V4L2M2MConverter::queueBuffers(FrameBuffer *input, - const std::map &outputs) + const std::map &outputs, + const V4L2Request *request) { std::set outputBufs; int ret; @@ -721,7 +725,7 @@ int V4L2M2MConverter::queueBuffers(FrameBuffer *input, /* Queue the input and output buffers to all the streams. */ for (auto [stream, buffer] : outputs) { - ret = streams_.at(stream)->queueBuffers(input, buffer); + ret = streams_.at(stream)->queueBuffers(input, buffer, request); if (ret < 0) return ret; } From patchwork Tue Sep 30 12:26:29 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24507 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id F392FC324C for ; Tue, 30 Sep 2025 12:48:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8C6526B5F0; Tue, 30 Sep 2025 14:48:05 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Q7ZYt92d"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2FFDC62C35 for ; Tue, 30 Sep 2025 14:48:02 +0200 (CEST) Received: from ideasonboard.com (unknown [94.31.94.171]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 1E6E2220; Tue, 30 Sep 2025 14:46:33 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1759236393; bh=ArMWQcxDRD0hwuC/YGbITjh8Pirv9WLsAcSFn+8u+4M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Q7ZYt92dhXHRue+xcIEftoBEOtANOYYmQHDusXNrCMKBZBn9PhlU3UKe2y7/YqKTb HnIYJhvqXL6WOudPa0oYAFaCM3jfT25URguquWg/Mi8cPhTJTE2xXiy1XuYRV9noMm bmYGIHsyIypVDFP9W1aAu2kRrPsCf+rhClcEjFbM= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 08/33] libcamera: converter_v4l2_m2m: Add suport for V4L2 requests Date: Tue, 30 Sep 2025 14:26:29 +0200 Message-ID: <20250930122726.1837524-9-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> References: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add support for V4L2 reuests. Signed-off-by: Stefan Klug --- ToDo: - Fix the place where the MediaObject is acquired/released --- .../internal/converter/converter_v4l2_m2m.h | 9 ++++- .../converter/converter_v4l2_m2m.cpp | 34 ++++++++++++++++++- 2 files changed, 41 insertions(+), 2 deletions(-) diff --git a/include/libcamera/internal/converter/converter_v4l2_m2m.h b/include/libcamera/internal/converter/converter_v4l2_m2m.h index 1b2a88c4a608..3b8fc5392b7d 100644 --- a/include/libcamera/internal/converter/converter_v4l2_m2m.h +++ b/include/libcamera/internal/converter/converter_v4l2_m2m.h @@ -73,7 +73,12 @@ public: std::pair inputCropBounds() override { return inputCropBounds_; } std::pair inputCropBounds(const Stream *stream) override; -private: + int allocateRequests(unsigned int count, + std::vector> *requests); + + bool supportsRequests(); + +protected: class V4L2M2MStream : protected Loggable { public: @@ -122,6 +127,8 @@ private: std::map> streams_; std::map queue_; std::pair inputCropBounds_; + + std::shared_ptr media_; }; } /* namespace libcamera */ diff --git a/src/libcamera/converter/converter_v4l2_m2m.cpp b/src/libcamera/converter/converter_v4l2_m2m.cpp index ff11a9735db7..c6153d728c9a 100644 --- a/src/libcamera/converter/converter_v4l2_m2m.cpp +++ b/src/libcamera/converter/converter_v4l2_m2m.cpp @@ -9,6 +9,7 @@ #include "libcamera/internal/converter/converter_v4l2_m2m.h" #include +#include #include #include #include @@ -266,7 +267,7 @@ void V4L2M2MConverter::V4L2M2MStream::captureBufferReady(FrameBuffer *buffer) * \param[in] media The media device implementing the converter */ V4L2M2MConverter::V4L2M2MConverter(std::shared_ptr media) - : Converter(media) + : Converter(media), media_(media) { if (deviceNode().empty()) return; @@ -742,6 +743,37 @@ int V4L2M2MConverter::queueBuffers(FrameBuffer *input, return 0; } +/** + * \copydoc libcamera::MediaDevice::allocateRequests + */ +int V4L2M2MConverter::allocateRequests(unsigned int count, + std::vector> *requests) +{ + /* \todo The acquire() must be moved to the right place. */ + media_->acquire(); + if (!media_->busy()) + LOG(Converter, Error) + << "MediaDevice must be valid."; + int ret = media_->allocateRequests(count, requests); + media_->release(); + return ret; +} + +/** + * \copydoc libcamera::MediaDevice::supportsRequests + */ +bool V4L2M2MConverter::supportsRequests() +{ + /* \todo The acquire() must be moved to the right place. */ + media_->acquire(); + if (!media_->busy()) + LOG(Converter, Error) + << "MediaDevice must be valid."; + bool ret = media_->supportsRequests(); + media_->release(); + return ret; +} + /* * \todo This should be extended to include Feature::Flag to denote * what each converter supports feature-wise. From patchwork Tue Sep 30 12:26:30 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24508 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9EBD7C324C for ; Tue, 30 Sep 2025 12:50:20 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 805076B5F0; Tue, 30 Sep 2025 14:50:19 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="eCpIFvGX"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 50BB362C35 for ; Tue, 30 Sep 2025 14:50:17 +0200 (CEST) Received: from ideasonboard.com (unknown [94.31.94.171]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 26C26220; Tue, 30 Sep 2025 14:48:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1759236529; bh=BTRXE+zMiu4Uw2jSOYzHjs6c77RqOLaf10zWYBQmzcE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=eCpIFvGX3KKooUkpnmaoiQXFaXQtyY02AdwpB71G9h27BhP0zydLE8jkXG/GDeg2N URw+vpxug/jHYXIHlM+HH491v/QTvgVufsI6hqf+433+AcmMw1sp0U5twF11CxTIme Inf9t3pJm0yRJajTb/6qPIA1QVll+SYbz81wdJZY= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 09/33] libcamera: converter_v4l2_m2m: Always set stride Date: Tue, 30 Sep 2025 14:26:30 +0200 Message-ID: <20250930122726.1837524-10-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> References: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Ensure the stride is properly set after a call to validateOutput(). Signed-off-by: Stefan Klug --- src/libcamera/converter/converter_v4l2_m2m.cpp | 1 + 1 file changed, 1 insertion(+) diff --git a/src/libcamera/converter/converter_v4l2_m2m.cpp b/src/libcamera/converter/converter_v4l2_m2m.cpp index c6153d728c9a..e57db8a438ab 100644 --- a/src/libcamera/converter/converter_v4l2_m2m.cpp +++ b/src/libcamera/converter/converter_v4l2_m2m.cpp @@ -680,6 +680,7 @@ int V4L2M2MConverter::validateOutput(StreamConfiguration *cfg, bool *adjusted, const Size cfgSize = cfg->size; cfg->size = adjustSizes(cfgSize, it->second, align); + cfg->stride = PixelFormatInfo::info(cfg->pixelFormat).stride(cfg->size.width, 0); if (cfg->size.isNull()) return -EINVAL; From patchwork Tue Sep 30 12:26:31 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24509 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9434AC328C for ; Tue, 30 Sep 2025 12:52:38 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9CE0B6B5F0; Tue, 30 Sep 2025 14:52:37 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="flBRP9i+"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4FCDB62C35 for ; Tue, 30 Sep 2025 14:52:36 +0200 (CEST) Received: from ideasonboard.com (unknown [94.31.94.171]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id ED0B6220; Tue, 30 Sep 2025 14:51:07 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1759236668; bh=SifEoJuCAlwRZWF+a4Ee1W4HI0QpOWoD9aAvLu2E1Qg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=flBRP9i+yz5OunatV7gg1Zd5XI6lhUsAgkenSeCZRu8Bd8xS0nACprnDKbCJj0ft3 vlpOs85iHkxPbLvHhyVRHVp9kBdIKUdSns77RuhrrlRed/J3kVL1ysvckMRd/3MPXj wsXSZd9ynGYY0i6KQYn0pXt/LQo8nE2jlFOSc4fU= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 10/33] pipeline: rkisp1: Use V4L2 requests for the dewarper Date: Tue, 30 Sep 2025 14:26:31 +0200 Message-ID: <20250930122726.1837524-11-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> References: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Use the V4L2 requests API if the dewarper supports it. Signed-off-by: Stefan Klug --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 61 ++++++++++++++++++++++-- 1 file changed, 57 insertions(+), 4 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index b52d1ff736cd..6048f028000a 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -44,6 +44,7 @@ #include "libcamera/internal/media_device.h" #include "libcamera/internal/media_pipeline.h" #include "libcamera/internal/pipeline_handler.h" +#include "libcamera/internal/v4l2_request.h" #include "libcamera/internal/v4l2_subdevice.h" #include "libcamera/internal/v4l2_videodevice.h" @@ -210,6 +211,7 @@ private: void imageBufferReady(FrameBuffer *buffer); void paramBufferReady(FrameBuffer *buffer); void statBufferReady(FrameBuffer *buffer); + void dewarpRequestReady(V4L2Request *request); void dewarpBufferReady(FrameBuffer *buffer); void frameStart(uint32_t sequence); @@ -239,6 +241,9 @@ private: std::vector> mainPathBuffers_; std::queue availableMainPathBuffers_; + std::vector> dewarpRequests_; + std::queue availableDewarpRequests_; + std::vector> paramBuffers_; std::vector> statBuffers_; std::queue availableParamBuffers_; @@ -1034,6 +1039,18 @@ int PipelineHandlerRkISP1::allocateBuffers(Camera *camera) for (std::unique_ptr &buffer : mainPathBuffers_) availableMainPathBuffers_.push(buffer.get()); + + if (dewarper_->supportsRequests()) { + ret = dewarper_->allocateRequests(kRkISP1MinBufferCount, + &dewarpRequests_); + if (ret < 0) + LOG(RkISP1, Error) << "Failed to allocate requests."; + } + + for (std::unique_ptr &request : dewarpRequests_) { + request->requestDone.connect(this, &PipelineHandlerRkISP1::dewarpRequestReady); + availableDewarpRequests_.push(request.get()); + } } auto pushBuffers = [&](const std::vector> &buffers, @@ -1075,6 +1092,11 @@ int PipelineHandlerRkISP1::freeBuffers(Camera *camera) statBuffers_.clear(); mainPathBuffers_.clear(); + while (!availableDewarpRequests_.empty()) + availableDewarpRequests_.pop(); + + dewarpRequests_.clear(); + std::vector ids; for (IPABuffer &ipabuf : data->ipaBuffers_) ids.push_back(ipabuf.id); @@ -1565,6 +1587,14 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) return; } + V4L2Request *dewarpRequest = nullptr; + if (!dewarpRequests_.empty()) { + /* If we have requests support, there must be one available */ + ASSERT(!availableDewarpRequests_.empty()); + dewarpRequest = availableDewarpRequests_.front(); + availableDewarpRequests_.pop(); + } + /* Handle scaler crop control. */ const auto &crop = request->controls().get(controls::ScalerCrop); if (crop) { @@ -1602,14 +1632,37 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) * buffers for the dewarper are the buffers of the request, supplied * by the application. */ - int ret = dewarper_->queueBuffers(buffer, request->buffers()); - if (ret < 0) - LOG(RkISP1, Error) << "Cannot queue buffers to dewarper: " - << strerror(-ret); + int ret = dewarper_->queueBuffers(buffer, request->buffers(), dewarpRequest); + if (ret < 0) { + LOG(RkISP1, Error) << "Failed to queue buffers to dewarper: -" + << strerrorname_np(-ret); + + /* Push it back into the queue. */ + if (dewarpRequest) + dewarpRequestReady(dewarpRequest); + + return; + } + + if (dewarpRequest) { + ret = dewarpRequest->queue(); + if (ret < 0) { + LOG(RkISP1, Error) << "Failed to queue dewarp request: -" + << strerrorname_np(-ret); + /* Push it back into the queue. */ + dewarpRequestReady(dewarpRequest); + } + } request->metadata().set(controls::ScalerCrop, activeCrop_.value()); } +void PipelineHandlerRkISP1::dewarpRequestReady(V4L2Request *request) +{ + request->reinit(); + availableDewarpRequests_.push(request); +} + void PipelineHandlerRkISP1::dewarpBufferReady(FrameBuffer *buffer) { ASSERT(activeCamera_); From patchwork Tue Sep 30 12:26:32 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24510 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id F1016C328C for ; Tue, 30 Sep 2025 12:54:54 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id F11536B5F0; Tue, 30 Sep 2025 14:54:53 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="hwZ/71/z"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 145A462C35 for ; Tue, 30 Sep 2025 14:54:52 +0200 (CEST) Received: from ideasonboard.com (unknown [94.31.94.171]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id A48ED220; Tue, 30 Sep 2025 14:53:23 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1759236803; bh=W9XQr5DS9xpvqVy+l0tC4HVURPUlRbZztdIcGOjy87c=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hwZ/71/zdZZLmeQ7iJGntfBFvWVmq7avdrVAgxBrJ76ENfLBaPopBytDqyfIkwAvn 0ijJwupx1pUYlG/KcVDncN5OavdN9m5M85IEshvkVS1w3UcteezgRY61ztNLeACPgm 2Q00oONqFks4c2hfjIMqERiIWiMpHOnBVkh+SL2g= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 11/33] libcamera: rkisp1: Properly cancel buffers in dewarp case Date: Tue, 30 Sep 2025 14:26:32 +0200 Message-ID: <20250930122726.1837524-12-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> References: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In case the buffer returned from the ISP was canceled, the upstream buffer was not correctly marked as canceled. Move the cancel functionality into an own helper function and correctly cancel the upstream buffers. Add missing cancellation in case queuing to the dewarper fails. Signed-off-by: Stefan Klug --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 48 +++++++++++++++--------- 1 file changed, 31 insertions(+), 17 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 6048f028000a..d5790864d026 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -208,6 +208,7 @@ private: int initLinks(Camera *camera, const RkISP1CameraConfiguration &config); int createCamera(MediaEntity *sensor); void tryCompleteRequest(RkISP1FrameInfo *info); + void cancelDewarpRequest(RkISP1FrameInfo *info); void imageBufferReady(FrameBuffer *buffer); void paramBufferReady(FrameBuffer *buffer); void statBufferReady(FrameBuffer *buffer); @@ -1526,6 +1527,31 @@ void PipelineHandlerRkISP1::tryCompleteRequest(RkISP1FrameInfo *info) completeRequest(request); } +void PipelineHandlerRkISP1::cancelDewarpRequest(RkISP1FrameInfo *info) +{ + RkISP1CameraData *data = cameraData(activeCamera_); + Request *request = info->request; + /* + * i.MX8MP is the only known platform with dewarper. It has + * no self path. Hence, only main path buffer completion is + * required. + * + * Also, we cannot completeBuffer(request, buffer) as buffer + * here, is an internal buffer (between ISP and dewarper) and + * is not associated to the any specific request. The request + * buffer associated with main path stream is the one that + * is required to be completed (not the internal buffer). + */ + for (auto [stream, buffer] : request->buffers()) { + if (stream == &data->mainPathStream_) { + buffer->_d()->cancel(); + completeBuffer(request, buffer); + } + } + + tryCompleteRequest(info); +} + void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) { ASSERT(activeCamera_); @@ -1567,23 +1593,7 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) /* Do not queue cancelled frames to dewarper. */ if (metadata.status == FrameMetadata::FrameCancelled) { - /* - * i.MX8MP is the only known platform with dewarper. It has - * no self path. Hence, only main path buffer completion is - * required. - * - * Also, we cannot completeBuffer(request, buffer) as buffer - * here, is an internal buffer (between ISP and dewarper) and - * is not associated to the any specific request. The request - * buffer associated with main path stream is the one that - * is required to be completed (not the internal buffer). - */ - for (auto it : request->buffers()) { - if (it.first == &data->mainPathStream_) - completeBuffer(request, it.second); - } - - tryCompleteRequest(info); + cancelDewarpRequest(info); return; } @@ -1641,6 +1651,8 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) if (dewarpRequest) dewarpRequestReady(dewarpRequest); + cancelDewarpRequest(info); + return; } @@ -1651,6 +1663,8 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) << strerrorname_np(-ret); /* Push it back into the queue. */ dewarpRequestReady(dewarpRequest); + + cancelDewarpRequest(info); } } From patchwork Tue Sep 30 12:26:33 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24511 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C9FF0C324C for ; Tue, 30 Sep 2025 12:57:13 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BC0C86B5F0; Tue, 30 Sep 2025 14:57:12 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Cspbb8I0"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E86BE62C35 for ; Tue, 30 Sep 2025 14:57:10 +0200 (CEST) Received: from ideasonboard.com (unknown [94.31.94.171]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 9DEAB220; Tue, 30 Sep 2025 14:55:42 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1759236942; bh=4F7ELLotJVJcHalhvwbW20jGoXCfsdYBPgrM9zGn19w=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Cspbb8I09fRy6ZjClDkWRIVXhVCaaXE0ZCxwg7zv9z0H540WEu0276al69/WXxJQk 97//hF3RKvo57LRhvSVig1A4I9ln+qQIspeM9eUWhrbWgaznR9rRDkUMKPqBn4EIl3 Nf0YF8c4BW/2m2EI6Lt47NmTyJt/iUIGsAk4PzEs= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Xavier Roumegue , Umang Jain , Jacopo Mondi Subject: [PATCH v1 12/33] include: linux: Update headers for Dw100 dewarper engine Date: Tue, 30 Sep 2025 14:26:33 +0200 Message-ID: <20250930122726.1837524-13-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> References: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Xavier Roumegue Include the DW100 dewarper header as part of linux headers. It will be used to set the V4L2_CID_DW100_DEWARPING_16x16_VERTEX_MAP control by ConverterDW100 Signed-off-by: Xavier Roumegue Signed-off-by: Umang Jain Acked-by: Jacopo Mondi --- include/linux/dw100.h | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100644 include/linux/dw100.h diff --git a/include/linux/dw100.h b/include/linux/dw100.h new file mode 100644 index 000000000000..3356496edd6b --- /dev/null +++ b/include/linux/dw100.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0-only WITH Linux-syscall-note */ +/* Copyright 2022 NXP */ + +#ifndef __UAPI_DW100_H__ +#define __UAPI_DW100_H__ + +#include + +/* + * Check Documentation/userspace-api/media/drivers/dw100.rst for control details. + */ +#define V4L2_CID_DW100_DEWARPING_16x16_VERTEX_MAP (V4L2_CID_USER_DW100_BASE + 1) + +#endif From patchwork Tue Sep 30 12:26:34 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24512 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 42F9BC324C for ; Tue, 30 Sep 2025 12:59:32 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4E3276B5F0; Tue, 30 Sep 2025 14:59:31 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="B3IoG+YR"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 708E662C35 for ; Tue, 30 Sep 2025 14:59:29 +0200 (CEST) Received: from ideasonboard.com (unknown [94.31.94.171]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 10D48220; Tue, 30 Sep 2025 14:58:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1759237081; bh=lBSgWBKYE8ufshhpqAOnTsznC+sPEavN+06FLfUpjLE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=B3IoG+YRz4qdcX2v7aC3Uo4K3GXKwPNv97O0eWGuMIMebKIVqD/pYBehGIlpTzSBe ABeKs1zEUQlS0Qxd2oEzNzhlbgByBygVj9HhYxfqpOgzwK8Ihg/PJQN6p/ExbMu3UG 0EvGE2a2yuFDP+dd0Nmk/R3aUTW+TC1y5AYhERJ0= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Umang Jain Subject: [PATCH v1 13/33] libcamera: converter_v4l2_m2m: Add helper to apply controls Date: Tue, 30 Sep 2025 14:26:34 +0200 Message-ID: <20250930122726.1837524-14-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> References: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Umang Jain Add applyControls() helper to apply controls for a stream. Signed-off-by: Umang Jain Signed-off-by: Stefan Klug --- Changes in v0.9 - Include request support in applyControls --- .../internal/converter/converter_v4l2_m2m.h | 5 +++++ .../converter/converter_v4l2_m2m.cpp | 19 +++++++++++++++++++ 2 files changed, 24 insertions(+) diff --git a/include/libcamera/internal/converter/converter_v4l2_m2m.h b/include/libcamera/internal/converter/converter_v4l2_m2m.h index 3b8fc5392b7d..615369c1a167 100644 --- a/include/libcamera/internal/converter/converter_v4l2_m2m.h +++ b/include/libcamera/internal/converter/converter_v4l2_m2m.h @@ -24,6 +24,7 @@ namespace libcamera { +class ControlList; class FrameBuffer; class MediaDevice; class Size; @@ -73,6 +74,8 @@ public: std::pair inputCropBounds() override { return inputCropBounds_; } std::pair inputCropBounds(const Stream *stream) override; + int applyControls(const Stream *stream, ControlList &ctrls, const V4L2Request *request = nullptr); + int allocateRequests(unsigned int count, std::vector> *requests); @@ -94,6 +97,8 @@ protected: int start(); void stop(); + int applyControls(ControlList &ctrls, const V4L2Request *request = nullptr); + int queueBuffers(FrameBuffer *input, FrameBuffer *output, const V4L2Request *request = nullptr); diff --git a/src/libcamera/converter/converter_v4l2_m2m.cpp b/src/libcamera/converter/converter_v4l2_m2m.cpp index e57db8a438ab..82e6d56e3d5c 100644 --- a/src/libcamera/converter/converter_v4l2_m2m.cpp +++ b/src/libcamera/converter/converter_v4l2_m2m.cpp @@ -18,6 +18,7 @@ #include #include +#include #include #include #include @@ -251,6 +252,12 @@ void V4L2M2MConverter::V4L2M2MStream::captureBufferReady(FrameBuffer *buffer) converter_->outputBufferReady.emit(buffer); } +int V4L2M2MConverter::V4L2M2MStream::applyControls(ControlList &ctrls, + const V4L2Request *request) +{ + return m2m_->capture()->setControls(&ctrls, request); +}; + /* ----------------------------------------------------------------------------- * V4L2M2MConverter */ @@ -744,6 +751,18 @@ int V4L2M2MConverter::queueBuffers(FrameBuffer *input, return 0; } +/** + * libcamera::Converter::applyControls + */ +int V4L2M2MConverter::applyControls(const Stream *stream, ControlList &ctrls, const V4L2Request *request) +{ + auto iter = streams_.find(stream); + if (iter == streams_.end()) + return -EINVAL; + + return iter->second->applyControls(ctrls, request); +} + /** * \copydoc libcamera::MediaDevice::allocateRequests */ From patchwork Tue Sep 30 12:26:35 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24513 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 84D3FC328C for ; Tue, 30 Sep 2025 13:01:47 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6DDB26B5F0; Tue, 30 Sep 2025 15:01:46 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="t0MPKGPc"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7F76969367 for ; Tue, 30 Sep 2025 15:01:45 +0200 (CEST) Received: from ideasonboard.com (unknown [94.31.94.171]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 1889B220; Tue, 30 Sep 2025 15:00:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1759237217; bh=aUat78G8Q83hmPMsMmfclvOr9DoJSyd/rPJXXalD/00=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=t0MPKGPcVpTnPeKujMp+btFmBw9A1FHx75e/vmJuNAwyrtsjG9044gWzcK6R9XQIv yj9BZrD4GyptkiOh+RjyuxXk5F33DcrUZlNvoJYLz89Z+gQCdBWflLwZJzIa9Y70Fe ISOtX72A58vUsPHHVo8wZcdoIOnyr8Mm+tJfz3Nk= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 14/33] libcamera: converter_v4l2_m2m: Add debug logging for formats Date: Tue, 30 Sep 2025 14:26:35 +0200 Message-ID: <20250930122726.1837524-15-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> References: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Getting the requested input/output format is crucial for debugging. Add log statements for that. Signed-off-by: Stefan Klug Reviewed-by: Kieran Bingham --- src/libcamera/converter/converter_v4l2_m2m.cpp | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/libcamera/converter/converter_v4l2_m2m.cpp b/src/libcamera/converter/converter_v4l2_m2m.cpp index 82e6d56e3d5c..43379b31881d 100644 --- a/src/libcamera/converter/converter_v4l2_m2m.cpp +++ b/src/libcamera/converter/converter_v4l2_m2m.cpp @@ -111,6 +111,7 @@ int V4L2M2MConverter::V4L2M2MStream::configure(const StreamConfiguration &inputC format.planesCount = 1; format.planes[0].bpl = inputCfg.stride; + LOG(Converter, Debug) << "Set input format to: " << format; int ret = m2m_->output()->setFormat(&format); if (ret < 0) { LOG(Converter, Error) @@ -133,6 +134,7 @@ int V4L2M2MConverter::V4L2M2MStream::configure(const StreamConfiguration &inputC format.fourcc = videoFormat; format.size = outputCfg.size; + LOG(Converter, Debug) << "Set output format to: " << format; ret = m2m_->capture()->setFormat(&format); if (ret < 0) { LOG(Converter, Error) From patchwork Tue Sep 30 12:26:36 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24514 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 788D6C328C for ; Tue, 30 Sep 2025 13:04:08 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 657F16B606; Tue, 30 Sep 2025 15:04:07 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Spb5jC3b"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 027BE69367 for ; Tue, 30 Sep 2025 15:04:05 +0200 (CEST) Received: from ideasonboard.com (unknown [94.31.94.171]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 96F42169; Tue, 30 Sep 2025 15:02:36 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1759237356; bh=Ka5l8jCVBgIEWsigXaUrubAdjbRqFAwcWR1C4LgeCZc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Spb5jC3bvXmk0iBl63PzyhDpQFXFA1uYjFCMF5cb5jXKR7VtNWQ330SlZbCx3zY+v HQKjIw2+Q+cLec+LjrSi3PcXhKp7Q3AJqm6qLQn/KZUisFbU1Dr3fiES7yaqQs7mGw MsZXnRF7+QzRZezP6eioxWpusKIBejlHED7BSlTQ= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 15/33] libcamera: rkisp1: Move useDewarper_ flag into RkISP1CameraData Date: Tue, 30 Sep 2025 14:26:36 +0200 Message-ID: <20250930122726.1837524-16-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> References: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The decision if the dewarper shall be used is not per pipeline but per camera and per configuration (raw streams can't use it). Move the corresponding flag into the camera data class. Rename the flag to "usesDewarper" which is easier understand when we later add the ability to enable/disable the dewarper on a per camera basis which will be expressed using a "canUseDewarper" flag. Signed-off-by: Stefan Klug --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 28 ++++++++++++------------ 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index d5790864d026..6550469842e1 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -123,6 +123,8 @@ public: */ MediaPipeline pipe_; + bool usesDewarper_; + private: void paramsComputed(unsigned int frame, unsigned int bytesused); void setSensorControls(unsigned int frame, @@ -234,7 +236,6 @@ private: std::unique_ptr dewarper_; Rectangle scalerMaxCrop_; - bool useDewarper_; std::optional activeCrop_; @@ -285,7 +286,7 @@ RkISP1FrameInfo *RkISP1Frames::create(const RkISP1CameraData *data, Request *req statBuffer = pipe_->availableStatBuffers_.front(); pipe_->availableStatBuffers_.pop(); - if (pipe_->useDewarper_) { + if (data->usesDewarper_) { mainPathBuffer = pipe_->availableMainPathBuffers_.front(); pipe_->availableMainPathBuffers_.pop(); } @@ -715,8 +716,7 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() */ PipelineHandlerRkISP1::PipelineHandlerRkISP1(CameraManager *manager) - : PipelineHandler(manager, kRkISP1MaxQueuedRequests), - hasSelfPath_(true), useDewarper_(false) + : PipelineHandler(manager, kRkISP1MaxQueuedRequests), hasSelfPath_(true) { } @@ -879,7 +879,7 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) const PixelFormat &streamFormat = config->at(0).pixelFormat; const PixelFormatInfo &info = PixelFormatInfo::info(streamFormat); isRaw_ = info.colourEncoding == PixelFormatInfo::ColourEncodingRAW; - useDewarper_ = dewarper_ && !isRaw_; + data->usesDewarper_ = dewarper_ && !isRaw_; /* YUYV8_2X8 is required on the ISP source path pad for YUV output. */ if (!isRaw_) @@ -893,7 +893,7 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) /* imx8mp has only a single path. */ const auto &cfg = config->at(0); Size ispCrop = format.size.boundedToAspectRatio(cfg.size); - if (useDewarper_) + if (data->usesDewarper_) ispCrop = dewarper_->adjustInputSize(cfg.pixelFormat, ispCrop); else @@ -934,7 +934,7 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) streamConfig[0] = IPAStream(cfg.pixelFormat, cfg.size); /* Configure dewarp */ - if (dewarper_ && !isRaw_) { + if (data->usesDewarper_) { outputCfgs.push_back(const_cast(cfg)); ret = dewarper_->configure(cfg, outputCfgs); if (ret) @@ -999,7 +999,7 @@ int PipelineHandlerRkISP1::exportFrameBuffers([[maybe_unused]] Camera *camera, S * It has mainpath and no self path. Hence, export buffers from * dewarper just for the main path stream, for now. */ - if (useDewarper_) + if (data->usesDewarper_) return dewarper_->exportBuffers(&data->mainPathStream_, count, buffers); else return mainPath_.exportBuffers(count, buffers); @@ -1033,7 +1033,7 @@ int PipelineHandlerRkISP1::allocateBuffers(Camera *camera) } /* If the dewarper is being used, allocate internal buffers for ISP. */ - if (useDewarper_) { + if (data->usesDewarper_) { ret = mainPath_.exportBuffers(kRkISP1MinBufferCount, &mainPathBuffers_); if (ret < 0) return ret; @@ -1158,7 +1158,7 @@ int PipelineHandlerRkISP1::start(Camera *camera, [[maybe_unused]] const ControlL } actions += [&]() { stat_->streamOff(); }; - if (useDewarper_) { + if (data->usesDewarper_) { ret = dewarper_->start(); if (ret) { LOG(RkISP1, Error) << "Failed to start dewarper"; @@ -1215,7 +1215,7 @@ void PipelineHandlerRkISP1::stopDevice(Camera *camera) LOG(RkISP1, Warning) << "Failed to stop parameters for " << camera->id(); - if (useDewarper_) + if (data->usesDewarper_) dewarper_->stop(); } @@ -1311,7 +1311,7 @@ int PipelineHandlerRkISP1::updateControls(RkISP1CameraData *data) { ControlInfoMap::Map controls; - if (dewarper_) { + if (data->usesDewarper_) { std::pair cropLimits; if (dewarper_->isConfigured(&data->mainPathStream_)) cropLimits = dewarper_->inputCropBounds(&data->mainPathStream_); @@ -1479,7 +1479,7 @@ bool PipelineHandlerRkISP1::match(DeviceEnumerator *enumerator) this, &PipelineHandlerRkISP1::dewarpBufferReady); LOG(RkISP1, Info) - << "Using DW100 dewarper " << dewarper_->deviceNode(); + << "Found DW100 dewarper " << dewarper_->deviceNode(); } else { LOG(RkISP1, Warning) << "Found DW100 dewarper " << dewarper_->deviceNode() @@ -1584,7 +1584,7 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) info->metadataProcessed = true; } - if (!useDewarper_) { + if (!data->usesDewarper_) { completeBuffer(request, buffer); tryCompleteRequest(info); From patchwork Tue Sep 30 12:26:37 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24515 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 79F8EC328C for ; Tue, 30 Sep 2025 13:06:27 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 24E7B6B606; Tue, 30 Sep 2025 15:06:26 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="fmlZBIUK"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 306E469367 for ; Tue, 30 Sep 2025 15:06:24 +0200 (CEST) Received: from ideasonboard.com (unknown [94.31.94.171]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id F11DA169; Tue, 30 Sep 2025 15:04:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1759237496; bh=Zppoy0qnTKtXwU5ds5A1HZMS96OVCZv+wKeA8NCVMug=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fmlZBIUKiaU+iH9MlDNKjWY9yJWDR28km26Awg6qT486GPGPGFeiwWXNegf/KMonI Fpz8S35CWPm+o/+Yre/pxuwuXNiEk/X6QLI/S/cf8sNa+PQTHpAc1sN9kyjpBPZQlE 68bk1xjCr55rVbLMtJ58PBNpBog0KXK7cT8cvwgg= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 16/33] libcamera: rkisp1: Scale down in dewarper instead of resizer Date: Tue, 30 Sep 2025 14:26:37 +0200 Message-ID: <20250930122726.1837524-17-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> References: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In order to allow digital zooming, scale down in the dewarper instead of the resizer. That means forwarding the full sensor size data to the dewarper. The ScalerCrop rectangle will also be applied at the dewarper. Signed-off-by: Stefan Klug --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 43 +++++++++++++++++++----- 1 file changed, 35 insertions(+), 8 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 6550469842e1..35a9d03198f1 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -888,16 +888,31 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) /* * On devices without DUAL_CROP (like the imx8mp) cropping needs to be * done on the ISP/IS output. + * + * If the dewarper is used, the cropping shall be done by the dewarper. */ if (media_->hwRevision() == RKISP1_V_IMX8MP) { /* imx8mp has only a single path. */ const auto &cfg = config->at(0); - Size ispCrop = format.size.boundedToAspectRatio(cfg.size); + /* + * If the dewarper is used, all cropping including aspect ratio + * preservation shall be done there. To ensure that the output + * format provided by the ISP is supported by the dewarper, a + * minimal crop still needs to be applied on the ISP output. + * + * \todo It might be possible to allocate bigger buffers + * (aligned to 8 pixels) with a stride matching format.size for + * the ISP. The not-filled border could later be ignored by the + * dewarper. This way we could skip the minimal crop here and + * the MaximumScalerCrop would always match the isp output. + */ + Size ispCrop; if (data->usesDewarper_) ispCrop = dewarper_->adjustInputSize(cfg.pixelFormat, - ispCrop); + format.size); else - ispCrop.alignUpTo(2, 2); + ispCrop = format.size.boundedToAspectRatio(cfg.size) + .alignedUpTo(2, 2); outputCrop = ispCrop.centeredTo(Rectangle(format.size).center()); format.size = ispCrop; @@ -930,13 +945,21 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) for (const StreamConfiguration &cfg : *config) { if (cfg.stream() == &data->mainPathStream_) { - ret = mainPath_.configure(cfg, format); - streamConfig[0] = IPAStream(cfg.pixelFormat, - cfg.size); - /* Configure dewarp */ + /* + * To allow for digital zoom, scaling down should happen + * in the dewarper, instead of the resizer. Configure + * the isp output to the same size as the sensor output. + */ + StreamConfiguration ispCfg = cfg; if (data->usesDewarper_) { outputCfgs.push_back(const_cast(cfg)); - ret = dewarper_->configure(cfg, outputCfgs); + + ispCfg.size = format.size; + ispCfg.stride = + PixelFormatInfo::info(ispCfg.pixelFormat) + .stride(ispCfg.size.width, 0); + + ret = dewarper_->configure(ispCfg, outputCfgs); if (ret) return ret; @@ -949,6 +972,10 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) outputCrop.transformedBetween(inputCrop, sensorInfo.analogCrop); } + + ret = mainPath_.configure(ispCfg, format); + streamConfig[0] = IPAStream(cfg.pixelFormat, + cfg.size); } else if (hasSelfPath_) { ret = selfPath_.configure(cfg, format); streamConfig[1] = IPAStream(cfg.pixelFormat, From patchwork Tue Sep 30 12:26:38 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24516 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EF0ACC324C for ; Tue, 30 Sep 2025 13:08:45 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 181356B60F; Tue, 30 Sep 2025 15:08:45 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="t1F+v7Pj"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E6E2B69367 for ; Tue, 30 Sep 2025 15:08:42 +0200 (CEST) Received: from ideasonboard.com (unknown [94.31.94.171]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 993D3169; Tue, 30 Sep 2025 15:07:14 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1759237634; bh=C7ViNxbYsmiuOIF/YlfGElK4yVzAkd+18FUpPuuHQpQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=t1F+v7PjwXcQOkVm0SpF+6cHWAbNkLnVVRfYy+4Z1qrzccrQbRksE+h9fxCnvX5oH iMa6LeTb4OM/6lab42Wtqxa+IT1fJJBzTuDlt81xP/XJheiS25/vEk2E6R+fuQ6IOK W4DLu7NQAwotLcouesDHU+9DA+/pPHwHnIyKqlWg= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 17/33] libcamera: rkisp1: Allow upscaling when the dewarper is present Date: Tue, 30 Sep 2025 14:26:38 +0200 Message-ID: <20250930122726.1837524-18-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> References: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When the dewarper is present, there is no need to forbid upscaling of the image data. Signed-off-by: Stefan Klug --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 35a9d03198f1..61eec1bb8c44 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -614,6 +614,11 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() return false; if (useDewarper) { + /* + * The dewarper output is independent of the ISP path. + * Reset to the originally requested size. + */ + tryCfg.size = cfg.size; bool adjusted; pipe->dewarper_->validateOutput(&tryCfg, &adjusted, From patchwork Tue Sep 30 12:26:39 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24517 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D08ECC324C for ; Tue, 30 Sep 2025 13:11:00 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0195B69367; Tue, 30 Sep 2025 15:11:00 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="d9BLLIo/"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E202362C35 for ; Tue, 30 Sep 2025 15:10:57 +0200 (CEST) Received: from ideasonboard.com (unknown [94.31.94.171]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 92E05BB; Tue, 30 Sep 2025 15:09:29 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1759237769; bh=R98BKcwzCyJIp3s8sEmNrjCk4iXFa/wrxpA4wKiYe7M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=d9BLLIo/eXLYxbY/euM/v+OA8+YRU2poWpVKKv+B9ncSMWmt2EQfD/68eUeJPAxwk hrP9W6tLXi8DiXAp73L2+d0HmkLJxQUCpkwukwIEauPYO8+Z3FEj9QqjmfJ0wIuzZX DOu0tUlZ36aKmS4uGN073/7pdtUo7Yv4JcdFz8Fc= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 18/33] libcamera: converter: converter_v4l2_m2m: Add makeStream() function Date: Tue, 30 Sep 2025 14:26:39 +0200 Message-ID: <20250930122726.1837524-19-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> References: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a makeStream() function that is called by the converter to create a corresponding stream. This can be overwritten by subclasses to be able to implement a custom stream type. Signed-off-by: Stefan Klug --- include/libcamera/internal/converter/converter_v4l2_m2m.h | 2 ++ src/libcamera/converter/converter_v4l2_m2m.cpp | 8 ++++++-- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/include/libcamera/internal/converter/converter_v4l2_m2m.h b/include/libcamera/internal/converter/converter_v4l2_m2m.h index 615369c1a167..033c4ba42292 100644 --- a/include/libcamera/internal/converter/converter_v4l2_m2m.h +++ b/include/libcamera/internal/converter/converter_v4l2_m2m.h @@ -127,6 +127,8 @@ protected: Size adjustSizes(const Size &size, const std::vector &ranges, Alignment align); + virtual std::unique_ptr makeStream(const Stream *stream); + std::unique_ptr m2m_; std::map> streams_; diff --git a/src/libcamera/converter/converter_v4l2_m2m.cpp b/src/libcamera/converter/converter_v4l2_m2m.cpp index 43379b31881d..d6b95fbe9105 100644 --- a/src/libcamera/converter/converter_v4l2_m2m.cpp +++ b/src/libcamera/converter/converter_v4l2_m2m.cpp @@ -548,8 +548,7 @@ int V4L2M2MConverter::configure(const StreamConfiguration &inputCfg, for (unsigned int i = 0; i < outputCfgs.size(); ++i) { const StreamConfiguration &cfg = outputCfgs[i]; - std::unique_ptr stream = - std::make_unique(this, cfg.stream()); + std::unique_ptr stream = makeStream(cfg.stream()); if (!stream->isValid()) { LOG(Converter, Error) @@ -796,6 +795,11 @@ bool V4L2M2MConverter::supportsRequests() return ret; } +std::unique_ptr V4L2M2MConverter::makeStream(const Stream *stream) +{ + return std::make_unique(this, stream); +} + /* * \todo This should be extended to include Feature::Flag to denote * what each converter supports feature-wise. From patchwork Tue Sep 30 12:26:40 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24518 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 01114C328C for ; Tue, 30 Sep 2025 13:13:16 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E36D66B5F3; Tue, 30 Sep 2025 15:13:15 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="D3IPmcG3"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 01D3362C35 for ; Tue, 30 Sep 2025 15:13:14 +0200 (CEST) Received: from ideasonboard.com (unknown [94.31.94.171]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 7D493169; Tue, 30 Sep 2025 15:11:45 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1759237905; bh=J6wdRKX4t1TTjP87zpFBoyEupSFZnyIrjRuRvJtHsyc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=D3IPmcG3/DXi0WddFTU3cFxTG5p4vKJfb5xZJskAfZfQ/EoMMIxv+x1Pt/fpvuYYR 21pK1s+nS+n2a8tH74Pth7NNlQptE2Rs58WNNA9Gu8Gv6bYKVS9IpK+419+CsJRzDT eX3FN3ymKhQimV+usQHxUmVFY5zAuBiBSVsEkFKE= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Umang Jain , Paul Elder Subject: [PATCH v1 19/33] libcamera: converter: Add dw100 converter class Date: Tue, 30 Sep 2025 14:26:40 +0200 Message-ID: <20250930122726.1837524-20-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> References: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Umang Jain DW100 Dewarp engine is present on i.MX8MP SoC. This patch provides a dedicated converter class (inherited from V4L2M2MConverter). The DW100 dewarp engine has scaling capabilites and can be used to set crop rectangles in the rkisp1 pipeline handler. Plumbing in the RkISP1 pipeline-handler is done in the subsequent patch. The ConverterDW100 class will be extended by upcoming patches to support the dewarping by setting a dewarp vertex map as well. Signed-off-by: Umang Jain Reviewed-by: Paul Elder Signed-off-by: Stefan Klug --- Changes in v0.9 - Use shared_ptr in constructor --- .../internal/converter/converter_dw100.h | 26 +++++++++++ .../libcamera/internal/converter/meson.build | 1 + src/libcamera/converter/converter_dw100.cpp | 43 +++++++++++++++++++ src/libcamera/converter/meson.build | 1 + 4 files changed, 71 insertions(+) create mode 100644 include/libcamera/internal/converter/converter_dw100.h create mode 100644 src/libcamera/converter/converter_dw100.cpp diff --git a/include/libcamera/internal/converter/converter_dw100.h b/include/libcamera/internal/converter/converter_dw100.h new file mode 100644 index 000000000000..deb0125a0dbc --- /dev/null +++ b/include/libcamera/internal/converter/converter_dw100.h @@ -0,0 +1,26 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (C) 2024, Ideas On Board Oy + * + * i.MX8MP Dewarp Engine integration + */ + +#pragma once + +#include "libcamera/internal/converter/converter_v4l2_m2m.h" + +namespace libcamera { + +class MediaDevice; +class Rectangle; +class Stream; + +class ConverterDW100 : public V4L2M2MConverter +{ +public: + ConverterDW100(std::shared_ptr media); + + void validateSize(Size *size) const; +}; + +} /* namespace libcamera */ diff --git a/include/libcamera/internal/converter/meson.build b/include/libcamera/internal/converter/meson.build index 891e79e7d493..85007a4b0f8b 100644 --- a/include/libcamera/internal/converter/meson.build +++ b/include/libcamera/internal/converter/meson.build @@ -1,5 +1,6 @@ # SPDX-License-Identifier: CC0-1.0 libcamera_internal_headers += files([ + 'converter_dw100.h', 'converter_v4l2_m2m.h', ]) diff --git a/src/libcamera/converter/converter_dw100.cpp b/src/libcamera/converter/converter_dw100.cpp new file mode 100644 index 000000000000..da1d88214a6e --- /dev/null +++ b/src/libcamera/converter/converter_dw100.cpp @@ -0,0 +1,43 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (C) 2024, Ideas On Board Oy + * + * i.MX8MP Dewarp Engine integration + */ + +#include "libcamera/internal/converter/converter_dw100.h" + +#include + +#include + +#include "libcamera/internal/media_device.h" +#include "libcamera/internal/v4l2_videodevice.h" + +namespace libcamera { + +LOG_DECLARE_CATEGORY(Converter) + +/** + * \class libcamera::ConverterDW100 + * \brief The i.MX8MP dewarp converter implements the converter interface based + * on V4L2 M2M device. +*/ + +/** + * \fn ConverterDW100::ConverterDW100 + * \brief Construct a ConverterDW100 instance + * \param[in] media The media device implementing the converter + */ +ConverterDW100::ConverterDW100(std::shared_ptr media) + : V4L2M2MConverter(media) +{ +} + +void ConverterDW100::validateSize(Size *size) const +{ + if (size) + size->alignDownTo(8, 8); +} + +} /* namespace libcamera */ diff --git a/src/libcamera/converter/meson.build b/src/libcamera/converter/meson.build index af1a80fec683..fe2dcebb67da 100644 --- a/src/libcamera/converter/meson.build +++ b/src/libcamera/converter/meson.build @@ -1,5 +1,6 @@ # SPDX-License-Identifier: CC0-1.0 libcamera_internal_sources += files([ + 'converter_dw100.cpp', 'converter_v4l2_m2m.cpp' ]) From patchwork Tue Sep 30 12:26:41 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24519 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CF00EC328C for ; Tue, 30 Sep 2025 13:15:35 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E205E6B599; Tue, 30 Sep 2025 15:15:34 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="iW50MnI5"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2C91462C35 for ; Tue, 30 Sep 2025 15:15:33 +0200 (CEST) Received: from ideasonboard.com (unknown [94.31.94.171]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id CEE01226; Tue, 30 Sep 2025 15:14:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1759238044; bh=iaZePcASm/J0ayqJaNJFOLLjY52UvzvyYVvuCKXP1Ws=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=iW50MnI5KYO6r1s07HZUB4uEsxTPfvQRAZ3NPoM8PRqPW5+3NYVoiekWOiQthtrra 3oSfgZO5qvJPQKA8VuiQz1BoFP8DAOQNn4KoaKNKQbtAyuT/wnlDa3W3Hnc+O8Pg9q QHiHS+wiDg2AUteFm+IwLOF0ym9oaShjWU+/vrgM= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 20/33] libcamera: converter: Add dw100 vertex map class Date: Tue, 30 Sep 2025 14:26:41 +0200 Message-ID: <20250930122726.1837524-21-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> References: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Using a custom vertex map the dw100 dewarper is capable of doing complex and useful transformations on the image data. This class implements a pipeline featuring: - Arbitrary ScalerCrop - Full transform support (Flip, 90deg rotations) - Arbitrary move, scale, rotate ScalerCrop and Transform is implemented to provide a interface that is standardized libcamera wide. The rest is implemented on top for more flexible dw100 specific features. Signed-off-by: Stefan Klug --- Changes in v0.9 - Include header in meson.build - Fix black line at top and left when rotation 180 degrees Changes in v0.8 - Cleanup & formatting Changes in v0.5 - Fix crash in std::clamp() due to rounding errors --- .../converter/converter_dw100_vertexmap.h | 80 ++++ .../libcamera/internal/converter/meson.build | 1 + .../converter/converter_dw100_vertexmap.cpp | 362 ++++++++++++++++++ src/libcamera/converter/meson.build | 1 + 4 files changed, 444 insertions(+) create mode 100644 include/libcamera/internal/converter/converter_dw100_vertexmap.h create mode 100644 src/libcamera/converter/converter_dw100_vertexmap.cpp diff --git a/include/libcamera/internal/converter/converter_dw100_vertexmap.h b/include/libcamera/internal/converter/converter_dw100_vertexmap.h new file mode 100644 index 000000000000..9c6e0ffa4513 --- /dev/null +++ b/include/libcamera/internal/converter/converter_dw100_vertexmap.h @@ -0,0 +1,80 @@ +#pragma once + +#include +#include +#include +#include +#include +#include + +#include + +#include +#include + +namespace libcamera { + +class Dw100VertexMap +{ +public: + enum ScaleMode { + Fill = 0, + Crop = 1, + }; + + void applyLimits(); + void setInputSize(const Size &size) + { + inputSize_ = size; + scalerCrop_ = Rectangle(size); + } + + void setSensorCrop(const Rectangle &rect) { sensorCrop_ = rect; } + + void setScalerCrop(const Rectangle &rect) { scalerCrop_ = rect; } + const Rectangle &effectiveScalerCrop() const { return effectiveScalerCrop_; } + std::pair scalerCropBounds() const + { + return { Rectangle(sensorCrop_.x, sensorCrop_.y, 1, 1), + sensorCrop_ }; + } + + void setOutputSize(const Size &size) { outputSize_ = size; } + const Size &outputSize() const { return outputSize_; } + + void setTransform(const Transform &transform) { transform_ = transform; } + const Transform &transform() const { return transform_; } + + void setScale(const float scale) { scale_ = scale; } + float effectiveScale() const { return (effectiveScaleX_ + effectiveScaleY_) * 0.5; } + + void setRotation(const float rotation) { rotation_ = rotation; } + float rotation() const { return rotation_; } + + void setOffset(const Point &offset) { offset_ = offset; } + const Point &effectiveOffset() const { return effectiveOffset_; } + + void setMode(const ScaleMode mode) { mode_ = mode; } + ScaleMode mode() const { return mode_; } + + std::vector getVertexMap(); + +private: + int getVerticesForLength(const int length); + + Rectangle scalerCrop_; + Rectangle sensorCrop_; + Transform transform_ = Transform::Identity; + Size inputSize_; + Size outputSize_; + Point offset_; + double scale_ = 1.0; + double rotation_ = 0.0; + ScaleMode mode_ = Fill; + double effectiveScaleX_; + double effectiveScaleY_; + Point effectiveOffset_; + Rectangle effectiveScalerCrop_; +}; + +} /* namespace libcamera */ diff --git a/include/libcamera/internal/converter/meson.build b/include/libcamera/internal/converter/meson.build index 85007a4b0f8b..128c644cb73f 100644 --- a/include/libcamera/internal/converter/meson.build +++ b/include/libcamera/internal/converter/meson.build @@ -2,5 +2,6 @@ libcamera_internal_headers += files([ 'converter_dw100.h', + 'converter_dw100_vertexmap.h', 'converter_v4l2_m2m.h', ]) diff --git a/src/libcamera/converter/converter_dw100_vertexmap.cpp b/src/libcamera/converter/converter_dw100_vertexmap.cpp new file mode 100644 index 000000000000..10d9e34d98c5 --- /dev/null +++ b/src/libcamera/converter/converter_dw100_vertexmap.cpp @@ -0,0 +1,362 @@ +#include "libcamera/internal/converter/converter_dw100_vertexmap.h" + +#include +#include +#include +#include +#include +#include + +#include +#include + +#include + +#include "libcamera/internal/vector.h" + +constexpr int kDw100BlockSize = 16; + +namespace libcamera { + +LOG_DECLARE_CATEGORY(Converter) + +using Vector2d = Vector; + +namespace { +void applyTransform(const Transform &t, const double w, const double h, + Vector2d &p) +{ + if (!!(t & Transform::HFlip)) { + p.x() = w - p.x(); + } + + if (!!(t & Transform::VFlip)) { + p.y() = h - p.y(); + } + + if (!!(t & Transform::Transpose)) { + std::swap(p.x(), p.y()); + } +} + +void applyTransform(const Transform &t, Rectangle &s) +{ + if (!!(t & Transform::Transpose)) { + std::swap(s.width, s.height); + } +} + +int roundTowardsZero(double v) +{ + double r = (v < 0) ? std::ceil(v) : std::floor(v); + return static_cast(r); +} + +Vector2d rotatedRectSize(const Vector2d &size, const double rad) +{ + double sa = sin(rad); + double ca = cos(rad); + + return { { std::abs(size.x() * ca) + std::abs(size.y() * sa), + std::abs(size.x() * sa) + std::abs(size.y() * ca) } }; +} + +Vector2d point2Vec2d(const Point &p) +{ + return { { static_cast(p.x), static_cast(p.y) } }; +} + +} /* namespace */ + +/** + * The vertex map class represents a helper for handling dewarper vertex maps. + * There are 3 important sizes in the system: + * + * - The Sensor size. The number of pixels of the whole sensor (Todo specify if + * OB lines are included) + * - The input Rectangle to the dewarper. Describes the pixel data + * flowing into the dewarper in sensor space + * - ScalerCrop rectangle. The rectangle that shall be used for all further + * stages. It is applied after lens dewarping but is in sensor coordinate + * space. + * - The output size. This defines the size, the dewarper should output. + * + * +------------------------+ + * |Sensor size | + * | | + * | +----------------+ | + * | | Input rect | | + * | | | | + * | +----------------+ | + * +------------------------+ + * + * This class implements a vertex map that forms the following pipeline: + * + * +-------------+ +-------------+ +-----------+ +--------------------+ + * | Lens Dewarp | -> | Scaler Crop | -> | Transform | -> | Offset, Scale, Rot | + * +-------------+ +-------------+ +-----------+ +--------------------+ + * + * ToDo: LensDewarp is not yet implemented. An identity map is used instead. + * + * All parameters are clamped to valid values before creating the vertex map. + * + * The constrains process works as follows: + * - The ScalerCrop rectangle is clamped to the input rectangle + * - The ScalerCrop rectangle is transformed by the specified transform + * forming ScalerCropT + * - A rectangle of output size is placed in the center of ScalerCropT + * (OutputRect). + * - Rotate gets applied to OutputRect, + * - Scale is applied, but clamped so that the OutputRect fits completely into + * ScalerCropT (Only regarding dimensions, not position) + * - Offset is clamped so that the OutputRect lies inside ScalerCropT + * + * The lens dewarp map is usually calibrated during tuning and is a map that + * maps from incoming pixels to dewarped pixels. + * + * Dewarp modes: + * - Crop: Fills the output frame with the data from ScalerCropT + * - Preserves aspect ratio + * - Rotate and Offset and Scale are taken into account within the limits. + * - Fill: + * - Fills the output frame with the data from ScalerCropT. + * - Does not preserve aspect ratio + * - Rotate is taken into account + * + */ +void Dw100VertexMap::applyLimits() +{ + int ow = outputSize_.width; + int oh = outputSize_.height; + effectiveScalerCrop_ = scalerCrop_.boundedTo(sensorCrop_); + + /* Map the scalerCrop to the input pixel space */ + Rectangle localScalerCropT = effectiveScalerCrop_.transformedBetween( + sensorCrop_, Rectangle(inputSize_)); + + applyTransform(transform_, localScalerCropT); + + double rad = rotation_ / 180.0 * M_PI; + double sa = sin(rad); + double ca = cos(rad); + + double scale = scale_; + Vector2d offset; + Vector2d size = rotatedRectSize(point2Vec2d(Point(ow, oh)), rad); + Vector2d t; + + /* + * Todo: All these rotations and calculations below are way easier using + * matrices. So reimplement using matrix class. + */ + + /* + * Calculate constraints + */ + if (mode_ == Crop) { + /* Scale up if needed */ + scale = std::max(scale, + std::max(size.x() / localScalerCropT.width, + size.y() / localScalerCropT.height)); + effectiveScaleX_ = scale; + effectiveScaleY_ = scale; + + size = size / scale; + + /* Transform offset to sensor space */ + offset.x() = ca * offset_.x - sa * offset_.y; + offset.y() = sa * offset_.x + ca * offset_.y; + offset = offset / scale; + } else if (mode_ == Fill) { + /* + * Calculate the best x and y scale values to fit the rotated + * localScalerCropT rectangle into the output rectangle. + */ + double diff = (static_cast(localScalerCropT.width) - + localScalerCropT.height) * + 0.5; + double middle = (static_cast(localScalerCropT.width) + + localScalerCropT.height) * + 0.5; + double w = middle + cos(rad * 2) * diff; + double h = middle - cos(rad * 2) * diff; + + size = rotatedRectSize(Vector2d{ { w, h } }, rad); + scale = std::max(size.x() / localScalerCropT.width, + size.y() / localScalerCropT.height); + + effectiveScaleX_ = (ow / w) * scale; + effectiveScaleY_ = (oh / h) * scale; + + size = size / scale; + + t = point2Vec2d(offset_); + t.x() /= effectiveScaleX_; + t.y() /= effectiveScaleY_; + + /* Transform offset to sensor space including local scale */ + offset.x() = ca * t.x() - sa * t.y(); + offset.y() = sa * t.x() + ca * t.y(); + } else { + LOG(Converter, Error) << "Unknown mode " << mode_; + return; + } + + /* Clamp offset in input space. */ + double maxoff; + /* + * Due to rounding errors, size might be slightly bigger than scaler + * crop. Clamp the offset to 0 to prevent a crash in the next clamp. + */ + maxoff = std::max(0.0, (localScalerCropT.width - size.x())) * 0.5; + offset.x() = std::clamp(offset.x(), -maxoff, maxoff); + maxoff = std::max(0.0, (localScalerCropT.height - size.y())) * 0.5; + offset.y() = std::clamp(offset.y(), -maxoff, maxoff); + + /* + * Transform offset back into output space. + * Note the transposed rotation matrix. + */ + t = offset; + offset.x() = ca * t.x() + sa * t.y(); + offset.y() = -sa * t.x() + ca * t.y(); + offset.x() *= effectiveScaleX_; + offset.y() *= effectiveScaleY_; + + effectiveOffset_ = Point(roundTowardsZero(offset.x()), + roundTowardsZero(offset.y())); +} + +std::vector Dw100VertexMap::getVertexMap() +{ + int ow = outputSize_.width; + int oh = outputSize_.height; + int tileCountW = getVerticesForLength(ow); + int tileCountH = getVerticesForLength(oh); + double rad = rotation_ / 180.0 * M_PI; + double sa = sin(rad); + double ca = cos(rad); + + applyLimits(); + + /* + * libcamera handles all crop rectangles in sensor space. But the + * dewarper "sees" only the pixels it gets passed. Note that these might + * not cover exactly the max sensor crop, as there might be a crop + * between ISP and dewarper to crop to a format supported by the + * dewarper. effectiveScalerCrop_ is the crop in sensor space that gets + * fed into the dewarper. localScalerCrop is the sensor crop mapped to + * the data that is fed into the dewarper. + */ + Rectangle localScalerCrop = effectiveScalerCrop_.transformedBetween( + sensorCrop_, Rectangle(inputSize_)); + Rectangle localScalerCropT = localScalerCrop; + applyTransform(transform_, localScalerCropT); + + /* + * The dw100 has a specialty in interpolation that has to be taken into + * account to use in a pixel perfect manner. To explain this, I will + * only use the x direction, the vertical axis behaves the same. + * + * Let's start with a pixel perfect 1:1 mapping of an image with a width + * of 64pixels. The coordinates of the vertex map would then be: + * 0 -- 16 -- 32 -- 48 -- 64 + * Note how the last coordinate lies outside the image (which ends at + * 63) as it is basically the beginning of the next macro block. + * + * if we zoom out a bit we might end up with something like + * -10 -- 0 -- 32 -- 64 -- 74 + * As the dewarper coordinates are unsigned it actually sees + * 0 -- 0 -- 32 -- 64 -- 74 + * Leading to stretched pixels at the beginning and black for everything + * > 63 + * + * Now lets rotate the image by 180 degrees. A trivial rotation would + * end up with: + * + * 64 -- 48 -- 32 -- 16 -- 0 + * + * But as the first column now points to pixel 64 we get a single black + * line. So for a proper 180* rotation, the coordinates need to be + * + * 63 -- 47 -- 31 -- 15 -- -1 + * + * The -1 is clamped to 0 again, leading to a theoretical slight + * interpolation error on the last 16 pixels. + * + * To create this proper transformation there are two things todo: + * + * 1. The rotation centers are offset by -0.5. This evens out for no rotation, + * and leads to a coordinate offset of -1 on 180 degree rotations. + * + * 2. The transformation (flip and transpose) need to act on a width-1 + * to get the same effect. + */ + Vector2d centerS{ { localScalerCropT.width * 0.5 - 0.5, + localScalerCropT.height * 0.5 - 0.5 } }; + Vector2d centerD{ { ow * 0.5 - 0.5, + oh * 0.5 - 0.5 } }; + + LOG(Converter, Debug) + << "Apply vertex map for" + << " inputSize: " << inputSize_ + << " outputSize: " << outputSize_ + << " Transform: " << transformToString(transform_) + << "\n effectiveScalerCrop: " << effectiveScalerCrop_ + << " localScalerCropT: " << localScalerCropT + << " scaleX: " << effectiveScaleX_ + << " scaleY: " << effectiveScaleX_ + << " rotation: " << rotation_ + << " offset: " << effectiveOffset_; + + /* + * For every output tile, calculate the position of the corners in the + * input image. + */ + std::vector res; + res.reserve(tileCountW * tileCountH); + for (int y = 0; y < tileCountH; y++) { + for (int x = 0; x < tileCountW; x++) { + Vector2d p{ { static_cast(x) * kDw100BlockSize, + static_cast(y) * kDw100BlockSize } }; + p = p.max(0.0).min(Vector2d{ { static_cast(ow), + static_cast(oh) } }); + + /* + * Transform into coordinates centered on the output + * rectangle + offset. + */ + p = p - centerD + point2Vec2d(effectiveOffset_); + p.x() /= effectiveScaleX_; + p.y() /= effectiveScaleY_; + + /* Rotate */ + Vector2d p2; + p2.x() = ca * p.x() - sa * p.y(); + p2.y() = sa * p.x() + ca * p.y(); + + /* Transform to localScalerCropT space. */ + p = p2 + centerS; + /* Subtract -1 for the mapping documented above. */ + applyTransform(-transform_, + localScalerCropT.width - 1, + localScalerCropT.height - 1, p); + p.x() += localScalerCrop.x; + p.y() += localScalerCrop.y; + + /* Convert to fixed point */ + uint32_t v = static_cast(p.y() * 16) << 16 | + (static_cast(p.x() * 16) & 0xffff); + res.push_back(v); + } + } + + return res; +} + +int Dw100VertexMap::getVerticesForLength(const int length) +{ + return (length + kDw100BlockSize - 1) / kDw100BlockSize + 1; +} + +} /* namespace libcamera */ diff --git a/src/libcamera/converter/meson.build b/src/libcamera/converter/meson.build index fe2dcebb67da..9f59b57c26b9 100644 --- a/src/libcamera/converter/meson.build +++ b/src/libcamera/converter/meson.build @@ -1,6 +1,7 @@ # SPDX-License-Identifier: CC0-1.0 libcamera_internal_sources += files([ + 'converter_dw100_vertexmap.cpp', 'converter_dw100.cpp', 'converter_v4l2_m2m.cpp' ]) From patchwork Tue Sep 30 12:26:42 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24520 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1D45EC324C for ; Tue, 30 Sep 2025 13:17:55 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 22A7669367; Tue, 30 Sep 2025 15:17:54 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="VxDXJZI7"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C11FE62C35 for ; Tue, 30 Sep 2025 15:17:51 +0200 (CEST) Received: from ideasonboard.com (unknown [94.31.94.171]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 72B6E16A; Tue, 30 Sep 2025 15:16:23 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1759238183; bh=6OOK1o9IMeYeiegQx9AWgicb0Xvhn5T5YIoQU0C2fsQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VxDXJZI71sfkrDMlvneosfZop/1eW8jmEdL8BbhuMnGsFSGhJr6gYHmKvyFyoP+hF zhnd8g+j9Dku4V022q430qU+CHz39CbWXqF62ZTuuTIs0ljjVusPaNIEnlOrQnQT38 MoSEuuPe+3W2dxu5BwD3KYON7MLxX8nCzP71jWZ0= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 21/33] libcamera: converter_dw100: Use vertex map Date: Tue, 30 Sep 2025 14:26:42 +0200 Message-ID: <20250930122726.1837524-22-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> References: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Use the new vertexmap class in the dw100 dewarper implementation. As the dw100 kernel driver does not use the requests API, controls can not be tied to a buffer. In the case that the dewarper has buffers queued for which processing has not yet started, applying the dewarp map would apply to an already queued buffer. Warn when this situation occurs. Signed-off-by: Stefan Klug --- Changes in v0.9: - Include requests support Changes in v0.8: - Added warning when queue is not empty --- .../internal/converter/converter_dw100.h | 19 +++++++ .../internal/converter/converter_v4l2_m2m.h | 4 +- src/libcamera/converter/converter_dw100.cpp | 56 +++++++++++++++++++ 3 files changed, 77 insertions(+), 2 deletions(-) diff --git a/include/libcamera/internal/converter/converter_dw100.h b/include/libcamera/internal/converter/converter_dw100.h index deb0125a0dbc..dabd442fcc98 100644 --- a/include/libcamera/internal/converter/converter_dw100.h +++ b/include/libcamera/internal/converter/converter_dw100.h @@ -7,6 +7,7 @@ #pragma once +#include "libcamera/internal/converter/converter_dw100_vertexmap.h" #include "libcamera/internal/converter/converter_v4l2_m2m.h" namespace libcamera { @@ -20,7 +21,25 @@ class ConverterDW100 : public V4L2M2MConverter public: ConverterDW100(std::shared_ptr media); + int applyVertexMap(const Stream *stream, const V4L2Request *request = nullptr); + Dw100VertexMap &vertexMap(const Stream *stream); + void validateSize(Size *size) const; + +private: + std::unique_ptr makeStream(const Stream *stream) override; + class DW100Stream : public V4L2M2MConverter::V4L2M2MStream + { + public: + DW100Stream(V4L2M2MConverter *converter, const Stream *stream); + + int configure(const StreamConfiguration &inputCfg, + const StreamConfiguration &outputCfg) override; + + int applyVertexMap(const V4L2Request *request = nullptr); + + Dw100VertexMap vertexMap_; + }; }; } /* namespace libcamera */ diff --git a/include/libcamera/internal/converter/converter_v4l2_m2m.h b/include/libcamera/internal/converter/converter_v4l2_m2m.h index 033c4ba42292..41b4fcc4c853 100644 --- a/include/libcamera/internal/converter/converter_v4l2_m2m.h +++ b/include/libcamera/internal/converter/converter_v4l2_m2m.h @@ -89,8 +89,8 @@ protected: bool isValid() const { return m2m_ != nullptr; } - int configure(const StreamConfiguration &inputCfg, - const StreamConfiguration &outputCfg); + virtual int configure(const StreamConfiguration &inputCfg, + const StreamConfiguration &outputCfg); int exportBuffers(unsigned int count, std::vector> *buffers); diff --git a/src/libcamera/converter/converter_dw100.cpp b/src/libcamera/converter/converter_dw100.cpp index da1d88214a6e..994b31d44d47 100644 --- a/src/libcamera/converter/converter_dw100.cpp +++ b/src/libcamera/converter/converter_dw100.cpp @@ -7,9 +7,12 @@ #include "libcamera/internal/converter/converter_dw100.h" +#include + #include #include +#include #include "libcamera/internal/media_device.h" #include "libcamera/internal/v4l2_videodevice.h" @@ -34,10 +37,63 @@ ConverterDW100::ConverterDW100(std::shared_ptr media) { } +int ConverterDW100::applyVertexMap(const Stream *stream, const V4L2Request *request) +{ + auto iter = streams_.find(stream); + if (iter == streams_.end()) + return -EINVAL; + + return dynamic_cast(iter->second.get())->applyVertexMap(request); +} + +Dw100VertexMap &ConverterDW100::vertexMap(const Stream *stream) +{ + auto iter = streams_.find(stream); + V4L2M2MConverter::V4L2M2MStream *s = iter->second.get(); + ASSERT(s); + return dynamic_cast(s)->vertexMap_; +} + void ConverterDW100::validateSize(Size *size) const { if (size) size->alignDownTo(8, 8); } +std::unique_ptr ConverterDW100::makeStream(const Stream *stream) +{ + return std::unique_ptr(new DW100Stream(this, stream)); +} + +ConverterDW100::DW100Stream::DW100Stream(V4L2M2MConverter *converter, + const Stream *stream) + : V4L2M2MConverter::V4L2M2MStream(converter, stream) +{ +} + +int ConverterDW100::DW100Stream::configure(const StreamConfiguration &inputCfg, + const StreamConfiguration &outputCfg) +{ + int ret; + ret = V4L2M2MConverter::V4L2M2MStream::configure(inputCfg, outputCfg); + if (ret) + return ret; + + /* Todo: is that comment still valid? + For actual dewarping we need the active area of the sensor mode here. */ + vertexMap_.setInputSize(inputCfg.size); + vertexMap_.setOutputSize(outputCfg.size); + return 0; +} + +int ConverterDW100::DW100Stream::applyVertexMap(const V4L2Request *request) +{ + std::vector map = vertexMap_.getVertexMap(); + auto value = Span(reinterpret_cast(&map[0]), map.size()); + + ControlList ctrls; + ctrls.set(V4L2_CID_DW100_DEWARPING_16x16_VERTEX_MAP, value); + return applyControls(ctrls, request); +} + } /* namespace libcamera */ From patchwork Tue Sep 30 12:26:43 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24521 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5D2ACC324C for ; Tue, 30 Sep 2025 13:20:10 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7764C6B5F8; Tue, 30 Sep 2025 15:20:09 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="fUm5Y2Su"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DD98C62C35 for ; Tue, 30 Sep 2025 15:20:06 +0200 (CEST) Received: from ideasonboard.com (unknown [94.31.94.171]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 7009816A; Tue, 30 Sep 2025 15:18:38 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1759238318; bh=0aKYFG4cW039XbhezNJ+c84C/W8TgmA48/Y7RpLjscw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fUm5Y2SuGU+hld8vPvAY9bo/see2vkLlfdook+pVNntkm/0HGYQRkDvKlxd9ncdVn Ufn7P6iElY9oFih5r4wd8ubVcwaHsFkLh5peC24nhB/APzC03nZT4TAez4GePrFr3r 4dqvXoINZ0NIhebF/YevVyZnQlL4sKEFb4YtCBVk= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 22/33] libcamera: rkisp1: Use the dw100 class instead of the generic v4l2 converter Date: Tue, 30 Sep 2025 14:26:43 +0200 Message-ID: <20250930122726.1837524-23-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> References: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" To be able to access the dw100 specific functions and as there is no known other dewarper in combination with the rkisp1, use the dw100 specific converter directly. Signed-off-by: Stefan Klug --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 61eec1bb8c44..8b78c7f213f6 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -36,7 +36,7 @@ #include "libcamera/internal/camera.h" #include "libcamera/internal/camera_sensor.h" #include "libcamera/internal/camera_sensor_properties.h" -#include "libcamera/internal/converter/converter_v4l2_m2m.h" +#include "libcamera/internal/converter/converter_dw100.h" #include "libcamera/internal/delayed_controls.h" #include "libcamera/internal/device_enumerator.h" #include "libcamera/internal/framebuffer.h" @@ -234,7 +234,7 @@ private: RkISP1MainPath mainPath_; RkISP1SelfPath selfPath_; - std::unique_ptr dewarper_; + std::unique_ptr dewarper_; Rectangle scalerMaxCrop_; std::optional activeCrop_; @@ -1505,7 +1505,7 @@ bool PipelineHandlerRkISP1::match(DeviceEnumerator *enumerator) std::shared_ptr dwpMediaDevice = enumerator->search(dwp); if (dwpMediaDevice) { - dewarper_ = std::make_unique(dwpMediaDevice); + dewarper_ = std::make_unique(dwpMediaDevice); if (dewarper_->isValid()) { dewarper_->outputBufferReady.connect( this, &PipelineHandlerRkISP1::dewarpBufferReady); From patchwork Tue Sep 30 12:26:44 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24522 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 36EB2C328C for ; Tue, 30 Sep 2025 13:22:26 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 40A9E6B5FB; Tue, 30 Sep 2025 15:22:25 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="sFerUjzC"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 54D1162C35 for ; Tue, 30 Sep 2025 15:22:23 +0200 (CEST) Received: from ideasonboard.com (unknown [94.31.94.171]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id B326416A; Tue, 30 Sep 2025 15:20:54 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1759238454; bh=7gVhXc/6VudosqCazp5WIjZFQ5QB+EbIqSMNrUWXtLQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sFerUjzCU0he9WntMzFEOZwAy3vFR/Go5F0Xx4pXJNuyMnpkzbaWNJBhuh1nXjQo1 fEU/p15RMS6XV7tqzDsb5kbFgnqF9uriKEQcQMW5z0Ltr5CcCCwmMnT/kE7e+/uzQe 7qfJP4vSKwx5dqffECzQwonbBpYzw3hXVI7xwsJs= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 23/33] libcamera: rkisp1: Implement dw100 specific features Date: Tue, 30 Sep 2025 14:26:44 +0200 Message-ID: <20250930122726.1837524-24-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> References: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The dw100 allows more features implemented in the dw100 vertex map. Implement these features for the rkisp1 pipeline. Signed-off-by: Stefan Klug --- src/libcamera/control_ids_draft.yaml | 39 ++++++++++++++++++- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 48 +++++++++++++++++++++++- 2 files changed, 85 insertions(+), 2 deletions(-) diff --git a/src/libcamera/control_ids_draft.yaml b/src/libcamera/control_ids_draft.yaml index 03309eeac34f..10ee68db6d8c 100644 --- a/src/libcamera/control_ids_draft.yaml +++ b/src/libcamera/control_ids_draft.yaml @@ -206,7 +206,44 @@ controls: available only on this camera device are at least this numeric value. All of the custom test patterns will be static (that is the raw image must not vary from frame to frame). - + - Dw100ScaleMode: + type: int32_t + direction: inout + description: | + Scale mode of the dewarper. + enum: + - name: Fill + value: 0 + description: | + Fills the given output size with the largest rectangle possible. + Aspect ratio is not preserved. Dw100Scale and Dw100Offset are + ignored. + - name: Crop + value: 1 + description: | + Crops to the given output size. The scale factor can be specified + using Dw100Scale. Aspect ratio is preserved. + - Dw100Scale: + type: float + direction: inout + description: | + Scale factor applied to the image when Dw100ScaleMode is set to Crop. + This value is clamped, so that all pixels have valid input data. + Therefore a value of 0 always provides the largest possible field of + view. + - Dw100Rotation: + type: float + direction: inout + description: | + Rotates the image by the given angle in degrees. + - Dw100Offset: + type: Point + direction: inout + description: | + Moves the image by the given values in x and y direction in output + coordinate space. This is clamped, so that all output pixels contain + valid data. The offset is therefore ignored when Dw100ScaleMode is set + to 'Fit' or the Dw100Scale value is too small. - FaceDetectMode: type: int32_t direction: inout diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 8b78c7f213f6..740791ac9c02 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -1364,6 +1364,17 @@ int PipelineHandlerRkISP1::updateControls(RkISP1CameraData *data) scalerMaxCrop_); data->properties_.set(properties::ScalerCropMaximum, scalerMaxCrop_); activeCrop_ = scalerMaxCrop_; + + if (dewarper_->supportsRequests()) { + controls[&controls::draft::Dw100Scale] = ControlInfo(0.2f, 8.0f, 1.0f); + controls[&controls::draft::Dw100Rotation] = ControlInfo(-180.0f, 180.0f, 0.0f); + controls[&controls::draft::Dw100Offset] = ControlInfo(Point(-10000, -10000), Point(10000, 10000), Point(0, 0)); + controls[&controls::draft::Dw100ScaleMode] = ControlInfo(controls::draft::Dw100ScaleModeValues, controls::draft::Fill); + } else { + LOG(RkISP1, Warning) + << "dw100 kernel driver has no requests support." + " No dynamic configuration possible."; + } } /* Add the IPA registered controls to list of camera controls. */ @@ -1637,6 +1648,37 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) availableDewarpRequests_.pop(); } + bool update = false; + auto &vertexMap = dewarper_->vertexMap(&data->mainPathStream_); + + const auto &scale = request->controls().get(controls::draft::Dw100Scale); + if (scale) { + vertexMap.setScale(*scale); + update = true; + } + + const auto &rotation = request->controls().get(controls::draft::Dw100Rotation); + if (rotation) { + vertexMap.setRotation(*rotation); + update = true; + } + + const auto &offset = request->controls().get(controls::draft::Dw100Offset); + if (offset) { + vertexMap.setOffset(*offset); + update = true; + } + + const auto &scaleMode = request->controls().get(controls::draft::Dw100ScaleMode); + if (scaleMode) { + vertexMap.setMode(static_cast(*scaleMode)); + update = true; + } + + if (update || info->frame == 0) { + dewarper_->applyVertexMap(&data->mainPathStream_, dewarpRequest); + } + /* Handle scaler crop control. */ const auto &crop = request->controls().get(controls::ScalerCrop); if (crop) { @@ -1700,7 +1742,11 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) } } - request->metadata().set(controls::ScalerCrop, activeCrop_.value()); + auto &meta = request->metadata(); + meta.set(controls::draft::Dw100Scale, vertexMap.effectiveScale()); + meta.set(controls::draft::Dw100Rotation, vertexMap.rotation()); + meta.set(controls::draft::Dw100Offset, vertexMap.effectiveOffset()); + meta.set(controls::ScalerCrop, activeCrop_.value()); } void PipelineHandlerRkISP1::dewarpRequestReady(V4L2Request *request) From patchwork Tue Sep 30 12:26:45 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24523 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B9252C328C for ; Tue, 30 Sep 2025 13:24:43 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id ED8436B5F3; Tue, 30 Sep 2025 15:24:42 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="A69g1Y2X"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BE29562C35 for ; Tue, 30 Sep 2025 15:24:41 +0200 (CEST) Received: from ideasonboard.com (unknown [94.31.94.171]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 5960B226; Tue, 30 Sep 2025 15:23:13 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1759238593; bh=IGSYJPNbfIhqPySXe9VsD4deKqwMpIi5GaiOcVc4Qvs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=A69g1Y2XQ0smoiu9eHcCue3ItqxwDX6UPSx/HlP+t7CEvhfso2dig5oCgb/ZMhovb +Kgea+RIMPA+Iap5GShcncjjt5QGvzXD6FIH6MR0I+cf0sDIRL2ddTspq74SEus4dC UUEumrdInfXPIBlgLAgNDSQTuLYMvYg6KV2jXk20= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 24/33] libcamera: rkisp1: Use vertex map to implement ScalerCrop Date: Tue, 30 Sep 2025 14:26:45 +0200 Message-ID: <20250930122726.1837524-25-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> References: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The input crop implemented in the dw100 kernel driver is quite limited in that it doesn't allow arbitrary crop rectangles but only scale factors quantized to the underlying fixed point representation and only aspect ratio preserving crops. The vertex map based implementation allows for pixel perfect crops. The only downside is that ScalerCrop can no longer be set dynamically on older kernels. Warn if the user tries to set ScalerCrop on such a setup. As the vertex map is now set on start() it no longer needs to be updated for frame==0. Signed-off-by: Stefan Klug --- Changes in 0.9 - Code cleanup - Update vertex map before start() to partially support old kernels Changes in 0.7: - Removed double call to applyVertexMap() --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 118 +++++++++++------------ 1 file changed, 59 insertions(+), 59 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 740791ac9c02..e8902ce66b70 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -235,9 +235,6 @@ private: RkISP1SelfPath selfPath_; std::unique_ptr dewarper_; - Rectangle scalerMaxCrop_; - - std::optional activeCrop_; /* Internal buffers used when dewarper is being used */ std::vector> mainPathBuffers_; @@ -969,13 +966,22 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) return ret; /* - * Calculate the crop rectangle of the data - * flowing into the dewarper in sensor - * coordinates. + * Apply the actual sensor crop, for proper + * dewarp map calculation + */ + Rectangle sensorCrop = outputCrop.transformedBetween( + inputCrop, sensorInfo.analogCrop); + auto &vertexMap = dewarper_->vertexMap(cfg.stream()); + vertexMap.setSensorCrop(sensorCrop); + data->properties_.set(properties::ScalerCropMaximum, sensorCrop); + + /* + * Apply a default sensor crop that keeps the + * aspect ratio. */ - scalerMaxCrop_ = - outputCrop.transformedBetween(inputCrop, - sensorInfo.analogCrop); + Size crop = format.size.boundedToAspectRatio(cfg.size); + vertexMap.setScalerCrop(crop.centeredTo( + Rectangle(format.size).center())); } ret = mainPath_.configure(ispCfg, format); @@ -1191,6 +1197,18 @@ int PipelineHandlerRkISP1::start(Camera *camera, [[maybe_unused]] const ControlL actions += [&]() { stat_->streamOff(); }; if (data->usesDewarper_) { + /* + * Apply the vertex map before start to partially + * support ScalerCrop on kernels with a dw100 driver + * that has no dynamic vertex map/requests support. + */ + if (controls && controls->contains(controls::ScalerCrop.id())) { + const auto &crop = controls->get(controls::ScalerCrop); + auto &vertexMap = dewarper_->vertexMap(&data->mainPathStream_); + vertexMap.setScalerCrop(*crop); + } + dewarper_->applyVertexMap(&data->mainPathStream_); + ret = dewarper_->start(); if (ret) { LOG(RkISP1, Error) << "Failed to start dewarper"; @@ -1345,25 +1363,31 @@ int PipelineHandlerRkISP1::updateControls(RkISP1CameraData *data) if (data->usesDewarper_) { std::pair cropLimits; - if (dewarper_->isConfigured(&data->mainPathStream_)) - cropLimits = dewarper_->inputCropBounds(&data->mainPathStream_); - else + if (dewarper_->isConfigured(&data->mainPathStream_)) { + auto &vertexMap = dewarper_->vertexMap(&data->mainPathStream_); + vertexMap.applyLimits(); + cropLimits = vertexMap.scalerCropBounds(); + controls[&controls::ScalerCrop] = ControlInfo(cropLimits.first, + cropLimits.second, + vertexMap.effectiveScalerCrop()); + } else { + /* This happens before configure() has run. Maybe we need a better solution.*/ + /* + * ScalerCrop is specified to be in Sensor coordinates. + * So we need to transform the limits to sensor coordinates. + * We can safely assume that the maximum crop limit contains the + * full fov of the dewarper. + */ cropLimits = dewarper_->inputCropBounds(); + Rectangle maxCrop = Rectangle(data->sensor_->resolution()); + Rectangle min = cropLimits.first.transformedBetween(cropLimits.second, + maxCrop); - /* - * ScalerCrop is specified to be in Sensor coordinates. - * So we need to transform the limits to sensor coordinates. - * We can safely assume that the maximum crop limit contains the - * full fov of the dewarper. - */ - Rectangle min = cropLimits.first.transformedBetween(cropLimits.second, - scalerMaxCrop_); + controls[&controls::ScalerCrop] = ControlInfo(min, + maxCrop, + maxCrop); + } - controls[&controls::ScalerCrop] = ControlInfo(min, - scalerMaxCrop_, - scalerMaxCrop_); - data->properties_.set(properties::ScalerCropMaximum, scalerMaxCrop_); - activeCrop_ = scalerMaxCrop_; if (dewarper_->supportsRequests()) { controls[&controls::draft::Dw100Scale] = ControlInfo(0.2f, 8.0f, 1.0f); @@ -1415,8 +1439,6 @@ int PipelineHandlerRkISP1::createCamera(MediaEntity *sensor) /* Initialize the camera properties. */ data->properties_ = data->sensor_->properties(); - scalerMaxCrop_ = Rectangle(data->sensor_->resolution()); - const CameraSensorProperties::SensorDelays &delays = data->sensor_->sensorDelays(); std::unordered_map params = { { V4L2_CID_ANALOGUE_GAIN, { delays.gainDelay, false } }, @@ -1675,42 +1697,20 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) update = true; } - if (update || info->frame == 0) { - dewarper_->applyVertexMap(&data->mainPathStream_, dewarpRequest); - } - /* Handle scaler crop control. */ const auto &crop = request->controls().get(controls::ScalerCrop); if (crop) { - Rectangle rect = crop.value(); - - /* - * ScalerCrop is specified to be in Sensor coordinates. - * So we need to transform it into dewarper coordinates. - * We can safely assume that the maximum crop limit contains the - * full fov of the dewarper. - */ - std::pair cropLimits = - dewarper_->inputCropBounds(&data->mainPathStream_); - - rect = rect.transformedBetween(scalerMaxCrop_, cropLimits.second); - int ret = dewarper_->setInputCrop(&data->mainPathStream_, - &rect); - rect = rect.transformedBetween(cropLimits.second, scalerMaxCrop_); - if (!ret && rect != crop.value()) { - /* - * If the rectangle is changed by setInputCrop on the - * dewarper, log a debug message and cache the actual - * applied rectangle for metadata reporting. - */ - LOG(RkISP1, Debug) - << "Applied rectangle " << rect.toString() - << " differs from requested " << crop.value().toString(); - } - - activeCrop_ = rect; + if (!dewarper_->supportsRequests()) + LOG(RkISP1, Error) + << "Dynamically setting ScalerCrop requires a " + "dw100 driver with requests support"; + vertexMap.setScalerCrop(*crop); + update = true; } + if (update) + dewarper_->applyVertexMap(&data->mainPathStream_, dewarpRequest); + /* * Queue input and output buffers to the dewarper. The output * buffers for the dewarper are the buffers of the request, supplied @@ -1746,7 +1746,7 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) meta.set(controls::draft::Dw100Scale, vertexMap.effectiveScale()); meta.set(controls::draft::Dw100Rotation, vertexMap.rotation()); meta.set(controls::draft::Dw100Offset, vertexMap.effectiveOffset()); - meta.set(controls::ScalerCrop, activeCrop_.value()); + meta.set(controls::ScalerCrop, vertexMap.effectiveScalerCrop()); } void PipelineHandlerRkISP1::dewarpRequestReady(V4L2Request *request) From patchwork Tue Sep 30 12:26:46 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24524 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 74EC3C324C for ; Tue, 30 Sep 2025 13:27:03 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5DF6E6B5F3; Tue, 30 Sep 2025 15:27:02 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ohE14hHX"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5AA3962C35 for ; Tue, 30 Sep 2025 15:27:00 +0200 (CEST) Received: from ideasonboard.com (unknown [94.31.94.171]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 1241516A; Tue, 30 Sep 2025 15:25:32 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1759238732; bh=kmpOB+ekNtmU6qe/0mAaXwEfiMpZ2boPqnqK7VWqS1k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ohE14hHXNt/q0PCQ1BPIUJrhBR5GRlm0uJPJY7J4i6nTuwNv1PIWOfcPdQQQydWms 5oWrSh/zXcGy30ZJaSX7ikgNYdz8IP+KWQfDlZPCg238QAqVZomsXGELlKDnbt+KK4 EfDyGpMZ5cBWz2Vqfm+VBGb1iEhsKnx83WcsH+zY= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 25/33] libcamera: internal: camera_sensor: Add accessor for mountingOrientation_ Date: Tue, 30 Sep 2025 14:26:46 +0200 Message-ID: <20250930122726.1837524-26-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> References: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" To properly handle the orientation in the dewarper, the mounting orientation of the sensor needs to be queryable. Add that. Signed-off-by: Stefan Klug --- include/libcamera/internal/camera_sensor.h | 1 + src/libcamera/sensor/camera_sensor_legacy.cpp | 1 + src/libcamera/sensor/camera_sensor_raw.cpp | 1 + 3 files changed, 3 insertions(+) diff --git a/include/libcamera/internal/camera_sensor.h b/include/libcamera/internal/camera_sensor.h index f6ef4df170d4..e6b72d22ac21 100644 --- a/include/libcamera/internal/camera_sensor.h +++ b/include/libcamera/internal/camera_sensor.h @@ -73,6 +73,7 @@ public: virtual int sensorInfo(IPACameraSensorInfo *info) const = 0; virtual Transform computeTransform(Orientation *orientation) const = 0; virtual BayerFormat::Order bayerOrder(Transform t) const = 0; + virtual Orientation mountingOrientation() const = 0; virtual const ControlInfoMap &controls() const = 0; virtual ControlList getControls(Span ids) = 0; diff --git a/src/libcamera/sensor/camera_sensor_legacy.cpp b/src/libcamera/sensor/camera_sensor_legacy.cpp index f9e685a9acc4..39c34200b0ff 100644 --- a/src/libcamera/sensor/camera_sensor_legacy.cpp +++ b/src/libcamera/sensor/camera_sensor_legacy.cpp @@ -88,6 +88,7 @@ public: int sensorInfo(IPACameraSensorInfo *info) const override; Transform computeTransform(Orientation *orientation) const override; BayerFormat::Order bayerOrder(Transform t) const override; + Orientation mountingOrientation() const override { return mountingOrientation_; } const ControlInfoMap &controls() const override; ControlList getControls(Span ids) override; diff --git a/src/libcamera/sensor/camera_sensor_raw.cpp b/src/libcamera/sensor/camera_sensor_raw.cpp index 8ea4423698cd..759cccafe4a9 100644 --- a/src/libcamera/sensor/camera_sensor_raw.cpp +++ b/src/libcamera/sensor/camera_sensor_raw.cpp @@ -94,6 +94,7 @@ public: int sensorInfo(IPACameraSensorInfo *info) const override; Transform computeTransform(Orientation *orientation) const override; BayerFormat::Order bayerOrder(Transform t) const override; + Orientation mountingOrientation() const override { return mountingOrientation_; } const ControlInfoMap &controls() const override; ControlList getControls(Span ids) override; From patchwork Tue Sep 30 12:26:47 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24525 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D3EC6C324C for ; Tue, 30 Sep 2025 13:29:19 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D891B6B5F0; Tue, 30 Sep 2025 15:29:18 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Z91/Ylb0"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7C01D62C35 for ; Tue, 30 Sep 2025 15:29:16 +0200 (CEST) Received: from ideasonboard.com (unknown [94.31.94.171]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 19B1316A; Tue, 30 Sep 2025 15:27:48 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1759238868; bh=S5kxuO0kE+eOLa3Pw+g2r8k/DuYrjE1NgSvbDcQ9I9c=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Z91/Ylb0YjZjHmZGAzagmZed48HIDNTS1sKuND6o/SbD6b6JNAIg5FotSr6HRG8KM vF28T+JgX2GFjSMaTWhuq83vUXnadsaQNItd5VisGxQ4wkNKJPTjyxbtjVh0/8335X 779WnACQXaKzLHmNnl76vscip3Bzx99goy3DeLSQ= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 26/33] libcamera: Add transpose() function to size Date: Tue, 30 Sep 2025 14:26:47 +0200 Message-ID: <20250930122726.1837524-27-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> References: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a transpose() function to size that applies the Transformation::Transpose operation in the size. This is useful when handling orientation adjustments. Signed-off-by: Stefan Klug Reviewed-by: Kieran Bingham --- include/libcamera/geometry.h | 6 ++++++ src/libcamera/geometry.cpp | 10 ++++++++++ 2 files changed, 16 insertions(+) diff --git a/include/libcamera/geometry.h b/include/libcamera/geometry.h index d9378efeca8c..572a07e2cf92 100644 --- a/include/libcamera/geometry.h +++ b/include/libcamera/geometry.h @@ -108,6 +108,12 @@ public: return *this; } + Size &transpose() + { + std::swap(width, height); + return *this; + } + [[nodiscard]] constexpr Size alignedDownTo(unsigned int hAlignment, unsigned int vAlignment) const { diff --git a/src/libcamera/geometry.cpp b/src/libcamera/geometry.cpp index de76d0c12a8c..2763967a6e57 100644 --- a/src/libcamera/geometry.cpp +++ b/src/libcamera/geometry.cpp @@ -209,6 +209,16 @@ std::string Size::toString() const * \return A reference to this object */ +/** + * \fn Size::transpose() + * \brief Transpose the size in place + * + * This function swaps width and height of this size. This effectively applies + * the \a Transform::Transpose transformation on this size. + * + * \return A reference to this object + */ + /** * \fn Size::alignedDownTo(unsigned int hAlignment, unsigned int vAlignment) * \brief Align the size down horizontally and vertically From patchwork Tue Sep 30 12:26:48 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24526 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 87C47C328C for ; Tue, 30 Sep 2025 13:31:38 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 484156B599; Tue, 30 Sep 2025 15:31:37 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ej7hp4N4"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3308D62C35 for ; Tue, 30 Sep 2025 15:31:35 +0200 (CEST) Received: from ideasonboard.com (unknown [94.31.94.171]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id E0274228; Tue, 30 Sep 2025 15:30:06 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1759239007; bh=nqlJJh22d5rsGn1yvoRpZ2weJ1i2js9fjhlM/L3T2pQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ej7hp4N4CiPsJyuwIzBWa4mET65zZ7NyIk+CgxqLpKX2CH3brqkridFDpMcPfdsoT 9woKM5eaTbt+EJzWlZxH4lM6fb7VnG7sdtC3COhcRw0d58ATmXXShOcM449D0kEFsi CBWhKQOwMQEsWkfdxYntGu/F146LPDNi4uREFy4o= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 27/33] libcamera: rkisp1: Handle requested orientation using dewarper Date: Tue, 30 Sep 2025 14:26:48 +0200 Message-ID: <20250930122726.1837524-28-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> References: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When the dewarper is present it can handle arbitrary orientations specified in the requested camera configuration. In that case handle all transformations inside the dewarper (even if the sensor supports some of them) because that makes it easier to handle coordinates for lens dewarping inside the dewarper. Signed-off-by: Stefan Klug --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 79 +++++++++++++++++++----- 1 file changed, 63 insertions(+), 16 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index e8902ce66b70..1046930857e1 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -550,11 +550,6 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() status = Adjusted; } - Orientation requestedOrientation = orientation; - combinedTransform_ = data_->sensor_->computeTransform(&orientation); - if (orientation != requestedOrientation) - status = Adjusted; - /* * Simultaneous capture of raw and processed streams isn't possible. If * there is any raw stream, cap the number of streams to one. @@ -570,6 +565,12 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() } } + /* + * If the dewarper supports orientation adjustments, apply that completely + * there. Even if the sensor supports flips, it is beneficial to do that + * in the dewarper so that lens dewarping happens on the unflipped image + */ + bool transposeAfterIsp = false; bool useDewarper = false; if (pipe->dewarper_) { /* @@ -580,6 +581,14 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() PixelFormatInfo::ColourEncodingRAW; if (!isRaw) useDewarper = true; + combinedTransform_ = orientation / data_->sensor_->mountingOrientation(); + if (!!(combinedTransform_ & Transform::Transpose)) + transposeAfterIsp = true; + } else { + Orientation requestedOrientation = orientation; + combinedTransform_ = data_->sensor_->computeTransform(&orientation); + if (orientation != requestedOrientation) + status = Adjusted; } /* @@ -610,6 +619,9 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() (expectedStatus == Valid && ret == Adjusted)) return false; + if (transposeAfterIsp) + tryCfg.size.transpose(); + if (useDewarper) { /* * The dewarper output is independent of the ISP path. @@ -694,6 +706,9 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() maxSize = std::max(maxSize, cfg.size); } + if (transposeAfterIsp) + maxSize.transpose(); + std::vector mbusCodes; if (rawFormat.isValid()) { @@ -739,6 +754,22 @@ PipelineHandlerRkISP1::generateConfiguration(Camera *camera, if (roles.empty()) return config; + Transform transform = Transform::Identity; + Size previewSize = kRkISP1PreviewSize; + bool transposeAfterIsp = false; + if (data->canUseDewarper_) { + transform = Orientation::Rotate0 / data->sensor_->mountingOrientation(); + if (!!(transform & Transform::Transpose)) + transposeAfterIsp = true; + } + + /* + * In case of a transpose transform we need to create a path for the + * transposed size. + */ + if (transposeAfterIsp) + previewSize.transpose(); + /* * As the ISP can't output different color spaces for the main and self * path, pick a sensible default color space based on the role of the @@ -767,7 +798,7 @@ PipelineHandlerRkISP1::generateConfiguration(Camera *camera, if (!colorSpace) colorSpace = ColorSpace::Sycc; - size = kRkISP1PreviewSize; + size = previewSize; break; case StreamRole::VideoRecording: @@ -775,7 +806,7 @@ PipelineHandlerRkISP1::generateConfiguration(Camera *camera, if (!colorSpace) colorSpace = ColorSpace::Rec709; - size = kRkISP1PreviewSize; + size = previewSize; break; case StreamRole::Raw: @@ -817,6 +848,9 @@ PipelineHandlerRkISP1::generateConfiguration(Camera *camera, if (!cfg.pixelFormat.isValid()) return nullptr; + if (transposeAfterIsp && role != StreamRole::Raw) + cfg.size.transpose(); + cfg.colorSpace = colorSpace; cfg.bufferCount = kRkISP1MinBufferCount; config->addConfiguration(cfg); @@ -839,6 +873,19 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) if (ret) return ret; + const PixelFormat &streamFormat = config->at(0).pixelFormat; + const PixelFormatInfo &info = PixelFormatInfo::info(streamFormat); + isRaw_ = info.colourEncoding == PixelFormatInfo::ColourEncodingRAW; + data->usesDewarper_ = data->canUseDewarper_ && !isRaw_; + + Transform transform = config->combinedTransform(); + bool transposeAfterIsp = false; + if (data->usesDewarper_) { + if (!!(transform & Transform::Transpose)) + transposeAfterIsp = true; + transform = Transform::Identity; + } + /* * Configure the format on the sensor output and propagate it through * the pipeline. @@ -848,10 +895,10 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) if (config->sensorConfig) ret = sensor->applyConfiguration(*config->sensorConfig, - config->combinedTransform(), + transform, &format); else - ret = sensor->setFormat(&format, config->combinedTransform()); + ret = sensor->setFormat(&format, transform); if (ret < 0) return ret; @@ -878,10 +925,6 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) << " crop " << inputCrop; Rectangle outputCrop = inputCrop; - const PixelFormat &streamFormat = config->at(0).pixelFormat; - const PixelFormatInfo &info = PixelFormatInfo::info(streamFormat); - isRaw_ = info.colourEncoding == PixelFormatInfo::ColourEncodingRAW; - data->usesDewarper_ = dewarper_ && !isRaw_; /* YUYV8_2X8 is required on the ISP source path pad for YUV output. */ if (!isRaw_) @@ -973,15 +1016,19 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) inputCrop, sensorInfo.analogCrop); auto &vertexMap = dewarper_->vertexMap(cfg.stream()); vertexMap.setSensorCrop(sensorCrop); + vertexMap.setTransform(config->combinedTransform()); data->properties_.set(properties::ScalerCropMaximum, sensorCrop); /* * Apply a default sensor crop that keeps the * aspect ratio. */ - Size crop = format.size.boundedToAspectRatio(cfg.size); - vertexMap.setScalerCrop(crop.centeredTo( - Rectangle(format.size).center())); + Size size = cfg.size; + if (transposeAfterIsp) + size.transpose(); + size = sensorCrop.size().boundedToAspectRatio(size); + vertexMap.setScalerCrop( + size.centeredTo(sensorCrop.center())); } ret = mainPath_.configure(ispCfg, format); From patchwork Tue Sep 30 12:26:49 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24527 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0BC29C328C for ; Tue, 30 Sep 2025 13:32:02 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id ABB676B5F0; Tue, 30 Sep 2025 15:32:01 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Ox2pzvk+"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 366626B599 for ; Tue, 30 Sep 2025 15:31:59 +0200 (CEST) Received: from ideasonboard.com (unknown [94.31.94.171]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id F3C7E16A; Tue, 30 Sep 2025 15:30:30 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1759239031; bh=ik1IurfUDpB0uQeOKgQlWDtvXgq2Tn5bYtZcu9frmRo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Ox2pzvk+ubytkKJwdyZKMvX2oidZxP3+ICCJggwaPa3hN9hrXQPxojB+Fc5ItlNaf CIuQqZfPtQpHQ+zhrTWxp4NHItZhi7VIMmVJ1jSrWT7qulHidhbV910+urTX6zPe5l 3jcXKsubXpCBBb1alPXnMLfnrUurH1uI0reHry5k= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 28/33] libcamera: dw100_vertexmap: Implement parametric dewarping Date: Tue, 30 Sep 2025 14:26:49 +0200 Message-ID: <20250930122726.1837524-29-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> References: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement functions to allow lens dewarping based on the common lens dewarp model used e.g. by OpenCV See https://docs.opencv.org/3.4/da/d54/group__imgproc__transform.html#ga7dfb72c9cf9780a347fbe3d1c47e5d5a Signed-off-by: Stefan Klug --- .../converter/converter_dw100_vertexmap.h | 17 ++++ .../converter/converter_dw100_vertexmap.cpp | 87 +++++++++++++++++++ 2 files changed, 104 insertions(+) diff --git a/include/libcamera/internal/converter/converter_dw100_vertexmap.h b/include/libcamera/internal/converter/converter_dw100_vertexmap.h index 9c6e0ffa4513..7a12c4cb7a50 100644 --- a/include/libcamera/internal/converter/converter_dw100_vertexmap.h +++ b/include/libcamera/internal/converter/converter_dw100_vertexmap.h @@ -12,6 +12,9 @@ #include #include +#include "libcamera/internal/matrix.h" +#include "libcamera/internal/vector.h" + namespace libcamera { class Dw100VertexMap @@ -57,11 +60,20 @@ public: void setMode(const ScaleMode mode) { mode_ = mode; } ScaleMode mode() const { return mode_; } + int loadDewarpParams(const YamlObject &dict); + int setDewarpParams(const Matrix &cm, const Span &coeffs); + bool dewarpParamsValid() { return dewarpParamsValid_; } + + void setLensDewarpEnable(bool enable) { lensDewarpEnable_ = enable; } + bool lensDewarpEnable() { return lensDewarpEnable_; } + std::vector getVertexMap(); private: int getVerticesForLength(const int length); + Vector dewarpPoint(const Vector &p); + Rectangle scalerCrop_; Rectangle sensorCrop_; Transform transform_ = Transform::Identity; @@ -75,6 +87,11 @@ private: double effectiveScaleY_; Point effectiveOffset_; Rectangle effectiveScalerCrop_; + + Matrix dewarpM_ = Matrix::identity(); + std::array dewarpCoeffs_; + bool lensDewarpEnable_ = true; + bool dewarpParamsValid_ = false; }; } /* namespace libcamera */ diff --git a/src/libcamera/converter/converter_dw100_vertexmap.cpp b/src/libcamera/converter/converter_dw100_vertexmap.cpp index 10d9e34d98c5..5581d53e57c7 100644 --- a/src/libcamera/converter/converter_dw100_vertexmap.cpp +++ b/src/libcamera/converter/converter_dw100_vertexmap.cpp @@ -66,6 +66,14 @@ Vector2d point2Vec2d(const Point &p) return { { static_cast(p.x), static_cast(p.y) } }; } +void transformPoint(const Rectangle &from, const Rectangle &to, Vector2d &p) +{ + double sx = to.width / static_cast(from.width); + double sy = to.height / static_cast(from.height); + p.x() = (p.x() - from.x) * sx + to.x; + p.y() = (p.y() - from.y) * sy + to.y; +} + } /* namespace */ /** @@ -344,6 +352,17 @@ std::vector Dw100VertexMap::getVertexMap() p.x() += localScalerCrop.x; p.y() += localScalerCrop.y; + if (dewarpParamsValid_ && lensDewarpEnable_) { + /* + * We are in localScalerCrop coordinates. + * Tranform to sensor coordinates. + */ + transformPoint(localScalerCrop, effectiveScalerCrop_, p); + p = dewarpPoint(p); + /* tarnsform back to localScalerCrop */ + transformPoint(effectiveScalerCrop_, localScalerCrop, p); + } + /* Convert to fixed point */ uint32_t v = static_cast(p.y() * 16) << 16 | (static_cast(p.x() * 16) & 0xffff); @@ -354,6 +373,74 @@ std::vector Dw100VertexMap::getVertexMap() return res; } +int Dw100VertexMap::loadDewarpParams(const YamlObject &dict) +{ + Matrix m; + const auto &cm = dict["cm"].get>(); + if (!cm) { + LOG(Converter, Error) << "Dewarp parameters are missing 'cm' value"; + return -EINVAL; + } + + m = *cm; + + const auto &coeffs = dict["coefficients"].getList(); + if (!coeffs) { + LOG(Converter, Error) << "Dewarp parameters 'coefficients' value is not a list"; + return -EINVAL; + } + + const Span span{ *coeffs }; + return setDewarpParams(m, span); +} + +int Dw100VertexMap::setDewarpParams(const Matrix &cm, const Span &coeffs) +{ + dewarpM_ = cm; + dewarpCoeffs_.fill(0.0); + + if (coeffs.size() != 4 && coeffs.size() != 5 && + coeffs.size() != 8 && coeffs.size() != 12) { + LOG(Converter, Error) << "Dewarp 'coefficients' must have 4, 5, 8 or 12 values"; + dewarpParamsValid_ = false; + return -EINVAL; + } + std::copy(coeffs.begin(), coeffs.end(), dewarpCoeffs_.begin()); + + dewarpParamsValid_ = true; + return 0; +} + +Vector2d Dw100VertexMap::dewarpPoint(const Vector2d &p) +{ + double x, y; + double k1 = dewarpCoeffs_[0]; + double k2 = dewarpCoeffs_[1]; + double p1 = dewarpCoeffs_[2]; + double p2 = dewarpCoeffs_[3]; + double k3 = dewarpCoeffs_[4]; + double k4 = dewarpCoeffs_[5]; + double k5 = dewarpCoeffs_[6]; + double k6 = dewarpCoeffs_[7]; + double s1 = dewarpCoeffs_[8]; + double s2 = dewarpCoeffs_[9]; + double s3 = dewarpCoeffs_[10]; + double s4 = dewarpCoeffs_[11]; + + y = (p.y() - dewarpM_[1][2]) / dewarpM_[1][1]; + x = (p.x() - dewarpM_[0][2] - y * dewarpM_[0][1]) / dewarpM_[0][0]; + + double r2 = x * x + y * y; + double d = (1 + k1 * r2 + k2 * r2 * r2 + k3 * r2 * r2 * r2) / + (1 + k4 * r2 + k5 * r2 * r2 + k6 * r2 * r2 * r2); + x = x * d + 2 * p1 * x * y + p2 * (r2 + 2 * x * x) + s1 * r2 + s2 * r2 * r2; + y = y * d + 2 * p2 * x * y + p1 * (r2 + 2 * y * y) + s3 * r2 + s4 * r2 * r2; + + Vector2d ret{ { x * dewarpM_[0][0] + y * dewarpM_[0][1] + dewarpM_[0][2], + y * dewarpM_[1][1] + dewarpM_[1][2] } }; + return ret; +} + int Dw100VertexMap::getVerticesForLength(const int length) { return (length + kDw100BlockSize - 1) / kDw100BlockSize + 1; From patchwork Tue Sep 30 12:26:50 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24528 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D1EABC328C for ; Tue, 30 Sep 2025 13:34:18 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E7CE76B5F0; Tue, 30 Sep 2025 15:34:17 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="lovTQ5pR"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1967C62C35 for ; Tue, 30 Sep 2025 15:34:15 +0200 (CEST) Received: from ideasonboard.com (unknown [94.31.94.171]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id BEA7E16A; Tue, 30 Sep 2025 15:32:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1759239166; bh=eauvXfNddQH20Q9V/RZ1gG3g7AsZ7qaeEL4sbl+f9T0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lovTQ5pRrtX7I7HetqOL6HI6jeqGhizq8S+e2ck2MItTYqaFraqLuWY2XDOhzf7Uy WELGZ48gckYOas5/db7Lk61ac5RfxdCVCEdO7M4ugPJ7KduaW2/svR1JvCNA0RKIUT DuOT+GmrAuElMKoFVHwxV4d98Bn5f6CMD1Je2PLU= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 29/33] pipeline: rkisp1: Enable the dewarper based on the tuning file Date: Tue, 30 Sep 2025 14:26:50 +0200 Message-ID: <20250930122726.1837524-30-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> References: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" To do actual lens dewarping, the dewarper will be configured based on the tuning file. As a first step implement the basic loading of the tuning file and enable/disable the dewarper for the given camera based on the existence of the "Dewarp" algorithm in the tuning file. Todo: This is an backwards incompatible change in that the dewarper is currently included in the chain unconditionally. Some users may want to not use the dewarper, so it is sensible to make that configurable. If it should be in the algorithms section or in a different one is open for debate. Signed-off-by: Stefan Klug --- src/ipa/libipa/module.h | 4 ++ src/libcamera/pipeline/rkisp1/rkisp1.cpp | 51 +++++++++++++++++++++++- 2 files changed, 54 insertions(+), 1 deletion(-) diff --git a/src/ipa/libipa/module.h b/src/ipa/libipa/module.h index 0fb51916fff6..84386f901534 100644 --- a/src/ipa/libipa/module.h +++ b/src/ipa/libipa/module.h @@ -74,6 +74,10 @@ private: int createAlgorithm(Context &context, const YamlObject &data) { const auto &[name, algoData] = *data.asDict().begin(); + + if (name == "Dewarp") + return 0; + std::unique_ptr> algo = createAlgorithm(name); if (!algo) { LOG(IPAModuleAlgo, Error) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 1046930857e1..07dd5dc8d99a 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -16,6 +16,7 @@ #include #include +#include #include #include @@ -47,6 +48,7 @@ #include "libcamera/internal/v4l2_request.h" #include "libcamera/internal/v4l2_subdevice.h" #include "libcamera/internal/v4l2_videodevice.h" +#include "libcamera/internal/yaml_parser.h" #include "rkisp1_path.h" @@ -123,6 +125,7 @@ public: */ MediaPipeline pipe_; + bool canUseDewarper_; bool usesDewarper_; private: @@ -131,6 +134,7 @@ private: const ControlList &sensorControls); void metadataReady(unsigned int frame, const ControlList &metadata); + int loadTuningFile(const std::string &file); }; class RkISP1CameraConfiguration : public CameraConfiguration @@ -416,6 +420,51 @@ int RkISP1CameraData::loadIPA(unsigned int hwRevision, uint32_t supportedBlocks) return ret; } + ret = loadTuningFile(ipaTuningFile); + if (ret < 0) { + LOG(RkISP1, Error) << "Failed to load tuning file"; + return ret; + } + + return 0; +} + +int RkISP1CameraData::loadTuningFile(const std::string &path) +{ + if (!pipe()->dewarper_) + /* Nothing to do without dewarper */ + return 0; + + LOG(RkISP1, Debug) << "Load tuning file " << path; + + File file(path); + if (!file.open(File::OpenModeFlag::ReadOnly)) { + int ret = file.error(); + LOG(RkISP1, Error) + << "Failed to open tuning file " + << path << ": " << strerror(-ret); + return ret; + } + + std::unique_ptr data = YamlParser::parse(file); + if (!data) + return -EINVAL; + + if (!data->contains("algorithms")) { + LOG(RkISP1, Error) + << "Tuning file doesn't contain any algorithm"; + return -EINVAL; + } + + const auto &algos = (*data)["algorithms"].asList(); + for (const auto &algo : algos) { + if (algo.contains("Dewarp")) { + const auto ¶ms = algo["Dewarp"]; + + canUseDewarper_ = true; + } + } + return 0; } @@ -572,7 +621,7 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() */ bool transposeAfterIsp = false; bool useDewarper = false; - if (pipe->dewarper_) { + if (data_->canUseDewarper_) { /* * Platforms with dewarper support, such as i.MX8MP, support * only a single stream. We can inspect config_[0] only here. From patchwork Tue Sep 30 12:26:51 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24529 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 80F66C324C for ; Tue, 30 Sep 2025 13:36:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6EEDF6B5F0; Tue, 30 Sep 2025 15:36:36 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ZE94ao2+"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E6EBF62C35 for ; Tue, 30 Sep 2025 15:36:34 +0200 (CEST) Received: from ideasonboard.com (unknown [94.31.94.171]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id A3852226; Tue, 30 Sep 2025 15:35:06 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1759239306; bh=1dktPjzZRlSD7DZj7eky+F8Y3+QEKO3Os8t0DGfV+SQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ZE94ao2+8mkk8TgUqUaDBDO11rGZXcJ7TtR7VmiLl/q7HcRv8aAKcsXzxZ1V3/FUK PoLo4VMR1a3AQnSm4k33KmCI7VEQlZQRAJ5txOKoPU+GSS9BFc1tP4D3iPidfxlYsp lj3wKPMRfAL/Ry2naPX9pmQFG6KELvOICQqfS3Rk= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 30/33] pipeline: rkisp1: Load dewarp parameters from tuning file Date: Tue, 30 Sep 2025 14:26:51 +0200 Message-ID: <20250930122726.1837524-31-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> References: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Load the dewarp parameters from the tuning file. Signed-off-by: Stefan Klug --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 30 ++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 07dd5dc8d99a..13d82c9957bc 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -125,6 +125,11 @@ public: */ MediaPipeline pipe_; + struct DewarpParms { + Matrix cm; + std::vector coeffs; + }; + std::optional dewarpParams_; bool canUseDewarper_; bool usesDewarper_; @@ -462,6 +467,27 @@ int RkISP1CameraData::loadTuningFile(const std::string &path) const auto ¶ms = algo["Dewarp"]; canUseDewarper_ = true; + Matrix m; + DewarpParms dp; + if (params["cm"]) { + const auto &cm = params["cm"].get>(); + if (!cm) { + LOG(RkISP1, Error) << "Dewarp parameters are missing 'cm' value"; + return -EINVAL; + } + dp.cm = *cm; + } + + if (params["coefficients"]) { + const auto &coeffs = params["coefficients"].getList(); + if (!coeffs) { + LOG(RkISP1, Error) << "Dewarp parameters 'coefficients' value is not a list"; + return -EINVAL; + } + dp.coeffs = *coeffs; + } + + dewarpParams_ = dp; } } @@ -1066,6 +1092,10 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) auto &vertexMap = dewarper_->vertexMap(cfg.stream()); vertexMap.setSensorCrop(sensorCrop); vertexMap.setTransform(config->combinedTransform()); + if (data->dewarpParams_) { + vertexMap.setDewarpParams(data->dewarpParams_->cm, + data->dewarpParams_->coeffs); + } data->properties_.set(properties::ScalerCropMaximum, sensorCrop); /* From patchwork Tue Sep 30 12:26:52 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24530 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 18B6CC324C for ; Tue, 30 Sep 2025 13:37:18 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C885C6B5FB; Tue, 30 Sep 2025 15:37:17 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="a8V3e9Ka"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 582266B599 for ; Tue, 30 Sep 2025 15:37:16 +0200 (CEST) Received: from ideasonboard.com (unknown [94.31.94.171]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 215DE16A; Tue, 30 Sep 2025 15:35:48 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1759239348; bh=u7YrCN8hDJnK+RlPnsKkCqYoV6UQ6gN2D8+nsX8Ttm4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=a8V3e9KaReKLcu0F92QbNgFY19vRtNcLDTn/PEw1nQpyf/1mZ7uWxEg5lXDkjO9ji s/UU2dASvJ8xqMCsn8zWdOEK41YYbH31WkfTOlxjyTDCFa+FLp4op9fpyKIomsSTWR afzdC3zEE9tsb+yPDhYjwzQfoVCPfNCBeB0Rkjj8= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 31/33] libcamera: Add LensDewarpEnable control Date: Tue, 30 Sep 2025 14:26:52 +0200 Message-ID: <20250930122726.1837524-32-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> References: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a LensDewarpEnable control to enable or disbale lens dewarping if it is configured. Signed-off-by: Stefan Klug --- src/libcamera/control_ids_core.yaml | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/src/libcamera/control_ids_core.yaml b/src/libcamera/control_ids_core.yaml index f781865859ac..d17c566cfba5 100644 --- a/src/libcamera/control_ids_core.yaml +++ b/src/libcamera/control_ids_core.yaml @@ -1346,4 +1346,10 @@ controls: reduces the WdrExposureValue until the amount of pixels that are close to saturation is lower than this value. + - LensDewarpEnable: + type: bool + direction: inout + description: | + Enable or disable the Lens dewarping. + ... From patchwork Tue Sep 30 12:26:53 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24531 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CAAF8C324C for ; Tue, 30 Sep 2025 13:39:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BFC656B5F0; Tue, 30 Sep 2025 15:39:36 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="sGLql67Z"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C4B0062C35 for ; Tue, 30 Sep 2025 15:39:34 +0200 (CEST) Received: from ideasonboard.com (unknown [94.31.94.171]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 7DBFD16A; Tue, 30 Sep 2025 15:38:06 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1759239486; bh=HYR0dFKeUd4S6eti4IokHq1N4OQaEVFRHX4DAIOKSxs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sGLql67ZUULZGeD/X8/z0Cbvg+OA0BTIhjm2LG3YeOjnvjXBnhJBW/9WjeQDC22j3 5kbUUPTaEalnYwuCp7jg807flGm3QAw4zMBVeavlImZsCr+ibaL0sK21qi/1QErb6j WS2YHysi0bBgC2OrtLTxS8hHX4iqZYno1mwGZylo= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 32/33] libcamera: rkisp1: Implement LensDewarpEnable control Date: Tue, 30 Sep 2025 14:26:53 +0200 Message-ID: <20250930122726.1837524-33-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> References: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement the LensDewarpEnable control for the rkisp1 pipeline using the dw100 converter. Signed-off-by: Stefan Klug --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 13d82c9957bc..98521c804d84 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -1514,12 +1514,13 @@ int PipelineHandlerRkISP1::updateControls(RkISP1CameraData *data) maxCrop); } - if (dewarper_->supportsRequests()) { controls[&controls::draft::Dw100Scale] = ControlInfo(0.2f, 8.0f, 1.0f); controls[&controls::draft::Dw100Rotation] = ControlInfo(-180.0f, 180.0f, 0.0f); controls[&controls::draft::Dw100Offset] = ControlInfo(Point(-10000, -10000), Point(10000, 10000), Point(0, 0)); controls[&controls::draft::Dw100ScaleMode] = ControlInfo(controls::draft::Dw100ScaleModeValues, controls::draft::Fill); + if (data->dewarpParams_.has_value()) + controls[&controls::LensDewarpEnable] = ControlInfo(false, true, true); } else { LOG(RkISP1, Warning) << "dw100 kernel driver has no requests support." @@ -1873,6 +1874,9 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) meta.set(controls::draft::Dw100Rotation, vertexMap.rotation()); meta.set(controls::draft::Dw100Offset, vertexMap.effectiveOffset()); meta.set(controls::ScalerCrop, vertexMap.effectiveScalerCrop()); + + if (vertexMap.dewarpParamsValid()) + meta.set(controls::LensDewarpEnable, vertexMap.lensDewarpEnable()); } void PipelineHandlerRkISP1::dewarpRequestReady(V4L2Request *request) From patchwork Tue Sep 30 12:26:54 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24532 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 62A37C328C for ; Tue, 30 Sep 2025 13:41:55 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0D0876B5F0; Tue, 30 Sep 2025 15:41:55 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="kTLFc7Z6"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B695262C35 for ; Tue, 30 Sep 2025 15:41:53 +0200 (CEST) Received: from ideasonboard.com (unknown [94.31.94.171]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 7D07816A; Tue, 30 Sep 2025 15:40:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1759239625; bh=x+vjD6onUSlvK78JMqMvix69fARA+qz6/JunZ/8/qCE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kTLFc7Z6FA+DKytS9NlKorj5rKSin6PiDV/7r+Na/C8GQREKfzqQTW2X7Vk36+8jX Hfa5Uq54MOGiywbsFac56AC8tlUOWoOoO8WghKwU/yWFyAvgiw5T+w6CJwESXlJLCi 4LzK8E9oDFrG+m7KIfHe1AbBYjXhI9Dvr20mjTGk= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 33/33] DNI pipeline: rkisp1: Workaround for customer kernels without requests Date: Tue, 30 Sep 2025 14:26:54 +0200 Message-ID: <20250930122726.1837524-34-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> References: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" There are a few kernels out there that have dynamic vertex map support, but no requests support. Allow full dewarp usage on these. Signed-off-by: Stefan Klug --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 98521c804d84..e50cf8c0f978 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -183,6 +183,13 @@ static constexpr unsigned int kRkISP1MaxQueuedRequests = 4; */ static constexpr unsigned int kRkISP1MinBufferCount = 4; +/* + * This flag allows to use dynamic dewarp maps to support pan, zoom, rotate when + * the kernel driver doesn't support requests. Only needed for legacy customer + * kernels. + */ +static constexpr bool kAllowDynamicDewarpMapsWithoutRequests = true; + } /* namespace */ class PipelineHandlerRkISP1 : public PipelineHandler @@ -1514,7 +1521,7 @@ int PipelineHandlerRkISP1::updateControls(RkISP1CameraData *data) maxCrop); } - if (dewarper_->supportsRequests()) { + if (dewarper_->supportsRequests() || kAllowDynamicDewarpMapsWithoutRequests) { controls[&controls::draft::Dw100Scale] = ControlInfo(0.2f, 8.0f, 1.0f); controls[&controls::draft::Dw100Rotation] = ControlInfo(-180.0f, 180.0f, 0.0f); controls[&controls::draft::Dw100Offset] = ControlInfo(Point(-10000, -10000), Point(10000, 10000), Point(0, 0));