From patchwork Thu Oct 23 14:48:02 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24729 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1E1DDC32CE for ; Thu, 23 Oct 2025 14:48:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id AD5F2607DB; Thu, 23 Oct 2025 16:48:50 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="MGU2p1oh"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 491FA607C9 for ; Thu, 23 Oct 2025 16:48:49 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7328:357b:4ce1:72b6]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 57EFC1127; Thu, 23 Oct 2025 16:47:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761230824; bh=il1MPl9gegwLqBk2WXmpryeIZ0Js/Yid7F7UPXStGWk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=MGU2p1ohPv/06gvmJWoCLIZW3bFqDEf+/i8u23EQdvgRPiRH744Gg4ulWjDELEZLi Og+7BAioytt3jWAJP1SSydUoc6K5Bqjh+6+vzBoB8CtDMerDx5iApne+TsTDUvRyz3 9Msz0EMd85kIPtyN74a0ZUpbzXFawYmpw7pW1+o8= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Kieran Bingham , Paul Elder Subject: [PATCH v2 01/35] libcamera: v4l2: Support fromEntityName with shared_ptr Date: Thu, 23 Oct 2025 16:48:02 +0200 Message-ID: <20251023144841.403689-2-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251023144841.403689-1-stefan.klug@ideasonboard.com> References: <20251023144841.403689-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham To facilitate expanding the use of a shared_ptr in preference of a raw MediaDevice* pointer, add an additional implementation for fromEntityName for both of v4l2_videodevice and v4l2_subdevice to support this type. Signed-off-by: Kieran Bingham Signed-off-by: Paul Elder --- include/libcamera/internal/v4l2_subdevice.h | 2 ++ include/libcamera/internal/v4l2_videodevice.h | 2 ++ src/libcamera/v4l2_subdevice.cpp | 15 +++++++++++++++ src/libcamera/v4l2_videodevice.cpp | 15 +++++++++++++++ 4 files changed, 34 insertions(+) diff --git a/include/libcamera/internal/v4l2_subdevice.h b/include/libcamera/internal/v4l2_subdevice.h index c1cde1df2e36..d1c8b838d49b 100644 --- a/include/libcamera/internal/v4l2_subdevice.h +++ b/include/libcamera/internal/v4l2_subdevice.h @@ -163,6 +163,8 @@ public: static std::unique_ptr fromEntityName(const MediaDevice *media, const std::string &entity); + static std::unique_ptr + fromEntityName(std::shared_ptr, const std::string &entity); protected: std::string logPrefix() const override; diff --git a/include/libcamera/internal/v4l2_videodevice.h b/include/libcamera/internal/v4l2_videodevice.h index 6caafc4dcf08..4f80f7176174 100644 --- a/include/libcamera/internal/v4l2_videodevice.h +++ b/include/libcamera/internal/v4l2_videodevice.h @@ -228,6 +228,8 @@ public: static std::unique_ptr fromEntityName(const MediaDevice *media, const std::string &entity); + static std::unique_ptr + fromEntityName(std::shared_ptr, const std::string &entity); V4L2PixelFormat toV4L2PixelFormat(const PixelFormat &pixelFormat) const; diff --git a/src/libcamera/v4l2_subdevice.cpp b/src/libcamera/v4l2_subdevice.cpp index 31a2ac72a7ee..1825e9e55ca4 100644 --- a/src/libcamera/v4l2_subdevice.cpp +++ b/src/libcamera/v4l2_subdevice.cpp @@ -1768,6 +1768,21 @@ V4L2Subdevice::fromEntityName(const MediaDevice *media, return std::make_unique(mediaEntity); } +/** + * \brief Create a new video subdevice instance from \a entity in media device + * \a media + * \param[in] media The media device where the entity is registered + * \param[in] entity The media entity name + * + * \return A newly created V4L2Subdevice on success, nullptr otherwise + */ +std::unique_ptr +V4L2Subdevice::fromEntityName(std::shared_ptr media, + const std::string &entity) +{ + return fromEntityName(media.get(), entity); +} + std::string V4L2Subdevice::logPrefix() const { return "'" + entity_->name() + "'"; diff --git a/src/libcamera/v4l2_videodevice.cpp b/src/libcamera/v4l2_videodevice.cpp index 7b48d911db73..9be5cfbc0395 100644 --- a/src/libcamera/v4l2_videodevice.cpp +++ b/src/libcamera/v4l2_videodevice.cpp @@ -2114,6 +2114,21 @@ V4L2VideoDevice::fromEntityName(const MediaDevice *media, return std::make_unique(mediaEntity); } +/** + * \brief Create a new video device instance from \a entity in media device + * \a media + * \param[in] media The media device where the entity is registered + * \param[in] entity The media entity name + * + * \return A newly created V4L2VideoDevice on success, nullptr otherwise + */ +std::unique_ptr +V4L2VideoDevice::fromEntityName(std::shared_ptr media, + const std::string &entity) +{ + return fromEntityName(media.get(), entity); +} + /** * \brief Convert \a PixelFormat to a V4L2PixelFormat supported by the device * \param[in] pixelFormat The PixelFormat to convert From patchwork Thu Oct 23 14:48:03 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24730 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1B635C3331 for ; Thu, 23 Oct 2025 14:48:53 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9EA03607E6; Thu, 23 Oct 2025 16:48:53 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="AisOJ9kK"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1E2D0607C9 for ; Thu, 23 Oct 2025 16:48:52 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7328:357b:4ce1:72b6]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 09A2D195E; Thu, 23 Oct 2025 16:47:06 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761230827; bh=0d5hrByq/o6Nd32z9ToB6bQsbJeSVhRqP1Xu8EyvMXM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=AisOJ9kKZTuwcR9fwVL3RfMFPaDI7hPx6GC6s01sqAUEdlCPUFHgr1ec3fSDrbyVa LyC+HN9AL5I3k8lM0Lw1uFrEiZjtZA47d08B95uj35YeI0/isezCL3703Aa/nmjrNO BsL8x/VaKuMLxesIttidII/Dn5BTp9yL/QkeHkeE= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Kieran Bingham , Paul Elder Subject: [PATCH v2 02/35] libcamera: pipeline: utilise shared MediaDevice pointers Date: Thu, 23 Oct 2025 16:48:03 +0200 Message-ID: <20251023144841.403689-3-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251023144841.403689-1-stefan.klug@ideasonboard.com> References: <20251023144841.403689-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham Adapt the PipelineHandler::acquireMediaDevice() support function to return a shared pointer instead of the underlying raw pointer. Propagate this update to all pipeline handlers that use the MediaDevice and store a std::shared_ptr accordingly. Signed-off-by: Kieran Bingham Signed-off-by: Paul Elder --- include/libcamera/internal/pipeline_handler.h | 8 ++++---- src/libcamera/pipeline/imx8-isi/imx8-isi.cpp | 2 +- src/libcamera/pipeline/ipu3/cio2.cpp | 2 +- src/libcamera/pipeline/ipu3/cio2.h | 2 +- src/libcamera/pipeline/ipu3/imgu.cpp | 3 ++- src/libcamera/pipeline/ipu3/imgu.h | 4 ++-- src/libcamera/pipeline/ipu3/ipu3.cpp | 4 ++-- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 2 +- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 2 +- src/libcamera/pipeline/rkisp1/rkisp1_path.cpp | 2 +- src/libcamera/pipeline/rkisp1/rkisp1_path.h | 2 +- .../pipeline/rpi/common/pipeline_base.cpp | 6 ++++-- src/libcamera/pipeline/rpi/common/pipeline_base.h | 9 ++++++--- src/libcamera/pipeline/rpi/pisp/pisp.cpp | 10 ++++++---- src/libcamera/pipeline/rpi/vc4/vc4.cpp | 13 +++++++++---- src/libcamera/pipeline/simple/simple.cpp | 13 +++++++------ src/libcamera/pipeline/uvcvideo/uvcvideo.cpp | 6 +++--- src/libcamera/pipeline/vimc/vimc.cpp | 6 +++--- src/libcamera/pipeline_handler.cpp | 14 ++++++++------ test/delayed_controls.cpp | 2 +- test/libtest/buffer_source.cpp | 2 +- 21 files changed, 65 insertions(+), 49 deletions(-) diff --git a/include/libcamera/internal/pipeline_handler.h b/include/libcamera/internal/pipeline_handler.h index e89d6a33e398..0b25cffb44e5 100644 --- a/include/libcamera/internal/pipeline_handler.h +++ b/include/libcamera/internal/pipeline_handler.h @@ -38,8 +38,8 @@ public: virtual ~PipelineHandler(); virtual bool match(DeviceEnumerator *enumerator) = 0; - MediaDevice *acquireMediaDevice(DeviceEnumerator *enumerator, - const DeviceMatch &dm); + std::shared_ptr acquireMediaDevice(DeviceEnumerator *enumerator, + const DeviceMatch &dm); bool acquire(Camera *camera); void release(Camera *camera); @@ -72,7 +72,7 @@ public: protected: void registerCamera(std::shared_ptr camera); - void hotplugMediaDevice(MediaDevice *media); + void hotplugMediaDevice(std::shared_ptr media); virtual int queueRequestDevice(Camera *camera, Request *request) = 0; virtual void stopDevice(Camera *camera) = 0; @@ -86,7 +86,7 @@ protected: private: void unlockMediaDevices(); - void mediaDeviceDisconnected(MediaDevice *media); + void mediaDeviceDisconnected(std::shared_ptr media); virtual void disconnect(); void doQueueRequest(Request *request); diff --git a/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp b/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp index de09431cb9b9..3f414676ec28 100644 --- a/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp +++ b/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp @@ -139,7 +139,7 @@ private: void bufferReady(FrameBuffer *buffer); - MediaDevice *isiDev_; + std::shared_ptr isiDev_; std::unique_ptr crossbar_; std::vector pipes_; diff --git a/src/libcamera/pipeline/ipu3/cio2.cpp b/src/libcamera/pipeline/ipu3/cio2.cpp index aa544d7b0303..ebbd424c56ea 100644 --- a/src/libcamera/pipeline/ipu3/cio2.cpp +++ b/src/libcamera/pipeline/ipu3/cio2.cpp @@ -112,7 +112,7 @@ std::vector CIO2Device::sizes(const PixelFormat &format) const * \return 0 on success or a negative error code otherwise * \retval -ENODEV No supported image sensor is connected to this CIO2 instance */ -int CIO2Device::init(const MediaDevice *media, unsigned int index) +int CIO2Device::init(std::shared_ptr media, unsigned int index) { int ret; diff --git a/src/libcamera/pipeline/ipu3/cio2.h b/src/libcamera/pipeline/ipu3/cio2.h index 3aa3a1ca8ef8..d844cb7ae831 100644 --- a/src/libcamera/pipeline/ipu3/cio2.h +++ b/src/libcamera/pipeline/ipu3/cio2.h @@ -38,7 +38,7 @@ public: std::vector formats() const; std::vector sizes(const PixelFormat &format) const; - int init(const MediaDevice *media, unsigned int index); + int init(std::shared_ptr media, unsigned int index); int configure(const Size &size, const Transform &transform, V4L2DeviceFormat *outputFormat); diff --git a/src/libcamera/pipeline/ipu3/imgu.cpp b/src/libcamera/pipeline/ipu3/imgu.cpp index 7be780913fae..391e000f6e12 100644 --- a/src/libcamera/pipeline/ipu3/imgu.cpp +++ b/src/libcamera/pipeline/ipu3/imgu.cpp @@ -330,7 +330,8 @@ FOV calcFOV(const Size &in, const ImgUDevice::PipeConfig &pipe) * * \return 0 on success or a negative error code otherwise */ -int ImgUDevice::init(MediaDevice *media, unsigned int index) +int ImgUDevice::init(std::shared_ptr media, + unsigned int index) { int ret; diff --git a/src/libcamera/pipeline/ipu3/imgu.h b/src/libcamera/pipeline/ipu3/imgu.h index fa508316b301..272f861f98c4 100644 --- a/src/libcamera/pipeline/ipu3/imgu.h +++ b/src/libcamera/pipeline/ipu3/imgu.h @@ -64,7 +64,7 @@ public: Size viewfinder; }; - int init(MediaDevice *media, unsigned int index); + int init(std::shared_ptr media, unsigned int index); PipeConfig calculatePipeConfig(Pipe *pipe); @@ -118,7 +118,7 @@ private: V4L2DeviceFormat *outputFormat); std::string name_; - MediaDevice *media_; + std::shared_ptr media_; }; } /* namespace libcamera */ diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index d6b7edcb5a7f..695762c750f8 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -164,8 +164,8 @@ private: ImgUDevice imgu0_; ImgUDevice imgu1_; - MediaDevice *cio2MediaDev_; - MediaDevice *imguMediaDev_; + std::shared_ptr cio2MediaDev_; + std::shared_ptr imguMediaDev_; std::vector ipaBuffers_; }; diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 38bdc6138ed1..434b2d5e8a5f 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -682,7 +682,7 @@ private: bool registerTPGCamera(MediaLink *link); bool registerSensorCamera(MediaLink *link); - MediaDevice *media_; + std::shared_ptr media_; std::unique_ptr isp_; std::unique_ptr stats_; std::unique_ptr params_; diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index ecd13831539f..959cdb41fe7e 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -218,7 +218,7 @@ private: int updateControls(RkISP1CameraData *data); - MediaDevice *media_; + std::shared_ptr media_; std::unique_ptr isp_; std::unique_ptr param_; std::unique_ptr stat_; diff --git a/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp b/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp index d318bea92446..8d5298c04833 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp @@ -64,7 +64,7 @@ RkISP1Path::RkISP1Path(const char *name, const Span &formats, { } -bool RkISP1Path::init(MediaDevice *media) +bool RkISP1Path::init(std::shared_ptr media) { std::string resizer = std::string("rkisp1_resizer_") + name_ + "path"; std::string video = std::string("rkisp1_") + name_ + "path"; diff --git a/src/libcamera/pipeline/rkisp1/rkisp1_path.h b/src/libcamera/pipeline/rkisp1/rkisp1_path.h index 0b60c499ac64..0c68e9eb99af 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1_path.h +++ b/src/libcamera/pipeline/rkisp1/rkisp1_path.h @@ -37,7 +37,7 @@ public: RkISP1Path(const char *name, const Span &formats, const Size &minResolution, const Size &maxResolution); - bool init(MediaDevice *media); + bool init(std::shared_ptr media); int setEnabled(bool enable) { return link_->setEnabled(enable); } bool isEnabled() const { return link_->flags() & MEDIA_LNK_FL_ENABLED; } diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index c209aa596311..9d65dc83573b 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -786,8 +786,10 @@ int PipelineHandlerBase::queueRequestDevice(Camera *camera, Request *request) } int PipelineHandlerBase::registerCamera(std::unique_ptr &cameraData, - MediaDevice *frontend, const std::string &frontendName, - MediaDevice *backend, MediaEntity *sensorEntity) + std::shared_ptr frontend, + const std::string &frontendName, + std::shared_ptr backend, + MediaEntity *sensorEntity) { CameraData *data = cameraData.get(); int ret; diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index 4bce4ec4f978..15628259afc6 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -224,13 +224,16 @@ public: protected: int registerCamera(std::unique_ptr &cameraData, - MediaDevice *frontent, const std::string &frontendName, - MediaDevice *backend, MediaEntity *sensorEntity); + std::shared_ptr frontend, + const std::string &frontendName, + std::shared_ptr backend, + MediaEntity *sensorEntity); void mapBuffers(Camera *camera, const BufferMap &buffers, unsigned int mask); virtual int platformRegister(std::unique_ptr &cameraData, - MediaDevice *unicam, MediaDevice *isp) = 0; + std::shared_ptr unicam, + std::shared_ptr isp) = 0; private: CameraData *cameraData(Camera *camera) diff --git a/src/libcamera/pipeline/rpi/pisp/pisp.cpp b/src/libcamera/pipeline/rpi/pisp/pisp.cpp index 082724c5a0fb..77acd2f64092 100644 --- a/src/libcamera/pipeline/rpi/pisp/pisp.cpp +++ b/src/libcamera/pipeline/rpi/pisp/pisp.cpp @@ -866,7 +866,8 @@ private: int prepareBuffers(Camera *camera) override; int platformRegister(std::unique_ptr &cameraData, - MediaDevice *cfe, MediaDevice *isp) override; + std::shared_ptr cfe, + std::shared_ptr isp) override; }; bool PipelineHandlerPiSP::match(DeviceEnumerator *enumerator) @@ -884,7 +885,7 @@ bool PipelineHandlerPiSP::match(DeviceEnumerator *enumerator) cfe.add("rp1-cfe-fe-image0"); cfe.add("rp1-cfe-fe-stats"); cfe.add("rp1-cfe-fe-config"); - MediaDevice *cfeDevice = acquireMediaDevice(enumerator, cfe); + std::shared_ptr cfeDevice = acquireMediaDevice(enumerator, cfe); if (!cfeDevice) { LOG(RPI, Debug) << "Unable to acquire a CFE instance"; @@ -900,7 +901,7 @@ bool PipelineHandlerPiSP::match(DeviceEnumerator *enumerator) isp.add("pispbe-tdn_input"); isp.add("pispbe-stitch_output"); isp.add("pispbe-stitch_input"); - MediaDevice *ispDevice = acquireMediaDevice(enumerator, isp); + std::shared_ptr ispDevice = acquireMediaDevice(enumerator, isp); if (!ispDevice) { LOG(RPI, Debug) << "Unable to acquire ISP instance"; @@ -1065,7 +1066,8 @@ int PipelineHandlerPiSP::prepareBuffers(Camera *camera) } int PipelineHandlerPiSP::platformRegister(std::unique_ptr &cameraData, - MediaDevice *cfe, MediaDevice *isp) + std::shared_ptr cfe, + std::shared_ptr isp) { PiSPCameraData *data = static_cast(cameraData.get()); int ret; diff --git a/src/libcamera/pipeline/rpi/vc4/vc4.cpp b/src/libcamera/pipeline/rpi/vc4/vc4.cpp index 99d43bd0a2f5..f0cb99d59e74 100644 --- a/src/libcamera/pipeline/rpi/vc4/vc4.cpp +++ b/src/libcamera/pipeline/rpi/vc4/vc4.cpp @@ -5,6 +5,8 @@ * Pipeline handler for VC4-based Raspberry Pi devices */ +#include + #include #include #include @@ -158,7 +160,8 @@ private: int prepareBuffers(Camera *camera) override; int platformRegister(std::unique_ptr &cameraData, - MediaDevice *unicam, MediaDevice *isp) override; + std::shared_ptr unicam, + std::shared_ptr isp) override; }; bool PipelineHandlerVc4::match(DeviceEnumerator *enumerator) @@ -173,7 +176,7 @@ bool PipelineHandlerVc4::match(DeviceEnumerator *enumerator) */ for (unsigned int i = 0; i < numUnicamDevices; i++) { DeviceMatch unicam("unicam"); - MediaDevice *unicamDevice = acquireMediaDevice(enumerator, unicam); + std::shared_ptr unicamDevice = acquireMediaDevice(enumerator, unicam); if (!unicamDevice) { LOG(RPI, Debug) << "Unable to acquire a Unicam instance"; @@ -181,7 +184,7 @@ bool PipelineHandlerVc4::match(DeviceEnumerator *enumerator) } DeviceMatch isp("bcm2835-isp"); - MediaDevice *ispDevice = acquireMediaDevice(enumerator, isp); + std::shared_ptr ispDevice = acquireMediaDevice(enumerator, isp); if (!ispDevice) { LOG(RPI, Debug) << "Unable to acquire ISP instance"; @@ -303,7 +306,9 @@ int PipelineHandlerVc4::prepareBuffers(Camera *camera) return 0; } -int PipelineHandlerVc4::platformRegister(std::unique_ptr &cameraData, MediaDevice *unicam, MediaDevice *isp) +int PipelineHandlerVc4::platformRegister(std::unique_ptr &cameraData, + std::shared_ptr unicam, + std::shared_ptr isp) { Vc4CameraData *data = static_cast(cameraData.get()); diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 7b0783cdbc32..1183be9a6b07 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -413,7 +413,7 @@ public: V4L2VideoDevice *video(const MediaEntity *entity); V4L2Subdevice *subdev(const MediaEntity *entity); - MediaDevice *converter() { return converter_; } + MediaDevice *converter() { return converter_.get(); } bool swIspEnabled() const { return swIspEnabled_; } protected: @@ -434,7 +434,8 @@ private: return static_cast(camera->_d()); } - bool matchDevice(MediaDevice *media, const SimplePipelineInfo &info, + bool matchDevice(std::shared_ptr media, + const SimplePipelineInfo &info, DeviceEnumerator *enumerator); std::vector locateSensors(MediaDevice *media); @@ -445,7 +446,7 @@ private: std::map entities_; - MediaDevice *converter_; + std::shared_ptr converter_; bool swIspEnabled_; }; @@ -1684,7 +1685,7 @@ int SimplePipelineHandler::resetRoutingTable(V4L2Subdevice *subdev) return 0; } -bool SimplePipelineHandler::matchDevice(MediaDevice *media, +bool SimplePipelineHandler::matchDevice(std::shared_ptr media, const SimplePipelineInfo &info, DeviceEnumerator *enumerator) { @@ -1715,7 +1716,7 @@ bool SimplePipelineHandler::matchDevice(MediaDevice *media, } /* Locate the sensors. */ - std::vector sensors = locateSensors(media); + std::vector sensors = locateSensors(media.get()); if (sensors.empty()) { LOG(SimplePipeline, Info) << "No sensor found for " << media->deviceNode(); return false; @@ -1833,7 +1834,7 @@ bool SimplePipelineHandler::matchDevice(MediaDevice *media, bool SimplePipelineHandler::match(DeviceEnumerator *enumerator) { - MediaDevice *media; + std::shared_ptr media; for (const SimplePipelineInfo &inf : supportedDevices) { DeviceMatch dm(inf.driver); diff --git a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp index 4b5816dfdde0..034f9c39d3e9 100644 --- a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp +++ b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp @@ -46,7 +46,7 @@ public: { } - int init(MediaDevice *media); + int init(std::shared_ptr media); void addControl(uint32_t cid, const ControlInfo &v4l2info, ControlInfoMap::Map *ctrls); void imageBufferReady(FrameBuffer *buffer); @@ -466,7 +466,7 @@ int PipelineHandlerUVC::queueRequestDevice(Camera *camera, Request *request) bool PipelineHandlerUVC::match(DeviceEnumerator *enumerator) { - MediaDevice *media; + std::shared_ptr media; DeviceMatch dm("uvcvideo"); media = acquireMediaDevice(enumerator, dm); @@ -508,7 +508,7 @@ void PipelineHandlerUVC::releaseDevice(Camera *camera) data->video_->close(); } -int UVCCameraData::init(MediaDevice *media) +int UVCCameraData::init(std::shared_ptr media) { int ret; diff --git a/src/libcamera/pipeline/vimc/vimc.cpp b/src/libcamera/pipeline/vimc/vimc.cpp index 5022101505a1..fdb92b4a82d9 100644 --- a/src/libcamera/pipeline/vimc/vimc.cpp +++ b/src/libcamera/pipeline/vimc/vimc.cpp @@ -49,7 +49,7 @@ LOG_DEFINE_CATEGORY(VIMC) class VimcCameraData : public Camera::Private { public: - VimcCameraData(PipelineHandler *pipe, MediaDevice *media) + VimcCameraData(PipelineHandler *pipe, std::shared_ptr media) : Camera::Private(pipe), media_(media) { } @@ -59,7 +59,7 @@ public: void imageBufferReady(FrameBuffer *buffer); void paramsComputed(unsigned int id, const Flags flags); - MediaDevice *media_; + std::shared_ptr media_; std::unique_ptr sensor_; std::unique_ptr debayer_; std::unique_ptr scaler_; @@ -479,7 +479,7 @@ bool PipelineHandlerVimc::match(DeviceEnumerator *enumerator) dm.add("RGB/YUV Input"); dm.add("Scaler"); - MediaDevice *media = acquireMediaDevice(enumerator, dm); + std::shared_ptr media = acquireMediaDevice(enumerator, dm); if (!media) return false; diff --git a/src/libcamera/pipeline_handler.cpp b/src/libcamera/pipeline_handler.cpp index e5f9e55c9783..825b8767fe3b 100644 --- a/src/libcamera/pipeline_handler.cpp +++ b/src/libcamera/pipeline_handler.cpp @@ -129,10 +129,12 @@ PipelineHandler::~PipelineHandler() * * \context This function shall be called from the CameraManager thread. * - * \return A pointer to the matching MediaDevice, or nullptr if no match is found + * \return A shared pointer to the matching MediaDevice, or nullptr if no match + * is found */ -MediaDevice *PipelineHandler::acquireMediaDevice(DeviceEnumerator *enumerator, - const DeviceMatch &dm) +std::shared_ptr +PipelineHandler::acquireMediaDevice(DeviceEnumerator *enumerator, + const DeviceMatch &dm) { std::shared_ptr media = enumerator->search(dm); if (!media) @@ -143,7 +145,7 @@ MediaDevice *PipelineHandler::acquireMediaDevice(DeviceEnumerator *enumerator, mediaDevices_.push_back(media); - return media.get(); + return media; } /** @@ -725,7 +727,7 @@ void PipelineHandler::registerCamera(std::shared_ptr camera) * handler gets notified and automatically disconnects all the cameras it has * registered without requiring any manual intervention. */ -void PipelineHandler::hotplugMediaDevice(MediaDevice *media) +void PipelineHandler::hotplugMediaDevice(std::shared_ptr media) { media->disconnected.connect(this, [this, media] { mediaDeviceDisconnected(media); }); } @@ -733,7 +735,7 @@ void PipelineHandler::hotplugMediaDevice(MediaDevice *media) /** * \brief Slot for the MediaDevice disconnected signal */ -void PipelineHandler::mediaDeviceDisconnected(MediaDevice *media) +void PipelineHandler::mediaDeviceDisconnected(std::shared_ptr media) { media->disconnected.disconnect(this); diff --git a/test/delayed_controls.cpp b/test/delayed_controls.cpp index 7bd30e7aead8..b305be48aafc 100644 --- a/test/delayed_controls.cpp +++ b/test/delayed_controls.cpp @@ -47,7 +47,7 @@ protected: return TestSkip; } - dev_ = V4L2VideoDevice::fromEntityName(media_.get(), "vivid-000-vid-cap"); + dev_ = V4L2VideoDevice::fromEntityName(media_, "vivid-000-vid-cap"); if (dev_->open()) { cerr << "Failed to open video device" << endl; return TestFail; diff --git a/test/libtest/buffer_source.cpp b/test/libtest/buffer_source.cpp index dde11f365e43..19b25fed50a0 100644 --- a/test/libtest/buffer_source.cpp +++ b/test/libtest/buffer_source.cpp @@ -52,7 +52,7 @@ int BufferSource::allocate(const StreamConfiguration &config) return TestSkip; } - std::unique_ptr video = V4L2VideoDevice::fromEntityName(media_.get(), videoDeviceName); + std::unique_ptr video = V4L2VideoDevice::fromEntityName(media_, videoDeviceName); if (!video) { std::cout << "Failed to get video device from entity " << videoDeviceName << std::endl; From patchwork Thu Oct 23 14:48:05 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24732 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 29552C3259 for ; Thu, 23 Oct 2025 14:49:01 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2C6DA607F8; Thu, 23 Oct 2025 16:49:00 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="nncI7e0w"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EF117606DE for ; Thu, 23 Oct 2025 16:48:57 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7328:357b:4ce1:72b6]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 0536319E5; Thu, 23 Oct 2025 16:47:12 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761230833; bh=DQ4Dk4FvJ+uYS9yZjd701CRURlrCQeMEmCNaJG8L/xw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nncI7e0wlZgQmrY9JpWSpnTV/ut6iPBlcOA0uEYcCJ92PKNcQ4zpc32pnQ1ie3VtM G1ZCWsT/1X9DruMswpVE4FDkihCJI9HxJyqReySYsmAeBkFXhGiOMeG3MEMqUmwrj4 drf4u1YnRDcuSw6D0ZrATDUAm7G2gX53FVXbxfoc= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 04/35] libcamera: converter: Utilise shared MediaDevice pointers Date: Thu, 23 Oct 2025 16:48:05 +0200 Message-ID: <20251023144841.403689-5-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251023144841.403689-1-stefan.klug@ideasonboard.com> References: <20251023144841.403689-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" With the upcoming addition of V4L2 requests support, the converters need to keep a handle to the corresponding media device. Prepare for that by changing the constructor parameter from a raw pointer to a shared pointer. Signed-off-by: Stefan Klug --- include/libcamera/internal/converter.h | 9 +++++---- .../libcamera/internal/converter/converter_v4l2_m2m.h | 2 +- src/libcamera/converter.cpp | 5 +++-- src/libcamera/converter/converter_v4l2_m2m.cpp | 4 ++-- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 2 +- src/libcamera/pipeline/simple/simple.cpp | 4 ++-- 6 files changed, 14 insertions(+), 12 deletions(-) diff --git a/include/libcamera/internal/converter.h b/include/libcamera/internal/converter.h index 644ec429bb25..4915af7ac5de 100644 --- a/include/libcamera/internal/converter.h +++ b/include/libcamera/internal/converter.h @@ -46,7 +46,7 @@ public: Up, }; - Converter(MediaDevice *media, Features features = Feature::None); + Converter(std::shared_ptr media, Features features = Feature::None); virtual ~Converter(); virtual int loadConfiguration(const std::string &filename) = 0; @@ -107,7 +107,7 @@ public: const std::vector &compatibles() const { return compatibles_; } - static std::unique_ptr create(MediaDevice *media); + static std::unique_ptr create(std::shared_ptr media); static std::vector &factories(); static std::vector names(); @@ -116,7 +116,8 @@ private: static void registerType(ConverterFactoryBase *factory); - virtual std::unique_ptr createInstance(MediaDevice *media) const = 0; + virtual std::unique_ptr + createInstance(std::shared_ptr media) const = 0; std::string name_; std::vector compatibles_; @@ -131,7 +132,7 @@ public: { } - std::unique_ptr createInstance(MediaDevice *media) const override + std::unique_ptr createInstance(std::shared_ptr media) const override { return std::make_unique<_Converter>(media); } diff --git a/include/libcamera/internal/converter/converter_v4l2_m2m.h b/include/libcamera/internal/converter/converter_v4l2_m2m.h index 0ad7bf7fdbe2..d316754040dd 100644 --- a/include/libcamera/internal/converter/converter_v4l2_m2m.h +++ b/include/libcamera/internal/converter/converter_v4l2_m2m.h @@ -36,7 +36,7 @@ class V4L2M2MDevice; class V4L2M2MConverter : public Converter { public: - V4L2M2MConverter(MediaDevice *media); + V4L2M2MConverter(std::shared_ptr media); int loadConfiguration([[maybe_unused]] const std::string &filename) override { return 0; } bool isValid() const override { return m2m_ != nullptr; } diff --git a/src/libcamera/converter.cpp b/src/libcamera/converter.cpp index d551b908d523..142fb29a1272 100644 --- a/src/libcamera/converter.cpp +++ b/src/libcamera/converter.cpp @@ -8,6 +8,7 @@ #include "libcamera/internal/converter.h" #include +#include #include @@ -68,7 +69,7 @@ LOG_DEFINE_CATEGORY(Converter) * This searches for the entity implementing the data streaming function in the * media graph entities and use its device node as the converter device node. */ -Converter::Converter(MediaDevice *media, Features features) +Converter::Converter(std::shared_ptr media, Features features) { const std::vector &entities = media->entities(); auto it = std::find_if(entities.begin(), entities.end(), @@ -332,7 +333,7 @@ ConverterFactoryBase::ConverterFactoryBase(const std::string name, std::initiali * \return A new instance of the converter subclass corresponding to the media * device, or null if the media device driver name doesn't match anything */ -std::unique_ptr ConverterFactoryBase::create(MediaDevice *media) +std::unique_ptr ConverterFactoryBase::create(std::shared_ptr media) { const std::vector &factories = ConverterFactoryBase::factories(); diff --git a/src/libcamera/converter/converter_v4l2_m2m.cpp b/src/libcamera/converter/converter_v4l2_m2m.cpp index 0ce7a7a67f11..b2bd54f368d8 100644 --- a/src/libcamera/converter/converter_v4l2_m2m.cpp +++ b/src/libcamera/converter/converter_v4l2_m2m.cpp @@ -10,6 +10,7 @@ #include #include +#include #include #include @@ -261,8 +262,7 @@ void V4L2M2MConverter::V4L2M2MStream::captureBufferReady(FrameBuffer *buffer) * \brief Construct a V4L2M2MConverter instance * \param[in] media The media device implementing the converter */ - -V4L2M2MConverter::V4L2M2MConverter(MediaDevice *media) +V4L2M2MConverter::V4L2M2MConverter(std::shared_ptr media) : Converter(media) { if (deviceNode().empty()) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 959cdb41fe7e..0e2a210e1e80 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -1445,7 +1445,7 @@ bool PipelineHandlerRkISP1::match(DeviceEnumerator *enumerator) std::shared_ptr dwpMediaDevice = enumerator->search(dwp); if (dwpMediaDevice) { - dewarper_ = std::make_unique(dwpMediaDevice.get()); + dewarper_ = std::make_unique(dwpMediaDevice); if (dewarper_->isValid()) { dewarper_->outputBufferReady.connect( this, &PipelineHandlerRkISP1::dewarpBufferReady); diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 1183be9a6b07..204b29f92907 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -413,7 +413,7 @@ public: V4L2VideoDevice *video(const MediaEntity *entity); V4L2Subdevice *subdev(const MediaEntity *entity); - MediaDevice *converter() { return converter_.get(); } + std::shared_ptr converter() { return converter_; } bool swIspEnabled() const { return swIspEnabled_; } protected: @@ -586,7 +586,7 @@ int SimpleCameraData::init() int ret; /* Open the converter, if any. */ - MediaDevice *converter = pipe->converter(); + std::shared_ptr converter = pipe->converter(); if (converter) { converter_ = ConverterFactoryBase::create(converter); if (!converter_) { From patchwork Thu Oct 23 14:48:06 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24733 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 43E24C3334 for ; Thu, 23 Oct 2025 14:49:03 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BC2F7607F3; Thu, 23 Oct 2025 16:49:02 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="NmfaY7Lb"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2D82E607ED for ; Thu, 23 Oct 2025 16:49:01 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7328:357b:4ce1:72b6]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 27C321127; Thu, 23 Oct 2025 16:47:16 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761230836; bh=ii1ugYfLMAI//VqbVDO7XTRzNSKRFP1iGseToSeFtkw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NmfaY7Lb3WYWdF6MM2HX1Y8krZR3gm8BTu13YZdjBz/Nn25eGO3ZqeTnBFdEEVSWo t+QTuuQK9REb1VgJzb44fgjs8G6JCeeiTepJcVhEWEBWvLOhg8Jqj6EMBjg/wwzyYQ KZsrajZfGRR/pHpgnH/NrQH56vDe7C6kCsgDkNb0= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 05/35] libcamera: Add support for V4L2 requests Date: Thu, 23 Oct 2025 16:48:06 +0200 Message-ID: <20251023144841.403689-6-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251023144841.403689-1-stefan.klug@ideasonboard.com> References: <20251023144841.403689-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The V4L2 requests API provides support to atomically tie controls to a set of buffers. This is especially common for m2m devices. Such a request is represented by a fd that is allocated vi MEDIA_IOC_REQUEST_ALLOC and then passed to various V4L2 function. Implement a V4L2Request class to wrap such an fd and add the corresponding utility functions. Signed-off-by: Stefan Klug --- Changes in v2: - Added documentation --- include/libcamera/internal/media_device.h | 7 + include/libcamera/internal/meson.build | 1 + include/libcamera/internal/v4l2_device.h | 5 +- include/libcamera/internal/v4l2_request.h | 49 +++++++ include/libcamera/internal/v4l2_videodevice.h | 3 +- src/libcamera/media_device.cpp | 47 +++++++ src/libcamera/meson.build | 1 + src/libcamera/v4l2_device.cpp | 28 +++- src/libcamera/v4l2_request.cpp | 128 ++++++++++++++++++ src/libcamera/v4l2_videodevice.cpp | 10 +- 10 files changed, 271 insertions(+), 8 deletions(-) create mode 100644 include/libcamera/internal/v4l2_request.h create mode 100644 src/libcamera/v4l2_request.cpp diff --git a/include/libcamera/internal/media_device.h b/include/libcamera/internal/media_device.h index b3a48b98d64b..74cb9ba1542d 100644 --- a/include/libcamera/internal/media_device.h +++ b/include/libcamera/internal/media_device.h @@ -18,6 +18,7 @@ #include #include "libcamera/internal/media_object.h" +#include "libcamera/internal/v4l2_request.h" namespace libcamera { @@ -57,6 +58,11 @@ public: std::vector locateEntities(unsigned int function); + int allocateRequests(unsigned int count, + std::vector> *requests); + + bool supportsRequests(); + protected: std::string logPrefix() const override; @@ -87,6 +93,7 @@ private: UniqueFD fd_; bool valid_; bool acquired_; + std::optional supportsRequests_; std::map objects_; std::vector entities_; diff --git a/include/libcamera/internal/meson.build b/include/libcamera/internal/meson.build index 45c299f6a332..e9540a2f734f 100644 --- a/include/libcamera/internal/meson.build +++ b/include/libcamera/internal/meson.build @@ -44,6 +44,7 @@ libcamera_internal_headers = files([ 'sysfs.h', 'v4l2_device.h', 'v4l2_pixelformat.h', + 'v4l2_request.h', 'v4l2_subdevice.h', 'v4l2_videodevice.h', 'vector.h', diff --git a/include/libcamera/internal/v4l2_device.h b/include/libcamera/internal/v4l2_device.h index 5bc9da96677d..dbbd118abd00 100644 --- a/include/libcamera/internal/v4l2_device.h +++ b/include/libcamera/internal/v4l2_device.h @@ -24,6 +24,7 @@ #include #include "libcamera/internal/formats.h" +#include "libcamera/internal/v4l2_request.h" namespace libcamera { @@ -37,8 +38,8 @@ public: const ControlInfoMap &controls() const { return controls_; } - ControlList getControls(Span ids); - int setControls(ControlList *ctrls); + ControlList getControls(Span ids, const V4L2Request *request = nullptr); + int setControls(ControlList *ctrls, const V4L2Request *request = nullptr); const struct v4l2_query_ext_ctrl *controlInfo(uint32_t id) const; diff --git a/include/libcamera/internal/v4l2_request.h b/include/libcamera/internal/v4l2_request.h new file mode 100644 index 000000000000..bf1bea3261af --- /dev/null +++ b/include/libcamera/internal/v4l2_request.h @@ -0,0 +1,49 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2025, Ideas On Board + * + * V4L2 requests + */ + +#pragma once + +#include + +#include + +#include +#include +#include +#include +#include + +#include +#include + +namespace libcamera { + +class V4L2Request : protected Loggable +{ +public: + bool isValid() const { return fd_.isValid(); } + int fd() const { return fd_.get(); } + + int reinit(); + int queue(); + + V4L2Request(int fd = -1); + ~V4L2Request() = default; + + Signal requestDone; + +private: + LIBCAMERA_DISABLE_COPY_AND_MOVE(V4L2Request) + + void requestReady(); + std::string logPrefix() const override; + + UniqueFD fd_; + EventNotifier fdNotifier_; +}; + +} /* namespace libcamera */ diff --git a/include/libcamera/internal/v4l2_videodevice.h b/include/libcamera/internal/v4l2_videodevice.h index 5a7dcfdda118..2d290971a0ee 100644 --- a/include/libcamera/internal/v4l2_videodevice.h +++ b/include/libcamera/internal/v4l2_videodevice.h @@ -33,6 +33,7 @@ #include "libcamera/internal/formats.h" #include "libcamera/internal/v4l2_device.h" #include "libcamera/internal/v4l2_pixelformat.h" +#include "libcamera/internal/v4l2_request.h" namespace libcamera { @@ -217,7 +218,7 @@ public: int importBuffers(unsigned int count); int releaseBuffers(); - int queueBuffer(FrameBuffer *buffer); + int queueBuffer(FrameBuffer *buffer, const V4L2Request *request = nullptr); Signal bufferReady; int streamOn(); diff --git a/src/libcamera/media_device.cpp b/src/libcamera/media_device.cpp index 353f34a81eca..673c53fefdd7 100644 --- a/src/libcamera/media_device.cpp +++ b/src/libcamera/media_device.cpp @@ -20,6 +20,7 @@ #include #include +#include "libcamera/internal/v4l2_request.h" /** * \file media_device.h @@ -851,4 +852,50 @@ std::vector MediaDevice::locateEntities(unsigned int function) return found; } +/** + * \brief Allocate requests + * \param[in] count Number of requests to allocate + * \param[out] requests Vector to store allocated requests + * + * Allocates and stores \a count requests in \a requests. If allocation fails, + * and error is returned and \a requests is cleared. + * + * \return 0 on success or a negative error code otherwise + */ +int MediaDevice::allocateRequests(unsigned int count, + std::vector> *requests) +{ + requests->resize(count); + for (unsigned int i = 0; i < count; i++) { + int requestFd; + int ret = ::ioctl(fd_.get(), MEDIA_IOC_REQUEST_ALLOC, &requestFd); + if (ret < 0) { + requests->clear(); + return -errno; + } + (*requests)[i] = std::make_unique(requestFd); + } + + return 0; +} + +/** + * \brief Check if requests are supported + * + * Checks if the device supports V4L2 requests by trying to allocate a single + * request. The result is cached, so the allocation is only tried once. + * + * \return True if the device supports requests, false otherwise + */ +bool MediaDevice::supportsRequests() +{ + if (supportsRequests_.has_value()) + return supportsRequests_.value(); + + std::vector> requests; + supportsRequests_ = (allocateRequests(1, &requests) == 0); + + return supportsRequests_.value(); +} + } /* namespace libcamera */ diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build index 5b9b86f211f1..34e20f557514 100644 --- a/src/libcamera/meson.build +++ b/src/libcamera/meson.build @@ -54,6 +54,7 @@ libcamera_internal_sources = files([ 'sysfs.cpp', 'v4l2_device.cpp', 'v4l2_pixelformat.cpp', + 'v4l2_request.cpp', 'v4l2_subdevice.cpp', 'v4l2_videodevice.cpp', 'vector.cpp', diff --git a/src/libcamera/v4l2_device.cpp b/src/libcamera/v4l2_device.cpp index 8c78b8c424e5..7a669a0303c1 100644 --- a/src/libcamera/v4l2_device.cpp +++ b/src/libcamera/v4l2_device.cpp @@ -162,6 +162,7 @@ void V4L2Device::close() /** * \brief Read controls from the device * \param[in] ids The list of controls to read, specified by their ID + * \param[in] request An optional request * * This function reads the value of all controls contained in \a ids, and * returns their values as a ControlList. @@ -171,10 +172,12 @@ void V4L2Device::close() * during validation of the requested controls, no control is read and this * function returns an empty control list. * + * If \a request is specified the controls tied to that request are read. + * * \return The control values in a ControlList on success, or an empty list on * error */ -ControlList V4L2Device::getControls(Span ids) +ControlList V4L2Device::getControls(Span ids, const V4L2Request *request) { if (ids.empty()) return {}; @@ -242,10 +245,16 @@ ControlList V4L2Device::getControls(Span ids) } struct v4l2_ext_controls v4l2ExtCtrls = {}; - v4l2ExtCtrls.which = V4L2_CTRL_WHICH_CUR_VAL; v4l2ExtCtrls.controls = v4l2Ctrls.data(); v4l2ExtCtrls.count = v4l2Ctrls.size(); + if (request) { + v4l2ExtCtrls.which = V4L2_CTRL_WHICH_REQUEST_VAL; + v4l2ExtCtrls.request_fd = request->fd(); + } else { + v4l2ExtCtrls.which = V4L2_CTRL_WHICH_CUR_VAL; + } + int ret = ioctl(VIDIOC_G_EXT_CTRLS, &v4l2ExtCtrls); if (ret) { unsigned int errorIdx = v4l2ExtCtrls.error_idx; @@ -273,6 +282,7 @@ ControlList V4L2Device::getControls(Span ids) /** * \brief Write controls to the device * \param[in] ctrls The list of controls to write + * \param[in] request And optional request * * This function writes the value of all controls contained in \a ctrls, and * stores the values actually applied to the device in the corresponding @@ -288,11 +298,15 @@ ControlList V4L2Device::getControls(Span ids) * are written and their values are updated in \a ctrls, while all other * controls are not written and their values are not changed. * + * If \a request is set, the controls will be applied to that request. If the + * device doesn't support requests, -EACCESS will be returned. If \a request is + * invalid, -EINVAL will be returned. + * * \return 0 on success or an error code otherwise * \retval -EINVAL One of the control is not supported or not accessible * \retval i The index of the control that failed */ -int V4L2Device::setControls(ControlList *ctrls) +int V4L2Device::setControls(ControlList *ctrls, const V4L2Request *request) { if (ctrls->empty()) return 0; @@ -377,10 +391,16 @@ int V4L2Device::setControls(ControlList *ctrls) } struct v4l2_ext_controls v4l2ExtCtrls = {}; - v4l2ExtCtrls.which = V4L2_CTRL_WHICH_CUR_VAL; v4l2ExtCtrls.controls = v4l2Ctrls.data(); v4l2ExtCtrls.count = v4l2Ctrls.size(); + if (request) { + v4l2ExtCtrls.which = V4L2_CTRL_WHICH_REQUEST_VAL; + v4l2ExtCtrls.request_fd = request->fd(); + } else { + v4l2ExtCtrls.which = V4L2_CTRL_WHICH_CUR_VAL; + } + int ret = ioctl(VIDIOC_S_EXT_CTRLS, &v4l2ExtCtrls); if (ret) { unsigned int errorIdx = v4l2ExtCtrls.error_idx; diff --git a/src/libcamera/v4l2_request.cpp b/src/libcamera/v4l2_request.cpp new file mode 100644 index 000000000000..708250d86f61 --- /dev/null +++ b/src/libcamera/v4l2_request.cpp @@ -0,0 +1,128 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2025, Ideas On Board + * + * V4L2 Request API + */ + +#include "libcamera/internal/v4l2_request.h" + +#include +#include +#include +#include +#include + +#include + +#include +#include + +/** + * \file v4l2_request.h + * \brief V4L2 Request + */ + +namespace libcamera { + +LOG_DECLARE_CATEGORY(V4L2) + +/** + * \class V4L2Request + * \brief V4L2Request object and API + * + * The V4L2Request class wraps a V4L2 request fd and provides some convenience + * functions to handle request. + * + * It is usually constructed by calling \a MediaDevice::allocateRequests(). + * + * A request can then be passed to the V4L2Device::setControls(), + * V4L2Device::getControls() and V4L2VideoDevice::queueBuffer(). + */ + +/** + * \brief Construct a V4L2Request + * \param[in] fd The request fd + */ +V4L2Request::V4L2Request(int fd) + : fd_(fd), fdNotifier_(fd, EventNotifier::Exception) +{ + if (!fd_.isValid()) + return; + + fdNotifier_.activated.connect(this, &V4L2Request::requestReady); + fdNotifier_.setEnabled(false); +} + +/** + * \fn V4L2Request::isValid() + * \brief Check if the request is valid + * + * Checks if the underlying fd is valid. + * + * \return True if the request is valid, false otherwise + */ + +/** + * \fn V4L2Request::fd() + * \brief Get the file descriptor + * + * \return The file descriptor wrapped by this V4L2Request + */ + +/** + * \var V4L2Request::requestDone + * \brief Signal that is emitted when the request is done + */ + +/** + * \brief Reinit the request + * + * Calls MEDIA_REQUEST_IOC_REINIT om the request fd. + * + * \return 0 on success or a negative error code otherwise + */ +int V4L2Request::reinit() +{ + fdNotifier_.setEnabled(false); + + if (::ioctl(fd_.get(), MEDIA_REQUEST_IOC_REINIT) < 0) + return -errno; + + return 0; +} + +/** + * \brief Reinit the request + * + * Calls MEDIA_REQUEST_IOC_QUEUE om the request fd. + * + * \return 0 on success or a negative error code otherwise + */ +int V4L2Request::queue() +{ + if (::ioctl(fd_.get(), MEDIA_REQUEST_IOC_QUEUE) < 0) + return -errno; + + fdNotifier_.setEnabled(true); + + return 0; +} + +std::string V4L2Request::logPrefix() const +{ + return "Request [" + std::to_string(fd()) + "]"; +} + +/** + * \brief Slot to handle request done events + * + * When this slot is called, the request is done and the requestDone will be + * emitted. + */ +void V4L2Request::requestReady() +{ + requestDone.emit(this); +} + +} /* namespace libcamera */ diff --git a/src/libcamera/v4l2_videodevice.cpp b/src/libcamera/v4l2_videodevice.cpp index bb57c1b76a5b..8ce739f4bc65 100644 --- a/src/libcamera/v4l2_videodevice.cpp +++ b/src/libcamera/v4l2_videodevice.cpp @@ -30,6 +30,7 @@ #include "libcamera/internal/framebuffer.h" #include "libcamera/internal/media_device.h" #include "libcamera/internal/media_object.h" +#include "libcamera/internal/v4l2_request.h" /** * \file v4l2_videodevice.h @@ -1629,6 +1630,7 @@ int V4L2VideoDevice::releaseBuffers() /** * \brief Queue a buffer to the video device * \param[in] buffer The buffer to be queued + * \param[in] request An optional request * * For capture video devices the \a buffer will be filled with data by the * device. For output video devices the \a buffer shall contain valid data and @@ -1641,9 +1643,11 @@ int V4L2VideoDevice::releaseBuffers() * Note that queueBuffer() will fail if the device is in the process of being * stopped from a streaming state through streamOff(). * + * If \a request is specified, the buffer will be tied to that request. + * * \return 0 on success or a negative error code otherwise */ -int V4L2VideoDevice::queueBuffer(FrameBuffer *buffer) +int V4L2VideoDevice::queueBuffer(FrameBuffer *buffer, const V4L2Request *request) { struct v4l2_plane v4l2Planes[VIDEO_MAX_PLANES] = {}; struct v4l2_buffer buf = {}; @@ -1674,6 +1678,10 @@ int V4L2VideoDevice::queueBuffer(FrameBuffer *buffer) buf.type = bufferType_; buf.memory = memoryType_; buf.field = V4L2_FIELD_NONE; + if (request) { + buf.flags = V4L2_BUF_FLAG_REQUEST_FD; + buf.request_fd = request->fd(); + } bool multiPlanar = V4L2_TYPE_IS_MULTIPLANAR(buf.type); Span planes = buffer->planes(); From patchwork Thu Oct 23 14:48:07 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24734 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CA314BE080 for ; Thu, 23 Oct 2025 14:49:06 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5E594607FE; Thu, 23 Oct 2025 16:49:06 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="hQW+jv+N"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A2ABF607ED for ; Thu, 23 Oct 2025 16:49:04 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7328:357b:4ce1:72b6]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id BB2A9192B; Thu, 23 Oct 2025 16:47:19 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761230839; bh=xBaT0D20jSTsUMVoqorXpecGMHSBSZegT7R0a+xjw0A=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hQW+jv+Nk4O3vYkDuyV0h7JTrnAGihFCvxzdHvt8UwWhmaBjSnhMZH17qkNV/ub35 rlEZ1EG3xVFs6QppGxon3V/vEELk8Hw90mtOb1CyBej2rwYsAnxM9IvdiijXBIT6xM H1pHUCGL126lwSLW6jukmXDsZzd3KLOx/xEJvqvI= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 06/35] libcamera: converter: Add V4L2 request support Date: Thu, 23 Oct 2025 16:48:07 +0200 Message-ID: <20251023144841.403689-7-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251023144841.403689-1-stefan.klug@ideasonboard.com> References: <20251023144841.403689-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add V4L2 request support to the V4L2M2MConverter class. Extend the functions related to buffer queuing with an optional request parameter that gets passed to the lower layers. Signed-off-by: Stefan Klug --- include/libcamera/internal/converter.h | 4 +++- .../internal/converter/converter_v4l2_m2m.h | 6 ++++-- src/libcamera/converter.cpp | 3 +++ src/libcamera/converter/converter_v4l2_m2m.cpp | 12 ++++++++---- 4 files changed, 18 insertions(+), 7 deletions(-) diff --git a/include/libcamera/internal/converter.h b/include/libcamera/internal/converter.h index 4915af7ac5de..4b811686fcf6 100644 --- a/include/libcamera/internal/converter.h +++ b/include/libcamera/internal/converter.h @@ -22,6 +22,7 @@ #include #include +#include "libcamera/internal/v4l2_request.h" namespace libcamera { @@ -79,7 +80,8 @@ public: virtual void stop() = 0; virtual int queueBuffers(FrameBuffer *input, - const std::map &outputs) = 0; + const std::map &outputs, + const V4L2Request *request = nullptr) = 0; virtual int setInputCrop(const Stream *stream, Rectangle *rect) = 0; virtual std::pair inputCropBounds() = 0; diff --git a/include/libcamera/internal/converter/converter_v4l2_m2m.h b/include/libcamera/internal/converter/converter_v4l2_m2m.h index d316754040dd..1b2a88c4a608 100644 --- a/include/libcamera/internal/converter/converter_v4l2_m2m.h +++ b/include/libcamera/internal/converter/converter_v4l2_m2m.h @@ -66,7 +66,8 @@ public: Alignment align = Alignment::Down) override; int queueBuffers(FrameBuffer *input, - const std::map &outputs) override; + const std::map &outputs, + const V4L2Request *request = nullptr) override; int setInputCrop(const Stream *stream, Rectangle *rect) override; std::pair inputCropBounds() override { return inputCropBounds_; } @@ -88,7 +89,8 @@ private: int start(); void stop(); - int queueBuffers(FrameBuffer *input, FrameBuffer *output); + int queueBuffers(FrameBuffer *input, FrameBuffer *output, + const V4L2Request *request = nullptr); int setInputSelection(unsigned int target, Rectangle *rect); int getInputSelection(unsigned int target, Rectangle *rect); diff --git a/src/libcamera/converter.cpp b/src/libcamera/converter.cpp index 142fb29a1272..ec0a6db6c035 100644 --- a/src/libcamera/converter.cpp +++ b/src/libcamera/converter.cpp @@ -205,11 +205,14 @@ Converter::~Converter() * \param[in] input The frame buffer to apply the conversion * \param[out] outputs The container holding the output stream pointers and * their respective frame buffer outputs. + * \param[in] request An optional request * * This function queues the \a input frame buffer on the output streams of the * \a outputs map key and retrieve the output frame buffer indicated by the * buffer map value. * + * If \a request is provided the buffers are tied to that request. + * * \return 0 on success or a negative error code otherwise */ diff --git a/src/libcamera/converter/converter_v4l2_m2m.cpp b/src/libcamera/converter/converter_v4l2_m2m.cpp index b2bd54f368d8..ff11a9735db7 100644 --- a/src/libcamera/converter/converter_v4l2_m2m.cpp +++ b/src/libcamera/converter/converter_v4l2_m2m.cpp @@ -22,6 +22,7 @@ #include #include "libcamera/internal/media_device.h" +#include "libcamera/internal/v4l2_request.h" #include "libcamera/internal/v4l2_videodevice.h" /** @@ -197,9 +198,11 @@ void V4L2M2MConverter::V4L2M2MStream::stop() m2m_->output()->releaseBuffers(); } -int V4L2M2MConverter::V4L2M2MStream::queueBuffers(FrameBuffer *input, FrameBuffer *output) +int V4L2M2MConverter::V4L2M2MStream::queueBuffers(FrameBuffer *input, + FrameBuffer *output, + const V4L2Request *request) { - int ret = m2m_->output()->queueBuffer(input); + int ret = m2m_->output()->queueBuffer(input, request); if (ret < 0) return ret; @@ -696,7 +699,8 @@ int V4L2M2MConverter::validateOutput(StreamConfiguration *cfg, bool *adjusted, * \copydoc libcamera::Converter::queueBuffers */ int V4L2M2MConverter::queueBuffers(FrameBuffer *input, - const std::map &outputs) + const std::map &outputs, + const V4L2Request *request) { std::set outputBufs; int ret; @@ -721,7 +725,7 @@ int V4L2M2MConverter::queueBuffers(FrameBuffer *input, /* Queue the input and output buffers to all the streams. */ for (auto [stream, buffer] : outputs) { - ret = streams_.at(stream)->queueBuffers(input, buffer); + ret = streams_.at(stream)->queueBuffers(input, buffer, request); if (ret < 0) return ret; } From patchwork Thu Oct 23 14:48:08 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24735 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CD207C3335 for ; Thu, 23 Oct 2025 14:49:09 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 157AA607FB; Thu, 23 Oct 2025 16:49:09 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="GCLMIe51"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C2C7B607F3 for ; Thu, 23 Oct 2025 16:49:07 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7328:357b:4ce1:72b6]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id D55AF195E; Thu, 23 Oct 2025 16:47:22 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761230842; bh=EsKEi/8heauykb2897O4r4q7gIAaQEdKGvmtRnyf6Rg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GCLMIe51rp4716Buj4kFdpcuxuzFIICS3DJMY3m/f9HErcAQTEwFTkRNGmXq0yL1z bAKLdbAgeE4YKYDVnTNg7D4suZZar20AMelRxKRJAcdMgU4JKM/3ezCjWExsA8anji GBQBvTWTFe7a6fAu2MmbHj9H+SPUZxRB1Dz0tCY8= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 07/35] libcamera: converter_v4l2_m2m: Add suport for V4L2 requests Date: Thu, 23 Oct 2025 16:48:08 +0200 Message-ID: <20251023144841.403689-8-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251023144841.403689-1-stefan.klug@ideasonboard.com> References: <20251023144841.403689-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" To actually use requests with the m2m device, requests need to be allocated on the underlying media device. This can only be done if the media device is opened which means acquiring it. Add a function to check if the m2m device supports requests by acquiring the media device, asking it and then releasing it again. Also add a function to allocate requests that acquires the internal media device and releases it after allocating the requests. Signed-off-by: Stefan Klug --- ToDo: - Fix the place where the MediaObject is acquired/released Changes in v2: - Improved commit message - Dropped the change from private to protected as that belongs to a different patch --- .../internal/converter/converter_v4l2_m2m.h | 7 ++++ .../converter/converter_v4l2_m2m.cpp | 34 ++++++++++++++++++- 2 files changed, 40 insertions(+), 1 deletion(-) diff --git a/include/libcamera/internal/converter/converter_v4l2_m2m.h b/include/libcamera/internal/converter/converter_v4l2_m2m.h index 1b2a88c4a608..bc85bff7a07b 100644 --- a/include/libcamera/internal/converter/converter_v4l2_m2m.h +++ b/include/libcamera/internal/converter/converter_v4l2_m2m.h @@ -73,6 +73,11 @@ public: std::pair inputCropBounds() override { return inputCropBounds_; } std::pair inputCropBounds(const Stream *stream) override; + int allocateRequests(unsigned int count, + std::vector> *requests); + + bool supportsRequests(); + private: class V4L2M2MStream : protected Loggable { @@ -122,6 +127,8 @@ private: std::map> streams_; std::map queue_; std::pair inputCropBounds_; + + std::shared_ptr media_; }; } /* namespace libcamera */ diff --git a/src/libcamera/converter/converter_v4l2_m2m.cpp b/src/libcamera/converter/converter_v4l2_m2m.cpp index ff11a9735db7..c6153d728c9a 100644 --- a/src/libcamera/converter/converter_v4l2_m2m.cpp +++ b/src/libcamera/converter/converter_v4l2_m2m.cpp @@ -9,6 +9,7 @@ #include "libcamera/internal/converter/converter_v4l2_m2m.h" #include +#include #include #include #include @@ -266,7 +267,7 @@ void V4L2M2MConverter::V4L2M2MStream::captureBufferReady(FrameBuffer *buffer) * \param[in] media The media device implementing the converter */ V4L2M2MConverter::V4L2M2MConverter(std::shared_ptr media) - : Converter(media) + : Converter(media), media_(media) { if (deviceNode().empty()) return; @@ -742,6 +743,37 @@ int V4L2M2MConverter::queueBuffers(FrameBuffer *input, return 0; } +/** + * \copydoc libcamera::MediaDevice::allocateRequests + */ +int V4L2M2MConverter::allocateRequests(unsigned int count, + std::vector> *requests) +{ + /* \todo The acquire() must be moved to the right place. */ + media_->acquire(); + if (!media_->busy()) + LOG(Converter, Error) + << "MediaDevice must be valid."; + int ret = media_->allocateRequests(count, requests); + media_->release(); + return ret; +} + +/** + * \copydoc libcamera::MediaDevice::supportsRequests + */ +bool V4L2M2MConverter::supportsRequests() +{ + /* \todo The acquire() must be moved to the right place. */ + media_->acquire(); + if (!media_->busy()) + LOG(Converter, Error) + << "MediaDevice must be valid."; + bool ret = media_->supportsRequests(); + media_->release(); + return ret; +} + /* * \todo This should be extended to include Feature::Flag to denote * what each converter supports feature-wise. From patchwork Thu Oct 23 14:48:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24736 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4F52DC3336 for ; Thu, 23 Oct 2025 14:49:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C51DA607FF; Thu, 23 Oct 2025 16:49:11 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="TZU89tRo"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 524AC607F3 for ; Thu, 23 Oct 2025 16:49:10 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7328:357b:4ce1:72b6]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 50EC919E5; Thu, 23 Oct 2025 16:47:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761230845; bh=mMVNHkgEIJLKeXte+Y3d0TMfOmEz54a8GFjmuD3f6Ag=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TZU89tRoMYMQeyrGvt9plyyIpgbs86DEwvIYV+nNysjLCqItAzzjk83QDBNsDbGB5 2a5neG7/bfuqIAxZNK1dv9W50r92a9BD90mahHLaJY61SqZeUWJOw1ItzEfWUEsED4 D3imIytq7OVcX7rto1sVHoKGhqB6wg05swCgJU/U= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Kieran Bingham Subject: [PATCH v2 08/35] libcamera: converter_v4l2_m2m: Always set stride Date: Thu, 23 Oct 2025 16:48:09 +0200 Message-ID: <20251023144841.403689-9-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251023144841.403689-1-stefan.klug@ideasonboard.com> References: <20251023144841.403689-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Ensure the stride is properly set after a call to validateOutput(). Signed-off-by: Stefan Klug Reviewed-by: Kieran Bingham Reviewed-by: Isaac Scott --- Changes in v2: - Collected tag --- src/libcamera/converter/converter_v4l2_m2m.cpp | 1 + 1 file changed, 1 insertion(+) diff --git a/src/libcamera/converter/converter_v4l2_m2m.cpp b/src/libcamera/converter/converter_v4l2_m2m.cpp index c6153d728c9a..e57db8a438ab 100644 --- a/src/libcamera/converter/converter_v4l2_m2m.cpp +++ b/src/libcamera/converter/converter_v4l2_m2m.cpp @@ -680,6 +680,7 @@ int V4L2M2MConverter::validateOutput(StreamConfiguration *cfg, bool *adjusted, const Size cfgSize = cfg->size; cfg->size = adjustSizes(cfgSize, it->second, align); + cfg->stride = PixelFormatInfo::info(cfg->pixelFormat).stride(cfg->size.width, 0); if (cfg->size.isNull()) return -EINVAL; From patchwork Thu Oct 23 14:48:10 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24737 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E86C3C32CE for ; Thu, 23 Oct 2025 14:49:14 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 90B3B607FF; Thu, 23 Oct 2025 16:49:14 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="KcEQqh4p"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4110D607F3 for ; Thu, 23 Oct 2025 16:49:13 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7328:357b:4ce1:72b6]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 52FFA1D2B; Thu, 23 Oct 2025 16:47:28 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761230848; bh=D4qT+SNg9hd+BYvSKgmM0erFPZxmbrAtSDyfQm6udVw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=KcEQqh4pkTnZbFsA560AglxRhQgpyuJ1UKltR1Y0IkmZxTm7jihvNeYx7bhN103PR /UWW1DXSRMV6wNnH8TUPB6v1Cr3LrrHF/STBQmbrR5Zf7mHYQySGqr3Z36tLCXmJr+ vsEm15+s3AVaOXNREVC442LVukBtXxwJDIxu40rQ= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 09/35] pipeline: rkisp1: Use V4L2 requests for the dewarper Date: Thu, 23 Oct 2025 16:48:10 +0200 Message-ID: <20251023144841.403689-10-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251023144841.403689-1-stefan.klug@ideasonboard.com> References: <20251023144841.403689-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Use the V4L2 requests API if the dewarper supports it. Signed-off-by: Stefan Klug --- Changes in v2: - Replace strerrorname_np() by strerror() because the former is not supported on all our targets --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 59 ++++++++++++++++++++++-- 1 file changed, 56 insertions(+), 3 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 0e2a210e1e80..dd3907e2fb5f 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -44,6 +44,7 @@ #include "libcamera/internal/media_device.h" #include "libcamera/internal/media_pipeline.h" #include "libcamera/internal/pipeline_handler.h" +#include "libcamera/internal/v4l2_request.h" #include "libcamera/internal/v4l2_subdevice.h" #include "libcamera/internal/v4l2_videodevice.h" @@ -210,6 +211,7 @@ private: void imageBufferReady(FrameBuffer *buffer); void paramBufferReady(FrameBuffer *buffer); void statBufferReady(FrameBuffer *buffer); + void dewarpRequestReady(V4L2Request *request); void dewarpBufferReady(FrameBuffer *buffer); void frameStart(uint32_t sequence); @@ -239,6 +241,9 @@ private: std::vector> mainPathBuffers_; std::queue availableMainPathBuffers_; + std::vector> dewarpRequests_; + std::queue availableDewarpRequests_; + std::vector> paramBuffers_; std::vector> statBuffers_; std::queue availableParamBuffers_; @@ -1034,6 +1039,18 @@ int PipelineHandlerRkISP1::allocateBuffers(Camera *camera) for (std::unique_ptr &buffer : mainPathBuffers_) availableMainPathBuffers_.push(buffer.get()); + + if (dewarper_->supportsRequests()) { + ret = dewarper_->allocateRequests(kRkISP1MinBufferCount, + &dewarpRequests_); + if (ret < 0) + LOG(RkISP1, Error) << "Failed to allocate requests."; + } + + for (std::unique_ptr &request : dewarpRequests_) { + request->requestDone.connect(this, &PipelineHandlerRkISP1::dewarpRequestReady); + availableDewarpRequests_.push(request.get()); + } } auto pushBuffers = [&](const std::vector> &buffers, @@ -1075,6 +1092,11 @@ int PipelineHandlerRkISP1::freeBuffers(Camera *camera) statBuffers_.clear(); mainPathBuffers_.clear(); + while (!availableDewarpRequests_.empty()) + availableDewarpRequests_.pop(); + + dewarpRequests_.clear(); + std::vector ids; for (IPABuffer &ipabuf : data->ipaBuffers_) ids.push_back(ipabuf.id); @@ -1560,6 +1582,14 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) return; } + V4L2Request *dewarpRequest = nullptr; + if (!dewarpRequests_.empty()) { + /* If we have requests support, there must be one available */ + ASSERT(!availableDewarpRequests_.empty()); + dewarpRequest = availableDewarpRequests_.front(); + availableDewarpRequests_.pop(); + } + /* Handle scaler crop control. */ const auto &crop = request->controls().get(controls::ScalerCrop); if (crop) { @@ -1597,14 +1627,37 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) * buffers for the dewarper are the buffers of the request, supplied * by the application. */ - int ret = dewarper_->queueBuffers(buffer, request->buffers()); - if (ret < 0) - LOG(RkISP1, Error) << "Cannot queue buffers to dewarper: " + int ret = dewarper_->queueBuffers(buffer, request->buffers(), dewarpRequest); + if (ret < 0) { + LOG(RkISP1, Error) << "Failed to queue buffers to dewarper: -" << strerror(-ret); + /* Push it back into the queue. */ + if (dewarpRequest) + dewarpRequestReady(dewarpRequest); + + return; + } + + if (dewarpRequest) { + ret = dewarpRequest->queue(); + if (ret < 0) { + LOG(RkISP1, Error) << "Failed to queue dewarp request: -" + << strerror(-ret); + /* Push it back into the queue. */ + dewarpRequestReady(dewarpRequest); + } + } + request->metadata().set(controls::ScalerCrop, activeCrop_.value()); } +void PipelineHandlerRkISP1::dewarpRequestReady(V4L2Request *request) +{ + request->reinit(); + availableDewarpRequests_.push(request); +} + void PipelineHandlerRkISP1::dewarpBufferReady(FrameBuffer *buffer) { ASSERT(activeCamera_); From patchwork Thu Oct 23 14:48:11 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24738 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A254DC3337 for ; Thu, 23 Oct 2025 14:49:17 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1C14E6080B; Thu, 23 Oct 2025 16:49:17 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="J8jFyfEm"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0CD56607F3 for ; Thu, 23 Oct 2025 16:49:16 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7328:357b:4ce1:72b6]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 19EB0177F; Thu, 23 Oct 2025 16:47:31 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761230851; bh=UPrfQAHB12PnuqPOGB1JyHi3DTC3VkxWf6/NoxilbYQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=J8jFyfEmY83ew1jVfXs22k+px7ZwakI50ZDIf1NaFzrB+KS9KRsv7yEjQQnzDz/jy lTIHr5vS2gxBOJ7ED5znxD4XxVeh59v1MIpjdIdRfH2WA/7Q8zbaVgCIGv0Dc5wQ90 nE4fYY0srhD+/GzefKUeqVeuBws2OV/tBM1R75JY= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 10/35] libcamera: rkisp1: Properly cancel buffers in dewarp case Date: Thu, 23 Oct 2025 16:48:11 +0200 Message-ID: <20251023144841.403689-11-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251023144841.403689-1-stefan.klug@ideasonboard.com> References: <20251023144841.403689-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In case the buffer returned from the ISP was canceled, the upstream buffer was not correctly marked as canceled. Move the cancel functionality into an own helper function and correctly cancel the upstream buffers. Add missing cancellation in case queuing to the dewarper fails. Signed-off-by: Stefan Klug --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 48 +++++++++++++++--------- 1 file changed, 31 insertions(+), 17 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index dd3907e2fb5f..05409bc46ee3 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -208,6 +208,7 @@ private: int initLinks(Camera *camera, const RkISP1CameraConfiguration &config); int createCamera(MediaEntity *sensor); void tryCompleteRequest(RkISP1FrameInfo *info); + void cancelDewarpRequest(RkISP1FrameInfo *info); void imageBufferReady(FrameBuffer *buffer); void paramBufferReady(FrameBuffer *buffer); void statBufferReady(FrameBuffer *buffer); @@ -1521,6 +1522,31 @@ void PipelineHandlerRkISP1::tryCompleteRequest(RkISP1FrameInfo *info) completeRequest(request); } +void PipelineHandlerRkISP1::cancelDewarpRequest(RkISP1FrameInfo *info) +{ + RkISP1CameraData *data = cameraData(activeCamera_); + Request *request = info->request; + /* + * i.MX8MP is the only known platform with dewarper. It has + * no self path. Hence, only main path buffer completion is + * required. + * + * Also, we cannot completeBuffer(request, buffer) as buffer + * here, is an internal buffer (between ISP and dewarper) and + * is not associated to the any specific request. The request + * buffer associated with main path stream is the one that + * is required to be completed (not the internal buffer). + */ + for (auto [stream, buffer] : request->buffers()) { + if (stream == &data->mainPathStream_) { + buffer->_d()->cancel(); + completeBuffer(request, buffer); + } + } + + tryCompleteRequest(info); +} + void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) { ASSERT(activeCamera_); @@ -1562,23 +1588,7 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) /* Do not queue cancelled frames to dewarper. */ if (metadata.status == FrameMetadata::FrameCancelled) { - /* - * i.MX8MP is the only known platform with dewarper. It has - * no self path. Hence, only main path buffer completion is - * required. - * - * Also, we cannot completeBuffer(request, buffer) as buffer - * here, is an internal buffer (between ISP and dewarper) and - * is not associated to the any specific request. The request - * buffer associated with main path stream is the one that - * is required to be completed (not the internal buffer). - */ - for (auto it : request->buffers()) { - if (it.first == &data->mainPathStream_) - completeBuffer(request, it.second); - } - - tryCompleteRequest(info); + cancelDewarpRequest(info); return; } @@ -1636,6 +1646,8 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) if (dewarpRequest) dewarpRequestReady(dewarpRequest); + cancelDewarpRequest(info); + return; } @@ -1646,6 +1658,8 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) << strerror(-ret); /* Push it back into the queue. */ dewarpRequestReady(dewarpRequest); + + cancelDewarpRequest(info); } } From patchwork Thu Oct 23 14:48:12 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24739 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A5C7EC3338 for ; Thu, 23 Oct 2025 14:49:20 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 020186080E; Thu, 23 Oct 2025 16:49:20 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="A1AWzmHN"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A572E607F3 for ; Thu, 23 Oct 2025 16:49:18 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7328:357b:4ce1:72b6]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id A86D51D2B; Thu, 23 Oct 2025 16:47:33 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761230853; bh=4F7ELLotJVJcHalhvwbW20jGoXCfsdYBPgrM9zGn19w=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=A1AWzmHNYpaBLzXI9PhTIADOXWU88ORtNnzbi70RAP9Q8Sp++9RQJpn3DsdZ4blpL UNoz07KUFbWy7SxYcvoA/EyFaYZqDD45Rk+xwlmazJIFncQHBYJfZxH2tV9VucggHU BFST2wy4m5I83FrnxCVd2GzPBPpFLoj+M0KWkk7M= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Xavier Roumegue , Umang Jain , Jacopo Mondi Subject: [PATCH v2 11/35] include: linux: Update headers for Dw100 dewarper engine Date: Thu, 23 Oct 2025 16:48:12 +0200 Message-ID: <20251023144841.403689-12-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251023144841.403689-1-stefan.klug@ideasonboard.com> References: <20251023144841.403689-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Xavier Roumegue Include the DW100 dewarper header as part of linux headers. It will be used to set the V4L2_CID_DW100_DEWARPING_16x16_VERTEX_MAP control by ConverterDW100 Signed-off-by: Xavier Roumegue Signed-off-by: Umang Jain Acked-by: Jacopo Mondi --- include/linux/dw100.h | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100644 include/linux/dw100.h diff --git a/include/linux/dw100.h b/include/linux/dw100.h new file mode 100644 index 000000000000..3356496edd6b --- /dev/null +++ b/include/linux/dw100.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0-only WITH Linux-syscall-note */ +/* Copyright 2022 NXP */ + +#ifndef __UAPI_DW100_H__ +#define __UAPI_DW100_H__ + +#include + +/* + * Check Documentation/userspace-api/media/drivers/dw100.rst for control details. + */ +#define V4L2_CID_DW100_DEWARPING_16x16_VERTEX_MAP (V4L2_CID_USER_DW100_BASE + 1) + +#endif From patchwork Thu Oct 23 14:48:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24740 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0FD40C3331 for ; Thu, 23 Oct 2025 14:49:25 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id AB5C360811; Thu, 23 Oct 2025 16:49:24 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Rnk0QDZh"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 43AD160803 for ; Thu, 23 Oct 2025 16:49:22 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7328:357b:4ce1:72b6]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 4E2F2EFE; Thu, 23 Oct 2025 16:47:37 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761230857; bh=7FkM8Y3TcCjkciHfIi7EET5p0hjejxGMzSOiDuFEngA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Rnk0QDZhyoMM2QxolnXI9z16eiDPYShX9nRbW5qLf+9LdXfgBQp/P6+XaA+NT214m sEM6FVQA9oV3D1ZcJMFgSIAO963POVR4I33gGdIRPGQBE6fzPiKFeMu7fxW7e5IVuH CTRVvkEmo3yxlfXjJV2GYjntKEzqJVkyEzGW/cb4= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Umang Jain Subject: [PATCH v2 12/35] libcamera: converter_v4l2_m2m: Add helper to apply controls Date: Thu, 23 Oct 2025 16:48:13 +0200 Message-ID: <20251023144841.403689-13-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251023144841.403689-1-stefan.klug@ideasonboard.com> References: <20251023144841.403689-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Umang Jain Add applyControls() helper to apply controls for a stream. Signed-off-by: Umang Jain Signed-off-by: Stefan Klug --- Changes in v2: - Removed stray semicolon - Added documentation Changes in v0.9 - Include request support in applyControls --- .../internal/converter/converter_v4l2_m2m.h | 5 +++ .../converter/converter_v4l2_m2m.cpp | 34 +++++++++++++++++++ 2 files changed, 39 insertions(+) diff --git a/include/libcamera/internal/converter/converter_v4l2_m2m.h b/include/libcamera/internal/converter/converter_v4l2_m2m.h index bc85bff7a07b..6d2453bb9e06 100644 --- a/include/libcamera/internal/converter/converter_v4l2_m2m.h +++ b/include/libcamera/internal/converter/converter_v4l2_m2m.h @@ -24,6 +24,7 @@ namespace libcamera { +class ControlList; class FrameBuffer; class MediaDevice; class Size; @@ -73,6 +74,8 @@ public: std::pair inputCropBounds() override { return inputCropBounds_; } std::pair inputCropBounds(const Stream *stream) override; + int applyControls(const Stream *stream, ControlList &ctrls, const V4L2Request *request = nullptr); + int allocateRequests(unsigned int count, std::vector> *requests); @@ -94,6 +97,8 @@ private: int start(); void stop(); + int applyControls(ControlList &ctrls, const V4L2Request *request = nullptr); + int queueBuffers(FrameBuffer *input, FrameBuffer *output, const V4L2Request *request = nullptr); diff --git a/src/libcamera/converter/converter_v4l2_m2m.cpp b/src/libcamera/converter/converter_v4l2_m2m.cpp index e57db8a438ab..48e49f143cf1 100644 --- a/src/libcamera/converter/converter_v4l2_m2m.cpp +++ b/src/libcamera/converter/converter_v4l2_m2m.cpp @@ -18,6 +18,7 @@ #include #include +#include #include #include #include @@ -251,6 +252,20 @@ void V4L2M2MConverter::V4L2M2MStream::captureBufferReady(FrameBuffer *buffer) converter_->outputBufferReady.emit(buffer); } +/** + * \brief Apply controls + * \param[in] ctrls The controls to apply + * \param[in] request An optional request + * + * \return 0 on success or a negative error code otherwise + * \see V4L2Device::setControls() + */ +int V4L2M2MConverter::V4L2M2MStream::applyControls(ControlList &ctrls, + const V4L2Request *request) +{ + return m2m_->capture()->setControls(&ctrls, request); +} + /* ----------------------------------------------------------------------------- * V4L2M2MConverter */ @@ -744,6 +759,25 @@ int V4L2M2MConverter::queueBuffers(FrameBuffer *input, return 0; } +/** + * \brief Apply controls + * \param[in] stream The stream on which to apply the controls + * \param[in] ctrls The controls to apply + * \param[in] request An optional request + * + * \return 0 on success or a negative error code otherwise + * \see V4L2Device::setControls() + */ +int V4L2M2MConverter::applyControls(const Stream *stream, ControlList &ctrls, + const V4L2Request *request) +{ + auto iter = streams_.find(stream); + if (iter == streams_.end()) + return -EINVAL; + + return iter->second->applyControls(ctrls, request); +} + /** * \copydoc libcamera::MediaDevice::allocateRequests */ From patchwork Thu Oct 23 14:48:14 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24741 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 033C0C32D4 for ; Thu, 23 Oct 2025 14:49:26 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 53CCF6080B; Thu, 23 Oct 2025 16:49:26 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="UU6CE6j3"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 31ED660809 for ; Thu, 23 Oct 2025 16:49:25 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7328:357b:4ce1:72b6]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 3B6A6EFE; Thu, 23 Oct 2025 16:47:40 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761230860; bh=IPeWjIvsK/SnFf3hCJYPEyCivN39LUmhW+Z5pIM09yA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UU6CE6j3RvO/Zw07Tws3/P1W5I//2vwgQRjYu1BCQmCPPQDQ/AZtw77rKJ0dKnyoY wA8fhWuxPctrRDRRGCv4Vrv53kgOfZrnrwrMLwwHl1Ahld59O2N9SGZxWcbgA8O1aU p+CUsMFdo5mmVn4l4HQpS/SOmPW40csT5XQktn+4= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Kieran Bingham Subject: [PATCH v2 13/35] libcamera: converter_v4l2_m2m: Add debug logging for formats Date: Thu, 23 Oct 2025 16:48:14 +0200 Message-ID: <20251023144841.403689-14-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251023144841.403689-1-stefan.klug@ideasonboard.com> References: <20251023144841.403689-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Getting the requested input/output format is crucial for debugging. Add log statements for that. Signed-off-by: Stefan Klug Reviewed-by: Kieran Bingham --- Changes in v2: - Collected tag --- src/libcamera/converter/converter_v4l2_m2m.cpp | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/libcamera/converter/converter_v4l2_m2m.cpp b/src/libcamera/converter/converter_v4l2_m2m.cpp index 48e49f143cf1..37377ab24952 100644 --- a/src/libcamera/converter/converter_v4l2_m2m.cpp +++ b/src/libcamera/converter/converter_v4l2_m2m.cpp @@ -111,6 +111,7 @@ int V4L2M2MConverter::V4L2M2MStream::configure(const StreamConfiguration &inputC format.planesCount = 1; format.planes[0].bpl = inputCfg.stride; + LOG(Converter, Debug) << "Set input format to: " << format; int ret = m2m_->output()->setFormat(&format); if (ret < 0) { LOG(Converter, Error) @@ -133,6 +134,7 @@ int V4L2M2MConverter::V4L2M2MStream::configure(const StreamConfiguration &inputC format.fourcc = videoFormat; format.size = outputCfg.size; + LOG(Converter, Debug) << "Set output format to: " << format; ret = m2m_->capture()->setFormat(&format); if (ret < 0) { LOG(Converter, Error) From patchwork Thu Oct 23 14:48:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24743 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 09BD5C3332 for ; Thu, 23 Oct 2025 14:49:32 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 60DBD60815; Thu, 23 Oct 2025 16:49:32 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="eaaP0Znq"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C41DD60805 for ; Thu, 23 Oct 2025 16:49:30 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7328:357b:4ce1:72b6]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id C615AEFE; Thu, 23 Oct 2025 16:47:45 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761230866; bh=YhJnWUezisdJ/mTNbzAkqL/6+LmXLgXb7afsnmrL9js=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=eaaP0Znq/mDxgmxA0jRZVWK1Q937OsPmu2747D9oyL9izbrBzt5OTAPKlXPV+KkVk mDyBm30oW06aKdtaeSBYPO0NweGkWSwmclZ8LmxAeodydTwSYvzYJoQWreGu6obQBu tGOCZhyZ1fNctAKKdiQI2vy4yVSiegWoZDWqZAB0= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 15/35] libcamera: rkisp1: Scale down in dewarper instead of resizer Date: Thu, 23 Oct 2025 16:48:16 +0200 Message-ID: <20251023144841.403689-16-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251023144841.403689-1-stefan.klug@ideasonboard.com> References: <20251023144841.403689-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In order to allow digital zooming, scale down in the dewarper instead of the resizer. That means forwarding the full sensor size data to the dewarper. The ScalerCrop rectangle will also be applied at the dewarper. Signed-off-by: Stefan Klug --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 43 +++++++++++++++++++----- 1 file changed, 35 insertions(+), 8 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 241c83f4125b..46bc81f2483c 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -888,16 +888,31 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) /* * On devices without DUAL_CROP (like the imx8mp) cropping needs to be * done on the ISP/IS output. + * + * If the dewarper is used, the cropping shall be done by the dewarper. */ if (media_->hwRevision() == RKISP1_V_IMX8MP) { /* imx8mp has only a single path. */ const auto &cfg = config->at(0); - Size ispCrop = format.size.boundedToAspectRatio(cfg.size); + /* + * If the dewarper is used, all cropping including aspect ratio + * preservation shall be done there. To ensure that the output + * format provided by the ISP is supported by the dewarper, a + * minimal crop still needs to be applied on the ISP output. + * + * \todo It might be possible to allocate bigger buffers + * (aligned to 8 pixels) with a stride matching format.size for + * the ISP. The not-filled border could later be ignored by the + * dewarper. This way we could skip the minimal crop here and + * the MaximumScalerCrop would always match the isp output. + */ + Size ispCrop; if (data->usesDewarper_) ispCrop = dewarper_->adjustInputSize(cfg.pixelFormat, - ispCrop); + format.size); else - ispCrop.alignUpTo(2, 2); + ispCrop = format.size.boundedToAspectRatio(cfg.size) + .alignedUpTo(2, 2); outputCrop = ispCrop.centeredTo(Rectangle(format.size).center()); format.size = ispCrop; @@ -930,13 +945,21 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) for (const StreamConfiguration &cfg : *config) { if (cfg.stream() == &data->mainPathStream_) { - ret = mainPath_.configure(cfg, format); - streamConfig[0] = IPAStream(cfg.pixelFormat, - cfg.size); - /* Configure dewarp */ + /* + * To allow for digital zoom, scaling down should happen + * in the dewarper, instead of the resizer. Configure + * the isp output to the same size as the sensor output. + */ + StreamConfiguration ispCfg = cfg; if (data->usesDewarper_) { outputCfgs.push_back(const_cast(cfg)); - ret = dewarper_->configure(cfg, outputCfgs); + + ispCfg.size = format.size; + ispCfg.stride = + PixelFormatInfo::info(ispCfg.pixelFormat) + .stride(ispCfg.size.width, 0); + + ret = dewarper_->configure(ispCfg, outputCfgs); if (ret) return ret; @@ -949,6 +972,10 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) outputCrop.transformedBetween(inputCrop, sensorInfo.analogCrop); } + + ret = mainPath_.configure(ispCfg, format); + streamConfig[0] = IPAStream(cfg.pixelFormat, + cfg.size); } else if (hasSelfPath_) { ret = selfPath_.configure(cfg, format); streamConfig[1] = IPAStream(cfg.pixelFormat, From patchwork Thu Oct 23 14:48:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24744 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9E1D3C333B for ; Thu, 23 Oct 2025 14:49:36 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 24A0060815; Thu, 23 Oct 2025 16:49:36 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="OvWcuz/z"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4806E60805 for ; Thu, 23 Oct 2025 16:49:34 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7328:357b:4ce1:72b6]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 59870EFE; Thu, 23 Oct 2025 16:47:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761230869; bh=kg+s632GFTl2fgzosj6Jq2H2VdCgjkp2sbqQIpqgexA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=OvWcuz/zUGaHB2OxVJGA9HrxFf8JfdkqiZTQ7SNAgf7bU/EmtQy2/L+p6AGcNPIfB TBg6AFS+HzSvTgUqiK/zrcHnjMg/97CivkMbXl2uU6fkoM1Ygvj+6tZBRm64WSPT+v M/FOhA1o57kIlyqhFUhHuZsc02rE3sB04+AEvFOg= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 16/35] libcamera: rkisp1: Allow upscaling when the dewarper is present Date: Thu, 23 Oct 2025 16:48:17 +0200 Message-ID: <20251023144841.403689-17-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251023144841.403689-1-stefan.klug@ideasonboard.com> References: <20251023144841.403689-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When the dewarper is present, there is no need to forbid upscaling of the image data. Signed-off-by: Stefan Klug --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 46bc81f2483c..77442b25b809 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -614,6 +614,11 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() return false; if (useDewarper) { + /* + * The dewarper output is independent of the ISP path. + * Reset to the originally requested size. + */ + tryCfg.size = cfg.size; bool adjusted; pipe->dewarper_->validateOutput(&tryCfg, &adjusted, From patchwork Thu Oct 23 14:48:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24745 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 03B18C333C for ; Thu, 23 Oct 2025 14:49:41 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5DA3B6081D; Thu, 23 Oct 2025 16:49:41 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Kr5+X9sk"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A3F8B60805 for ; Thu, 23 Oct 2025 16:49:40 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7328:357b:4ce1:72b6]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id A3C7E591; Thu, 23 Oct 2025 16:47:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761230875; bh=pd+DFxuBTqdfIbDhvLPeoBe2m2oY+1pIzgQ0LsDtZz8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Kr5+X9skKniY8p7UGXojnuCkhkaXcwDeXJ8HxQ/M1+C6BiJAfEbBE47q0BicFVlP0 UD08tIcLcuKqyrfwIgzKwUI3S0pzcg2FGoxA3qmJWqAqsKzuj7Jgee2yIdoyU8V3Hu 9TnohXoiCRmb7uhMOiFOi0hR5wSoxCF6CEdF+TiU= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 18/35] libcamera: converter_m2m: Make some parts protected Date: Thu, 23 Oct 2025 16:48:19 +0200 Message-ID: <20251023144841.403689-19-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251023144841.403689-1-stefan.klug@ideasonboard.com> References: <20251023144841.403689-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In order to derive from V4L2M2MConverter and also from V4L2M2MStream in a subclass, raise the visibility of most of the private parts to protected and document them. Signed-off-by: Stefan Klug --- Changes in v2: - Added this patch --- .../internal/converter/converter_v4l2_m2m.h | 7 +- .../converter/converter_v4l2_m2m.cpp | 107 ++++++++++++++++++ 2 files changed, 111 insertions(+), 3 deletions(-) diff --git a/include/libcamera/internal/converter/converter_v4l2_m2m.h b/include/libcamera/internal/converter/converter_v4l2_m2m.h index 2d6f361c17e1..79d008fc6918 100644 --- a/include/libcamera/internal/converter/converter_v4l2_m2m.h +++ b/include/libcamera/internal/converter/converter_v4l2_m2m.h @@ -81,7 +81,7 @@ public: bool supportsRequests(); -private: +protected: class V4L2M2MStream : protected Loggable { public: @@ -130,12 +130,13 @@ private: virtual std::unique_ptr makeStream(const Stream *stream); std::unique_ptr m2m_; + std::shared_ptr media_; std::map> streams_; + +private: std::map queue_; std::pair inputCropBounds_; - - std::shared_ptr media_; }; } /* namespace libcamera */ diff --git a/src/libcamera/converter/converter_v4l2_m2m.cpp b/src/libcamera/converter/converter_v4l2_m2m.cpp index 2e3966444361..3a05e45ac33c 100644 --- a/src/libcamera/converter/converter_v4l2_m2m.cpp +++ b/src/libcamera/converter/converter_v4l2_m2m.cpp @@ -86,6 +86,28 @@ int getCropBounds(V4L2VideoDevice *device, Rectangle &minCrop, * V4L2M2MConverter::V4L2M2MStream */ +/** + * \class libcamera::V4L2M2MConverter::V4L2M2MStream + * \brief The V4L2 M2M stream represents a stream in a V4L2M2MConverter + * + * The Converter interface allows to create multiple output image streams from + * a single input image stream. This class represents one such stream. + */ + +/** + * \fn V4L2M2MConverter::V4L2M2MStream::isValid + * \brief Checks if the stream is valid + * + * This function checks if the underlying m2m device is valid. + * + * \return True if the stream is valid, false otherwise + */ + +/** + * \brief Constructs a V4L2M2MStream + * \param[in] converter The converter this stream belongs to + * \param[in] stream The stream this V4L2M2MStream represents + */ V4L2M2MConverter::V4L2M2MStream::V4L2M2MStream(V4L2M2MConverter *converter, const Stream *stream) : converter_(converter), stream_(stream) { @@ -99,6 +121,13 @@ V4L2M2MConverter::V4L2M2MStream::V4L2M2MStream(V4L2M2MConverter *converter, cons m2m_.reset(); } +/** + * \brief Configure the stream + * \param[in] inputCfg The input config + * \param[in] outputCfg The output config + * + * \return 0 on success or a negative error code otherwise + */ int V4L2M2MConverter::V4L2M2MStream::configure(const StreamConfiguration &inputCfg, const StreamConfiguration &outputCfg) { @@ -161,12 +190,24 @@ int V4L2M2MConverter::V4L2M2MStream::configure(const StreamConfiguration &inputC return 0; } +/** + * \brief Export buffers + * \param[in] count The number of buffers + * \param[out] buffers The exported buffers + * + * \return 0 on success or a negative error code otherwise + */ int V4L2M2MConverter::V4L2M2MStream::exportBuffers(unsigned int count, std::vector> *buffers) { return m2m_->capture()->exportBuffers(count, buffers); } +/** + * \brief Start the stream + * + * \return 0 on success or a negative error code otherwise + */ int V4L2M2MConverter::V4L2M2MStream::start() { int ret = m2m_->output()->importBuffers(inputBufferCount_); @@ -194,6 +235,9 @@ int V4L2M2MConverter::V4L2M2MStream::start() return 0; } +/** + * \brief Stop the stream + */ void V4L2M2MConverter::V4L2M2MStream::stop() { m2m_->capture()->streamOff(); @@ -202,6 +246,14 @@ void V4L2M2MConverter::V4L2M2MStream::stop() m2m_->output()->releaseBuffers(); } +/** + * \brief Queue buffers + * \param[in] input The input buffer + * \param[in] output The output buffer + * \param[in] request An optional request + * + * \return 0 on success or a negative error code otherwise + */ int V4L2M2MConverter::V4L2M2MStream::queueBuffers(FrameBuffer *input, FrameBuffer *output, const V4L2Request *request) @@ -217,16 +269,35 @@ int V4L2M2MConverter::V4L2M2MStream::queueBuffers(FrameBuffer *input, return 0; } +/** + * \brief Get the input selection rectangle + * \param[in] target The selection target + * \param[out] rect The selection rectangle + * + * \return 0 on success or a negative error code otherwise + */ int V4L2M2MConverter::V4L2M2MStream::getInputSelection(unsigned int target, Rectangle *rect) { return m2m_->output()->getSelection(target, rect); } +/** + * \brief Set the input selection rectangle + * \param[in] target The selection target + * \param[in] rect The selection rectangle + * + * \return 0 on success or a negative error code otherwise + */ int V4L2M2MConverter::V4L2M2MStream::setInputSelection(unsigned int target, Rectangle *rect) { return m2m_->output()->setSelection(target, rect); } +/** + * \brief Get the input crop bounds + * + * \return A pair of rectangles representing the min and max input crop bounds + */ std::pair V4L2M2MConverter::V4L2M2MStream::inputCropBounds() { return inputCropBounds_; @@ -461,6 +532,19 @@ Size V4L2M2MConverter::adjustOutputSize(const PixelFormat &pixFmt, return adjustSizes(size, it->second, align); } +/** + * \brief Adjust the converter output \a size to a valid value + * \param[in] cfgSize The requested size + * \param[in] ranges The possible sizes + * \param[in] align The desired alignment + * + * Selects the best fitting size from \a ranges according to \a align and + * returns that. + * + * \return The adjusted converter output size or a null Size if \a size cannot + * be adjusted + * \see libcamera::Converter::adjustOutputSize + */ Size V4L2M2MConverter::adjustSizes(const Size &cfgSize, const std::vector &ranges, Alignment align) @@ -810,11 +894,34 @@ bool V4L2M2MConverter::supportsRequests() return ret; } +/** + * \brief Create a V4L2M2MStream + * \param[in] stream The stream to wrap + * + * This function creates a new V4L2M2MConverter::V4L2M2MStream for this + * converter and the provided \a stream. + * + * This function can be overwritten by subclasses to be able provide their own + * stream implementation. + * + * \return The created V4L2M2MStream + */ std::unique_ptr V4L2M2MConverter::makeStream(const Stream *stream) { return std::make_unique(this, stream); } +/** + * \var V4L2M2MConverter::m2m_ + * \brief The underlying m2m device + * + * \var V4L2M2MConverter::media_ + * \brief The underlying media device + * + * \var V4L2M2MConverter::streams_ + * \brief Map of Stream pointers to V4L2M2MStream unique pointers + */ + /* * \todo This should be extended to include Feature::Flag to denote * what each converter supports feature-wise. From patchwork Thu Oct 23 14:48:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24746 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B7B4EC333D for ; Thu, 23 Oct 2025 14:49:45 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3BFCD60827; Thu, 23 Oct 2025 16:49:45 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Qk28LQCt"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B0E2F6081B for ; Thu, 23 Oct 2025 16:49:43 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7328:357b:4ce1:72b6]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id BB8D0177F; Thu, 23 Oct 2025 16:47:58 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761230878; bh=swYmmu87kl8GkVUrs4QNf1ckrkILpGD5Fx54UnIycSA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Qk28LQCt487Y6ToVxr4Ezl1MhvzuprcD/2BOEPAjsAn1UTTY0CAsfZfewgU3yQ+UQ c/+MkC0X7yd/Gtt2gzqlkCpIWAEqOcWqStMdyiMQvZnv6Yz+61SteYGJwdC1H2Hv3g j5QSLLqP0B4b8fyyxVBY94MVxPlBTL3gisBjUbXA= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Umang Jain , Paul Elder Subject: [PATCH v2 19/35] libcamera: converter: Add dw100 converter class Date: Thu, 23 Oct 2025 16:48:20 +0200 Message-ID: <20251023144841.403689-20-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251023144841.403689-1-stefan.klug@ideasonboard.com> References: <20251023144841.403689-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Umang Jain DW100 Dewarp engine is present on i.MX8MP SoC. This patch provides a dedicated converter class (inherited from V4L2M2MConverter). The DW100 dewarp engine has scaling capabilites and can be used to set crop rectangles in the rkisp1 pipeline handler. Plumbing in the RkISP1 pipeline-handler is done in the subsequent patch. The ConverterDW100 class will be extended by upcoming patches to support the dewarping by setting a dewarp vertex map as well. Signed-off-by: Umang Jain Reviewed-by: Paul Elder Signed-off-by: Stefan Klug --- Changes in v2: - Drop validateSize() as it is not used anywhere Changes in v0.9 - Use shared_ptr in constructor --- .../internal/converter/converter_dw100.h | 24 ++++++++++++ .../libcamera/internal/converter/meson.build | 1 + src/libcamera/converter/converter_dw100.cpp | 37 +++++++++++++++++++ src/libcamera/converter/meson.build | 1 + 4 files changed, 63 insertions(+) create mode 100644 include/libcamera/internal/converter/converter_dw100.h create mode 100644 src/libcamera/converter/converter_dw100.cpp diff --git a/include/libcamera/internal/converter/converter_dw100.h b/include/libcamera/internal/converter/converter_dw100.h new file mode 100644 index 000000000000..dc41f365b5c7 --- /dev/null +++ b/include/libcamera/internal/converter/converter_dw100.h @@ -0,0 +1,24 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (C) 2024, Ideas On Board Oy + * + * i.MX8MP Dewarp Engine integration + */ + +#pragma once + +#include "libcamera/internal/converter/converter_v4l2_m2m.h" + +namespace libcamera { + +class MediaDevice; +class Rectangle; +class Stream; + +class ConverterDW100 : public V4L2M2MConverter +{ +public: + ConverterDW100(std::shared_ptr media); +}; + +} /* namespace libcamera */ diff --git a/include/libcamera/internal/converter/meson.build b/include/libcamera/internal/converter/meson.build index 891e79e7d493..85007a4b0f8b 100644 --- a/include/libcamera/internal/converter/meson.build +++ b/include/libcamera/internal/converter/meson.build @@ -1,5 +1,6 @@ # SPDX-License-Identifier: CC0-1.0 libcamera_internal_headers += files([ + 'converter_dw100.h', 'converter_v4l2_m2m.h', ]) diff --git a/src/libcamera/converter/converter_dw100.cpp b/src/libcamera/converter/converter_dw100.cpp new file mode 100644 index 000000000000..1d2d0435b08e --- /dev/null +++ b/src/libcamera/converter/converter_dw100.cpp @@ -0,0 +1,37 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (C) 2024, Ideas On Board Oy + * + * i.MX8MP Dewarp Engine integration + */ + +#include "libcamera/internal/converter/converter_dw100.h" + +#include + +#include + +#include "libcamera/internal/media_device.h" +#include "libcamera/internal/v4l2_videodevice.h" + +namespace libcamera { + +LOG_DECLARE_CATEGORY(Converter) + +/** + * \class libcamera::ConverterDW100 + * \brief The i.MX8MP dewarp converter implements the converter interface based + * on V4L2 M2M device. +*/ + +/** + * \fn ConverterDW100::ConverterDW100 + * \brief Construct a ConverterDW100 instance + * \param[in] media The media device implementing the converter + */ +ConverterDW100::ConverterDW100(std::shared_ptr media) + : V4L2M2MConverter(media) +{ +} + +} /* namespace libcamera */ diff --git a/src/libcamera/converter/meson.build b/src/libcamera/converter/meson.build index af1a80fec683..fe2dcebb67da 100644 --- a/src/libcamera/converter/meson.build +++ b/src/libcamera/converter/meson.build @@ -1,5 +1,6 @@ # SPDX-License-Identifier: CC0-1.0 libcamera_internal_sources += files([ + 'converter_dw100.cpp', 'converter_v4l2_m2m.cpp' ]) From patchwork Thu Oct 23 14:48:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24747 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4AADFC3334 for ; Thu, 23 Oct 2025 14:49:48 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 01C586083A; Thu, 23 Oct 2025 16:49:48 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="q2j6+grh"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9138F6081B for ; Thu, 23 Oct 2025 16:49:46 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7328:357b:4ce1:72b6]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 841631127; Thu, 23 Oct 2025 16:48:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761230881; bh=LbhVCKPmpquMvTodiUYckjrhEt0/lz50k6XNAmtyidg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=q2j6+grh5C3iKJIJZ4WL2LYKeoyr945BNugpty5zDM7Q7rZI8NCH+W4FGpyu9JkcI aqyGpXpIL0yMIJiptXu1hNZ7RN98AMSFlYop3OsHtoWLzQxH4RTLrFpbGxpuWEdPV2 dlbIekmMQGzCPuR/bh8g2vubwLFNoRUNwUUKRfGM= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 20/35] libcamera: converter: Add dw100 vertex map class Date: Thu, 23 Oct 2025 16:48:21 +0200 Message-ID: <20251023144841.403689-21-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251023144841.403689-1-stefan.klug@ideasonboard.com> References: <20251023144841.403689-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Using a custom vertex map the dw100 dewarper is capable of doing complex and useful transformations on the image data. This class implements a pipeline featuring: - Arbitrary ScalerCrop - Full transform support (Flip, 90deg rotations) - Arbitrary move, scale, rotate ScalerCrop and Transform is implemented to provide a interface that is standardized libcamera wide. The rest is implemented on top for more flexible dw100 specific features. Signed-off-by: Stefan Klug --- Changes in v2: - Replaced manual transforms with an affine transformation matrix - Changed rotation direction to be in sync with the rotation in CameraConfiguration::orientation - Changed offset parameter to be in ScalerCrop coordinates. This is easier to explain and has the added benefit, that Scale/Rotate is always centered to the visible image. - Improved code comments - Make dw100VerticesForLength a local function - Dropped unnecessary includes - Added documentation Changes in v0.9 - Include header in meson.build - Fix black line at top and left when rotation 180 degrees Changes in v0.8 - Cleanup & formatting Changes in v0.5 - Fix crash in std::clamp() due to rounding errors --- .../converter/converter_dw100_vertexmap.h | 76 +++ .../libcamera/internal/converter/meson.build | 1 + .../converter/converter_dw100_vertexmap.cpp | 566 ++++++++++++++++++ src/libcamera/converter/meson.build | 1 + 4 files changed, 644 insertions(+) create mode 100644 include/libcamera/internal/converter/converter_dw100_vertexmap.h create mode 100644 src/libcamera/converter/converter_dw100_vertexmap.cpp diff --git a/include/libcamera/internal/converter/converter_dw100_vertexmap.h b/include/libcamera/internal/converter/converter_dw100_vertexmap.h new file mode 100644 index 000000000000..e72cb72bb9f1 --- /dev/null +++ b/include/libcamera/internal/converter/converter_dw100_vertexmap.h @@ -0,0 +1,76 @@ +#pragma once + +#include +#include +#include +#include + +#include + +#include +#include + +namespace libcamera { + +class Dw100VertexMap +{ +public: + enum ScaleMode { + Fill = 0, + Crop = 1, + }; + + void applyLimits(); + void setInputSize(const Size &size) + { + inputSize_ = size; + scalerCrop_ = Rectangle(size); + } + + void setSensorCrop(const Rectangle &rect) { sensorCrop_ = rect; } + + void setScalerCrop(const Rectangle &rect) { scalerCrop_ = rect; } + const Rectangle &effectiveScalerCrop() const { return effectiveScalerCrop_; } + std::pair scalerCropBounds() const + { + return { Rectangle(sensorCrop_.x, sensorCrop_.y, 1, 1), + sensorCrop_ }; + } + + void setOutputSize(const Size &size) { outputSize_ = size; } + const Size &outputSize() const { return outputSize_; } + + void setTransform(const Transform &transform) { transform_ = transform; } + const Transform &transform() const { return transform_; } + + void setScale(const float scale) { scale_ = scale; } + float effectiveScale() const { return (effectiveScaleX_ + effectiveScaleY_) * 0.5; } + + void setRotation(const float rotation) { rotation_ = rotation; } + float rotation() const { return rotation_; } + + void setOffset(const Point &offset) { offset_ = offset; } + const Point &effectiveOffset() const { return effectiveOffset_; } + + void setMode(const ScaleMode mode) { mode_ = mode; } + ScaleMode mode() const { return mode_; } + + std::vector getVertexMap(); + +private: + Rectangle scalerCrop_; + Rectangle sensorCrop_; + Transform transform_ = Transform::Identity; + Size inputSize_; + Size outputSize_; + Point offset_; + double scale_ = 1.0; + double rotation_ = 0.0; + ScaleMode mode_ = Fill; + double effectiveScaleX_; + double effectiveScaleY_; + Point effectiveOffset_; + Rectangle effectiveScalerCrop_; +}; + +} /* namespace libcamera */ diff --git a/include/libcamera/internal/converter/meson.build b/include/libcamera/internal/converter/meson.build index 85007a4b0f8b..128c644cb73f 100644 --- a/include/libcamera/internal/converter/meson.build +++ b/include/libcamera/internal/converter/meson.build @@ -2,5 +2,6 @@ libcamera_internal_headers += files([ 'converter_dw100.h', + 'converter_dw100_vertexmap.h', 'converter_v4l2_m2m.h', ]) diff --git a/src/libcamera/converter/converter_dw100_vertexmap.cpp b/src/libcamera/converter/converter_dw100_vertexmap.cpp new file mode 100644 index 000000000000..0e930479b6f7 --- /dev/null +++ b/src/libcamera/converter/converter_dw100_vertexmap.cpp @@ -0,0 +1,566 @@ +#include "libcamera/internal/converter/converter_dw100_vertexmap.h" + +#include +#include +#include +#include +#include +#include + +#include +#include + +#include +#include + +#include "libcamera/internal/vector.h" + +constexpr int kDw100BlockSize = 16; + +namespace libcamera { + +LOG_DECLARE_CATEGORY(Converter) +namespace { + +using Vector2d = Vector; +using Vector3d = Vector; +using Matrix3x3 = Matrix; + +Matrix3x3 makeTranslate(const double tx, const double ty) +{ + Matrix3x3 m = Matrix3x3::identity(); + m[0][2] = tx; + m[1][2] = ty; + return m; +} + +Matrix3x3 makeTranslate(const Vector2d &t) +{ + return makeTranslate(t.x(), t.y()); +} + +Matrix3x3 makeRotate(const double degrees) +{ + double rad = degrees / 180.0 * M_PI; + double sa = std::sin(rad); + double ca = std::cos(rad); + + Matrix3x3 m = Matrix3x3::identity(); + m[0][0] = ca; + m[0][1] = -sa; + m[1][0] = sa; + m[1][1] = ca; + return m; +} + +Matrix3x3 makeScale(const double sx, const double sy) +{ + Matrix3x3 m = Matrix3x3::identity(); + m[0][0] = sx; + m[1][1] = sy; + return m; +} + +/** + * \param t The transform to apply + * \param size The size of the rectangle that is transformed + * + * Create a matrix that represents the transform done by the \a t. It assumes + * that the origin of the coordinate system is at the top left corner of of the + * rectangle. + */ +Matrix3x3 makeTransform(const Transform &t, const Size &size) +{ + Matrix3x3 m = Matrix3x3::identity(); + double wm = size.width * 0.5; + double hm = size.height * 0.5; + m = makeTranslate(-wm, -hm) * m; + + if (!!(t & Transform::HFlip)) + m = makeScale(-1, 1) * m; + + if (!!(t & Transform::VFlip)) + m = makeScale(1, -1) * m; + + if (!!(t & Transform::Transpose)) { + m = makeRotate(-90) * m; + m = makeScale(1, -1) * m; + std::swap(wm, hm); + } + + m = makeTranslate(wm, hm) * m; + + return m; +} + +/** + * \param from The source rectangle + * \param to The destination rectangle + * + * Create a matrix that transforms from the coordinate system of rectangle \a + * from into the coordinate system of rectangle \a to, by overlaying the + * rectangles. + * + * \see Rectangle::transformedBetween() + */ +Matrix3x3 makeTransform(const Rectangle &from, const Rectangle &to) +{ + Matrix3x3 m = Matrix3x3::identity(); + double sx = to.width / static_cast(from.width); + double sy = to.height / static_cast(from.height); + m = makeTranslate(-from.x, -from.y) * m; + m = makeScale(sx, sy) * m; + m = makeTranslate(to.x, to.y) * m; + return m; +} + +Vector2d transformPoint(const Matrix3x3 &m, const Vector2d &p) +{ + Vector3d p2{ { p.x(), p.y(), 1.0 } }; + p2 = m * p2; + return { { p2.x() / p2.z(), p2.y() / p2.z() } }; +} + +Vector2d transformVector(const Matrix3x3 &m, const Vector2d &p) +{ + Vector3d p2{ { p.x(), p.y(), 0.0 } }; + p2 = m * p2; + return { { p2.x(), p2.y() } }; +} + +Vector2d rotatedRectSize(const Vector2d &size, const double degrees) +{ + double rad = degrees / 180.0 * M_PI; + double sa = sin(rad); + double ca = cos(rad); + + return { { std::abs(size.x() * ca) + std::abs(size.y() * sa), + std::abs(size.x() * sa) + std::abs(size.y() * ca) } }; +} + +Vector2d point2Vec2d(const Point &p) +{ + return { { static_cast(p.x), static_cast(p.y) } }; +} + +int dw100VerticesForLength(const int length) +{ + return (length + kDw100BlockSize - 1) / kDw100BlockSize + 1; +} + +} /* namespace */ + +/** + * \class libcamera::Dw100VertexMap + * \brief Helper class to compute dw100 vertex maps + * + * The vertex map class represents a helper for handling dewarper vertex maps. + * There are 3 important sizes in the system: + * + * - The sensor size. The number of pixels of the whole sensor (\todo specify + * the crop rectangle). + * - The input rectangle to the dewarper. Describes the pixel data flowing into + * the dewarper in sensor coordinates. + * - ScalerCrop rectangle. The rectangle that shall be used for all further + * stages. It is applied after lens dewarping but is in sensor coordinate + * space. + * - The output size. This defines the size, the dewarper should output. + * + * +------------------------+ + * |Sensor size | + * | +----------------+ | + * | | Input rect | | + * | | +-------------+ | + * | | | ScalerCrop | | + * | | | | | + * | +--+-------------+ | + * +------------------------+ + * + * This class implements a vertex map that forms the following pipeline: + * + * +-------------+ +-------------+ +------------+ +-----------------+ + * | | -> | | | Transform | | Pan/Zoom | + * | Lens Dewarp | -> | Scaler Crop | -> | (H/V Flip, | -> | (Offset, Scale, | + * | | | | | Transpose) | | Rotate) | + * +-------------+ +-------------+ +------------+ +-----------------+ + * + * \todo Lens dewarp is not yet implemented. An identity map is used instead. + * + * All parameters are clamped to valid values before creating the vertex map. + * + * The constrains process works as follows: + * - The ScalerCrop rectangle is clamped to the input rectangle + * - The ScalerCrop rectangle is transformed by the specified transform + * forming ScalerCropT + * - A rectangle of output size is placed in the center of ScalerCropT + * (OutputRect). + * - Rotate gets applied to OutputRect, + * - Scale is applied, but clamped so that the OutputRect fits completely into + * ScalerCropT (Only regarding dimensions, not position) + * - Offset is clamped so that the OutputRect lies inside ScalerCropT + * + * The lens dewarp map is usually calibrated during tuning and is a map that + * maps from incoming pixels to dewarped pixels. + */ + +/** + * \enum Dw100VertexMap::ScaleMode + * \brief The scale modes available for a vertex map + * + * \var Dw100VertexMap::Fill + * \brief Scale the input to fill the output + * + * This scale mode does not preserve aspect ratio. Offset and rotation are taken + * into account. + * + * \var Dw100VertexMap::Crop + * \brief Crop the input + * + * This scale mode preserves the aspect ratio. Offset, scale, rotation are taken + * into account within the possible limits. + */ + +/** + * \brief Apply limits on scale and offset + * + * This function calculates \a effectiveScalerCrop_, \a effectiveScale_ and \a + * effectiveOffset_ based on the requested scaler crop, scale, rotation, offset + * and the selected scale mode, so that the whole output area is filled with + * valid input data. + */ +void Dw100VertexMap::applyLimits() +{ + int ow = outputSize_.width; + int oh = outputSize_.height; + effectiveScalerCrop_ = scalerCrop_.boundedTo(sensorCrop_); + + /* Map the scalerCrop to the input pixel space */ + Rectangle localScalerCrop = effectiveScalerCrop_.transformedBetween( + sensorCrop_, Rectangle(inputSize_)); + + Size localCropSizeT = localScalerCrop.size(); + if (!!(transform_ & Transform::Transpose)) + std::swap(localCropSizeT.width, localCropSizeT.height); + + Vector2d size = rotatedRectSize(point2Vec2d({ ow, oh }), rotation_); + + /* Calculate constraints */ + double scale = scale_; + if (mode_ == Crop) { + /* Scale up if needed */ + scale = std::max(scale, + std::max(size.x() / localCropSizeT.width, + size.y() / localCropSizeT.height)); + effectiveScaleX_ = scale; + effectiveScaleY_ = scale; + + size = size / scale; + + } else if (mode_ == Fill) { + effectiveScaleX_ = size.x() / localCropSizeT.width; + effectiveScaleY_ = size.y() / localCropSizeT.height; + + size.x() /= effectiveScaleX_; + size.y() /= effectiveScaleY_; + } else { + LOG(Converter, Error) << "Unknown mode " << mode_; + return; + } + + /* + * Clamp offset. Due to rounding errors, size might be slightly bigger + * than scaler crop. Clamp the offset to 0 to prevent a crash in the + * next clamp. + */ + double maxoffX, maxoffY; + maxoffX = std::max(0.0, (localCropSizeT.width - size.x())) * 0.5; + maxoffY = std::max(0.0, (localCropSizeT.height - size.y())) * 0.5; + if (!!(transform_ & Transform::Transpose)) + std::swap(maxoffX, maxoffY); + + /* + * Transform the offset from sensor space to local space, apply the + * limit and transform back. + */ + Vector2d offset = point2Vec2d(offset_); + Matrix3x3 m; + + m = makeTransform(effectiveScalerCrop_, localScalerCrop); + offset = transformVector(m, offset); + offset.x() = std::clamp(offset.x(), -maxoffX, maxoffX); + offset.y() = std::clamp(offset.y(), -maxoffY, maxoffY); + m = makeTransform(localScalerCrop, effectiveScalerCrop_); + offset = transformVector(m, offset); + effectiveOffset_.x = offset.x(); + effectiveOffset_.y = offset.y(); +} + +/** + * \fn Dw100VertexMap::setInputSize() + * \brief Set the size of the input data + * \param[in] size The input size + * + * To calculate a proper vertex map, the size of the input images must be set. + */ + +/** + * \fn Dw100VertexMap::setSensorCrop() + * \brief Set the crop rectangle that represents the input data + * \param[in] rect + * + * Set the rectangle that represents the input data in sensor coordinates. This + * must be specified to properly calculate the vertex map. + */ + +/** + * \fn Dw100VertexMap::setScalerCrop() + * \brief Set the requested scaler crop + * \param[in] rect + * + * Set the requested scaler crop. The actually applied scaler crop can be + * queried using \a Dw100VertexMap::effectiveScalerCrop() after calling + * Dw100VertexMap::applyLimits(). + */ + +/** + * \fn Dw100VertexMap::effectiveScalerCrop() + * \brief Get the effective scaler crop + * + * \return The effective scaler crop + */ + +/** + * \fn Dw100VertexMap::scalerCropBounds() + * \brief Get the min and max values for the scaler crop + * + * \return A pair of rectangles that represent the scaler crop min/max values + */ + +/** + * \fn Dw100VertexMap::setOutputSize() + * \brief Set the output size + * \param[in] size The size of the output images + */ + +/** + * \fn Dw100VertexMap::outputSize() + * \brief Get the output size + * \return The output size + */ + +/** + * \fn Dw100VertexMap::setTransform() + * \brief Sets the transform to apply + * \param[in] transform The transform + */ + +/** + * \fn Dw100VertexMap::transform() + * \brief Get the transform + * \return The transform + */ + +/** + * \fn Dw100VertexMap::setScale() + * \brief Sets the scale to apply + * \param[in] scale The scale + * + * Set the requested scale. The actually applied scale can be queried using \a + * Dw100VertexMap::effectiveScale() after calling \a + * Dw100VertexMap::applyLimits(). + */ + +/** + * \fn Dw100VertexMap::effectiveScale() + * \brief Get the effective scale + * + * Returns the actual scale applied to the input pixels in x and y direction. So + * a value of [2.0, 1.5] means that every input pixel is scaled to cover 2 + * output pixels in x-direction and 1.5 in y-direction. + * + * \return The effective scale + */ + +/** + * \fn Dw100VertexMap::setRotation() + * \brief Sets the rotation to apply + * \param[in] rotation The rotation in degrees + * + * The rotation is in clockwise direction to allow the same transform as + * CameraConfiguration::orientation + */ + +/** + * \fn Dw100VertexMap::rotation() + * \brief Get the rotation + * \return The rotation in degrees + */ + +/** + * \fn Dw100VertexMap::setOffset() + * \brief Sets the offset to apply + * \param[in] offset The offset + * + * Set the requested offset. The actually applied offset can be queried using \a + * Dw100VertexMap::effectiveOffset() after calling \a + * Dw100VertexMap::applyLimits(). + */ + +/** + * \fn Dw100VertexMap::effectiveOffset() + * \brief Get the effective offset + * + * Returns the actual offset applied to the input pixels in ScalerCrop + * coordinates. + * + * \return The effective offset + */ + +/** + * \fn Dw100VertexMap::setMode() + * \brief Sets the scaling mode to apply + * \param[in] mode The mode + */ + +/** + * \fn Dw100VertexMap::mode() + * \brief Get the scaling mode + * \return The scaling mode + */ + +/** + * \brief Get the dw100 vertex map + * + * Calculates the vertex map as a vector of hardware specific entries. + * + * \return The vertex map + */ +std::vector Dw100VertexMap::getVertexMap() +{ + int ow = outputSize_.width; + int oh = outputSize_.height; + int tileCountW = dw100VerticesForLength(ow); + int tileCountH = dw100VerticesForLength(oh); + + applyLimits(); + + /* + * libcamera handles all crop rectangles in sensor space. But the + * dewarper "sees" only the pixels it gets passed. Note that these might + * not cover exactly the max sensor crop, as there might be a crop + * between ISP and dewarper to crop to a format supported by the + * dewarper. effectiveScalerCrop_ is the crop in sensor space that gets + * fed into the dewarper. localScalerCrop is the sensor crop mapped to + * the data that is fed into the dewarper. + */ + Rectangle localScalerCrop = effectiveScalerCrop_.transformedBetween( + sensorCrop_, Rectangle(inputSize_)); + Size localCropSizeT = localScalerCrop.size(); + if (!!(transform_ & Transform::Transpose)) + std::swap(localCropSizeT.width, localCropSizeT.height); + + /* + * The dw100 has a specialty in interpolation that has to be taken into + * account to use in a pixel perfect manner. To explain this, I will + * only use the x direction, the vertical axis behaves the same. + * + * Let's start with a pixel perfect 1:1 mapping of an image with a width + * of 64pixels. The coordinates of the vertex map would then be: + * 0 -- 16 -- 32 -- 48 -- 64 + * Note how the last coordinate lies outside the image (which ends at + * 63) as it is basically the beginning of the next macro block. + * + * if we zoom out a bit we might end up with something like + * -10 -- 0 -- 32 -- 64 -- 74 + * As the dewarper coordinates are unsigned it actually sees + * 0 -- 0 -- 32 -- 64 -- 74 + * Leading to stretched pixels at the beginning and black for everything + * > 63 + * + * Now lets rotate the image by 180 degrees. A trivial rotation would + * end up with: + * + * 64 -- 48 -- 32 -- 16 -- 0 + * + * But as the first column now points to pixel 64 we get a single black + * line. So for a proper 180* rotation, the coordinates need to be + * + * 63 -- 47 -- 31 -- 15 -- -1 + * + * The -1 is clamped to 0 again, leading to a theoretical slight + * interpolation error on the last 16 pixels. + * + * To create this proper transformation there are two things todo: + * + * 1. The rotation centers are offset by -0.5. This evens out for no + * rotation, and leads to a coordinate offset of -1 on 180 degree + * rotations. + * 2. The transformation (flip and transpose) need to act on a size-1 + * to get the same effect. + */ + Vector2d centerS{ { localCropSizeT.width * 0.5 - 0.5, + localCropSizeT.height * 0.5 - 0.5 } }; + Vector2d centerD{ { ow * 0.5 - 0.5, + oh * 0.5 - 0.5 } }; + + LOG(Converter, Debug) + << "Apply vertex map for" + << " inputSize: " << inputSize_ + << " outputSize: " << outputSize_ + << " Transform: " << transformToString(transform_) + << "\n effectiveScalerCrop: " << effectiveScalerCrop_ + << " localCropSizeT: " << localCropSizeT + << " scaleX: " << effectiveScaleX_ + << " scaleY: " << effectiveScaleX_ + << " rotation: " << rotation_ + << " offset: " << effectiveOffset_; + + Matrix3x3 outputToSensor = Matrix3x3::identity(); + /* Move to center of output */ + outputToSensor = makeTranslate(-centerD) * outputToSensor; + outputToSensor = makeRotate(-rotation_) * outputToSensor; + outputToSensor = makeScale(1.0 / effectiveScaleX_, 1.0 / effectiveScaleY_) * outputToSensor; + /* Move to top left of localScalerCropT */ + outputToSensor = makeTranslate(centerS) * outputToSensor; + outputToSensor = makeTransform(-transform_, localCropSizeT.shrunkBy({ 1, 1 })) * + outputToSensor; + /* Transform from "within localScalerCrop" to input reference frame */ + outputToSensor = makeTranslate(localScalerCrop.x, localScalerCrop.y) * outputToSensor; + outputToSensor = makeTransform(localScalerCrop, effectiveScalerCrop_) * outputToSensor; + outputToSensor = makeTranslate(point2Vec2d(effectiveOffset_)) * outputToSensor; + + Matrix3x3 sensorToInput = makeTransform(effectiveScalerCrop_, localScalerCrop); + + /* + * For every output tile, calculate the position of the corners in the + * input image. + */ + std::vector res; + res.reserve(tileCountW * tileCountH); + for (int y = 0; y < tileCountH; y++) { + for (int x = 0; x < tileCountW; x++) { + Vector2d p{ { static_cast(x) * kDw100BlockSize, + static_cast(y) * kDw100BlockSize } }; + p = p.max(0.0).min(Vector2d{ { static_cast(ow), + static_cast(oh) } }); + + p = transformPoint(outputToSensor, p); + + /* + * \todo: Transformations in sensor space to be added + * here. + */ + + p = transformPoint(sensorToInput, p); + + /* Convert to fixed point */ + uint32_t v = static_cast(p.y() * 16) << 16 | + (static_cast(p.x() * 16) & 0xffff); + res.push_back(v); + } + } + + return res; +} + +} /* namespace libcamera */ diff --git a/src/libcamera/converter/meson.build b/src/libcamera/converter/meson.build index fe2dcebb67da..9f59b57c26b9 100644 --- a/src/libcamera/converter/meson.build +++ b/src/libcamera/converter/meson.build @@ -1,6 +1,7 @@ # SPDX-License-Identifier: CC0-1.0 libcamera_internal_sources += files([ + 'converter_dw100_vertexmap.cpp', 'converter_dw100.cpp', 'converter_v4l2_m2m.cpp' ]) From patchwork Thu Oct 23 14:48:22 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24748 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 36CB2C333E for ; Thu, 23 Oct 2025 14:49:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9EA4B6083E; Thu, 23 Oct 2025 16:49:50 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="hW2jcGXj"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BA4116081B for ; Thu, 23 Oct 2025 16:49:49 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7328:357b:4ce1:72b6]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id C795CEFE; Thu, 23 Oct 2025 16:48:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761230884; bh=zMPONoLmMpfnqpDZLSi9vEhOn9O6IhoX/4F6zyJ6XB4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hW2jcGXjNJhyrMljDGOkxg3sNheakHeq4Mkxae8ti9yyZrwK1fh4ICrIqt8ttqIxb nd5FDIfrPftlDOWreqF1DOqrhRXYpAatJS1B/E43CvrIzF0XghG3JKOx8FPRImkXct B3Gmz8OqGXo6bQaqyNky3QePizhxvCrFIyRiqFtg= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 21/35] libcamera: converter_dw100: Use vertex map Date: Thu, 23 Oct 2025 16:48:22 +0200 Message-ID: <20251023144841.403689-22-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251023144841.403689-1-stefan.klug@ideasonboard.com> References: <20251023144841.403689-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Use the new vertexmap class in the dw100 dewarper implementation. As the dw100 kernel driver does not use the requests API, controls can not be tied to a buffer. In the case that the dewarper has buffers queued for which processing has not yet started, applying the dewarp map would apply to an already queued buffer. Warn when this situation occurs. Signed-off-by: Stefan Klug --- Changes in v2: - Added documentation Changes in v0.9: - Include requests support Changes in v0.8: - Added warning when queue is not empty --- .../internal/converter/converter_dw100.h | 19 +++++ .../internal/converter/converter_v4l2_m2m.h | 4 +- src/libcamera/converter/converter_dw100.cpp | 74 +++++++++++++++++++ 3 files changed, 95 insertions(+), 2 deletions(-) diff --git a/include/libcamera/internal/converter/converter_dw100.h b/include/libcamera/internal/converter/converter_dw100.h index dc41f365b5c7..54bc7de4de77 100644 --- a/include/libcamera/internal/converter/converter_dw100.h +++ b/include/libcamera/internal/converter/converter_dw100.h @@ -7,6 +7,7 @@ #pragma once +#include "libcamera/internal/converter/converter_dw100_vertexmap.h" #include "libcamera/internal/converter/converter_v4l2_m2m.h" namespace libcamera { @@ -19,6 +20,24 @@ class ConverterDW100 : public V4L2M2MConverter { public: ConverterDW100(std::shared_ptr media); + + int applyVertexMap(const Stream *stream, const V4L2Request *request = nullptr); + Dw100VertexMap &vertexMap(const Stream *stream); + +private: + std::unique_ptr makeStream(const Stream *stream) override; + class DW100Stream : public V4L2M2MConverter::V4L2M2MStream + { + public: + DW100Stream(V4L2M2MConverter *converter, const Stream *stream); + + int configure(const StreamConfiguration &inputCfg, + const StreamConfiguration &outputCfg) override; + + int applyVertexMap(const V4L2Request *request = nullptr); + + Dw100VertexMap vertexMap_; + }; }; } /* namespace libcamera */ diff --git a/include/libcamera/internal/converter/converter_v4l2_m2m.h b/include/libcamera/internal/converter/converter_v4l2_m2m.h index 79d008fc6918..03b6670daf22 100644 --- a/include/libcamera/internal/converter/converter_v4l2_m2m.h +++ b/include/libcamera/internal/converter/converter_v4l2_m2m.h @@ -89,8 +89,8 @@ protected: bool isValid() const { return m2m_ != nullptr; } - int configure(const StreamConfiguration &inputCfg, - const StreamConfiguration &outputCfg); + virtual int configure(const StreamConfiguration &inputCfg, + const StreamConfiguration &outputCfg); int exportBuffers(unsigned int count, std::vector> *buffers); diff --git a/src/libcamera/converter/converter_dw100.cpp b/src/libcamera/converter/converter_dw100.cpp index 1d2d0435b08e..f0764ac8229e 100644 --- a/src/libcamera/converter/converter_dw100.cpp +++ b/src/libcamera/converter/converter_dw100.cpp @@ -7,9 +7,12 @@ #include "libcamera/internal/converter/converter_dw100.h" +#include + #include #include +#include #include "libcamera/internal/media_device.h" #include "libcamera/internal/v4l2_videodevice.h" @@ -34,4 +37,75 @@ ConverterDW100::ConverterDW100(std::shared_ptr media) { } +/** + * \brief Apply the vertex map for a given stream + * \param[in] stream The stream to update + * \param[in] request An optional request + * + * This function updates the vertex map on the stream \a stream. If \a request + * is provided, the updated happens in that request. + * + * \return 0 on success or a negative error code otherwise + */ +int ConverterDW100::applyVertexMap(const Stream *stream, const V4L2Request *request) +{ + auto iter = streams_.find(stream); + if (iter == streams_.end()) + return -EINVAL; + + return dynamic_cast(iter->second.get())->applyVertexMap(request); +} + +/** + * \brief Get the vertex map for a stream + * \param[in] stream The stream + * + * This function returns a reference to the vertex map of the stream \a stream. + * + * \return The vertex map + */ +Dw100VertexMap &ConverterDW100::vertexMap(const Stream *stream) +{ + auto iter = streams_.find(stream); + V4L2M2MConverter::V4L2M2MStream *s = iter->second.get(); + ASSERT(s); + return dynamic_cast(s)->vertexMap_; +} + +std::unique_ptr ConverterDW100::makeStream(const Stream *stream) +{ + return std::unique_ptr(new DW100Stream(this, stream)); +} + +ConverterDW100::DW100Stream::DW100Stream(V4L2M2MConverter *converter, + const Stream *stream) + : V4L2M2MConverter::V4L2M2MStream(converter, stream) +{ +} + +int ConverterDW100::DW100Stream::configure(const StreamConfiguration &inputCfg, + const StreamConfiguration &outputCfg) +{ + int ret; + ret = V4L2M2MConverter::V4L2M2MStream::configure(inputCfg, outputCfg); + if (ret) + return ret; + + /* Todo: is that comment still valid? + For actual dewarping we need the active area of the sensor mode here. */ + vertexMap_.setInputSize(inputCfg.size); + vertexMap_.setOutputSize(outputCfg.size); + return 0; +} + +int ConverterDW100::DW100Stream::applyVertexMap(const V4L2Request *request) +{ + std::vector map = vertexMap_.getVertexMap(); + auto value = Span(reinterpret_cast(&map[0]), map.size()); + + ControlList ctrls; + ctrls.set(V4L2_CID_DW100_DEWARPING_16x16_VERTEX_MAP, value); + return applyControls(ctrls, request); +} + } /* namespace libcamera */ From patchwork Thu Oct 23 14:48:23 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24749 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1D973C333F for ; Thu, 23 Oct 2025 14:49:53 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7EE3060837; Thu, 23 Oct 2025 16:49:53 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="c4nnMWGO"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A49896082F for ; Thu, 23 Oct 2025 16:49:52 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7328:357b:4ce1:72b6]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id AF56B1127; Thu, 23 Oct 2025 16:48:07 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761230887; bh=B3Raujcg+ETvVR9IhtXNlejI19RXE7D07mOmJSXM8nY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=c4nnMWGOZPEu3fGzEWV92VvMrZKv5+cx0tWqxLlFBfnVK2w/8KN22LNEe5JLye2Sx Zyx5pit+T/faI7AvplZ3gJz+k8tdRGlP4GrVIZA0MT5dFYnpMpOIm0HXiecKs3AJO6 4OwMpu/UTD9Gl0799+QYnDwvjquzUw2VyfiHKs8w= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 22/35] libcamera: rkisp1: Use the dw100 class instead of the generic v4l2 converter Date: Thu, 23 Oct 2025 16:48:23 +0200 Message-ID: <20251023144841.403689-23-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251023144841.403689-1-stefan.klug@ideasonboard.com> References: <20251023144841.403689-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" To be able to access the dw100 specific functions and as there is no known other dewarper in combination with the rkisp1, use the dw100 specific converter directly. Signed-off-by: Stefan Klug --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 77442b25b809..e22f05408931 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -36,7 +36,7 @@ #include "libcamera/internal/camera.h" #include "libcamera/internal/camera_sensor.h" #include "libcamera/internal/camera_sensor_properties.h" -#include "libcamera/internal/converter/converter_v4l2_m2m.h" +#include "libcamera/internal/converter/converter_dw100.h" #include "libcamera/internal/delayed_controls.h" #include "libcamera/internal/device_enumerator.h" #include "libcamera/internal/framebuffer.h" @@ -234,7 +234,7 @@ private: RkISP1MainPath mainPath_; RkISP1SelfPath selfPath_; - std::unique_ptr dewarper_; + std::unique_ptr dewarper_; Rectangle scalerMaxCrop_; std::optional activeCrop_; @@ -1500,7 +1500,7 @@ bool PipelineHandlerRkISP1::match(DeviceEnumerator *enumerator) std::shared_ptr dwpMediaDevice = enumerator->search(dwp); if (dwpMediaDevice) { - dewarper_ = std::make_unique(dwpMediaDevice); + dewarper_ = std::make_unique(dwpMediaDevice); if (dewarper_->isValid()) { dewarper_->outputBufferReady.connect( this, &PipelineHandlerRkISP1::dewarpBufferReady); From patchwork Thu Oct 23 14:48:24 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24750 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 82BADC3340 for ; Thu, 23 Oct 2025 14:49:57 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 080EE6083D; Thu, 23 Oct 2025 16:49:57 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="vZWi5pnd"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A560E6082F for ; Thu, 23 Oct 2025 16:49:55 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7328:357b:4ce1:72b6]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id B0182177F; Thu, 23 Oct 2025 16:48:10 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761230890; bh=8ze/wVtYSCNyH2pXLXiMKgc7r4dl8SlTkahzl1pW36g=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=vZWi5pndber5GuXZapm/DJ7qOCHUti03yyWnUq4N1TnSocbAcic9z9/TrpOEBAEaV PwNVUb0lKtXfg722rQ6gpBcaHFyux+jrJM5v8wwJa7qCWJgS9I+w1I/AsbEGbs27dJ njAz1QEazRV+kUYkv375Jwp1X0aDq4NYvvX4qltg= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 23/35] libcamera: rkisp1: Use vertex map to implement ScalerCrop Date: Thu, 23 Oct 2025 16:48:24 +0200 Message-ID: <20251023144841.403689-24-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251023144841.403689-1-stefan.klug@ideasonboard.com> References: <20251023144841.403689-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The input crop implemented in the dw100 kernel driver is quite limited in that it doesn't allow arbitrary crop rectangles but only scale factors quantized to the underlying fixed point representation and only aspect ratio preserving crops. The vertex map based implementation allows for pixel perfect crops. The only downside is that ScalerCrop can no longer be set dynamically on older kernels. Warn if the user tries to set ScalerCrop on such a setup. As the vertex map is now set on start() it no longer needs to be updated for frame==0. Signed-off-by: Stefan Klug --- Changes in 0.9 - Code cleanup - Update vertex map before start() to partially support old kernels Changes in 0.7: - Removed double call to applyVertexMap() --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 115 ++++++++++++----------- 1 file changed, 59 insertions(+), 56 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index e22f05408931..0b31b8077c8d 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -235,9 +235,6 @@ private: RkISP1SelfPath selfPath_; std::unique_ptr dewarper_; - Rectangle scalerMaxCrop_; - - std::optional activeCrop_; /* Internal buffers used when dewarper is being used */ std::vector> mainPathBuffers_; @@ -969,13 +966,22 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) return ret; /* - * Calculate the crop rectangle of the data - * flowing into the dewarper in sensor - * coordinates. + * Apply the actual sensor crop, for proper + * dewarp map calculation + */ + Rectangle sensorCrop = outputCrop.transformedBetween( + inputCrop, sensorInfo.analogCrop); + auto &vertexMap = dewarper_->vertexMap(cfg.stream()); + vertexMap.setSensorCrop(sensorCrop); + data->properties_.set(properties::ScalerCropMaximum, sensorCrop); + + /* + * Apply a default sensor crop that keeps the + * aspect ratio. */ - scalerMaxCrop_ = - outputCrop.transformedBetween(inputCrop, - sensorInfo.analogCrop); + Size crop = format.size.boundedToAspectRatio(cfg.size); + vertexMap.setScalerCrop(crop.centeredTo( + Rectangle(format.size).center())); } ret = mainPath_.configure(ispCfg, format); @@ -1186,6 +1192,18 @@ int PipelineHandlerRkISP1::start(Camera *camera, [[maybe_unused]] const ControlL actions += [&]() { stat_->streamOff(); }; if (data->usesDewarper_) { + /* + * Apply the vertex map before start to partially + * support ScalerCrop on kernels with a dw100 driver + * that has no dynamic vertex map/requests support. + */ + if (controls && controls->contains(controls::ScalerCrop.id())) { + const auto &crop = controls->get(controls::ScalerCrop); + auto &vertexMap = dewarper_->vertexMap(&data->mainPathStream_); + vertexMap.setScalerCrop(*crop); + } + dewarper_->applyVertexMap(&data->mainPathStream_); + ret = dewarper_->start(); if (ret) { LOG(RkISP1, Error) << "Failed to start dewarper"; @@ -1338,25 +1356,31 @@ int PipelineHandlerRkISP1::updateControls(RkISP1CameraData *data) if (data->usesDewarper_) { std::pair cropLimits; - if (dewarper_->isConfigured(&data->mainPathStream_)) - cropLimits = dewarper_->inputCropBounds(&data->mainPathStream_); - else + if (dewarper_->isConfigured(&data->mainPathStream_)) { + auto &vertexMap = dewarper_->vertexMap(&data->mainPathStream_); + vertexMap.applyLimits(); + cropLimits = vertexMap.scalerCropBounds(); + controls[&controls::ScalerCrop] = ControlInfo(cropLimits.first, + cropLimits.second, + vertexMap.effectiveScalerCrop()); + } else { + /* This happens before configure() has run. Maybe we need a better solution.*/ + /* + * ScalerCrop is specified to be in Sensor coordinates. + * So we need to transform the limits to sensor coordinates. + * We can safely assume that the maximum crop limit contains the + * full fov of the dewarper. + */ cropLimits = dewarper_->inputCropBounds(); + Rectangle maxCrop = Rectangle(data->sensor_->resolution()); + Rectangle min = cropLimits.first.transformedBetween(cropLimits.second, + maxCrop); + + controls[&controls::ScalerCrop] = ControlInfo(min, + maxCrop, + maxCrop); + } - /* - * ScalerCrop is specified to be in Sensor coordinates. - * So we need to transform the limits to sensor coordinates. - * We can safely assume that the maximum crop limit contains the - * full fov of the dewarper. - */ - Rectangle min = cropLimits.first.transformedBetween(cropLimits.second, - scalerMaxCrop_); - - controls[&controls::ScalerCrop] = ControlInfo(min, - scalerMaxCrop_, - scalerMaxCrop_); - data->properties_.set(properties::ScalerCropMaximum, scalerMaxCrop_); - activeCrop_ = scalerMaxCrop_; } /* Add the IPA registered controls to list of camera controls. */ @@ -1397,8 +1421,6 @@ int PipelineHandlerRkISP1::createCamera(MediaEntity *sensor) /* Initialize the camera properties. */ data->properties_ = data->sensor_->properties(); - scalerMaxCrop_ = Rectangle(data->sensor_->resolution()); - const CameraSensorProperties::SensorDelays &delays = data->sensor_->sensorDelays(); std::unordered_map params = { { V4L2_CID_ANALOGUE_GAIN, { delays.gainDelay, false } }, @@ -1632,36 +1654,17 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) availableDewarpRequests_.pop(); } + auto &vertexMap = dewarper_->vertexMap(&data->mainPathStream_); + /* Handle scaler crop control. */ const auto &crop = request->controls().get(controls::ScalerCrop); if (crop) { - Rectangle rect = crop.value(); - - /* - * ScalerCrop is specified to be in Sensor coordinates. - * So we need to transform it into dewarper coordinates. - * We can safely assume that the maximum crop limit contains the - * full fov of the dewarper. - */ - std::pair cropLimits = - dewarper_->inputCropBounds(&data->mainPathStream_); - - rect = rect.transformedBetween(scalerMaxCrop_, cropLimits.second); - int ret = dewarper_->setInputCrop(&data->mainPathStream_, - &rect); - rect = rect.transformedBetween(cropLimits.second, scalerMaxCrop_); - if (!ret && rect != crop.value()) { - /* - * If the rectangle is changed by setInputCrop on the - * dewarper, log a debug message and cache the actual - * applied rectangle for metadata reporting. - */ - LOG(RkISP1, Debug) - << "Applied rectangle " << rect.toString() - << " differs from requested " << crop.value().toString(); - } - - activeCrop_ = rect; + if (!dewarper_->supportsRequests()) + LOG(RkISP1, Error) + << "Dynamically setting ScalerCrop requires a " + "dw100 driver with requests support"; + vertexMap.setScalerCrop(*crop); + dewarper_->applyVertexMap(&data->mainPathStream_, dewarpRequest); } /* @@ -1695,7 +1698,7 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) } } - request->metadata().set(controls::ScalerCrop, activeCrop_.value()); + request->metadata().set(controls::ScalerCrop, vertexMap.effectiveScalerCrop()); } void PipelineHandlerRkISP1::dewarpRequestReady(V4L2Request *request) From patchwork Thu Oct 23 14:48:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24751 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0C468BE080 for ; Thu, 23 Oct 2025 14:50:00 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 802446083D; Thu, 23 Oct 2025 16:50:00 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="MZpv9SG9"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 71ECE6082F for ; Thu, 23 Oct 2025 16:49:58 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7328:357b:4ce1:72b6]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 8267D177F; Thu, 23 Oct 2025 16:48:13 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761230893; bh=YD8/Fc4sV3lxt4nPYvG9lORTvpOnOCp/4zQKsFVq1Jc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=MZpv9SG94Fw5wGlr4NFeq39MUwPlq0TsdwKlwr8JlAZ7cM0x9gQ4UASEF6md0yIjv bGap88PaG+QnAJkMkHlSgoKXIG7rkUOilcifAR2+E7k3j6r7S3OtfBUhKRgbZgP4SZ 8oeG66tfuQg6y+N1I59RXkPQVHwmxrn3sd9afe6w= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 24/35] pipeline: rksip1: Move isRaw up in scope Date: Thu, 23 Oct 2025 16:48:25 +0200 Message-ID: <20251023144841.403689-25-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251023144841.403689-1-stefan.klug@ideasonboard.com> References: <20251023144841.403689-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In raw mode the number of configurations is actively limited to 1. It is therefore safe to move isRaw up one level to simplify the code and prepare for later use. Signed-off-by: Stefan Klug --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 14 ++++---------- 1 file changed, 4 insertions(+), 10 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 0b31b8077c8d..35cb54c812bc 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -559,28 +559,22 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() * Simultaneous capture of raw and processed streams isn't possible. If * there is any raw stream, cap the number of streams to one. */ + bool isRaw = false; if (config_.size() > 1) { for (const auto &cfg : config_) { if (PixelFormatInfo::info(cfg.pixelFormat).colourEncoding == PixelFormatInfo::ColourEncodingRAW) { config_.resize(1); status = Adjusted; + isRaw = true; break; } } } bool useDewarper = false; - if (pipe->dewarper_) { - /* - * Platforms with dewarper support, such as i.MX8MP, support - * only a single stream. We can inspect config_[0] only here. - */ - bool isRaw = PixelFormatInfo::info(config_[0].pixelFormat).colourEncoding == - PixelFormatInfo::ColourEncodingRAW; - if (!isRaw) - useDewarper = true; - } + if (pipe->dewarper_ && !isRaw) + useDewarper = true; /* * If there are more than one stream in the configuration figure out the From patchwork Thu Oct 23 14:48:26 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24752 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CC74DC3341 for ; Thu, 23 Oct 2025 14:50:03 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0E53760844; Thu, 23 Oct 2025 16:50:03 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="kcZiKNGt"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B92426082F for ; Thu, 23 Oct 2025 16:50:01 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7328:357b:4ce1:72b6]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 99B9F177F; Thu, 23 Oct 2025 16:48:16 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761230896; bh=j/3WsFB4D2Uta3xMCM/dsUMW3xNcA+oJ/oLjccHCkWw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kcZiKNGtnHGy0KqC+44eRhTEqii1l2/9oXsOkwKZRh1HfGmErqvC7CaH2sxPUxkkn U3Lu43P2pAcwAij9F/+xmQE5EUm1J+dhFB3/2X27COB922gj20zEvXltVlmbbLhmAS TyaYeBV60E+iNshCJz/+OJrWDEAYenMFqGaTYwGU= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 25/35] pipeline: rkisp1: Drop rawFormat variable Date: Thu, 23 Oct 2025 16:48:26 +0200 Message-ID: <20251023144841.403689-26-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251023144841.403689-1-stefan.klug@ideasonboard.com> References: <20251023144841.403689-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In raw mode we know there is only a single configuration so there is no need to iterate over all configurations to find the format. Drop that. Signed-off-by: Stefan Klug --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 35cb54c812bc..d7bb30f20668 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -677,21 +677,16 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() } /* Select the sensor format. */ - PixelFormat rawFormat; Size maxSize; for (const StreamConfiguration &cfg : config_) { - const PixelFormatInfo &info = PixelFormatInfo::info(cfg.pixelFormat); - if (info.colourEncoding == PixelFormatInfo::ColourEncodingRAW) - rawFormat = cfg.pixelFormat; - maxSize = std::max(maxSize, cfg.size); } std::vector mbusCodes; - if (rawFormat.isValid()) { - mbusCodes = { rawFormats.at(rawFormat) }; + if (isRaw) { + mbusCodes = { rawFormats.at(config_[0].pixelFormat) }; } else { std::transform(rawFormats.begin(), rawFormats.end(), std::back_inserter(mbusCodes), From patchwork Thu Oct 23 14:48:27 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24753 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 738D6C3335 for ; Thu, 23 Oct 2025 14:50:06 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0912160854; Thu, 23 Oct 2025 16:50:06 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="fhupAZZ1"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 125056082F for ; Thu, 23 Oct 2025 16:50:05 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7328:357b:4ce1:72b6]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id C3AE5192B; Thu, 23 Oct 2025 16:48:19 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761230900; bh=5JSMbVo9ptEj9giMgsIXR+/AE/uoToOidXbRBBZR38E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fhupAZZ1hlHtnIvWlE21zf8BzfT7Mq9SwI+9xCC7BIVK/tO5m4dj8Cv98w40mw295 xFi4h1Uudtumz1NZdPOkQbEYsKgnaD2PeBaOZPOQIbeMwTU94W2o86Ku6w+o7F71kk YDwtq/OyJTNYO0BVCIeEPHdZQh2ywtVw4mBO9JPI= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 26/35] pipeline: rkisp1: Enable the dewarper based on the tuning file Date: Thu, 23 Oct 2025 16:48:27 +0200 Message-ID: <20251023144841.403689-27-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251023144841.403689-1-stefan.klug@ideasonboard.com> References: <20251023144841.403689-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" To do actual lens dewarping, the dewarper will be configured based on the tuning file. As a first step implement the basic loading of the tuning file and enable/disable the dewarper for the given camera based on the existence of the "Dewarp" algorithm in the tuning file. Todo: This is an backwards incompatible change in that the dewarper is currently included in the chain unconditionally. Some users may want to not use the dewarper, so it is sensible to make that configurable. If it should be in the algorithms section or in a different one is open for debate. Signed-off-by: Stefan Klug --- Changes in v2: - Drop unused params variable - Moved patch a bit earlier so canUseDewarper is available for handling the orientation --- src/ipa/libipa/module.h | 4 ++ src/libcamera/pipeline/rkisp1/rkisp1.cpp | 48 +++++++++++++++++++++++- 2 files changed, 51 insertions(+), 1 deletion(-) diff --git a/src/ipa/libipa/module.h b/src/ipa/libipa/module.h index 0fb51916fff6..84386f901534 100644 --- a/src/ipa/libipa/module.h +++ b/src/ipa/libipa/module.h @@ -74,6 +74,10 @@ private: int createAlgorithm(Context &context, const YamlObject &data) { const auto &[name, algoData] = *data.asDict().begin(); + + if (name == "Dewarp") + return 0; + std::unique_ptr> algo = createAlgorithm(name); if (!algo) { LOG(IPAModuleAlgo, Error) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index d7bb30f20668..943f26ece974 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -16,6 +16,7 @@ #include #include +#include #include #include @@ -47,6 +48,7 @@ #include "libcamera/internal/v4l2_request.h" #include "libcamera/internal/v4l2_subdevice.h" #include "libcamera/internal/v4l2_videodevice.h" +#include "libcamera/internal/yaml_parser.h" #include "rkisp1_path.h" @@ -123,6 +125,7 @@ public: */ MediaPipeline pipe_; + bool canUseDewarper_; bool usesDewarper_; private: @@ -131,6 +134,7 @@ private: const ControlList &sensorControls); void metadataReady(unsigned int frame, const ControlList &metadata); + int loadTuningFile(const std::string &file); }; class RkISP1CameraConfiguration : public CameraConfiguration @@ -416,6 +420,48 @@ int RkISP1CameraData::loadIPA(unsigned int hwRevision, uint32_t supportedBlocks) return ret; } + ret = loadTuningFile(ipaTuningFile); + if (ret < 0) { + LOG(RkISP1, Error) << "Failed to load tuning file"; + return ret; + } + + return 0; +} + +int RkISP1CameraData::loadTuningFile(const std::string &path) +{ + if (!pipe()->dewarper_) + /* Nothing to do without dewarper */ + return 0; + + LOG(RkISP1, Debug) << "Load tuning file " << path; + + File file(path); + if (!file.open(File::OpenModeFlag::ReadOnly)) { + int ret = file.error(); + LOG(RkISP1, Error) + << "Failed to open tuning file " + << path << ": " << strerror(-ret); + return ret; + } + + std::unique_ptr data = YamlParser::parse(file); + if (!data) + return -EINVAL; + + if (!data->contains("algorithms")) { + LOG(RkISP1, Error) + << "Tuning file doesn't contain any algorithm"; + return -EINVAL; + } + + const auto &algos = (*data)["algorithms"].asList(); + for (const auto &algo : algos) { + if (algo.contains("Dewarp")) + canUseDewarper_ = true; + } + return 0; } @@ -573,7 +619,7 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() } bool useDewarper = false; - if (pipe->dewarper_ && !isRaw) + if (data_->canUseDewarper_ && !isRaw) useDewarper = true; /* From patchwork Thu Oct 23 14:48:28 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24754 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3F479C3342 for ; Thu, 23 Oct 2025 14:50:10 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B125160858; Thu, 23 Oct 2025 16:50:09 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="e3CHCrC5"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5E45A60844 for ; Thu, 23 Oct 2025 16:50:08 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7328:357b:4ce1:72b6]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 2AA05195E; Thu, 23 Oct 2025 16:48:23 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761230903; bh=rkOkPBkXLjQV0VPKMB8Po4PHfV5o2kMxP1oYCdtChjk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=e3CHCrC5fAZyxnz9ZmUzNUSU9Td541xBQagxMZVj93sH06y8KifZmSxt3Ay3Tph3M tp610Y3Ej/B4kL6KJiVaUt8PI5s4P4uVSUs+qkY3CHEq4EGjkPhYrq0jT14O2H0u1I FU2Ya5qDMetAvknG/eQzba+8BDH0qhHzqLJkvros= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Kieran Bingham Subject: [PATCH v2 27/35] libcamera: internal: camera_sensor: Add accessor for mountingOrientation_ Date: Thu, 23 Oct 2025 16:48:28 +0200 Message-ID: <20251023144841.403689-28-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251023144841.403689-1-stefan.klug@ideasonboard.com> References: <20251023144841.403689-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" To properly handle the orientation in the dewarper, the mounting orientation of the sensor needs to be queryable. Add that. Signed-off-by: Stefan Klug Reviewed-by: Kieran Bingham --- Changes in v2: - Collected tag - Add documentation --- include/libcamera/internal/camera_sensor.h | 1 + src/libcamera/sensor/camera_sensor.cpp | 7 +++++++ src/libcamera/sensor/camera_sensor_legacy.cpp | 1 + src/libcamera/sensor/camera_sensor_raw.cpp | 1 + 4 files changed, 10 insertions(+) diff --git a/include/libcamera/internal/camera_sensor.h b/include/libcamera/internal/camera_sensor.h index f6ef4df170d4..e6b72d22ac21 100644 --- a/include/libcamera/internal/camera_sensor.h +++ b/include/libcamera/internal/camera_sensor.h @@ -73,6 +73,7 @@ public: virtual int sensorInfo(IPACameraSensorInfo *info) const = 0; virtual Transform computeTransform(Orientation *orientation) const = 0; virtual BayerFormat::Order bayerOrder(Transform t) const = 0; + virtual Orientation mountingOrientation() const = 0; virtual const ControlInfoMap &controls() const = 0; virtual ControlList getControls(Span ids) = 0; diff --git a/src/libcamera/sensor/camera_sensor.cpp b/src/libcamera/sensor/camera_sensor.cpp index 4f2fd2690eb5..05390d1e1b60 100644 --- a/src/libcamera/sensor/camera_sensor.cpp +++ b/src/libcamera/sensor/camera_sensor.cpp @@ -331,6 +331,13 @@ int CameraSensor::setEmbeddedDataEnabled(bool enable) * \return The Bayer order produced by the sensor when the Transform is applied */ +/** + * \fn CameraSensor::mountingOrientation() + * \brief Fetch the mounting orientation of the sensor + * + * \return The mounting orientation of the sensor + */ + /** * \fn CameraSensor::controls() * \brief Retrieve the supported V4L2 controls and their information diff --git a/src/libcamera/sensor/camera_sensor_legacy.cpp b/src/libcamera/sensor/camera_sensor_legacy.cpp index f9e685a9acc4..39c34200b0ff 100644 --- a/src/libcamera/sensor/camera_sensor_legacy.cpp +++ b/src/libcamera/sensor/camera_sensor_legacy.cpp @@ -88,6 +88,7 @@ public: int sensorInfo(IPACameraSensorInfo *info) const override; Transform computeTransform(Orientation *orientation) const override; BayerFormat::Order bayerOrder(Transform t) const override; + Orientation mountingOrientation() const override { return mountingOrientation_; } const ControlInfoMap &controls() const override; ControlList getControls(Span ids) override; diff --git a/src/libcamera/sensor/camera_sensor_raw.cpp b/src/libcamera/sensor/camera_sensor_raw.cpp index 8ea4423698cd..759cccafe4a9 100644 --- a/src/libcamera/sensor/camera_sensor_raw.cpp +++ b/src/libcamera/sensor/camera_sensor_raw.cpp @@ -94,6 +94,7 @@ public: int sensorInfo(IPACameraSensorInfo *info) const override; Transform computeTransform(Orientation *orientation) const override; BayerFormat::Order bayerOrder(Transform t) const override; + Orientation mountingOrientation() const override { return mountingOrientation_; } const ControlInfoMap &controls() const override; ControlList getControls(Span ids) override; From patchwork Thu Oct 23 14:48:29 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24755 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EEB91C3343 for ; Thu, 23 Oct 2025 14:50:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 638DD6085B; Thu, 23 Oct 2025 16:50:12 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="VEOBQ41x"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0D6E760847 for ; Thu, 23 Oct 2025 16:50:11 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7328:357b:4ce1:72b6]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 1720E19E5; Thu, 23 Oct 2025 16:48:26 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761230906; bh=S5kxuO0kE+eOLa3Pw+g2r8k/DuYrjE1NgSvbDcQ9I9c=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VEOBQ41x3nht/f2XHvoni99+8wbrsoViEEWprPQcmhwbKoKsOSqWqMiLWdRIaYINc g3xNUGDCy+nRTad++U9KD3kWxNLFxmgEQyuk9T9IEl4/ilu4GMcbxv+4n080NnFbaY WlIk6DUbC+sYihKWNrCP0H49LVP/g6Z06bh/8NNM= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 28/35] libcamera: Add transpose() function to size Date: Thu, 23 Oct 2025 16:48:29 +0200 Message-ID: <20251023144841.403689-29-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251023144841.403689-1-stefan.klug@ideasonboard.com> References: <20251023144841.403689-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a transpose() function to size that applies the Transformation::Transpose operation in the size. This is useful when handling orientation adjustments. Signed-off-by: Stefan Klug --- include/libcamera/geometry.h | 6 ++++++ src/libcamera/geometry.cpp | 10 ++++++++++ 2 files changed, 16 insertions(+) diff --git a/include/libcamera/geometry.h b/include/libcamera/geometry.h index d9378efeca8c..572a07e2cf92 100644 --- a/include/libcamera/geometry.h +++ b/include/libcamera/geometry.h @@ -108,6 +108,12 @@ public: return *this; } + Size &transpose() + { + std::swap(width, height); + return *this; + } + [[nodiscard]] constexpr Size alignedDownTo(unsigned int hAlignment, unsigned int vAlignment) const { diff --git a/src/libcamera/geometry.cpp b/src/libcamera/geometry.cpp index de76d0c12a8c..2763967a6e57 100644 --- a/src/libcamera/geometry.cpp +++ b/src/libcamera/geometry.cpp @@ -209,6 +209,16 @@ std::string Size::toString() const * \return A reference to this object */ +/** + * \fn Size::transpose() + * \brief Transpose the size in place + * + * This function swaps width and height of this size. This effectively applies + * the \a Transform::Transpose transformation on this size. + * + * \return A reference to this object + */ + /** * \fn Size::alignedDownTo(unsigned int hAlignment, unsigned int vAlignment) * \brief Align the size down horizontally and vertically From patchwork Thu Oct 23 14:48:30 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24756 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5772AC3336 for ; Thu, 23 Oct 2025 14:50:16 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id ED33260854; Thu, 23 Oct 2025 16:50:15 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="bp+BjQYY"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 59B6E607ED for ; Thu, 23 Oct 2025 16:50:14 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7328:357b:4ce1:72b6]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 62D8E1AA6; Thu, 23 Oct 2025 16:48:29 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761230909; bh=fnekDP4Y6VsfeMeZXhHHLYR7osZb/stTo6p0peHGrrs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bp+BjQYYUp2//GnqxcbOuy2icDKtD0OlHM1sjAZM3XKoEONO8RZAlFYTb0St1O8Zd ezlQ4TyVTEhjm34iVMYF64DXORI46w0K1ROu5zOI6gO6c4v5aLwkXfzg7geOhGOzni trLJAMVnZdGsHIR+tIF1vuaI0OGibb82PKjsvhcs= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 29/35] libcamera: rkisp1: Handle requested orientation using dewarper Date: Thu, 23 Oct 2025 16:48:30 +0200 Message-ID: <20251023144841.403689-30-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251023144841.403689-1-stefan.klug@ideasonboard.com> References: <20251023144841.403689-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When the dewarper is present it can handle arbitrary orientations specified in the requested camera configuration. In that case handle all transformations inside the dewarper (even if the sensor supports some of them) because that makes it easier to handle coordinates for lens dewarping inside the dewarper. This complicates the path selection a bit, as for transformations that include a transpose, the format before the dewarper has swapped width/height. Signed-off-by: Stefan Klug --- Changes in v2: - Fix path validation for cases where a transpose is involved --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 97 +++++++++++++++++------- 1 file changed, 71 insertions(+), 26 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 943f26ece974..eaf82d0f1097 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -596,11 +596,6 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() status = Adjusted; } - Orientation requestedOrientation = orientation; - combinedTransform_ = data_->sensor_->computeTransform(&orientation); - if (orientation != requestedOrientation) - status = Adjusted; - /* * Simultaneous capture of raw and processed streams isn't possible. If * there is any raw stream, cap the number of streams to one. @@ -618,9 +613,24 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() } } + /* + * If the dewarper supports orientation adjustments, apply that completely + * there. Even if the sensor supports flips, it is beneficial to do that + * in the dewarper so that lens dewarping happens on the unflipped image + */ + bool transposeAfterIsp = false; bool useDewarper = false; - if (data_->canUseDewarper_ && !isRaw) + if (data_->canUseDewarper_ && !isRaw) { useDewarper = true; + combinedTransform_ = orientation / data_->sensor_->mountingOrientation(); + if (!!(combinedTransform_ & Transform::Transpose)) + transposeAfterIsp = true; + } else { + Orientation requestedOrientation = orientation; + combinedTransform_ = data_->sensor_->computeTransform(&orientation); + if (orientation != requestedOrientation) + status = Adjusted; + } /* * If there are more than one stream in the configuration figure out the @@ -636,12 +646,18 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() /* * Validate the configuration against the desired path and, if the - * platform supports it, the dewarper. + * platform supports it, the dewarper. While iterating over the + * configurations collect the smallest common sensor format. */ + Size accumulatedSensorSize; auto validateConfig = [&](StreamConfiguration &cfg, RkISP1Path *path, Stream *stream, Status expectedStatus) { StreamConfiguration tryCfg = cfg; + /* Need to validate the path before the transpose */ + if (transposeAfterIsp) + tryCfg.size.transpose(); + Status ret = path->validate(sensor, sensorConfig, &tryCfg); if (ret == Invalid) return false; @@ -650,6 +666,8 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() (expectedStatus == Valid && ret == Adjusted)) return false; + Size sensorSize = tryCfg.size; + if (useDewarper) { /* * The dewarper output is independent of the ISP path. @@ -672,6 +690,8 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() cfg = tryCfg; cfg.setStream(stream); + + accumulatedSensorSize = std::max(accumulatedSensorSize, sensorSize); return true; }; @@ -722,13 +742,6 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() return Invalid; } - /* Select the sensor format. */ - Size maxSize; - - for (const StreamConfiguration &cfg : config_) { - maxSize = std::max(maxSize, cfg.size); - } - std::vector mbusCodes; if (isRaw) { @@ -739,7 +752,7 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() [](const auto &value) { return value.second; }); } - sensorFormat_ = sensor->getFormat(mbusCodes, maxSize, + sensorFormat_ = sensor->getFormat(mbusCodes, accumulatedSensorSize, mainPath->maxResolution()); if (sensorFormat_.size.isNull()) @@ -774,6 +787,22 @@ PipelineHandlerRkISP1::generateConfiguration(Camera *camera, if (roles.empty()) return config; + Transform transform = Transform::Identity; + Size previewSize = kRkISP1PreviewSize; + bool transposeAfterIsp = false; + if (data->canUseDewarper_) { + transform = Orientation::Rotate0 / data->sensor_->mountingOrientation(); + if (!!(transform & Transform::Transpose)) + transposeAfterIsp = true; + } + + /* + * In case of a transpose transform we need to create a path for the + * transposed size. + */ + if (transposeAfterIsp) + previewSize.transpose(); + /* * As the ISP can't output different color spaces for the main and self * path, pick a sensible default color space based on the role of the @@ -802,7 +831,7 @@ PipelineHandlerRkISP1::generateConfiguration(Camera *camera, if (!colorSpace) colorSpace = ColorSpace::Sycc; - size = kRkISP1PreviewSize; + size = previewSize; break; case StreamRole::VideoRecording: @@ -810,7 +839,7 @@ PipelineHandlerRkISP1::generateConfiguration(Camera *camera, if (!colorSpace) colorSpace = ColorSpace::Rec709; - size = kRkISP1PreviewSize; + size = previewSize; break; case StreamRole::Raw: @@ -852,6 +881,9 @@ PipelineHandlerRkISP1::generateConfiguration(Camera *camera, if (!cfg.pixelFormat.isValid()) return nullptr; + if (transposeAfterIsp && role != StreamRole::Raw) + cfg.size.transpose(); + cfg.colorSpace = colorSpace; cfg.bufferCount = kRkISP1MinBufferCount; config->addConfiguration(cfg); @@ -874,6 +906,19 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) if (ret) return ret; + const PixelFormat &streamFormat = config->at(0).pixelFormat; + const PixelFormatInfo &info = PixelFormatInfo::info(streamFormat); + isRaw_ = info.colourEncoding == PixelFormatInfo::ColourEncodingRAW; + data->usesDewarper_ = data->canUseDewarper_ && !isRaw_; + + Transform transform = config->combinedTransform(); + bool transposeAfterIsp = false; + if (data->usesDewarper_) { + if (!!(transform & Transform::Transpose)) + transposeAfterIsp = true; + transform = Transform::Identity; + } + /* * Configure the format on the sensor output and propagate it through * the pipeline. @@ -883,10 +928,10 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) if (config->sensorConfig) ret = sensor->applyConfiguration(*config->sensorConfig, - config->combinedTransform(), + transform, &format); else - ret = sensor->setFormat(&format, config->combinedTransform()); + ret = sensor->setFormat(&format, transform); if (ret < 0) return ret; @@ -913,10 +958,6 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) << " crop " << inputCrop; Rectangle outputCrop = inputCrop; - const PixelFormat &streamFormat = config->at(0).pixelFormat; - const PixelFormatInfo &info = PixelFormatInfo::info(streamFormat); - isRaw_ = info.colourEncoding == PixelFormatInfo::ColourEncodingRAW; - data->usesDewarper_ = dewarper_ && !isRaw_; /* YUYV8_2X8 is required on the ISP source path pad for YUV output. */ if (!isRaw_) @@ -1008,15 +1049,19 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) inputCrop, sensorInfo.analogCrop); auto &vertexMap = dewarper_->vertexMap(cfg.stream()); vertexMap.setSensorCrop(sensorCrop); + vertexMap.setTransform(config->combinedTransform()); data->properties_.set(properties::ScalerCropMaximum, sensorCrop); /* * Apply a default sensor crop that keeps the * aspect ratio. */ - Size crop = format.size.boundedToAspectRatio(cfg.size); - vertexMap.setScalerCrop(crop.centeredTo( - Rectangle(format.size).center())); + Size size = cfg.size; + if (transposeAfterIsp) + size.transpose(); + size = sensorCrop.size().boundedToAspectRatio(size); + vertexMap.setScalerCrop( + size.centeredTo(sensorCrop.center())); } ret = mainPath_.configure(ispCfg, format); From patchwork Thu Oct 23 14:48:31 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24757 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 031DFC3344 for ; Thu, 23 Oct 2025 14:50:18 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 80B9D60854; Thu, 23 Oct 2025 16:50:18 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="d84fuRgi"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CC1FE607ED for ; Thu, 23 Oct 2025 16:50:16 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7328:357b:4ce1:72b6]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id CB6821127; Thu, 23 Oct 2025 16:48:31 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761230911; bh=UpEvhD8cXzD3/aA5BBffZLonOzc8X/lKUqsIeTlFF1c=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=d84fuRgiFEbQ2YLovBKaoAUFtUazVMkpGoihP0eySAHDVqeFvTmp1ZrAzadC54VGi Ka1r7Q8A73d82Fm59sVSTt0uVzfIgrmOmJjvkGEOyR1U2RB74MxGyj4tRE5xvlt7ii Vnra/SqtEMohIX7N5W4a+XwaDDwZ6YOtRTS6jMn8= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 30/35] libcamera: dw100_vertexmap: Implement parametric dewarping Date: Thu, 23 Oct 2025 16:48:31 +0200 Message-ID: <20251023144841.403689-31-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251023144841.403689-1-stefan.klug@ideasonboard.com> References: <20251023144841.403689-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement functions to allow lens dewarping based on the common lens dewarp model used e.g. by OpenCV. See https://docs.opencv.org/4.12.0/d9/d0c/group__calib3d.html for an in depth explanation of the parameters. Signed-off-by: Stefan Klug --- Changes in v2: - Adapt to vertex map implementation based on affine transforms - Improved documentation - dropped loadDewarpParams function as it was never used --- .../converter/converter_dw100_vertexmap.h | 16 +++ .../converter/converter_dw100_vertexmap.cpp | 98 +++++++++++++++++-- 2 files changed, 108 insertions(+), 6 deletions(-) diff --git a/include/libcamera/internal/converter/converter_dw100_vertexmap.h b/include/libcamera/internal/converter/converter_dw100_vertexmap.h index e72cb72bb9f1..b05ed8338a23 100644 --- a/include/libcamera/internal/converter/converter_dw100_vertexmap.h +++ b/include/libcamera/internal/converter/converter_dw100_vertexmap.h @@ -10,6 +10,9 @@ #include #include +#include "libcamera/internal/matrix.h" +#include "libcamera/internal/vector.h" + namespace libcamera { class Dw100VertexMap @@ -55,9 +58,17 @@ public: void setMode(const ScaleMode mode) { mode_ = mode; } ScaleMode mode() const { return mode_; } + int setDewarpParams(const Matrix &cm, const Span &coeffs); + bool dewarpParamsValid() { return dewarpParamsValid_; } + + void setLensDewarpEnable(bool enable) { lensDewarpEnable_ = enable; } + bool lensDewarpEnable() { return lensDewarpEnable_; } + std::vector getVertexMap(); private: + Vector dewarpPoint(const Vector &p); + Rectangle scalerCrop_; Rectangle sensorCrop_; Transform transform_ = Transform::Identity; @@ -71,6 +82,11 @@ private: double effectiveScaleY_; Point effectiveOffset_; Rectangle effectiveScalerCrop_; + + Matrix dewarpM_ = Matrix::identity(); + std::array dewarpCoeffs_; + bool lensDewarpEnable_ = true; + bool dewarpParamsValid_ = false; }; } /* namespace libcamera */ diff --git a/src/libcamera/converter/converter_dw100_vertexmap.cpp b/src/libcamera/converter/converter_dw100_vertexmap.cpp index 0e930479b6f7..f2be9a697ff3 100644 --- a/src/libcamera/converter/converter_dw100_vertexmap.cpp +++ b/src/libcamera/converter/converter_dw100_vertexmap.cpp @@ -184,8 +184,6 @@ int dw100VerticesForLength(const int length) * | | | | | Transpose) | | Rotate) | * +-------------+ +-------------+ +------------+ +-----------------+ * - * \todo Lens dewarp is not yet implemented. An identity map is used instead. - * * All parameters are clamped to valid values before creating the vertex map. * * The constrains process works as follows: @@ -546,10 +544,8 @@ std::vector Dw100VertexMap::getVertexMap() p = transformPoint(outputToSensor, p); - /* - * \todo: Transformations in sensor space to be added - * here. - */ + if (dewarpParamsValid_ && lensDewarpEnable_) + p = dewarpPoint(p); p = transformPoint(sensorToInput, p); @@ -563,4 +559,94 @@ std::vector Dw100VertexMap::getVertexMap() return res; } +/** + * \brief Set the dewarp parameters + * \param cm The camera matrix + * \param coeffs The dewarp coefficients + * + * Sets the dewarp parameters according to the commonly used dewarp model. See + * https://docs.opencv.org/4.12.0/d9/d0c/group__calib3d.html for further details + * on the model. The parameter \a coeffs must either hold 4,5,8 or 12 values. + * They represent the parameters k1,k2,p1,p2[,k3[,k4,k5,k6[,s1,s2,s3,s4]]] in + * the model. + * + * \return A negative number on error, 0 otherwise + */ +int Dw100VertexMap::setDewarpParams(const Matrix &cm, + const Span &coeffs) +{ + dewarpM_ = cm; + dewarpCoeffs_.fill(0.0); + + if (coeffs.size() != 4 && coeffs.size() != 5 && + coeffs.size() != 8 && coeffs.size() != 12) { + LOG(Converter, Error) + << "Dewarp 'coefficients' must have 4, 5, 8 or 12 values"; + dewarpParamsValid_ = false; + return -EINVAL; + } + std::copy(coeffs.begin(), coeffs.end(), dewarpCoeffs_.begin()); + + dewarpParamsValid_ = true; + return 0; +} + +/** + * \fn Dw100VertexMap::dewarpParamsValid() + * \brief Returns if the dewarp parameters are valid + * + * \return True if the dewarp parameters are valid, fals otherwise + */ + +/** + * \fn Dw100VertexMap::setLensDewarpEnable() + * \brief Enables or disabled lens dewarping + * \param[in] enable Enable or disable lens dewarping + */ + +/** + * \fn Dw100VertexMap::lensDewarpEnable() + * \brief Returns if lens dewarping is enabled + */ + +/** + * \brief Apply dewarp calculation to a point + * \param p The point to dewarp + * + * Applies the dewarp transformation to point \a p according to the commonly + * used dewarp model. See + * https://docs.opencv.org/4.12.0/d9/d0c/group__calib3d.html for further details + * on the model. + * + * \return The dewarped point + */ +Vector2d Dw100VertexMap::dewarpPoint(const Vector2d &p) +{ + double x, y; + double k1 = dewarpCoeffs_[0]; + double k2 = dewarpCoeffs_[1]; + double p1 = dewarpCoeffs_[2]; + double p2 = dewarpCoeffs_[3]; + double k3 = dewarpCoeffs_[4]; + double k4 = dewarpCoeffs_[5]; + double k5 = dewarpCoeffs_[6]; + double k6 = dewarpCoeffs_[7]; + double s1 = dewarpCoeffs_[8]; + double s2 = dewarpCoeffs_[9]; + double s3 = dewarpCoeffs_[10]; + double s4 = dewarpCoeffs_[11]; + + y = (p.y() - dewarpM_[1][2]) / dewarpM_[1][1]; + x = (p.x() - dewarpM_[0][2] - y * dewarpM_[0][1]) / dewarpM_[0][0]; + + double r2 = x * x + y * y; + double d = (1 + k1 * r2 + k2 * r2 * r2 + k3 * r2 * r2 * r2) / + (1 + k4 * r2 + k5 * r2 * r2 + k6 * r2 * r2 * r2); + x = x * d + 2 * p1 * x * y + p2 * (r2 + 2 * x * x) + s1 * r2 + s2 * r2 * r2; + y = y * d + 2 * p2 * x * y + p1 * (r2 + 2 * y * y) + s3 * r2 + s4 * r2 * r2; + + return { { x * dewarpM_[0][0] + y * dewarpM_[0][1] + dewarpM_[0][2], + y * dewarpM_[1][1] + dewarpM_[1][2] } }; +} + } /* namespace libcamera */ From patchwork Thu Oct 23 14:48:32 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24758 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id AD7F2C3345 for ; Thu, 23 Oct 2025 14:50:21 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3C26F60854; Thu, 23 Oct 2025 16:50:21 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="rjZt7oqs"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EB1D0607ED for ; Thu, 23 Oct 2025 16:50:19 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7328:357b:4ce1:72b6]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id E9D741D2B; Thu, 23 Oct 2025 16:48:34 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761230915; bh=X/atkDhziiN20cDoi0BOYNgnetYnqG9uNQt3YQGpzB0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rjZt7oqssZ0l9tqK/gPPgeEO1gmvYLMUdlYePc4UzShTFsj7OiNNXLu11Fv7l9MDI GBBnzKvWPSq5V73ucLFUO69wQkKFpVm0W6zqt9ximW5W3YI2xqYR6lcn1IEFkOpQ39 HOpjGytt79UQz1zbJXlsH1nxaFzQiwikZpqnxgUQ= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 31/35] pipeline: rkisp1: Load dewarp parameters from tuning file Date: Thu, 23 Oct 2025 16:48:32 +0200 Message-ID: <20251023144841.403689-32-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251023144841.403689-1-stefan.klug@ideasonboard.com> References: <20251023144841.403689-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Load the dewarp parameters from the tuning file. Signed-off-by: Stefan Klug --- Changes in v2: - Dropped unused variable --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 33 +++++++++++++++++++++++- 1 file changed, 32 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index eaf82d0f1097..2280a5554f5a 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -125,6 +125,11 @@ public: */ MediaPipeline pipe_; + struct DewarpParms { + Matrix cm; + std::vector coeffs; + }; + std::optional dewarpParams_; bool canUseDewarper_; bool usesDewarper_; @@ -458,8 +463,30 @@ int RkISP1CameraData::loadTuningFile(const std::string &path) const auto &algos = (*data)["algorithms"].asList(); for (const auto &algo : algos) { - if (algo.contains("Dewarp")) + const auto ¶ms = algo["Dewarp"]; + if (params) { canUseDewarper_ = true; + DewarpParms dp; + if (params["cm"]) { + const auto &cm = params["cm"].get>(); + if (!cm) { + LOG(RkISP1, Error) << "Dewarp parameters are missing 'cm' value"; + return -EINVAL; + } + dp.cm = *cm; + } + + if (params["coefficients"]) { + const auto &coeffs = params["coefficients"].getList(); + if (!coeffs) { + LOG(RkISP1, Error) << "Dewarp parameters 'coefficients' value is not a list"; + return -EINVAL; + } + dp.coeffs = *coeffs; + } + + dewarpParams_ = dp; + } } return 0; @@ -1050,6 +1077,10 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) auto &vertexMap = dewarper_->vertexMap(cfg.stream()); vertexMap.setSensorCrop(sensorCrop); vertexMap.setTransform(config->combinedTransform()); + if (data->dewarpParams_) { + vertexMap.setDewarpParams(data->dewarpParams_->cm, + data->dewarpParams_->coeffs); + } data->properties_.set(properties::ScalerCropMaximum, sensorCrop); /* From patchwork Thu Oct 23 14:48:33 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24759 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D4E7AC3346 for ; Thu, 23 Oct 2025 14:50:25 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 15BE560861; Thu, 23 Oct 2025 16:50:25 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="XDi+9YIC"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id ACDFF607ED for ; Thu, 23 Oct 2025 16:50:22 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7328:357b:4ce1:72b6]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id BA53A1D2B; Thu, 23 Oct 2025 16:48:37 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761230917; bh=9UZZaXIngu5Dle1/LCtUV/S6Q1vKVbwh4WpHag52gY0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XDi+9YICZg/R3BeX9Md8+S4EX5EVbm3h5oAANFpDNvGDRZB+x76PtmJ5PGX2bfs0g is2b2FhVu6773JSRHoiCsxo0ygoNgM1dG3frCsCAt+Mz4ZEfWtWPH+9k/xQHkOCCDy tp9nFG1cZf8B0ITQ/zXGfniIzxCp90TLXlstJjKg= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 32/35] libcamera: Add and implement LensDewarpEnable control Date: Thu, 23 Oct 2025 16:48:33 +0200 Message-ID: <20251023144841.403689-33-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251023144841.403689-1-stefan.klug@ideasonboard.com> References: <20251023144841.403689-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a LensDewarpEnable control to enable or disable lens dewarping if it is configured. Implement it for the dw100 converter. Signed-off-by: Stefan Klug --- Changes in v2: - Merged control definition and usage into one patch - Improved control description - Fixed actual handling of LensDewarpEnable control that got lost in a previous rebase --- src/libcamera/control_ids_core.yaml | 7 +++++++ src/libcamera/pipeline/rkisp1/rkisp1.cpp | 26 ++++++++++++++++++++++-- 2 files changed, 31 insertions(+), 2 deletions(-) diff --git a/src/libcamera/control_ids_core.yaml b/src/libcamera/control_ids_core.yaml index f781865859ac..3bcb475fd102 100644 --- a/src/libcamera/control_ids_core.yaml +++ b/src/libcamera/control_ids_core.yaml @@ -1346,4 +1346,11 @@ controls: reduces the WdrExposureValue until the amount of pixels that are close to saturation is lower than this value. + - LensDewarpEnable: + type: bool + direction: inout + description: | + Enable or disable lens dewarping. This control is only available if lens + dewarp parameters are configured in the tuning file. + ... diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 2280a5554f5a..da3cfc0ffdeb 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -1492,6 +1492,14 @@ int PipelineHandlerRkISP1::updateControls(RkISP1CameraData *data) maxCrop); } + if (data->dewarpParams_.has_value()) { + if (dewarper_->supportsRequests()) + controls[&controls::LensDewarpEnable] = ControlInfo(false, true, true); + else + LOG(RkISP1, Warning) + << "dw100 kernel driver has no requests support." + " No dynamic configuration possible."; + } } /* Add the IPA registered controls to list of camera controls. */ @@ -1765,8 +1773,15 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) availableDewarpRequests_.pop(); } + bool update = false; auto &vertexMap = dewarper_->vertexMap(&data->mainPathStream_); + const auto &lensDewarpEnable = request->controls().get(controls::LensDewarpEnable); + if (lensDewarpEnable) { + vertexMap.setLensDewarpEnable(*lensDewarpEnable); + update = true; + } + /* Handle scaler crop control. */ const auto &crop = request->controls().get(controls::ScalerCrop); if (crop) { @@ -1775,9 +1790,12 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) << "Dynamically setting ScalerCrop requires a " "dw100 driver with requests support"; vertexMap.setScalerCrop(*crop); - dewarper_->applyVertexMap(&data->mainPathStream_, dewarpRequest); + update = true; } + if (update) + dewarper_->applyVertexMap(&data->mainPathStream_, dewarpRequest); + /* * Queue input and output buffers to the dewarper. The output * buffers for the dewarper are the buffers of the request, supplied @@ -1809,7 +1827,11 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) } } - request->metadata().set(controls::ScalerCrop, vertexMap.effectiveScalerCrop()); + auto &meta = request->metadata(); + meta.set(controls::ScalerCrop, vertexMap.effectiveScalerCrop()); + + if (vertexMap.dewarpParamsValid()) + meta.set(controls::LensDewarpEnable, vertexMap.lensDewarpEnable()); } void PipelineHandlerRkISP1::dewarpRequestReady(V4L2Request *request) From patchwork Thu Oct 23 14:48:34 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24760 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 429B5C32CE for ; Thu, 23 Oct 2025 14:50:27 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9833960870; Thu, 23 Oct 2025 16:50:27 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="UNQ5BgwP"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E3C51607ED for ; Thu, 23 Oct 2025 16:50:25 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7328:357b:4ce1:72b6]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id D320B1D2B; Thu, 23 Oct 2025 16:48:40 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761230920; bh=ZTwgWxl6cSIDcJw51iwQGL/F9McIFEHTaxZMDRQ8utE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UNQ5BgwPhP8xfBLyFRKjxoim/XuGI2UpAfsLxNAkKuB0uhLGEpindRoXVvzEswa/E 8o6/fLQRWdbDTqLqE2EcMyfT+8iZrNW3fC4JL9H5pVLAgCre2hMzD1jmDvrqfagcGy HRL0k3GO/g5PRiJkLXREsjvO2aG2r1sZDhyG3QYw= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 33/35] pipeline: rkisp1: Add warning to unclear format handling Date: Thu, 23 Oct 2025 16:48:34 +0200 Message-ID: <20251023144841.403689-34-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251023144841.403689-1-stefan.klug@ideasonboard.com> References: <20251023144841.403689-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" It is not mentioned when the sensorFormat is allowed to have zero size. Add a warning and todo to that case but leave it in place for now. Signed-off-by: Stefan Klug --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index da3cfc0ffdeb..9a156d4a01f2 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -782,8 +782,16 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() sensorFormat_ = sensor->getFormat(mbusCodes, accumulatedSensorSize, mainPath->maxResolution()); - if (sensorFormat_.size.isNull()) + if (sensorFormat_.size.isNull()) { + /* + * \todo When can this happen? Should we return a failure in + * this case? + */ sensorFormat_.size = sensor->resolution(); + LOG(RkISP1, Warning) + << "Failed to select sensor format. Default to " + << sensorFormat_; + } return status; } From patchwork Thu Oct 23 14:48:35 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24761 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C0BEFC3347 for ; Thu, 23 Oct 2025 14:50:29 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4C7D260865; Thu, 23 Oct 2025 16:50:29 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="R8QkNssV"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8C648607ED for ; Thu, 23 Oct 2025 16:50:28 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7328:357b:4ce1:72b6]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 91B42192B; Thu, 23 Oct 2025 16:48:43 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761230923; bh=jNkATD4IVKZDk3Nr2jeaeEoD0w69FOUxdlyhHGDPwO0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=R8QkNssVKikSvpXWkrqqtmvJr8iwFzOXea/+WXHv5lxNxCh+EWj+iIQSeZDu9rnAl f1GGvs4l6tZBRbh6iqgKiQxGuZT64dm1utGmAIB7GexyYJPxlAlIs9xmoHyWIjMf5C 9RfFvbegOx3TLcMq1rzH/sLiKrqHWi5SZwkdOeiY= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 34/35] DNI libcamera: rkisp1: Implement dw100 specific features Date: Thu, 23 Oct 2025 16:48:35 +0200 Message-ID: <20251023144841.403689-35-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251023144841.403689-1-stefan.klug@ideasonboard.com> References: <20251023144841.403689-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The dw100 allows more features implemented in the dw100 vertex map. Implement these features for the rkisp1 pipeline. Signed-off-by: Stefan Klug --- Changes in v2: - Marked as DoNotIntegrate because the puplic interface needs further discussions - Added Dw100EffectiveScale metadata --- .../converter/converter_dw100_vertexmap.h | 2 +- src/libcamera/control_ids_draft.yaml | 45 ++++++++++++++++++ src/libcamera/pipeline/rkisp1/rkisp1.cpp | 46 ++++++++++++++++--- 3 files changed, 86 insertions(+), 7 deletions(-) diff --git a/include/libcamera/internal/converter/converter_dw100_vertexmap.h b/include/libcamera/internal/converter/converter_dw100_vertexmap.h index b05ed8338a23..5fa46452f9ef 100644 --- a/include/libcamera/internal/converter/converter_dw100_vertexmap.h +++ b/include/libcamera/internal/converter/converter_dw100_vertexmap.h @@ -47,7 +47,7 @@ public: const Transform &transform() const { return transform_; } void setScale(const float scale) { scale_ = scale; } - float effectiveScale() const { return (effectiveScaleX_ + effectiveScaleY_) * 0.5; } + std::array effectiveScale() const { return { static_cast(effectiveScaleX_), static_cast(effectiveScaleY_) }; } void setRotation(const float rotation) { rotation_ = rotation; } float rotation() const { return rotation_; } diff --git a/src/libcamera/control_ids_draft.yaml b/src/libcamera/control_ids_draft.yaml index 03309eeac34f..30c832e32159 100644 --- a/src/libcamera/control_ids_draft.yaml +++ b/src/libcamera/control_ids_draft.yaml @@ -206,6 +206,51 @@ controls: available only on this camera device are at least this numeric value. All of the custom test patterns will be static (that is the raw image must not vary from frame to frame). + - Dw100ScaleMode: + type: int32_t + direction: inout + description: | + Scale mode of the dewarper. + enum: + - name: Fill + value: 0 + description: | + Fills the given output size with the largest rectangle possible. + Aspect ratio is not preserved. Dw100Scale is ignored. + - name: Crop + value: 1 + description: | + Crops to the given output size. The scale factor can be specified + using Dw100Scale. Aspect ratio is preserved. + - Dw100Scale: + type: float + direction: inout + description: | + Scale factor applied to the image when Dw100ScaleMode is set to Crop. + This value is clamped, so that all pixels have valid input data. + Therefore a value of 0 always provides the largest possible field of + view. Scaling happens centered around the center of the ScalerCrop + + Dw10Offset. + - Dw100Rotation: + type: float + direction: inout + description: | + Rotates the image by the given angle in degrees in clockwise direction + around the center of ScalerCrop + Dw100Offset + - Dw100Offset: + type: Point + direction: inout + description: | + Moves the image by the given values in x and y direction in sensor + coordinate space. This is clamped, so that all output pixels contain + valid data. The offset is therefore ignored when the Dw100Scale value is + too small. + - Dw100EffectiveScale: + type: float + direction: out + description: | + Effective scale + size: [2] - FaceDetectMode: type: int32_t diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 9a156d4a01f2..6538fbcc9698 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -1500,13 +1500,18 @@ int PipelineHandlerRkISP1::updateControls(RkISP1CameraData *data) maxCrop); } - if (data->dewarpParams_.has_value()) { - if (dewarper_->supportsRequests()) + if (dewarper_->supportsRequests()) { + controls[&controls::draft::Dw100Scale] = ControlInfo(0.2f, 8.0f, 1.0f); + controls[&controls::draft::Dw100Rotation] = ControlInfo(-180.0f, 180.0f, 0.0f); + controls[&controls::draft::Dw100Offset] = ControlInfo(Point(-10000, -10000), Point(10000, 10000), Point(0, 0)); + controls[&controls::draft::Dw100ScaleMode] = ControlInfo(controls::draft::Dw100ScaleModeValues, controls::draft::Fill); + + if (data->dewarpParams_.has_value()) controls[&controls::LensDewarpEnable] = ControlInfo(false, true, true); - else - LOG(RkISP1, Warning) - << "dw100 kernel driver has no requests support." - " No dynamic configuration possible."; + } else { + LOG(RkISP1, Warning) + << "dw100 kernel driver has no requests support." + " No dynamic configuration possible."; } } @@ -1784,6 +1789,30 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) bool update = false; auto &vertexMap = dewarper_->vertexMap(&data->mainPathStream_); + const auto &scale = request->controls().get(controls::draft::Dw100Scale); + if (scale) { + vertexMap.setScale(*scale); + update = true; + } + + const auto &rotation = request->controls().get(controls::draft::Dw100Rotation); + if (rotation) { + vertexMap.setRotation(*rotation); + update = true; + } + + const auto &offset = request->controls().get(controls::draft::Dw100Offset); + if (offset) { + vertexMap.setOffset(*offset); + update = true; + } + + const auto &scaleMode = request->controls().get(controls::draft::Dw100ScaleMode); + if (scaleMode) { + vertexMap.setMode(static_cast(*scaleMode)); + update = true; + } + const auto &lensDewarpEnable = request->controls().get(controls::LensDewarpEnable); if (lensDewarpEnable) { vertexMap.setLensDewarpEnable(*lensDewarpEnable); @@ -1836,6 +1865,11 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) } auto &meta = request->metadata(); + std::array effectiveScale = vertexMap.effectiveScale(); + meta.set(controls::draft::Dw100EffectiveScale, effectiveScale); + meta.set(controls::draft::Dw100Scale, (effectiveScale[0] + effectiveScale[1]) / 2.0); + meta.set(controls::draft::Dw100Rotation, vertexMap.rotation()); + meta.set(controls::draft::Dw100Offset, vertexMap.effectiveOffset()); meta.set(controls::ScalerCrop, vertexMap.effectiveScalerCrop()); if (vertexMap.dewarpParamsValid()) From patchwork Thu Oct 23 14:48:36 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24762 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BF636C3348 for ; Thu, 23 Oct 2025 14:50:33 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C603860878; Thu, 23 Oct 2025 16:50:32 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="gIZdZStQ"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8F4AB607ED for ; Thu, 23 Oct 2025 16:50:31 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7328:357b:4ce1:72b6]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 90F08195E; Thu, 23 Oct 2025 16:48:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761230926; bh=RIGwM8C3GNF2z+62dNHs6SG07ZN4k8+IAHkjnhUvA+Q=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=gIZdZStQkRmHpFyafrqitY23kas5Fa25fuv/relqqylgltHroWnF+s3iE7WeesLtD o3gvu0WfABQWlFna+hiOFPf/G6cvEZqkj3mpxRDbzKL1zsQX6OUdtKbiwoPu3ySzuX QrGDsdXgvTBfWvguEk7osc5Ke15NBVB5NAQW48Aw= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 35/35] DNI pipeline: rkisp1: Workaround for customer kernels without requests Date: Thu, 23 Oct 2025 16:48:36 +0200 Message-ID: <20251023144841.403689-36-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251023144841.403689-1-stefan.klug@ideasonboard.com> References: <20251023144841.403689-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" There are a few kernels out there that have dynamic vertex map support, but no requests support. Allow full dewarp usage on these. Signed-off-by: Stefan Klug --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 6538fbcc9698..fef6e23e39b7 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -183,6 +183,13 @@ static constexpr unsigned int kRkISP1MaxQueuedRequests = 4; */ static constexpr unsigned int kRkISP1MinBufferCount = 4; +/* + * This flag allows to use dynamic dewarp maps to support pan, zoom, rotate when + * the kernel driver doesn't support requests. Only needed for legacy customer + * kernels. + */ +static constexpr bool kAllowDynamicDewarpMapsWithoutRequests = true; + } /* namespace */ class PipelineHandlerRkISP1 : public PipelineHandler @@ -1500,7 +1507,7 @@ int PipelineHandlerRkISP1::updateControls(RkISP1CameraData *data) maxCrop); } - if (dewarper_->supportsRequests()) { + if (dewarper_->supportsRequests() || kAllowDynamicDewarpMapsWithoutRequests) { controls[&controls::draft::Dw100Scale] = ControlInfo(0.2f, 8.0f, 1.0f); controls[&controls::draft::Dw100Rotation] = ControlInfo(-180.0f, 180.0f, 0.0f); controls[&controls::draft::Dw100Offset] = ControlInfo(Point(-10000, -10000), Point(10000, 10000), Point(0, 0));