From patchwork Tue Nov 25 16:28:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 25180 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8CBCBC333C for ; Tue, 25 Nov 2025 16:29:00 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 34E0760AA2; Tue, 25 Nov 2025 17:29:00 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="V8rH3ZlR"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4CBD0609E0 for ; Tue, 25 Nov 2025 17:28:58 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bae1:340c:573c:570b]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 5B69E524; Tue, 25 Nov 2025 17:26:49 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1764088009; bh=A0gbPQ56Ijn8abbRp394XVOou8XJIhZmdsHUv8sJx2g=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=V8rH3ZlRBNKfOD/W7+T53bOklP59meEAuhaw3XHFm29j4Mr77KKT+LJq6DPX/6y+m WORAO1xbB2ph5FMYhya9QG+W/UVCJDtii0fWcJ8kg567Nl3t0PYU3xFsbqiK+vzZ82 wAXgrsZfN/ENEW57C74M99x0MszgCxT2EELlwsYw= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Kieran Bingham , Paul Elder , Isaac Scott Subject: [PATCH v3 01/29] libcamera: pipeline: utilise shared MediaDevice pointers Date: Tue, 25 Nov 2025 17:28:13 +0100 Message-ID: <20251125162851.2301793-2-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> References: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham Adapt the PipelineHandler::acquireMediaDevice() support function to return a shared pointer instead of the underlying raw pointer. Propagate this update to all pipeline handlers that use the MediaDevice and store a std::shared_ptr accordingly. This is required to support media devices that are potentially shared among multiple pipeline handlers, like a dewarper implemented as v4l2 m2m device. Signed-off-by: Kieran Bingham Signed-off-by: Paul Elder Signed-off-by: Stefan Klug Reviewed-by: Isaac Scott --- Changes in v3: - Call fromEntityName() with a pointer, as the shared_ptr overload was dropped --- include/libcamera/internal/pipeline_handler.h | 8 ++++---- src/libcamera/pipeline/imx8-isi/imx8-isi.cpp | 8 ++++---- src/libcamera/pipeline/ipu3/cio2.cpp | 4 ++-- src/libcamera/pipeline/ipu3/cio2.h | 2 +- src/libcamera/pipeline/ipu3/imgu.cpp | 15 ++++++++------- src/libcamera/pipeline/ipu3/imgu.h | 4 ++-- src/libcamera/pipeline/ipu3/ipu3.cpp | 4 ++-- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 16 ++++++++-------- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 8 ++++---- src/libcamera/pipeline/rkisp1/rkisp1_path.cpp | 6 +++--- src/libcamera/pipeline/rkisp1/rkisp1_path.h | 2 +- .../pipeline/rpi/common/pipeline_base.cpp | 6 ++++-- .../pipeline/rpi/common/pipeline_base.h | 9 ++++++--- src/libcamera/pipeline/rpi/pisp/pisp.cpp | 10 ++++++---- src/libcamera/pipeline/rpi/vc4/vc4.cpp | 13 +++++++++---- src/libcamera/pipeline/simple/simple.cpp | 13 +++++++------ src/libcamera/pipeline/uvcvideo/uvcvideo.cpp | 6 +++--- src/libcamera/pipeline/vimc/vimc.cpp | 14 +++++++------- src/libcamera/pipeline_handler.cpp | 14 ++++++++------ test/libtest/buffer_source.cpp | 3 ++- 20 files changed, 91 insertions(+), 74 deletions(-) diff --git a/include/libcamera/internal/pipeline_handler.h b/include/libcamera/internal/pipeline_handler.h index b2078577f5a4..b4f97477acec 100644 --- a/include/libcamera/internal/pipeline_handler.h +++ b/include/libcamera/internal/pipeline_handler.h @@ -38,8 +38,8 @@ public: virtual ~PipelineHandler(); virtual bool match(DeviceEnumerator *enumerator) = 0; - MediaDevice *acquireMediaDevice(DeviceEnumerator *enumerator, - const DeviceMatch &dm); + std::shared_ptr acquireMediaDevice(DeviceEnumerator *enumerator, + const DeviceMatch &dm); bool acquire(Camera *camera); void release(Camera *camera); @@ -72,7 +72,7 @@ public: protected: void registerCamera(std::shared_ptr camera); - void hotplugMediaDevice(MediaDevice *media); + void hotplugMediaDevice(std::shared_ptr media); unsigned int useCount() const { return useCount_; } virtual int queueRequestDevice(Camera *camera, Request *request) = 0; @@ -87,7 +87,7 @@ protected: private: void unlockMediaDevices(); - void mediaDeviceDisconnected(MediaDevice *media); + void mediaDeviceDisconnected(std::shared_ptr media); virtual void disconnect(); void doQueueRequest(Request *request); diff --git a/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp b/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp index 9550f54600c4..049d9b1c44cb 100644 --- a/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp +++ b/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp @@ -141,7 +141,7 @@ private: void bufferReady(FrameBuffer *buffer); - MediaDevice *isiDev_; + std::shared_ptr isiDev_; std::unique_ptr crossbar_; std::vector pipes_; @@ -996,7 +996,7 @@ bool PipelineHandlerISI::match(DeviceEnumerator *enumerator) * Acquire the subdevs and video nodes for the crossbar switch and the * processing pipelines. */ - crossbar_ = V4L2Subdevice::fromEntityName(isiDev_, "crossbar"); + crossbar_ = V4L2Subdevice::fromEntityName(isiDev_.get(), "crossbar"); if (!crossbar_) return false; @@ -1007,7 +1007,7 @@ bool PipelineHandlerISI::match(DeviceEnumerator *enumerator) for (unsigned int i = 0; ; ++i) { std::string entityName = "mxc_isi." + std::to_string(i); std::unique_ptr isi = - V4L2Subdevice::fromEntityName(isiDev_, entityName); + V4L2Subdevice::fromEntityName(isiDev_.get(), entityName); if (!isi) break; @@ -1017,7 +1017,7 @@ bool PipelineHandlerISI::match(DeviceEnumerator *enumerator) entityName += ".capture"; std::unique_ptr capture = - V4L2VideoDevice::fromEntityName(isiDev_, entityName); + V4L2VideoDevice::fromEntityName(isiDev_.get(), entityName); if (!capture) return false; diff --git a/src/libcamera/pipeline/ipu3/cio2.cpp b/src/libcamera/pipeline/ipu3/cio2.cpp index aa544d7b0303..6f9c055f4d12 100644 --- a/src/libcamera/pipeline/ipu3/cio2.cpp +++ b/src/libcamera/pipeline/ipu3/cio2.cpp @@ -112,7 +112,7 @@ std::vector CIO2Device::sizes(const PixelFormat &format) const * \return 0 on success or a negative error code otherwise * \retval -ENODEV No supported image sensor is connected to this CIO2 instance */ -int CIO2Device::init(const MediaDevice *media, unsigned int index) +int CIO2Device::init(std::shared_ptr media, unsigned int index) { int ret; @@ -170,7 +170,7 @@ int CIO2Device::init(const MediaDevice *media, unsigned int index) return ret; std::string cio2Name = "ipu3-cio2 " + std::to_string(index); - output_ = V4L2VideoDevice::fromEntityName(media, cio2Name); + output_ = V4L2VideoDevice::fromEntityName(media.get(), cio2Name); return output_->open(); } diff --git a/src/libcamera/pipeline/ipu3/cio2.h b/src/libcamera/pipeline/ipu3/cio2.h index 3aa3a1ca8ef8..d844cb7ae831 100644 --- a/src/libcamera/pipeline/ipu3/cio2.h +++ b/src/libcamera/pipeline/ipu3/cio2.h @@ -38,7 +38,7 @@ public: std::vector formats() const; std::vector sizes(const PixelFormat &format) const; - int init(const MediaDevice *media, unsigned int index); + int init(std::shared_ptr media, unsigned int index); int configure(const Size &size, const Transform &transform, V4L2DeviceFormat *outputFormat); diff --git a/src/libcamera/pipeline/ipu3/imgu.cpp b/src/libcamera/pipeline/ipu3/imgu.cpp index 7be780913fae..bb36c06e8dea 100644 --- a/src/libcamera/pipeline/ipu3/imgu.cpp +++ b/src/libcamera/pipeline/ipu3/imgu.cpp @@ -330,7 +330,8 @@ FOV calcFOV(const Size &in, const ImgUDevice::PipeConfig &pipe) * * \return 0 on success or a negative error code otherwise */ -int ImgUDevice::init(MediaDevice *media, unsigned int index) +int ImgUDevice::init(std::shared_ptr media, + unsigned int index) { int ret; @@ -342,32 +343,32 @@ int ImgUDevice::init(MediaDevice *media, unsigned int index) * by the match() function: no need to check for newly created * video devices and subdevice validity here. */ - imgu_ = V4L2Subdevice::fromEntityName(media, name_); + imgu_ = V4L2Subdevice::fromEntityName(media.get(), name_); ret = imgu_->open(); if (ret) return ret; - input_ = V4L2VideoDevice::fromEntityName(media, name_ + " input"); + input_ = V4L2VideoDevice::fromEntityName(media.get(), name_ + " input"); ret = input_->open(); if (ret) return ret; - output_ = V4L2VideoDevice::fromEntityName(media, name_ + " output"); + output_ = V4L2VideoDevice::fromEntityName(media.get(), name_ + " output"); ret = output_->open(); if (ret) return ret; - viewfinder_ = V4L2VideoDevice::fromEntityName(media, name_ + " viewfinder"); + viewfinder_ = V4L2VideoDevice::fromEntityName(media.get(), name_ + " viewfinder"); ret = viewfinder_->open(); if (ret) return ret; - param_ = V4L2VideoDevice::fromEntityName(media, name_ + " parameters"); + param_ = V4L2VideoDevice::fromEntityName(media.get(), name_ + " parameters"); ret = param_->open(); if (ret) return ret; - stat_ = V4L2VideoDevice::fromEntityName(media, name_ + " 3a stat"); + stat_ = V4L2VideoDevice::fromEntityName(media.get(), name_ + " 3a stat"); ret = stat_->open(); if (ret) return ret; diff --git a/src/libcamera/pipeline/ipu3/imgu.h b/src/libcamera/pipeline/ipu3/imgu.h index fa508316b301..272f861f98c4 100644 --- a/src/libcamera/pipeline/ipu3/imgu.h +++ b/src/libcamera/pipeline/ipu3/imgu.h @@ -64,7 +64,7 @@ public: Size viewfinder; }; - int init(MediaDevice *media, unsigned int index); + int init(std::shared_ptr media, unsigned int index); PipeConfig calculatePipeConfig(Pipe *pipe); @@ -118,7 +118,7 @@ private: V4L2DeviceFormat *outputFormat); std::string name_; - MediaDevice *media_; + std::shared_ptr media_; }; } /* namespace libcamera */ diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index d6b7edcb5a7f..695762c750f8 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -164,8 +164,8 @@ private: ImgUDevice imgu0_; ImgUDevice imgu1_; - MediaDevice *cio2MediaDev_; - MediaDevice *imguMediaDev_; + std::shared_ptr cio2MediaDev_; + std::shared_ptr imguMediaDev_; std::vector ipaBuffers_; }; diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 38bdc6138ed1..cf0cb15f8bb3 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -682,7 +682,7 @@ private: bool registerTPGCamera(MediaLink *link); bool registerSensorCamera(MediaLink *link); - MediaDevice *media_; + std::shared_ptr media_; std::unique_ptr isp_; std::unique_ptr stats_; std::unique_ptr params_; @@ -1647,24 +1647,24 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) if (!media_) return false; - isp_ = V4L2Subdevice::fromEntityName(media_, "mali-c55 isp"); + isp_ = V4L2Subdevice::fromEntityName(media_.get(), "mali-c55 isp"); if (isp_->open() < 0) return false; - stats_ = V4L2VideoDevice::fromEntityName(media_, "mali-c55 3a stats"); + stats_ = V4L2VideoDevice::fromEntityName(media_.get(), "mali-c55 3a stats"); if (stats_->open() < 0) return false; - params_ = V4L2VideoDevice::fromEntityName(media_, "mali-c55 3a params"); + params_ = V4L2VideoDevice::fromEntityName(media_.get(), "mali-c55 3a params"); if (params_->open() < 0) return false; MaliC55Pipe *frPipe = &pipes_[MaliC55FR]; - frPipe->resizer = V4L2Subdevice::fromEntityName(media_, "mali-c55 resizer fr"); + frPipe->resizer = V4L2Subdevice::fromEntityName(media_.get(), "mali-c55 resizer fr"); if (frPipe->resizer->open() < 0) return false; - frPipe->cap = V4L2VideoDevice::fromEntityName(media_, "mali-c55 fr"); + frPipe->cap = V4L2VideoDevice::fromEntityName(media_.get(), "mali-c55 fr"); if (frPipe->cap->open() < 0) return false; @@ -1682,11 +1682,11 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) MaliC55Pipe *dsPipe = &pipes_[MaliC55DS]; - dsPipe->resizer = V4L2Subdevice::fromEntityName(media_, "mali-c55 resizer ds"); + dsPipe->resizer = V4L2Subdevice::fromEntityName(media_.get(), "mali-c55 resizer ds"); if (dsPipe->resizer->open() < 0) return false; - dsPipe->cap = V4L2VideoDevice::fromEntityName(media_, "mali-c55 ds"); + dsPipe->cap = V4L2VideoDevice::fromEntityName(media_.get(), "mali-c55 ds"); if (dsPipe->cap->open() < 0) return false; diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index d7195b6c484d..d108acfbad9f 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -218,7 +218,7 @@ private: int updateControls(RkISP1CameraData *data); - MediaDevice *media_; + std::shared_ptr media_; std::unique_ptr isp_; std::unique_ptr param_; std::unique_ptr stat_; @@ -1412,16 +1412,16 @@ bool PipelineHandlerRkISP1::match(DeviceEnumerator *enumerator) hasSelfPath_ = !!media_->getEntityByName("rkisp1_selfpath"); /* Create the V4L2 subdevices we will need. */ - isp_ = V4L2Subdevice::fromEntityName(media_, "rkisp1_isp"); + isp_ = V4L2Subdevice::fromEntityName(media_.get(), "rkisp1_isp"); if (isp_->open() < 0) return false; /* Locate and open the stats and params video nodes. */ - stat_ = V4L2VideoDevice::fromEntityName(media_, "rkisp1_stats"); + stat_ = V4L2VideoDevice::fromEntityName(media_.get(), "rkisp1_stats"); if (stat_->open() < 0) return false; - param_ = V4L2VideoDevice::fromEntityName(media_, "rkisp1_params"); + param_ = V4L2VideoDevice::fromEntityName(media_.get(), "rkisp1_params"); if (param_->open() < 0) return false; diff --git a/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp b/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp index d318bea92446..ef9cfbdc327a 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp @@ -64,16 +64,16 @@ RkISP1Path::RkISP1Path(const char *name, const Span &formats, { } -bool RkISP1Path::init(MediaDevice *media) +bool RkISP1Path::init(std::shared_ptr media) { std::string resizer = std::string("rkisp1_resizer_") + name_ + "path"; std::string video = std::string("rkisp1_") + name_ + "path"; - resizer_ = V4L2Subdevice::fromEntityName(media, resizer); + resizer_ = V4L2Subdevice::fromEntityName(media.get(), resizer); if (resizer_->open() < 0) return false; - video_ = V4L2VideoDevice::fromEntityName(media, video); + video_ = V4L2VideoDevice::fromEntityName(media.get(), video); if (video_->open() < 0) return false; diff --git a/src/libcamera/pipeline/rkisp1/rkisp1_path.h b/src/libcamera/pipeline/rkisp1/rkisp1_path.h index 0b60c499ac64..0c68e9eb99af 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1_path.h +++ b/src/libcamera/pipeline/rkisp1/rkisp1_path.h @@ -37,7 +37,7 @@ public: RkISP1Path(const char *name, const Span &formats, const Size &minResolution, const Size &maxResolution); - bool init(MediaDevice *media); + bool init(std::shared_ptr media); int setEnabled(bool enable) { return link_->setEnabled(enable); } bool isEnabled() const { return link_->flags() & MEDIA_LNK_FL_ENABLED; } diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index c209aa596311..9d65dc83573b 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -786,8 +786,10 @@ int PipelineHandlerBase::queueRequestDevice(Camera *camera, Request *request) } int PipelineHandlerBase::registerCamera(std::unique_ptr &cameraData, - MediaDevice *frontend, const std::string &frontendName, - MediaDevice *backend, MediaEntity *sensorEntity) + std::shared_ptr frontend, + const std::string &frontendName, + std::shared_ptr backend, + MediaEntity *sensorEntity) { CameraData *data = cameraData.get(); int ret; diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index 4bce4ec4f978..15628259afc6 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -224,13 +224,16 @@ public: protected: int registerCamera(std::unique_ptr &cameraData, - MediaDevice *frontent, const std::string &frontendName, - MediaDevice *backend, MediaEntity *sensorEntity); + std::shared_ptr frontend, + const std::string &frontendName, + std::shared_ptr backend, + MediaEntity *sensorEntity); void mapBuffers(Camera *camera, const BufferMap &buffers, unsigned int mask); virtual int platformRegister(std::unique_ptr &cameraData, - MediaDevice *unicam, MediaDevice *isp) = 0; + std::shared_ptr unicam, + std::shared_ptr isp) = 0; private: CameraData *cameraData(Camera *camera) diff --git a/src/libcamera/pipeline/rpi/pisp/pisp.cpp b/src/libcamera/pipeline/rpi/pisp/pisp.cpp index 082724c5a0fb..77acd2f64092 100644 --- a/src/libcamera/pipeline/rpi/pisp/pisp.cpp +++ b/src/libcamera/pipeline/rpi/pisp/pisp.cpp @@ -866,7 +866,8 @@ private: int prepareBuffers(Camera *camera) override; int platformRegister(std::unique_ptr &cameraData, - MediaDevice *cfe, MediaDevice *isp) override; + std::shared_ptr cfe, + std::shared_ptr isp) override; }; bool PipelineHandlerPiSP::match(DeviceEnumerator *enumerator) @@ -884,7 +885,7 @@ bool PipelineHandlerPiSP::match(DeviceEnumerator *enumerator) cfe.add("rp1-cfe-fe-image0"); cfe.add("rp1-cfe-fe-stats"); cfe.add("rp1-cfe-fe-config"); - MediaDevice *cfeDevice = acquireMediaDevice(enumerator, cfe); + std::shared_ptr cfeDevice = acquireMediaDevice(enumerator, cfe); if (!cfeDevice) { LOG(RPI, Debug) << "Unable to acquire a CFE instance"; @@ -900,7 +901,7 @@ bool PipelineHandlerPiSP::match(DeviceEnumerator *enumerator) isp.add("pispbe-tdn_input"); isp.add("pispbe-stitch_output"); isp.add("pispbe-stitch_input"); - MediaDevice *ispDevice = acquireMediaDevice(enumerator, isp); + std::shared_ptr ispDevice = acquireMediaDevice(enumerator, isp); if (!ispDevice) { LOG(RPI, Debug) << "Unable to acquire ISP instance"; @@ -1065,7 +1066,8 @@ int PipelineHandlerPiSP::prepareBuffers(Camera *camera) } int PipelineHandlerPiSP::platformRegister(std::unique_ptr &cameraData, - MediaDevice *cfe, MediaDevice *isp) + std::shared_ptr cfe, + std::shared_ptr isp) { PiSPCameraData *data = static_cast(cameraData.get()); int ret; diff --git a/src/libcamera/pipeline/rpi/vc4/vc4.cpp b/src/libcamera/pipeline/rpi/vc4/vc4.cpp index 99d43bd0a2f5..f0cb99d59e74 100644 --- a/src/libcamera/pipeline/rpi/vc4/vc4.cpp +++ b/src/libcamera/pipeline/rpi/vc4/vc4.cpp @@ -5,6 +5,8 @@ * Pipeline handler for VC4-based Raspberry Pi devices */ +#include + #include #include #include @@ -158,7 +160,8 @@ private: int prepareBuffers(Camera *camera) override; int platformRegister(std::unique_ptr &cameraData, - MediaDevice *unicam, MediaDevice *isp) override; + std::shared_ptr unicam, + std::shared_ptr isp) override; }; bool PipelineHandlerVc4::match(DeviceEnumerator *enumerator) @@ -173,7 +176,7 @@ bool PipelineHandlerVc4::match(DeviceEnumerator *enumerator) */ for (unsigned int i = 0; i < numUnicamDevices; i++) { DeviceMatch unicam("unicam"); - MediaDevice *unicamDevice = acquireMediaDevice(enumerator, unicam); + std::shared_ptr unicamDevice = acquireMediaDevice(enumerator, unicam); if (!unicamDevice) { LOG(RPI, Debug) << "Unable to acquire a Unicam instance"; @@ -181,7 +184,7 @@ bool PipelineHandlerVc4::match(DeviceEnumerator *enumerator) } DeviceMatch isp("bcm2835-isp"); - MediaDevice *ispDevice = acquireMediaDevice(enumerator, isp); + std::shared_ptr ispDevice = acquireMediaDevice(enumerator, isp); if (!ispDevice) { LOG(RPI, Debug) << "Unable to acquire ISP instance"; @@ -303,7 +306,9 @@ int PipelineHandlerVc4::prepareBuffers(Camera *camera) return 0; } -int PipelineHandlerVc4::platformRegister(std::unique_ptr &cameraData, MediaDevice *unicam, MediaDevice *isp) +int PipelineHandlerVc4::platformRegister(std::unique_ptr &cameraData, + std::shared_ptr unicam, + std::shared_ptr isp) { Vc4CameraData *data = static_cast(cameraData.get()); diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 118b4186c8bf..5e81a6b25920 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -413,7 +413,7 @@ public: V4L2VideoDevice *video(const MediaEntity *entity); V4L2Subdevice *subdev(const MediaEntity *entity); - MediaDevice *converter() { return converter_; } + MediaDevice *converter() { return converter_.get(); } bool swIspEnabled() const { return swIspEnabled_; } protected: @@ -434,7 +434,8 @@ private: return static_cast(camera->_d()); } - bool matchDevice(MediaDevice *media, const SimplePipelineInfo &info, + bool matchDevice(std::shared_ptr media, + const SimplePipelineInfo &info, DeviceEnumerator *enumerator); std::vector locateSensors(MediaDevice *media); @@ -445,7 +446,7 @@ private: std::map entities_; - MediaDevice *converter_; + std::shared_ptr converter_; bool swIspEnabled_; }; @@ -1690,7 +1691,7 @@ int SimplePipelineHandler::resetRoutingTable(V4L2Subdevice *subdev) return 0; } -bool SimplePipelineHandler::matchDevice(MediaDevice *media, +bool SimplePipelineHandler::matchDevice(std::shared_ptr media, const SimplePipelineInfo &info, DeviceEnumerator *enumerator) { @@ -1721,7 +1722,7 @@ bool SimplePipelineHandler::matchDevice(MediaDevice *media, } /* Locate the sensors. */ - std::vector sensors = locateSensors(media); + std::vector sensors = locateSensors(media.get()); if (sensors.empty()) { LOG(SimplePipeline, Info) << "No sensor found for " << media->deviceNode(); return false; @@ -1839,7 +1840,7 @@ bool SimplePipelineHandler::matchDevice(MediaDevice *media, bool SimplePipelineHandler::match(DeviceEnumerator *enumerator) { - MediaDevice *media; + std::shared_ptr media; for (const SimplePipelineInfo &inf : supportedDevices) { DeviceMatch dm(inf.driver); diff --git a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp index 4b5816dfdde0..034f9c39d3e9 100644 --- a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp +++ b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp @@ -46,7 +46,7 @@ public: { } - int init(MediaDevice *media); + int init(std::shared_ptr media); void addControl(uint32_t cid, const ControlInfo &v4l2info, ControlInfoMap::Map *ctrls); void imageBufferReady(FrameBuffer *buffer); @@ -466,7 +466,7 @@ int PipelineHandlerUVC::queueRequestDevice(Camera *camera, Request *request) bool PipelineHandlerUVC::match(DeviceEnumerator *enumerator) { - MediaDevice *media; + std::shared_ptr media; DeviceMatch dm("uvcvideo"); media = acquireMediaDevice(enumerator, dm); @@ -508,7 +508,7 @@ void PipelineHandlerUVC::releaseDevice(Camera *camera) data->video_->close(); } -int UVCCameraData::init(MediaDevice *media) +int UVCCameraData::init(std::shared_ptr media) { int ret; diff --git a/src/libcamera/pipeline/vimc/vimc.cpp b/src/libcamera/pipeline/vimc/vimc.cpp index 5022101505a1..cd3370e3d419 100644 --- a/src/libcamera/pipeline/vimc/vimc.cpp +++ b/src/libcamera/pipeline/vimc/vimc.cpp @@ -49,7 +49,7 @@ LOG_DEFINE_CATEGORY(VIMC) class VimcCameraData : public Camera::Private { public: - VimcCameraData(PipelineHandler *pipe, MediaDevice *media) + VimcCameraData(PipelineHandler *pipe, std::shared_ptr media) : Camera::Private(pipe), media_(media) { } @@ -59,7 +59,7 @@ public: void imageBufferReady(FrameBuffer *buffer); void paramsComputed(unsigned int id, const Flags flags); - MediaDevice *media_; + std::shared_ptr media_; std::unique_ptr sensor_; std::unique_ptr debayer_; std::unique_ptr scaler_; @@ -479,7 +479,7 @@ bool PipelineHandlerVimc::match(DeviceEnumerator *enumerator) dm.add("RGB/YUV Input"); dm.add("Scaler"); - MediaDevice *media = acquireMediaDevice(enumerator, dm); + std::shared_ptr media = acquireMediaDevice(enumerator, dm); if (!media) return false; @@ -539,21 +539,21 @@ int VimcCameraData::init() if (!sensor_) return -ENODEV; - debayer_ = V4L2Subdevice::fromEntityName(media_, "Debayer B"); + debayer_ = V4L2Subdevice::fromEntityName(media_.get(), "Debayer B"); if (debayer_->open()) return -ENODEV; - scaler_ = V4L2Subdevice::fromEntityName(media_, "Scaler"); + scaler_ = V4L2Subdevice::fromEntityName(media_.get(), "Scaler"); if (scaler_->open()) return -ENODEV; - video_ = V4L2VideoDevice::fromEntityName(media_, "RGB/YUV Capture"); + video_ = V4L2VideoDevice::fromEntityName(media_.get(), "RGB/YUV Capture"); if (video_->open()) return -ENODEV; video_->bufferReady.connect(this, &VimcCameraData::imageBufferReady); - raw_ = V4L2VideoDevice::fromEntityName(media_, "Raw Capture 1"); + raw_ = V4L2VideoDevice::fromEntityName(media_.get(), "Raw Capture 1"); if (raw_->open()) return -ENODEV; diff --git a/src/libcamera/pipeline_handler.cpp b/src/libcamera/pipeline_handler.cpp index ca6215ec69ee..5c469e5bad24 100644 --- a/src/libcamera/pipeline_handler.cpp +++ b/src/libcamera/pipeline_handler.cpp @@ -129,10 +129,12 @@ PipelineHandler::~PipelineHandler() * * \context This function shall be called from the CameraManager thread. * - * \return A pointer to the matching MediaDevice, or nullptr if no match is found + * \return A shared pointer to the matching MediaDevice, or nullptr if no match + * is found */ -MediaDevice *PipelineHandler::acquireMediaDevice(DeviceEnumerator *enumerator, - const DeviceMatch &dm) +std::shared_ptr +PipelineHandler::acquireMediaDevice(DeviceEnumerator *enumerator, + const DeviceMatch &dm) { std::shared_ptr media = enumerator->search(dm); if (!media) @@ -143,7 +145,7 @@ MediaDevice *PipelineHandler::acquireMediaDevice(DeviceEnumerator *enumerator, mediaDevices_.push_back(media); - return media.get(); + return media; } /** @@ -734,7 +736,7 @@ void PipelineHandler::registerCamera(std::shared_ptr camera) * handler gets notified and automatically disconnects all the cameras it has * registered without requiring any manual intervention. */ -void PipelineHandler::hotplugMediaDevice(MediaDevice *media) +void PipelineHandler::hotplugMediaDevice(std::shared_ptr media) { media->disconnected.connect(this, [this, media] { mediaDeviceDisconnected(media); }); } @@ -742,7 +744,7 @@ void PipelineHandler::hotplugMediaDevice(MediaDevice *media) /** * \brief Slot for the MediaDevice disconnected signal */ -void PipelineHandler::mediaDeviceDisconnected(MediaDevice *media) +void PipelineHandler::mediaDeviceDisconnected(std::shared_ptr media) { media->disconnected.disconnect(this); diff --git a/test/libtest/buffer_source.cpp b/test/libtest/buffer_source.cpp index dde11f365e43..97c8ae1cbd31 100644 --- a/test/libtest/buffer_source.cpp +++ b/test/libtest/buffer_source.cpp @@ -52,7 +52,8 @@ int BufferSource::allocate(const StreamConfiguration &config) return TestSkip; } - std::unique_ptr video = V4L2VideoDevice::fromEntityName(media_.get(), videoDeviceName); + std::unique_ptr video = V4L2VideoDevice::fromEntityName(media_.get(), + videoDeviceName); if (!video) { std::cout << "Failed to get video device from entity " << videoDeviceName << std::endl; From patchwork Tue Nov 25 16:28:14 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 25181 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 91A2EC333C for ; Tue, 25 Nov 2025 16:29:03 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4335D60AA1; Tue, 25 Nov 2025 17:29:03 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="pu0xY3Oo"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 54D7C609E0 for ; Tue, 25 Nov 2025 17:29:01 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bae1:340c:573c:570b]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 5C733FDB; Tue, 25 Nov 2025 17:26:52 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1764088012; bh=n+gNvg8g0oaH8LMVT8N2zaukIHp892CqNUtWD6tdjjo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=pu0xY3OoaVgt0S7WeKVL4kwnos/aXxZTjOFPfKNffPdzbtgl/WXa69x0dCrMm7+Ph kx6AKlOOqWUoPt2qPaxXvCd9VgDtX17tUcmT4uW1mfmocYVPDJdnSGIQb4Ba6DcgtD o8GX5gSZVSy9f9pi8gzm33MtB87nmVCJMVUnjO6E= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Paul Elder , Isaac Scott Subject: [PATCH v3 02/29] libcamera: converter: Utilise shared MediaDevice pointers Date: Tue, 25 Nov 2025 17:28:14 +0100 Message-ID: <20251125162851.2301793-3-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> References: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" With the upcoming addition of V4L2 requests support, the converters need to keep a handle to the corresponding media device. Prepare for that by changing the constructor parameter from a raw pointer to a shared pointer. Signed-off-by: Stefan Klug Reviewed-by: Paul Elder Reviewed-by: Isaac Scott --- include/libcamera/internal/converter.h | 9 +++++---- .../libcamera/internal/converter/converter_v4l2_m2m.h | 2 +- src/libcamera/converter.cpp | 5 +++-- src/libcamera/converter/converter_v4l2_m2m.cpp | 4 ++-- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 2 +- src/libcamera/pipeline/simple/simple.cpp | 4 ++-- 6 files changed, 14 insertions(+), 12 deletions(-) diff --git a/include/libcamera/internal/converter.h b/include/libcamera/internal/converter.h index 644ec429bb25..4915af7ac5de 100644 --- a/include/libcamera/internal/converter.h +++ b/include/libcamera/internal/converter.h @@ -46,7 +46,7 @@ public: Up, }; - Converter(MediaDevice *media, Features features = Feature::None); + Converter(std::shared_ptr media, Features features = Feature::None); virtual ~Converter(); virtual int loadConfiguration(const std::string &filename) = 0; @@ -107,7 +107,7 @@ public: const std::vector &compatibles() const { return compatibles_; } - static std::unique_ptr create(MediaDevice *media); + static std::unique_ptr create(std::shared_ptr media); static std::vector &factories(); static std::vector names(); @@ -116,7 +116,8 @@ private: static void registerType(ConverterFactoryBase *factory); - virtual std::unique_ptr createInstance(MediaDevice *media) const = 0; + virtual std::unique_ptr + createInstance(std::shared_ptr media) const = 0; std::string name_; std::vector compatibles_; @@ -131,7 +132,7 @@ public: { } - std::unique_ptr createInstance(MediaDevice *media) const override + std::unique_ptr createInstance(std::shared_ptr media) const override { return std::make_unique<_Converter>(media); } diff --git a/include/libcamera/internal/converter/converter_v4l2_m2m.h b/include/libcamera/internal/converter/converter_v4l2_m2m.h index 0ad7bf7fdbe2..d316754040dd 100644 --- a/include/libcamera/internal/converter/converter_v4l2_m2m.h +++ b/include/libcamera/internal/converter/converter_v4l2_m2m.h @@ -36,7 +36,7 @@ class V4L2M2MDevice; class V4L2M2MConverter : public Converter { public: - V4L2M2MConverter(MediaDevice *media); + V4L2M2MConverter(std::shared_ptr media); int loadConfiguration([[maybe_unused]] const std::string &filename) override { return 0; } bool isValid() const override { return m2m_ != nullptr; } diff --git a/src/libcamera/converter.cpp b/src/libcamera/converter.cpp index d551b908d523..142fb29a1272 100644 --- a/src/libcamera/converter.cpp +++ b/src/libcamera/converter.cpp @@ -8,6 +8,7 @@ #include "libcamera/internal/converter.h" #include +#include #include @@ -68,7 +69,7 @@ LOG_DEFINE_CATEGORY(Converter) * This searches for the entity implementing the data streaming function in the * media graph entities and use its device node as the converter device node. */ -Converter::Converter(MediaDevice *media, Features features) +Converter::Converter(std::shared_ptr media, Features features) { const std::vector &entities = media->entities(); auto it = std::find_if(entities.begin(), entities.end(), @@ -332,7 +333,7 @@ ConverterFactoryBase::ConverterFactoryBase(const std::string name, std::initiali * \return A new instance of the converter subclass corresponding to the media * device, or null if the media device driver name doesn't match anything */ -std::unique_ptr ConverterFactoryBase::create(MediaDevice *media) +std::unique_ptr ConverterFactoryBase::create(std::shared_ptr media) { const std::vector &factories = ConverterFactoryBase::factories(); diff --git a/src/libcamera/converter/converter_v4l2_m2m.cpp b/src/libcamera/converter/converter_v4l2_m2m.cpp index 0ce7a7a67f11..b2bd54f368d8 100644 --- a/src/libcamera/converter/converter_v4l2_m2m.cpp +++ b/src/libcamera/converter/converter_v4l2_m2m.cpp @@ -10,6 +10,7 @@ #include #include +#include #include #include @@ -261,8 +262,7 @@ void V4L2M2MConverter::V4L2M2MStream::captureBufferReady(FrameBuffer *buffer) * \brief Construct a V4L2M2MConverter instance * \param[in] media The media device implementing the converter */ - -V4L2M2MConverter::V4L2M2MConverter(MediaDevice *media) +V4L2M2MConverter::V4L2M2MConverter(std::shared_ptr media) : Converter(media) { if (deviceNode().empty()) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index d108acfbad9f..dbd6bf9673ff 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -1445,7 +1445,7 @@ bool PipelineHandlerRkISP1::match(DeviceEnumerator *enumerator) std::shared_ptr dwpMediaDevice = enumerator->search(dwp); if (dwpMediaDevice) { - dewarper_ = std::make_unique(dwpMediaDevice.get()); + dewarper_ = std::make_unique(dwpMediaDevice); if (dewarper_->isValid()) { dewarper_->outputBufferReady.connect( this, &PipelineHandlerRkISP1::dewarpBufferReady); diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 5e81a6b25920..7cebc9ac3824 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -413,7 +413,7 @@ public: V4L2VideoDevice *video(const MediaEntity *entity); V4L2Subdevice *subdev(const MediaEntity *entity); - MediaDevice *converter() { return converter_.get(); } + std::shared_ptr converter() { return converter_; } bool swIspEnabled() const { return swIspEnabled_; } protected: @@ -586,7 +586,7 @@ int SimpleCameraData::init() int ret; /* Open the converter, if any. */ - MediaDevice *converter = pipe->converter(); + std::shared_ptr converter = pipe->converter(); if (converter) { converter_ = ConverterFactoryBase::create(converter); if (!converter_) { From patchwork Tue Nov 25 16:28:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 25182 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9197BC333C for ; Tue, 25 Nov 2025 16:29:06 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 394E860AA2; Tue, 25 Nov 2025 17:29:06 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="V2/+SzEe"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A5D83609E0 for ; Tue, 25 Nov 2025 17:29:04 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bae1:340c:573c:570b]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 7AD721E2A; Tue, 25 Nov 2025 17:26:55 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1764088015; bh=pawi4pqbrOgvyYKGn7ePAXaGTNuE7Yk0khC5rT9bJAU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=V2/+SzEeNrY69HslYGYUbAUpmfi6nFRAWuGCQ27EHp84NjLfUfY/ycUgyNfst0Daq AMZniCBfBsuS1YgzZSgYvl2Qlmgkgc+mRRoflC3i/nm51j7FRO1wz9NlHvlk16lDcd Lc3sI5eRyAFbgsyVmfGAP9FrQbER5VZc5uOlpXLM= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v3 03/29] libcamera: Add support for V4L2 requests Date: Tue, 25 Nov 2025 17:28:15 +0100 Message-ID: <20251125162851.2301793-4-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> References: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The V4L2 requests API provides support to atomically tie controls to a set of buffers. This is especially common for m2m devices. Such a request is represented by an fd that is allocated via MEDIA_IOC_REQUEST_ALLOC and then passed to the various V4L2 functions. Implement a V4L2Request class to wrap such an fd and add the corresponding utility functions. Signed-off-by: Stefan Klug Reviewed-by: Kieran Bingham --- Changes in v3: - Replaced int by UniqueFD - Fixed lots if typos from review Changes in v2: - Added documentation --- include/libcamera/internal/media_device.h | 8 ++ include/libcamera/internal/meson.build | 1 + include/libcamera/internal/v4l2_device.h | 5 +- include/libcamera/internal/v4l2_request.h | 44 ++++++ include/libcamera/internal/v4l2_videodevice.h | 3 +- src/libcamera/media_device.cpp | 47 +++++++ src/libcamera/meson.build | 1 + src/libcamera/v4l2_device.cpp | 30 +++- src/libcamera/v4l2_request.cpp | 128 ++++++++++++++++++ src/libcamera/v4l2_videodevice.cpp | 10 +- 10 files changed, 268 insertions(+), 9 deletions(-) create mode 100644 include/libcamera/internal/v4l2_request.h create mode 100644 src/libcamera/v4l2_request.cpp diff --git a/include/libcamera/internal/media_device.h b/include/libcamera/internal/media_device.h index b3a48b98d64b..2eb3ad988b09 100644 --- a/include/libcamera/internal/media_device.h +++ b/include/libcamera/internal/media_device.h @@ -8,6 +8,7 @@ #pragma once #include +#include #include #include @@ -18,6 +19,7 @@ #include #include "libcamera/internal/media_object.h" +#include "libcamera/internal/v4l2_request.h" namespace libcamera { @@ -57,6 +59,11 @@ public: std::vector locateEntities(unsigned int function); + int allocateRequests(unsigned int count, + std::vector> *requests); + + bool supportsRequests(); + protected: std::string logPrefix() const override; @@ -87,6 +94,7 @@ private: UniqueFD fd_; bool valid_; bool acquired_; + std::optional supportsRequests_; std::map objects_; std::vector entities_; diff --git a/include/libcamera/internal/meson.build b/include/libcamera/internal/meson.build index 45c299f6a332..e9540a2f734f 100644 --- a/include/libcamera/internal/meson.build +++ b/include/libcamera/internal/meson.build @@ -44,6 +44,7 @@ libcamera_internal_headers = files([ 'sysfs.h', 'v4l2_device.h', 'v4l2_pixelformat.h', + 'v4l2_request.h', 'v4l2_subdevice.h', 'v4l2_videodevice.h', 'vector.h', diff --git a/include/libcamera/internal/v4l2_device.h b/include/libcamera/internal/v4l2_device.h index 5bc9da96677d..dbbd118abd00 100644 --- a/include/libcamera/internal/v4l2_device.h +++ b/include/libcamera/internal/v4l2_device.h @@ -24,6 +24,7 @@ #include #include "libcamera/internal/formats.h" +#include "libcamera/internal/v4l2_request.h" namespace libcamera { @@ -37,8 +38,8 @@ public: const ControlInfoMap &controls() const { return controls_; } - ControlList getControls(Span ids); - int setControls(ControlList *ctrls); + ControlList getControls(Span ids, const V4L2Request *request = nullptr); + int setControls(ControlList *ctrls, const V4L2Request *request = nullptr); const struct v4l2_query_ext_ctrl *controlInfo(uint32_t id) const; diff --git a/include/libcamera/internal/v4l2_request.h b/include/libcamera/internal/v4l2_request.h new file mode 100644 index 000000000000..376c79ceedba --- /dev/null +++ b/include/libcamera/internal/v4l2_request.h @@ -0,0 +1,44 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2025, Ideas On Board + * + * V4L2 requests + */ + +#pragma once + +#include + +#include + +#include +#include +#include +#include + +namespace libcamera { + +class V4L2Request : protected Loggable +{ +public: + bool isValid() const { return fd_.isValid(); } + int fd() const { return fd_.get(); } + + int reinit(); + int queue(); + + V4L2Request(UniqueFD &&fd); + + Signal requestDone; + +private: + LIBCAMERA_DISABLE_COPY_AND_MOVE(V4L2Request) + + void requestReady(); + std::string logPrefix() const override; + + UniqueFD fd_; + EventNotifier fdNotifier_; +}; + +} /* namespace libcamera */ diff --git a/include/libcamera/internal/v4l2_videodevice.h b/include/libcamera/internal/v4l2_videodevice.h index 6caafc4dcf08..57db0036db6b 100644 --- a/include/libcamera/internal/v4l2_videodevice.h +++ b/include/libcamera/internal/v4l2_videodevice.h @@ -33,6 +33,7 @@ #include "libcamera/internal/formats.h" #include "libcamera/internal/v4l2_device.h" #include "libcamera/internal/v4l2_pixelformat.h" +#include "libcamera/internal/v4l2_request.h" namespace libcamera { @@ -217,7 +218,7 @@ public: int importBuffers(unsigned int count); int releaseBuffers(); - int queueBuffer(FrameBuffer *buffer); + int queueBuffer(FrameBuffer *buffer, const V4L2Request *request = nullptr); Signal bufferReady; int streamOn(); diff --git a/src/libcamera/media_device.cpp b/src/libcamera/media_device.cpp index 353f34a81eca..2a848ebed998 100644 --- a/src/libcamera/media_device.cpp +++ b/src/libcamera/media_device.cpp @@ -20,6 +20,7 @@ #include #include +#include "libcamera/internal/v4l2_request.h" /** * \file media_device.h @@ -851,4 +852,50 @@ std::vector MediaDevice::locateEntities(unsigned int function) return found; } +/** + * \brief Allocate requests + * \param[in] count Number of requests to allocate + * \param[out] requests Vector to store allocated requests + * + * Allocates and stores \a count requests in \a requests. If allocation fails, + * an error is returned and \a requests is cleared. + * + * \return 0 on success or a negative error code otherwise + */ +int MediaDevice::allocateRequests(unsigned int count, + std::vector> *requests) +{ + requests->resize(count); + for (unsigned int i = 0; i < count; i++) { + int requestFd; + int ret = ::ioctl(fd_.get(), MEDIA_IOC_REQUEST_ALLOC, &requestFd); + if (ret < 0) { + requests->clear(); + return -errno; + } + (*requests)[i] = std::make_unique(UniqueFD(requestFd)); + } + + return 0; +} + +/** + * \brief Check if requests are supported + * + * Checks if the device supports V4L2 requests by trying to allocate a single + * request. The result is cached, so the allocation is only tried once. + * + * \return True if the device supports requests, false otherwise + */ +bool MediaDevice::supportsRequests() +{ + if (supportsRequests_.has_value()) + return supportsRequests_.value(); + + std::vector> requests; + supportsRequests_ = (allocateRequests(1, &requests) == 0); + + return supportsRequests_.value(); +} + } /* namespace libcamera */ diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build index 5b9b86f211f1..34e20f557514 100644 --- a/src/libcamera/meson.build +++ b/src/libcamera/meson.build @@ -54,6 +54,7 @@ libcamera_internal_sources = files([ 'sysfs.cpp', 'v4l2_device.cpp', 'v4l2_pixelformat.cpp', + 'v4l2_request.cpp', 'v4l2_subdevice.cpp', 'v4l2_videodevice.cpp', 'vector.cpp', diff --git a/src/libcamera/v4l2_device.cpp b/src/libcamera/v4l2_device.cpp index 8c78b8c424e5..8dcd5e618938 100644 --- a/src/libcamera/v4l2_device.cpp +++ b/src/libcamera/v4l2_device.cpp @@ -162,6 +162,7 @@ void V4L2Device::close() /** * \brief Read controls from the device * \param[in] ids The list of controls to read, specified by their ID + * \param[in] request An optional request * * This function reads the value of all controls contained in \a ids, and * returns their values as a ControlList. @@ -171,10 +172,12 @@ void V4L2Device::close() * during validation of the requested controls, no control is read and this * function returns an empty control list. * + * If \a request is specified the controls tied to that request are read. + * * \return The control values in a ControlList on success, or an empty list on * error */ -ControlList V4L2Device::getControls(Span ids) +ControlList V4L2Device::getControls(Span ids, const V4L2Request *request) { if (ids.empty()) return {}; @@ -242,10 +245,16 @@ ControlList V4L2Device::getControls(Span ids) } struct v4l2_ext_controls v4l2ExtCtrls = {}; - v4l2ExtCtrls.which = V4L2_CTRL_WHICH_CUR_VAL; v4l2ExtCtrls.controls = v4l2Ctrls.data(); v4l2ExtCtrls.count = v4l2Ctrls.size(); + if (request) { + v4l2ExtCtrls.which = V4L2_CTRL_WHICH_REQUEST_VAL; + v4l2ExtCtrls.request_fd = request->fd(); + } else { + v4l2ExtCtrls.which = V4L2_CTRL_WHICH_CUR_VAL; + } + int ret = ioctl(VIDIOC_G_EXT_CTRLS, &v4l2ExtCtrls); if (ret) { unsigned int errorIdx = v4l2ExtCtrls.error_idx; @@ -273,6 +282,7 @@ ControlList V4L2Device::getControls(Span ids) /** * \brief Write controls to the device * \param[in] ctrls The list of controls to write + * \param[in] request An optional request * * This function writes the value of all controls contained in \a ctrls, and * stores the values actually applied to the device in the corresponding @@ -288,11 +298,15 @@ ControlList V4L2Device::getControls(Span ids) * are written and their values are updated in \a ctrls, while all other * controls are not written and their values are not changed. * + * If \a request is set, the controls will be applied to that request. If the + * device doesn't support requests, -EACCESS will be returned. If \a request is + * invalid, -EINVAL will be returned. + * * \return 0 on success or an error code otherwise - * \retval -EINVAL One of the control is not supported or not accessible + * \retval -EINVAL One of the controls is not supported or not accessible * \retval i The index of the control that failed */ -int V4L2Device::setControls(ControlList *ctrls) +int V4L2Device::setControls(ControlList *ctrls, const V4L2Request *request) { if (ctrls->empty()) return 0; @@ -377,10 +391,16 @@ int V4L2Device::setControls(ControlList *ctrls) } struct v4l2_ext_controls v4l2ExtCtrls = {}; - v4l2ExtCtrls.which = V4L2_CTRL_WHICH_CUR_VAL; v4l2ExtCtrls.controls = v4l2Ctrls.data(); v4l2ExtCtrls.count = v4l2Ctrls.size(); + if (request) { + v4l2ExtCtrls.which = V4L2_CTRL_WHICH_REQUEST_VAL; + v4l2ExtCtrls.request_fd = request->fd(); + } else { + v4l2ExtCtrls.which = V4L2_CTRL_WHICH_CUR_VAL; + } + int ret = ioctl(VIDIOC_S_EXT_CTRLS, &v4l2ExtCtrls); if (ret) { unsigned int errorIdx = v4l2ExtCtrls.error_idx; diff --git a/src/libcamera/v4l2_request.cpp b/src/libcamera/v4l2_request.cpp new file mode 100644 index 000000000000..9441d7c4017b --- /dev/null +++ b/src/libcamera/v4l2_request.cpp @@ -0,0 +1,128 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2025, Ideas On Board + * + * V4L2 Request API + */ + +#include "libcamera/internal/v4l2_request.h" + +#include +#include +#include +#include +#include + +#include + +#include +#include + +/** + * \file v4l2_request.h + * \brief V4L2 Request + */ + +namespace libcamera { + +LOG_DECLARE_CATEGORY(V4L2) + +/** + * \class V4L2Request + * \brief V4L2Request object and API + * + * The V4L2Request class wraps a V4L2 request fd and provides some convenience + * functions to handle request. + * + * It is usually constructed by calling \a MediaDevice::allocateRequests(). + * + * A request can then be passed to the V4L2Device::setControls(), + * V4L2Device::getControls() and V4L2VideoDevice::queueBuffer(). + */ + +/** + * \brief Construct a V4L2Request + * \param[in] fd The request fd + */ +V4L2Request::V4L2Request(UniqueFD &&fd) + : fd_(std::move(fd)), fdNotifier_(fd_.get(), EventNotifier::Exception) +{ + if (!fd_.isValid()) + return; + + fdNotifier_.activated.connect(this, &V4L2Request::requestReady); + fdNotifier_.setEnabled(false); +} + +/** + * \fn V4L2Request::isValid() + * \brief Check if the request is valid + * + * Checks if the underlying fd is valid. + * + * \return True if the request is valid, false otherwise + */ + +/** + * \fn V4L2Request::fd() + * \brief Get the file descriptor + * + * \return The file descriptor wrapped by this V4L2Request + */ + +/** + * \var V4L2Request::requestDone + * \brief Signal that is emitted when the request is done + */ + +/** + * \brief Reinit the request + * + * Calls MEDIA_REQUEST_IOC_REINIT on the request fd. + * + * \return 0 on success or a negative error code otherwise + */ +int V4L2Request::reinit() +{ + fdNotifier_.setEnabled(false); + + if (::ioctl(fd_.get(), MEDIA_REQUEST_IOC_REINIT) < 0) + return -errno; + + return 0; +} + +/** + * \brief Queue the request + * + * Calls MEDIA_REQUEST_IOC_QUEUE on the request fd. + * + * \return 0 on success or a negative error code otherwise + */ +int V4L2Request::queue() +{ + if (::ioctl(fd_.get(), MEDIA_REQUEST_IOC_QUEUE) < 0) + return -errno; + + fdNotifier_.setEnabled(true); + + return 0; +} + +std::string V4L2Request::logPrefix() const +{ + return "Request [" + std::to_string(fd()) + "]"; +} + +/** + * \brief Slot to handle request done events + * + * When this slot is called, the request is done and the requestDone will be + * emitted. + */ +void V4L2Request::requestReady() +{ + requestDone.emit(this); +} + +} /* namespace libcamera */ diff --git a/src/libcamera/v4l2_videodevice.cpp b/src/libcamera/v4l2_videodevice.cpp index 7b48d911db73..25b61d049a0e 100644 --- a/src/libcamera/v4l2_videodevice.cpp +++ b/src/libcamera/v4l2_videodevice.cpp @@ -30,6 +30,7 @@ #include "libcamera/internal/framebuffer.h" #include "libcamera/internal/media_device.h" #include "libcamera/internal/media_object.h" +#include "libcamera/internal/v4l2_request.h" /** * \file v4l2_videodevice.h @@ -1629,6 +1630,7 @@ int V4L2VideoDevice::releaseBuffers() /** * \brief Queue a buffer to the video device * \param[in] buffer The buffer to be queued + * \param[in] request An optional request * * For capture video devices the \a buffer will be filled with data by the * device. For output video devices the \a buffer shall contain valid data and @@ -1641,9 +1643,11 @@ int V4L2VideoDevice::releaseBuffers() * Note that queueBuffer() will fail if the device is in the process of being * stopped from a streaming state through streamOff(). * + * If \a request is specified, the buffer will be tied to that request. + * * \return 0 on success or a negative error code otherwise */ -int V4L2VideoDevice::queueBuffer(FrameBuffer *buffer) +int V4L2VideoDevice::queueBuffer(FrameBuffer *buffer, const V4L2Request *request) { struct v4l2_plane v4l2Planes[VIDEO_MAX_PLANES] = {}; struct v4l2_buffer buf = {}; @@ -1674,6 +1678,10 @@ int V4L2VideoDevice::queueBuffer(FrameBuffer *buffer) buf.type = bufferType_; buf.memory = memoryType_; buf.field = V4L2_FIELD_NONE; + if (request) { + buf.flags = V4L2_BUF_FLAG_REQUEST_FD; + buf.request_fd = request->fd(); + } bool multiPlanar = V4L2_TYPE_IS_MULTIPLANAR(buf.type); Span planes = buffer->planes(); From patchwork Tue Nov 25 16:28:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 25183 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 96FE6C333C for ; Tue, 25 Nov 2025 16:29:09 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 34BCE60AA9; Tue, 25 Nov 2025 17:29:09 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="CWWHAZ7b"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 83B53609E0 for ; Tue, 25 Nov 2025 17:29:07 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bae1:340c:573c:570b]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 90F711E2A; Tue, 25 Nov 2025 17:26:58 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1764088018; bh=R/OeGakh4E74RaUScqVa/Jadhx0yfwzbVkg+ww+ql0g=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CWWHAZ7b+3rrNMPCfI+rzJJz8GdYPGchpkMKHUMwjidfGwNluMJqdXkBExpfoPYDr 3/qHNo7uDCIY7hXggK3TKWMFfX22Ocx3a1ohDg7eadx2Y9Fxnr4chdqGBUz9iW9Hv5 FHuJFWhuWl8yD+rkQtzeIWqYzmpNgzVfI/dod6hY= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Paul Elder , Isaac Scott Subject: [PATCH v3 04/29] libcamera: converter: Add V4L2 request support Date: Tue, 25 Nov 2025 17:28:16 +0100 Message-ID: <20251125162851.2301793-5-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> References: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add V4L2 request support to the V4L2M2MConverter class. Extend the functions related to buffer queuing with an optional request parameter that gets passed to the lower layers. Signed-off-by: Stefan Klug Reviewed-by: Paul Elder Reviewed-by: Isaac Scott --- Changes in v3: - Collected tags --- include/libcamera/internal/converter.h | 4 +++- .../internal/converter/converter_v4l2_m2m.h | 6 ++++-- src/libcamera/converter.cpp | 3 +++ src/libcamera/converter/converter_v4l2_m2m.cpp | 12 ++++++++---- 4 files changed, 18 insertions(+), 7 deletions(-) diff --git a/include/libcamera/internal/converter.h b/include/libcamera/internal/converter.h index 4915af7ac5de..4b811686fcf6 100644 --- a/include/libcamera/internal/converter.h +++ b/include/libcamera/internal/converter.h @@ -22,6 +22,7 @@ #include #include +#include "libcamera/internal/v4l2_request.h" namespace libcamera { @@ -79,7 +80,8 @@ public: virtual void stop() = 0; virtual int queueBuffers(FrameBuffer *input, - const std::map &outputs) = 0; + const std::map &outputs, + const V4L2Request *request = nullptr) = 0; virtual int setInputCrop(const Stream *stream, Rectangle *rect) = 0; virtual std::pair inputCropBounds() = 0; diff --git a/include/libcamera/internal/converter/converter_v4l2_m2m.h b/include/libcamera/internal/converter/converter_v4l2_m2m.h index d316754040dd..1b2a88c4a608 100644 --- a/include/libcamera/internal/converter/converter_v4l2_m2m.h +++ b/include/libcamera/internal/converter/converter_v4l2_m2m.h @@ -66,7 +66,8 @@ public: Alignment align = Alignment::Down) override; int queueBuffers(FrameBuffer *input, - const std::map &outputs) override; + const std::map &outputs, + const V4L2Request *request = nullptr) override; int setInputCrop(const Stream *stream, Rectangle *rect) override; std::pair inputCropBounds() override { return inputCropBounds_; } @@ -88,7 +89,8 @@ private: int start(); void stop(); - int queueBuffers(FrameBuffer *input, FrameBuffer *output); + int queueBuffers(FrameBuffer *input, FrameBuffer *output, + const V4L2Request *request = nullptr); int setInputSelection(unsigned int target, Rectangle *rect); int getInputSelection(unsigned int target, Rectangle *rect); diff --git a/src/libcamera/converter.cpp b/src/libcamera/converter.cpp index 142fb29a1272..ec0a6db6c035 100644 --- a/src/libcamera/converter.cpp +++ b/src/libcamera/converter.cpp @@ -205,11 +205,14 @@ Converter::~Converter() * \param[in] input The frame buffer to apply the conversion * \param[out] outputs The container holding the output stream pointers and * their respective frame buffer outputs. + * \param[in] request An optional request * * This function queues the \a input frame buffer on the output streams of the * \a outputs map key and retrieve the output frame buffer indicated by the * buffer map value. * + * If \a request is provided the buffers are tied to that request. + * * \return 0 on success or a negative error code otherwise */ diff --git a/src/libcamera/converter/converter_v4l2_m2m.cpp b/src/libcamera/converter/converter_v4l2_m2m.cpp index b2bd54f368d8..ff11a9735db7 100644 --- a/src/libcamera/converter/converter_v4l2_m2m.cpp +++ b/src/libcamera/converter/converter_v4l2_m2m.cpp @@ -22,6 +22,7 @@ #include #include "libcamera/internal/media_device.h" +#include "libcamera/internal/v4l2_request.h" #include "libcamera/internal/v4l2_videodevice.h" /** @@ -197,9 +198,11 @@ void V4L2M2MConverter::V4L2M2MStream::stop() m2m_->output()->releaseBuffers(); } -int V4L2M2MConverter::V4L2M2MStream::queueBuffers(FrameBuffer *input, FrameBuffer *output) +int V4L2M2MConverter::V4L2M2MStream::queueBuffers(FrameBuffer *input, + FrameBuffer *output, + const V4L2Request *request) { - int ret = m2m_->output()->queueBuffer(input); + int ret = m2m_->output()->queueBuffer(input, request); if (ret < 0) return ret; @@ -696,7 +699,8 @@ int V4L2M2MConverter::validateOutput(StreamConfiguration *cfg, bool *adjusted, * \copydoc libcamera::Converter::queueBuffers */ int V4L2M2MConverter::queueBuffers(FrameBuffer *input, - const std::map &outputs) + const std::map &outputs, + const V4L2Request *request) { std::set outputBufs; int ret; @@ -721,7 +725,7 @@ int V4L2M2MConverter::queueBuffers(FrameBuffer *input, /* Queue the input and output buffers to all the streams. */ for (auto [stream, buffer] : outputs) { - ret = streams_.at(stream)->queueBuffers(input, buffer); + ret = streams_.at(stream)->queueBuffers(input, buffer, request); if (ret < 0) return ret; } From patchwork Tue Nov 25 16:28:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 25184 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 89EC7C333C for ; Tue, 25 Nov 2025 16:29:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3547E60A86; Tue, 25 Nov 2025 17:29:11 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="SSuB9UQo"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 35594609E0 for ; Tue, 25 Nov 2025 17:29:10 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bae1:340c:573c:570b]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 33FA7524; Tue, 25 Nov 2025 17:27:01 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1764088021; bh=RqNAgiDN7nrpVVoNAiWQAGMUqF3E12PxCBnpsW8jzso=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=SSuB9UQoz5D2/iM5ldY+e2FZOsn7S38eg0GepwOyLew0OJofGkGg25A6LNFSwoFIr J4PPLbWXwC5dHDJazxdXhNvR6V+NLA2uQ7+TIqW/QbKBfBxELd9J/urtA12KgyQW8Y 0afSrf6jQMlC7+iCuogy/yKL7nrX96I3sj5PRGPs= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v3 05/29] libcamera: converter_v4l2_m2m: Add suport for V4L2 requests Date: Tue, 25 Nov 2025 17:28:17 +0100 Message-ID: <20251125162851.2301793-6-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> References: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" To actually use requests with the m2m device, requests need to be allocated on the underlying media device. This can only be done if the media device is opened which means acquiring it. Add a function to check if the m2m device supports requests by acquiring the media device, asking it and then releasing it again. Also add a function to allocate requests that acquires the internal media device and releases it after allocating the requests. Signed-off-by: Stefan Klug Reviewed-by: Kieran Bingham --- ToDo: - Fix the place where the MediaObject is acquired/released Changes in v2: - Improved commit message - Dropped the change from private to protected as that belongs to a different patch --- .../internal/converter/converter_v4l2_m2m.h | 7 ++++ .../converter/converter_v4l2_m2m.cpp | 34 ++++++++++++++++++- 2 files changed, 40 insertions(+), 1 deletion(-) diff --git a/include/libcamera/internal/converter/converter_v4l2_m2m.h b/include/libcamera/internal/converter/converter_v4l2_m2m.h index 1b2a88c4a608..bc85bff7a07b 100644 --- a/include/libcamera/internal/converter/converter_v4l2_m2m.h +++ b/include/libcamera/internal/converter/converter_v4l2_m2m.h @@ -73,6 +73,11 @@ public: std::pair inputCropBounds() override { return inputCropBounds_; } std::pair inputCropBounds(const Stream *stream) override; + int allocateRequests(unsigned int count, + std::vector> *requests); + + bool supportsRequests(); + private: class V4L2M2MStream : protected Loggable { @@ -122,6 +127,8 @@ private: std::map> streams_; std::map queue_; std::pair inputCropBounds_; + + std::shared_ptr media_; }; } /* namespace libcamera */ diff --git a/src/libcamera/converter/converter_v4l2_m2m.cpp b/src/libcamera/converter/converter_v4l2_m2m.cpp index ff11a9735db7..c6153d728c9a 100644 --- a/src/libcamera/converter/converter_v4l2_m2m.cpp +++ b/src/libcamera/converter/converter_v4l2_m2m.cpp @@ -9,6 +9,7 @@ #include "libcamera/internal/converter/converter_v4l2_m2m.h" #include +#include #include #include #include @@ -266,7 +267,7 @@ void V4L2M2MConverter::V4L2M2MStream::captureBufferReady(FrameBuffer *buffer) * \param[in] media The media device implementing the converter */ V4L2M2MConverter::V4L2M2MConverter(std::shared_ptr media) - : Converter(media) + : Converter(media), media_(media) { if (deviceNode().empty()) return; @@ -742,6 +743,37 @@ int V4L2M2MConverter::queueBuffers(FrameBuffer *input, return 0; } +/** + * \copydoc libcamera::MediaDevice::allocateRequests + */ +int V4L2M2MConverter::allocateRequests(unsigned int count, + std::vector> *requests) +{ + /* \todo The acquire() must be moved to the right place. */ + media_->acquire(); + if (!media_->busy()) + LOG(Converter, Error) + << "MediaDevice must be valid."; + int ret = media_->allocateRequests(count, requests); + media_->release(); + return ret; +} + +/** + * \copydoc libcamera::MediaDevice::supportsRequests + */ +bool V4L2M2MConverter::supportsRequests() +{ + /* \todo The acquire() must be moved to the right place. */ + media_->acquire(); + if (!media_->busy()) + LOG(Converter, Error) + << "MediaDevice must be valid."; + bool ret = media_->supportsRequests(); + media_->release(); + return ret; +} + /* * \todo This should be extended to include Feature::Flag to denote * what each converter supports feature-wise. From patchwork Tue Nov 25 16:28:18 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 25185 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3AB20C333C for ; Tue, 25 Nov 2025 16:29:15 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CE64560AAC; Tue, 25 Nov 2025 17:29:14 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Y1MwhneN"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8DC87609E0 for ; Tue, 25 Nov 2025 17:29:13 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bae1:340c:573c:570b]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id B36D41E4D; Tue, 25 Nov 2025 17:27:03 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1764088023; bh=KvkXT6rO8BXj40x8YoebfMZJGCSM/KBAKph2QcZzUz4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Y1MwhneNpXwf6owTwzPMGFTXHFYjeBuPlurxGzLnI/YONBVtrg46N54AFLf9VDmsi bCL75o+cbkNlqeVlSrvimGOO5GK8/dqYNxGOWq4IfzYvlikrQvI3NGa2SfGlS1HI2i e8wjklcNYAMM+ZM6UaXKtRHfNzAzG6W7sSTQPCB8= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Kieran Bingham , Isaac Scott , Umang Jain , Paul Elder Subject: [PATCH v3 06/29] libcamera: converter_v4l2_m2m: Always set stride Date: Tue, 25 Nov 2025 17:28:18 +0100 Message-ID: <20251125162851.2301793-7-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> References: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Ensure the stride is properly set after a call to validateOutput(). Signed-off-by: Stefan Klug Reviewed-by: Kieran Bingham Reviewed-by: Isaac Scott Reviewed-by: Umang Jain Reviewed-by: Paul Elder --- Changes in v3: - Collected tags Changes in v2: - Collected tag --- src/libcamera/converter/converter_v4l2_m2m.cpp | 1 + 1 file changed, 1 insertion(+) diff --git a/src/libcamera/converter/converter_v4l2_m2m.cpp b/src/libcamera/converter/converter_v4l2_m2m.cpp index c6153d728c9a..e57db8a438ab 100644 --- a/src/libcamera/converter/converter_v4l2_m2m.cpp +++ b/src/libcamera/converter/converter_v4l2_m2m.cpp @@ -680,6 +680,7 @@ int V4L2M2MConverter::validateOutput(StreamConfiguration *cfg, bool *adjusted, const Size cfgSize = cfg->size; cfg->size = adjustSizes(cfgSize, it->second, align); + cfg->stride = PixelFormatInfo::info(cfg->pixelFormat).stride(cfg->size.width, 0); if (cfg->size.isNull()) return -EINVAL; From patchwork Tue Nov 25 16:28:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 25186 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 45808C333C for ; Tue, 25 Nov 2025 16:29:18 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id DED8160AAE; Tue, 25 Nov 2025 17:29:17 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="iNiECHMZ"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 513F260AAE for ; Tue, 25 Nov 2025 17:29:15 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bae1:340c:573c:570b]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 511EFFDB; Tue, 25 Nov 2025 17:27:06 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1764088026; bh=aU5zk8ArWYFA+RK5tVzh9KXDENjW4DmhFkf9a2qozjk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=iNiECHMZhmdnDNUraa2sPvEwGS2ZHnA+St1tDKFUILOgTihKr7TZdGqWfaCH8xBmB 9eb9OSag5yAERejpysNSShsozYmcKqmMD55K8O7r7XHQb/J9fmN/M5pp2n/bV42I6h 9Th8lHD2Su5eBNIAnEqsM476z6QJEPj75NWuDUcU= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Paul Elder , Isaac Scott Subject: [PATCH v3 07/29] libcamera: rkisp1: Properly cancel buffers in dewarp case Date: Tue, 25 Nov 2025 17:28:19 +0100 Message-ID: <20251125162851.2301793-8-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> References: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In case the buffer returned from the ISP was canceled, the upstream buffer was not correctly marked as canceled. Move the cancel functionality into an own helper function and correctly cancel the upstream buffers. Add missing cancellation in case queuing to the dewarper fails. Signed-off-by: Stefan Klug Reviewed-by: Paul Elder Reviewed-by: Isaac Scott --- Changes in v3: - Collected tags - Adapted code a bit as the request handling is no longer part of rkisp1, but was moved to the dw100 implementation --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 51 +++++++++++++++--------- 1 file changed, 33 insertions(+), 18 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index dbd6bf9673ff..b56b306f1ad1 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -207,6 +207,7 @@ private: int initLinks(Camera *camera, const RkISP1CameraConfiguration &config); int createCamera(MediaEntity *sensor); void tryCompleteRequest(RkISP1FrameInfo *info); + void cancelDewarpRequest(RkISP1FrameInfo *info); void imageBufferReady(FrameBuffer *buffer); void paramBufferReady(FrameBuffer *buffer); void statBufferReady(FrameBuffer *buffer); @@ -1499,6 +1500,31 @@ void PipelineHandlerRkISP1::tryCompleteRequest(RkISP1FrameInfo *info) completeRequest(request); } +void PipelineHandlerRkISP1::cancelDewarpRequest(RkISP1FrameInfo *info) +{ + RkISP1CameraData *data = cameraData(activeCamera_); + Request *request = info->request; + /* + * i.MX8MP is the only known platform with dewarper. It has + * no self path. Hence, only main path buffer completion is + * required. + * + * Also, we cannot completeBuffer(request, buffer) as buffer + * here, is an internal buffer (between ISP and dewarper) and + * is not associated to the any specific request. The request + * buffer associated with main path stream is the one that + * is required to be completed (not the internal buffer). + */ + for (auto [stream, buffer] : request->buffers()) { + if (stream == &data->mainPathStream_) { + buffer->_d()->cancel(); + completeBuffer(request, buffer); + } + } + + tryCompleteRequest(info); +} + void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) { ASSERT(activeCamera_); @@ -1540,23 +1566,7 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) /* Do not queue cancelled frames to dewarper. */ if (metadata.status == FrameMetadata::FrameCancelled) { - /* - * i.MX8MP is the only known platform with dewarper. It has - * no self path. Hence, only main path buffer completion is - * required. - * - * Also, we cannot completeBuffer(request, buffer) as buffer - * here, is an internal buffer (between ISP and dewarper) and - * is not associated to the any specific request. The request - * buffer associated with main path stream is the one that - * is required to be completed (not the internal buffer). - */ - for (auto it : request->buffers()) { - if (it.first == &data->mainPathStream_) - completeBuffer(request, it.second); - } - - tryCompleteRequest(info); + cancelDewarpRequest(info); return; } @@ -1598,10 +1608,15 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) * by the application. */ int ret = dewarper_->queueBuffers(buffer, request->buffers()); - if (ret < 0) + if (ret < 0) { LOG(RkISP1, Error) << "Cannot queue buffers to dewarper: " << strerror(-ret); + cancelDewarpRequest(info); + + return; + } + request->metadata().set(controls::ScalerCrop, activeCrop_.value()); } From patchwork Tue Nov 25 16:28:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 25187 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3BC4EC333C for ; Tue, 25 Nov 2025 16:29:20 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E1FAB60AB1; Tue, 25 Nov 2025 17:29:19 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="cIFhWJhi"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EAB3960AAF for ; Tue, 25 Nov 2025 17:29:17 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bae1:340c:573c:570b]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 0CBD66AF; Tue, 25 Nov 2025 17:27:09 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1764088029; bh=3utOiDjUycBRYhjvpT74wbXMeu2BGNuKY7W0F5eotXM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cIFhWJhihrWsTc4VRIAFIEexj7TDVCQHoMUHu0h4/LgmnzW3iRMBEKgxR6xFoJhpk Ynqi2ynfFSBJoQVhzcX/QzsaEjceq4BgPtrfcghXmNPrxu61DA5ypJdz7WOSNSuv5M G4t1oeAhPAiEP2gZpmyFzL5fjDN89MYx6gJ+s3h0= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Xavier Roumegue , Umang Jain , Jacopo Mondi , Paul Elder Subject: [PATCH v3 08/29] include: linux: Update headers for Dw100 dewarper engine Date: Tue, 25 Nov 2025 17:28:20 +0100 Message-ID: <20251125162851.2301793-9-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> References: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Xavier Roumegue Include the DW100 dewarper header as part of linux headers. It will be used to set the V4L2_CID_DW100_DEWARPING_16x16_VERTEX_MAP control by ConverterDW100 Signed-off-by: Xavier Roumegue Signed-off-by: Umang Jain Acked-by: Jacopo Mondi Acked-by: Paul Elder --- Changes in v3: - Collected tags --- include/linux/dw100.h | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100644 include/linux/dw100.h diff --git a/include/linux/dw100.h b/include/linux/dw100.h new file mode 100644 index 000000000000..3356496edd6b --- /dev/null +++ b/include/linux/dw100.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0-only WITH Linux-syscall-note */ +/* Copyright 2022 NXP */ + +#ifndef __UAPI_DW100_H__ +#define __UAPI_DW100_H__ + +#include + +/* + * Check Documentation/userspace-api/media/drivers/dw100.rst for control details. + */ +#define V4L2_CID_DW100_DEWARPING_16x16_VERTEX_MAP (V4L2_CID_USER_DW100_BASE + 1) + +#endif From patchwork Tue Nov 25 16:28:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 25188 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2380EC333C for ; Tue, 25 Nov 2025 16:29:23 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C7AB660AB1; Tue, 25 Nov 2025 17:29:22 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="oTcV64Eh"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5348960A85 for ; Tue, 25 Nov 2025 17:29:21 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bae1:340c:573c:570b]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 5571B1E2A; Tue, 25 Nov 2025 17:27:12 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1764088032; bh=klZ9dbavnx5ai/wUySH711DwmVvRN/fwBS7jblr1XFo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=oTcV64Ehhu4F60PZSjei/2+2cMRzUcUaOjLXIZQHP6cu1atyibikmt4xRHWY8/+zC J7OCZ1Aq/ucn7gmEiLecT12ulZL7Ugay6UMYei2JILbORJW7QiuwVb+Hc3aUZPrOF1 O2hZ/UZcbNcWsLQoGKNXYL6mCVGT1zYz5uw9SAVU= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Umang Jain , Isaac Scott , Paul Elder Subject: [PATCH v3 09/29] libcamera: converter_v4l2_m2m: Add helper to apply controls Date: Tue, 25 Nov 2025 17:28:21 +0100 Message-ID: <20251125162851.2301793-10-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> References: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Umang Jain Add applyControls() helper to apply controls for a stream. Signed-off-by: Umang Jain Signed-off-by: Stefan Klug Reviewed-by: Isaac Scott Reviewed-by: Paul Elder --- Changes in v3: - Collected tags Changes in v2: - Removed stray semicolon - Added documentation Changes in v0.9 - Include request support in applyControls --- .../internal/converter/converter_v4l2_m2m.h | 5 +++ .../converter/converter_v4l2_m2m.cpp | 34 +++++++++++++++++++ 2 files changed, 39 insertions(+) diff --git a/include/libcamera/internal/converter/converter_v4l2_m2m.h b/include/libcamera/internal/converter/converter_v4l2_m2m.h index bc85bff7a07b..6d2453bb9e06 100644 --- a/include/libcamera/internal/converter/converter_v4l2_m2m.h +++ b/include/libcamera/internal/converter/converter_v4l2_m2m.h @@ -24,6 +24,7 @@ namespace libcamera { +class ControlList; class FrameBuffer; class MediaDevice; class Size; @@ -73,6 +74,8 @@ public: std::pair inputCropBounds() override { return inputCropBounds_; } std::pair inputCropBounds(const Stream *stream) override; + int applyControls(const Stream *stream, ControlList &ctrls, const V4L2Request *request = nullptr); + int allocateRequests(unsigned int count, std::vector> *requests); @@ -94,6 +97,8 @@ private: int start(); void stop(); + int applyControls(ControlList &ctrls, const V4L2Request *request = nullptr); + int queueBuffers(FrameBuffer *input, FrameBuffer *output, const V4L2Request *request = nullptr); diff --git a/src/libcamera/converter/converter_v4l2_m2m.cpp b/src/libcamera/converter/converter_v4l2_m2m.cpp index e57db8a438ab..48e49f143cf1 100644 --- a/src/libcamera/converter/converter_v4l2_m2m.cpp +++ b/src/libcamera/converter/converter_v4l2_m2m.cpp @@ -18,6 +18,7 @@ #include #include +#include #include #include #include @@ -251,6 +252,20 @@ void V4L2M2MConverter::V4L2M2MStream::captureBufferReady(FrameBuffer *buffer) converter_->outputBufferReady.emit(buffer); } +/** + * \brief Apply controls + * \param[in] ctrls The controls to apply + * \param[in] request An optional request + * + * \return 0 on success or a negative error code otherwise + * \see V4L2Device::setControls() + */ +int V4L2M2MConverter::V4L2M2MStream::applyControls(ControlList &ctrls, + const V4L2Request *request) +{ + return m2m_->capture()->setControls(&ctrls, request); +} + /* ----------------------------------------------------------------------------- * V4L2M2MConverter */ @@ -744,6 +759,25 @@ int V4L2M2MConverter::queueBuffers(FrameBuffer *input, return 0; } +/** + * \brief Apply controls + * \param[in] stream The stream on which to apply the controls + * \param[in] ctrls The controls to apply + * \param[in] request An optional request + * + * \return 0 on success or a negative error code otherwise + * \see V4L2Device::setControls() + */ +int V4L2M2MConverter::applyControls(const Stream *stream, ControlList &ctrls, + const V4L2Request *request) +{ + auto iter = streams_.find(stream); + if (iter == streams_.end()) + return -EINVAL; + + return iter->second->applyControls(ctrls, request); +} + /** * \copydoc libcamera::MediaDevice::allocateRequests */ From patchwork Tue Nov 25 16:28:22 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 25189 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B87A1C333C for ; Tue, 25 Nov 2025 16:29:25 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6863260AA9; Tue, 25 Nov 2025 17:29:25 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="HpMmBx7q"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3485660A86 for ; Tue, 25 Nov 2025 17:29:24 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bae1:340c:573c:570b]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 3DCA22106; Tue, 25 Nov 2025 17:27:15 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1764088035; bh=3a1hYayUKIPR784V6RG9gGQmR5U4Pd2Ul/cvHZ6nPuM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HpMmBx7qX2pFqLKyTsfgCvi8YAKvQiR+6Lv6SaO9+bRkFn/GC1KSL5KxkBmtk2VNt Eg8UQalzPC3CT9jeao0GyChXAXG1S9GWrOTf0t0muExDItsysEeeDTRWwI77wZye8P 2wI70RX91+V5zXHcPaLM+XXidSg1FMkKfkKBzPEY= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder Subject: [PATCH v3 10/29] libcamera: converter_v4l2_m2m: Add debug logging for formats Date: Tue, 25 Nov 2025 17:28:22 +0100 Message-ID: <20251125162851.2301793-11-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> References: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Getting the requested input/output format is crucial for debugging. Add log statements for that. Signed-off-by: Stefan Klug Reviewed-by: Kieran Bingham Reviewed-by: Barnabás PÅ‘cze Reviewed-by: Paul Elder --- Changes in v3: - Collected tags Changes in v2: - Collected tag --- src/libcamera/converter/converter_v4l2_m2m.cpp | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/libcamera/converter/converter_v4l2_m2m.cpp b/src/libcamera/converter/converter_v4l2_m2m.cpp index 48e49f143cf1..37377ab24952 100644 --- a/src/libcamera/converter/converter_v4l2_m2m.cpp +++ b/src/libcamera/converter/converter_v4l2_m2m.cpp @@ -111,6 +111,7 @@ int V4L2M2MConverter::V4L2M2MStream::configure(const StreamConfiguration &inputC format.planesCount = 1; format.planes[0].bpl = inputCfg.stride; + LOG(Converter, Debug) << "Set input format to: " << format; int ret = m2m_->output()->setFormat(&format); if (ret < 0) { LOG(Converter, Error) @@ -133,6 +134,7 @@ int V4L2M2MConverter::V4L2M2MStream::configure(const StreamConfiguration &inputC format.fourcc = videoFormat; format.size = outputCfg.size; + LOG(Converter, Debug) << "Set output format to: " << format; ret = m2m_->capture()->setFormat(&format); if (ret < 0) { LOG(Converter, Error) From patchwork Tue Nov 25 16:28:23 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 25190 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 49498C333C for ; Tue, 25 Nov 2025 16:29:29 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0631A60AA9; Tue, 25 Nov 2025 17:29:29 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="VVw1fHUd"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2EA7D60A86 for ; Tue, 25 Nov 2025 17:29:27 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bae1:340c:573c:570b]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 3C8E61E4D; Tue, 25 Nov 2025 17:27:18 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1764088038; bh=iFDSPjubyYBD2SEB3gWp3Hvtpge3dVnIDQbTk0mhPyI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VVw1fHUdqHLVMMEkNLNdHDHZnsHAhN7GevCH+D6iSN0Ubc8xYpKeym24ZrZFj182X pmQrJwY0x/GWJgHnUGmPG6F/l0MFpWepcbWwhAmZTqCmZbtZc9s/ljEyho5MUkeFfo mjz8pR52yNS8xgdDlrjN+p5r/GvaIkmy4DfOBUN4= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Paul Elder Subject: [PATCH v3 11/29] libcamera: rkisp1: Move useDewarper_ flag into RkISP1CameraData Date: Tue, 25 Nov 2025 17:28:23 +0100 Message-ID: <20251125162851.2301793-12-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> References: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The decision if the dewarper shall be used is not per pipeline but per camera and per configuration (raw streams can't use it). Move the corresponding flag into the camera data class. Rename the flag to "usesDewarper" which is easier understand when we later add the ability to enable/disable the dewarper on a per camera basis which will be expressed using a "canUseDewarper" flag. Signed-off-by: Stefan Klug Reviewed-by: Paul Elder Reviewed-by: Kieran Bingham --- Changes in v3: - Collected tags --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 28 ++++++++++++------------ 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index b56b306f1ad1..a3b92804c69f 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -122,6 +122,8 @@ public: */ MediaPipeline pipe_; + bool usesDewarper_; + private: void paramsComputed(unsigned int frame, unsigned int bytesused); void setSensorControls(unsigned int frame, @@ -232,7 +234,6 @@ private: std::unique_ptr dewarper_; Rectangle scalerMaxCrop_; - bool useDewarper_; std::optional activeCrop_; @@ -280,7 +281,7 @@ RkISP1FrameInfo *RkISP1Frames::create(const RkISP1CameraData *data, Request *req statBuffer = pipe_->availableStatBuffers_.front(); pipe_->availableStatBuffers_.pop(); - if (pipe_->useDewarper_) { + if (data->usesDewarper_) { mainPathBuffer = pipe_->availableMainPathBuffers_.front(); pipe_->availableMainPathBuffers_.pop(); } @@ -710,8 +711,7 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() */ PipelineHandlerRkISP1::PipelineHandlerRkISP1(CameraManager *manager) - : PipelineHandler(manager, kRkISP1MaxQueuedRequests), - hasSelfPath_(true), useDewarper_(false) + : PipelineHandler(manager, kRkISP1MaxQueuedRequests), hasSelfPath_(true) { } @@ -874,7 +874,7 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) const PixelFormat &streamFormat = config->at(0).pixelFormat; const PixelFormatInfo &info = PixelFormatInfo::info(streamFormat); isRaw_ = info.colourEncoding == PixelFormatInfo::ColourEncodingRAW; - useDewarper_ = dewarper_ && !isRaw_; + data->usesDewarper_ = dewarper_ && !isRaw_; /* YUYV8_2X8 is required on the ISP source path pad for YUV output. */ if (!isRaw_) @@ -888,7 +888,7 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) /* imx8mp has only a single path. */ const auto &cfg = config->at(0); Size ispCrop = format.size.boundedToAspectRatio(cfg.size); - if (useDewarper_) + if (data->usesDewarper_) ispCrop = dewarper_->adjustInputSize(cfg.pixelFormat, ispCrop); else @@ -929,7 +929,7 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) streamConfig[0] = IPAStream(cfg.pixelFormat, cfg.size); /* Configure dewarp */ - if (dewarper_ && !isRaw_) { + if (data->usesDewarper_) { outputCfgs.push_back(const_cast(cfg)); ret = dewarper_->configure(cfg, outputCfgs); if (ret) @@ -994,7 +994,7 @@ int PipelineHandlerRkISP1::exportFrameBuffers([[maybe_unused]] Camera *camera, S * It has mainpath and no self path. Hence, export buffers from * dewarper just for the main path stream, for now. */ - if (useDewarper_) + if (data->usesDewarper_) return dewarper_->exportBuffers(&data->mainPathStream_, count, buffers); else return mainPath_.exportBuffers(count, buffers); @@ -1028,7 +1028,7 @@ int PipelineHandlerRkISP1::allocateBuffers(Camera *camera) } /* If the dewarper is being used, allocate internal buffers for ISP. */ - if (useDewarper_) { + if (data->usesDewarper_) { ret = mainPath_.exportBuffers(kRkISP1MinBufferCount, &mainPathBuffers_); if (ret < 0) return ret; @@ -1131,7 +1131,7 @@ int PipelineHandlerRkISP1::start(Camera *camera, [[maybe_unused]] const ControlL } actions += [&]() { stat_->streamOff(); }; - if (useDewarper_) { + if (data->usesDewarper_) { ret = dewarper_->start(); if (ret) { LOG(RkISP1, Error) << "Failed to start dewarper"; @@ -1186,7 +1186,7 @@ void PipelineHandlerRkISP1::stopDevice(Camera *camera) LOG(RkISP1, Warning) << "Failed to stop parameters for " << camera->id(); - if (useDewarper_) + if (data->usesDewarper_) dewarper_->stop(); } @@ -1282,7 +1282,7 @@ int PipelineHandlerRkISP1::updateControls(RkISP1CameraData *data) { ControlInfoMap::Map controls; - if (dewarper_) { + if (data->usesDewarper_) { std::pair cropLimits; if (dewarper_->isConfigured(&data->mainPathStream_)) cropLimits = dewarper_->inputCropBounds(&data->mainPathStream_); @@ -1452,7 +1452,7 @@ bool PipelineHandlerRkISP1::match(DeviceEnumerator *enumerator) this, &PipelineHandlerRkISP1::dewarpBufferReady); LOG(RkISP1, Info) - << "Using DW100 dewarper " << dewarper_->deviceNode(); + << "Found DW100 dewarper " << dewarper_->deviceNode(); } else { LOG(RkISP1, Warning) << "Found DW100 dewarper " << dewarper_->deviceNode() @@ -1557,7 +1557,7 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) info->metadataProcessed = true; } - if (!useDewarper_) { + if (!data->usesDewarper_) { completeBuffer(request, buffer); tryCompleteRequest(info); From patchwork Tue Nov 25 16:28:24 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 25191 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7CFF4C333C for ; Tue, 25 Nov 2025 16:29:32 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 26D4C60AB7; Tue, 25 Nov 2025 17:29:32 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="fe+vPOq+"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 11F7760A9D for ; Tue, 25 Nov 2025 17:29:30 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bae1:340c:573c:570b]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 18578FDB; Tue, 25 Nov 2025 17:27:21 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1764088041; bh=2wH5ApziXpLBKYGLtiTpn+3C8UB8yEnJIX+9DqsXAW4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fe+vPOq+kIDtsa0LN7u3LPXo6zFCpP1B4gdGcpJnR3ZsqFkbu3D3WHYCveIPR3IGE frZglSyfl8kjFp70/uWqi5blihqazDbcMJ0CWlXJVGkdI4//zOlT6U/ojj51I6uWss jbnAqMruThJp+DfMoFB7D0myWldZUrQ5IfJFEEV8= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Paul Elder Subject: [PATCH v3 12/29] libcamera: rkisp1: Scale down in dewarper instead of resizer Date: Tue, 25 Nov 2025 17:28:24 +0100 Message-ID: <20251125162851.2301793-13-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> References: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In order to allow digital zooming, scale down in the dewarper instead of the resizer. That means forwarding the full sensor size data to the dewarper. The ScalerCrop rectangle will also be applied at the dewarper. Signed-off-by: Stefan Klug Reviewed-by: Paul Elder Reviewed-by: Kieran Bingham --- Changes in v3: - Collected tag --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 43 +++++++++++++++++++----- 1 file changed, 35 insertions(+), 8 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index a3b92804c69f..2d89b0b41958 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -883,16 +883,31 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) /* * On devices without DUAL_CROP (like the imx8mp) cropping needs to be * done on the ISP/IS output. + * + * If the dewarper is used, the cropping shall be done by the dewarper. */ if (media_->hwRevision() == RKISP1_V_IMX8MP) { /* imx8mp has only a single path. */ const auto &cfg = config->at(0); - Size ispCrop = format.size.boundedToAspectRatio(cfg.size); + /* + * If the dewarper is used, all cropping including aspect ratio + * preservation shall be done there. To ensure that the output + * format provided by the ISP is supported by the dewarper, a + * minimal crop still needs to be applied on the ISP output. + * + * \todo It might be possible to allocate bigger buffers + * (aligned to 8 pixels) with a stride matching format.size for + * the ISP. The not-filled border could later be ignored by the + * dewarper. This way we could skip the minimal crop here and + * the MaximumScalerCrop would always match the isp output. + */ + Size ispCrop; if (data->usesDewarper_) ispCrop = dewarper_->adjustInputSize(cfg.pixelFormat, - ispCrop); + format.size); else - ispCrop.alignUpTo(2, 2); + ispCrop = format.size.boundedToAspectRatio(cfg.size) + .alignedUpTo(2, 2); outputCrop = ispCrop.centeredTo(Rectangle(format.size).center()); format.size = ispCrop; @@ -925,13 +940,21 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) for (const StreamConfiguration &cfg : *config) { if (cfg.stream() == &data->mainPathStream_) { - ret = mainPath_.configure(cfg, format); - streamConfig[0] = IPAStream(cfg.pixelFormat, - cfg.size); - /* Configure dewarp */ + /* + * To allow for digital zoom, scaling down should happen + * in the dewarper, instead of the resizer. Configure + * the isp output to the same size as the sensor output. + */ + StreamConfiguration ispCfg = cfg; if (data->usesDewarper_) { outputCfgs.push_back(const_cast(cfg)); - ret = dewarper_->configure(cfg, outputCfgs); + + ispCfg.size = format.size; + ispCfg.stride = + PixelFormatInfo::info(ispCfg.pixelFormat) + .stride(ispCfg.size.width, 0); + + ret = dewarper_->configure(ispCfg, outputCfgs); if (ret) return ret; @@ -944,6 +967,10 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) outputCrop.transformedBetween(inputCrop, sensorInfo.analogCrop); } + + ret = mainPath_.configure(ispCfg, format); + streamConfig[0] = IPAStream(cfg.pixelFormat, + cfg.size); } else if (hasSelfPath_) { ret = selfPath_.configure(cfg, format); streamConfig[1] = IPAStream(cfg.pixelFormat, From patchwork Tue Nov 25 16:28:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 25192 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B5BF1C333C for ; Tue, 25 Nov 2025 16:29:36 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 51C6B60ABB; Tue, 25 Nov 2025 17:29:36 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="QCrpfaAs"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5D2D760A9D for ; Tue, 25 Nov 2025 17:29:33 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bae1:340c:573c:570b]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 70E17524; Tue, 25 Nov 2025 17:27:24 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1764088044; bh=54seuIsjJEBIcBROwnfmlkpCovzEqz1s9O3NMQZYXm8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QCrpfaAs1eOI2rYYs8Wet4FjuK5qtBsoMpnGkh0WsWXQZe+9QIlKzIVxflObrXnvx 0xcDohiPWEnDf9Owt37MSeVHXL/KcNf5hXAEIVcwNxyNaOOq1IHKG9cR0rmvBdFKQX YqIwwBu67ngQnFq9jLYCJh3pKiETqDWFOoLQ13M0= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Paul Elder Subject: [PATCH v3 13/29] libcamera: rkisp1: Allow upscaling when the dewarper is present Date: Tue, 25 Nov 2025 17:28:25 +0100 Message-ID: <20251125162851.2301793-14-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> References: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When the dewarper is present, there is no need to forbid upscaling of the image data. Signed-off-by: Stefan Klug Reviewed-by: Paul Elder Reviewed-by: Kieran Bingham --- Changes in v3: - Collected tag --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 2d89b0b41958..79c8717c839a 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -609,6 +609,11 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() return false; if (useDewarper) { + /* + * The dewarper output is independent of the ISP path. + * Reset to the originally requested size. + */ + tryCfg.size = cfg.size; bool adjusted; pipe->dewarper_->validateOutput(&tryCfg, &adjusted, From patchwork Tue Nov 25 16:28:26 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 25193 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E8B8CC333C for ; Tue, 25 Nov 2025 16:29:38 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 928EE60AB4; Tue, 25 Nov 2025 17:29:38 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="DsKPn+nv"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 94AF360AC4 for ; Tue, 25 Nov 2025 17:29:36 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bae1:340c:573c:570b]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 9F7D86AF; Tue, 25 Nov 2025 17:27:27 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1764088047; bh=h3miCZmgaN6PDBGl9AKd+RDzNgW3R0M2XkJkpFniOyU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DsKPn+nvZcEDXxlDGKwe7sbtZZTAd1M/6lbirqy9gNNjqOhAeAh++PPM9rO/tUlc9 sDW+sE2gzoO8LXMz4+bFT/Wrz95nKg13hw0W26RRYwmhRCvl5dkgYzA5k+BA4yjknk SVy9NzsPK+P+yQMrJM377HM0mFRdr7zw9K+n2ha4= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v3 14/29] pipeline: rkisp1: Fix number of buffers imported into the dewarper Date: Tue, 25 Nov 2025 17:28:26 +0100 Message-ID: <20251125162851.2301793-15-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> References: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When the dewarper is used, an addition buffer loop with kRkISP1MinBufferCount buffers is created between ISP and dewarper. When the dewarper is configured, it stores the bufferCount value of the requested stream configurations. This number of buffers is then imported when the dewarper is started. On the input stream of the dewarper the bufferCount is currently left unchanged, meaning it carries the bufferCount as supplied by the user instead of the bufferCount of the additional loop. Fix that by setting the bufferCount to kRkISP1MinBufferCount. Signed-off-by: Stefan Klug Reviewed-by: Kieran Bingham --- Changes in v3: - Added this patch --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 1 + 1 file changed, 1 insertion(+) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 79c8717c839a..138e1d5bf06b 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -954,6 +954,7 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) if (data->usesDewarper_) { outputCfgs.push_back(const_cast(cfg)); + ispCfg.bufferCount = kRkISP1MinBufferCount; ispCfg.size = format.size; ispCfg.stride = PixelFormatInfo::info(ispCfg.pixelFormat) From patchwork Tue Nov 25 16:28:27 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 25194 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8F0EBC333C for ; Tue, 25 Nov 2025 16:29:42 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3C49B60AA0; Tue, 25 Nov 2025 17:29:42 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="HhIqmzCN"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B060260A9E for ; Tue, 25 Nov 2025 17:29:39 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bae1:340c:573c:570b]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 936086AF; Tue, 25 Nov 2025 17:27:30 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1764088050; bh=srkXm2mnlY6xWBjtN23qdQXjrVgugwEL9ETORL812Ko=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HhIqmzCNX9E05cWXNHmBcmS8IjHNI5W0aZDYGw+rnRtSkG1oeRLgE9ZajlraAOVeK KFJjeRhxEbsqdMXqsqVUTtJx37r6WyiSEkSRzzvOWP56B0y+6Hca1zaGd9q47HvqCE cuMz41Q+7JyvFgBmZqT/maXAwgJ2UDiTCSXfZixg= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v3 15/29] libcamera: converter: Add dw100 vertex map class Date: Tue, 25 Nov 2025 17:28:27 +0100 Message-ID: <20251125162851.2301793-16-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> References: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Using a custom vertex map the dw100 dewarper is capable of doing complex and useful transformations on the image data. This class implements a pipeline featuring: - Arbitrary ScalerCrop - Full transform support (Flip, 90deg rotations) - Arbitrary move, scale, rotate ScalerCrop and Transform is implemented to provide a interface that is standardized libcamera wide. The rest is implemented on top for more flexible dw100 specific features. Signed-off-by: Stefan Klug Reviewed-by: Kieran Bingham --- Changes in v3: - Dropped scalerCropBounds() function as it is not needed anymore - Modified applyLimits() to always run until the end - Small changes in docs Changes in v2: - Replaced manual transforms with an affine transformation matrix - Changed rotation direction to be in sync with the rotation in CameraConfiguration::orientation - Changed offset parameter to be in ScalerCrop coordinates. This is easier to explain and has the added benefit, that Scale/Rotate is always centered to the visible image. - Improved code comments - Make dw100VerticesForLength a local function - Dropped unnecessary includes - Added documentation Changes in v0.9 - Include header in meson.build - Fix black line at top and left when rotation 180 degrees Changes in v0.8 - Cleanup & formatting Changes in v0.5 - Fix crash in std::clamp() due to rounding errors --- .../converter/converter_dw100_vertexmap.h | 71 +++ .../libcamera/internal/converter/meson.build | 1 + .../converter/converter_dw100_vertexmap.cpp | 564 ++++++++++++++++++ src/libcamera/converter/meson.build | 1 + 4 files changed, 637 insertions(+) create mode 100644 include/libcamera/internal/converter/converter_dw100_vertexmap.h create mode 100644 src/libcamera/converter/converter_dw100_vertexmap.cpp diff --git a/include/libcamera/internal/converter/converter_dw100_vertexmap.h b/include/libcamera/internal/converter/converter_dw100_vertexmap.h new file mode 100644 index 000000000000..428b3d74d4d2 --- /dev/null +++ b/include/libcamera/internal/converter/converter_dw100_vertexmap.h @@ -0,0 +1,71 @@ +#pragma once + +#include +#include +#include +#include + +#include + +#include +#include + +namespace libcamera { + +class Dw100VertexMap +{ +public: + enum ScaleMode { + Fill = 0, + Crop = 1, + }; + + void applyLimits(); + void setInputSize(const Size &size) + { + inputSize_ = size; + scalerCrop_ = Rectangle(size); + } + + void setSensorCrop(const Rectangle &rect) { sensorCrop_ = rect; } + + void setScalerCrop(const Rectangle &rect) { scalerCrop_ = rect; } + const Rectangle &effectiveScalerCrop() const { return effectiveScalerCrop_; } + + void setOutputSize(const Size &size) { outputSize_ = size; } + const Size &outputSize() const { return outputSize_; } + + void setTransform(const Transform &transform) { transform_ = transform; } + const Transform &transform() const { return transform_; } + + void setScale(const float scale) { scale_ = scale; } + float effectiveScale() const { return (effectiveScaleX_ + effectiveScaleY_) * 0.5; } + + void setRotation(const float rotation) { rotation_ = rotation; } + float rotation() const { return rotation_; } + + void setOffset(const Point &offset) { offset_ = offset; } + const Point &effectiveOffset() const { return effectiveOffset_; } + + void setMode(const ScaleMode mode) { mode_ = mode; } + ScaleMode mode() const { return mode_; } + + std::vector getVertexMap(); + +private: + Rectangle scalerCrop_; + Rectangle sensorCrop_; + Transform transform_ = Transform::Identity; + Size inputSize_; + Size outputSize_; + Point offset_; + double scale_ = 1.0; + double rotation_ = 0.0; + ScaleMode mode_ = Fill; + double effectiveScaleX_; + double effectiveScaleY_; + Point effectiveOffset_; + Rectangle effectiveScalerCrop_; +}; + +} /* namespace libcamera */ diff --git a/include/libcamera/internal/converter/meson.build b/include/libcamera/internal/converter/meson.build index 891e79e7d493..9d586293f63a 100644 --- a/include/libcamera/internal/converter/meson.build +++ b/include/libcamera/internal/converter/meson.build @@ -1,5 +1,6 @@ # SPDX-License-Identifier: CC0-1.0 libcamera_internal_headers += files([ + 'converter_dw100_vertexmap.h', 'converter_v4l2_m2m.h', ]) diff --git a/src/libcamera/converter/converter_dw100_vertexmap.cpp b/src/libcamera/converter/converter_dw100_vertexmap.cpp new file mode 100644 index 000000000000..427d710743b8 --- /dev/null +++ b/src/libcamera/converter/converter_dw100_vertexmap.cpp @@ -0,0 +1,564 @@ +#include "libcamera/internal/converter/converter_dw100_vertexmap.h" + +#include +#include +#include +#include +#include +#include + +#include +#include + +#include +#include + +#include "libcamera/internal/vector.h" + +constexpr int kDw100BlockSize = 16; + +namespace libcamera { + +LOG_DECLARE_CATEGORY(Converter) +namespace { + +using Vector2d = Vector; +using Vector3d = Vector; +using Matrix3x3 = Matrix; + +Matrix3x3 makeTranslate(const double tx, const double ty) +{ + Matrix3x3 m = Matrix3x3::identity(); + m[0][2] = tx; + m[1][2] = ty; + return m; +} + +Matrix3x3 makeTranslate(const Vector2d &t) +{ + return makeTranslate(t.x(), t.y()); +} + +Matrix3x3 makeRotate(const double degrees) +{ + double rad = degrees / 180.0 * M_PI; + double sa = std::sin(rad); + double ca = std::cos(rad); + + Matrix3x3 m = Matrix3x3::identity(); + m[0][0] = ca; + m[0][1] = -sa; + m[1][0] = sa; + m[1][1] = ca; + return m; +} + +Matrix3x3 makeScale(const double sx, const double sy) +{ + Matrix3x3 m = Matrix3x3::identity(); + m[0][0] = sx; + m[1][1] = sy; + return m; +} + +/** + * \param t The transform to apply + * \param size The size of the rectangle that is transformed + * + * Create a matrix that represents the transform done by the \a t. It assumes + * that the origin of the coordinate system is at the top left corner of of the + * rectangle. + */ +Matrix3x3 makeTransform(const Transform &t, const Size &size) +{ + Matrix3x3 m = Matrix3x3::identity(); + double wm = size.width * 0.5; + double hm = size.height * 0.5; + m = makeTranslate(-wm, -hm) * m; + + if (!!(t & Transform::HFlip)) + m = makeScale(-1, 1) * m; + + if (!!(t & Transform::VFlip)) + m = makeScale(1, -1) * m; + + if (!!(t & Transform::Transpose)) { + m = makeRotate(-90) * m; + m = makeScale(1, -1) * m; + std::swap(wm, hm); + } + + m = makeTranslate(wm, hm) * m; + + return m; +} + +/** + * \param from The source rectangle + * \param to The destination rectangle + * + * Create a matrix that transforms from the coordinate system of rectangle \a + * from into the coordinate system of rectangle \a to, by overlaying the + * rectangles. + * + * \see Rectangle::transformedBetween() + */ +Matrix3x3 makeTransform(const Rectangle &from, const Rectangle &to) +{ + Matrix3x3 m = Matrix3x3::identity(); + double sx = to.width / static_cast(from.width); + double sy = to.height / static_cast(from.height); + m = makeTranslate(-from.x, -from.y) * m; + m = makeScale(sx, sy) * m; + m = makeTranslate(to.x, to.y) * m; + return m; +} + +Vector2d transformPoint(const Matrix3x3 &m, const Vector2d &p) +{ + Vector3d p2{ { p.x(), p.y(), 1.0 } }; + p2 = m * p2; + return { { p2.x() / p2.z(), p2.y() / p2.z() } }; +} + +Vector2d transformVector(const Matrix3x3 &m, const Vector2d &p) +{ + Vector3d p2{ { p.x(), p.y(), 0.0 } }; + p2 = m * p2; + return { { p2.x(), p2.y() } }; +} + +Vector2d rotatedRectSize(const Vector2d &size, const double degrees) +{ + double rad = degrees / 180.0 * M_PI; + double sa = sin(rad); + double ca = cos(rad); + + return { { std::abs(size.x() * ca) + std::abs(size.y() * sa), + std::abs(size.x() * sa) + std::abs(size.y() * ca) } }; +} + +Vector2d point2Vec2d(const Point &p) +{ + return { { static_cast(p.x), static_cast(p.y) } }; +} + +int dw100VerticesForLength(const int length) +{ + return (length + kDw100BlockSize - 1) / kDw100BlockSize + 1; +} + +} /* namespace */ + +/** + * \class libcamera::Dw100VertexMap + * \brief Helper class to compute dw100 vertex maps + * + * The vertex map class represents a helper for handling dewarper vertex maps. + * There are 3 important sizes in the system: + * + * - The sensor size. The number of pixels of the whole sensor. + * - The input rectangle to the dewarper. Describes the pixel data flowing into + * the dewarper in sensor coordinates. + * - ScalerCrop rectangle. The rectangle that shall be used for all further + * stages. It is applied after lens dewarping but is in sensor coordinate + * space. + * - The output size. This defines the size, the dewarper should output. + * + * +------------------------+ + * |Sensor size | + * | +----------------+ | + * | | Input rect | | + * | | +-------------+ | + * | | | ScalerCrop | | + * | | | | | + * | +--+-------------+ | + * +------------------------+ + * + * This class implements a vertex map that forms the following pipeline: + * + * +-------------+ +-------------+ +------------+ +-----------------+ + * | | | | | Transform | | Pan/Zoom | + * | Lens Dewarp | -> | Scaler Crop | -> | (H/V Flip, | -> | (Offset, Scale, | + * | | | | | Transpose) | | Rotate) | + * +-------------+ +-------------+ +------------+ +-----------------+ + * + * \todo Lens dewarp is not yet implemented. An identity map is used instead. + * + * All parameters are clamped to valid values before creating the vertex map. + * + * The constrains process works as follows: + * - The ScalerCrop rectangle is clamped to the input rectangle + * - The ScalerCrop rectangle is transformed by the specified transform + * forming ScalerCropT + * - A rectangle of output size is placed in the center of ScalerCropT + * (OutputRect). + * - Rotate gets applied to OutputRect, + * - Scale is applied, but clamped so that the OutputRect fits completely into + * ScalerCropT (Only regarding dimensions, not position) + * - Offset is clamped so that the OutputRect lies inside ScalerCropT + * + * After applying the limits, the actual values used for processing are stored + * effectiveXXX members and can be queried using the corresponding functions. + * + * The lens dewarp map is usually calibrated during tuning and is a map that + * maps from incoming pixels to dewarped pixels. + */ + +/** + * \enum Dw100VertexMap::ScaleMode + * \brief The scale modes available for a vertex map + * + * \var Dw100VertexMap::Fill + * \brief Scale the input to fill the output + * + * This scale mode does not preserve aspect ratio. Offset and rotation are taken + * into account. + * + * \var Dw100VertexMap::Crop + * \brief Crop the input + * + * This scale mode preserves the aspect ratio. Offset, scale, rotation are taken + * into account within the possible limits. + */ + +/** + * \brief Apply limits on scale and offset + * + * This function calculates \a effectiveScalerCrop_, \a effectiveScale_ and \a + * effectiveOffset_ based on the requested scaler crop, scale, rotation, offset + * and the selected scale mode, so that the whole output area is filled with + * valid input data. + */ +void Dw100VertexMap::applyLimits() +{ + int ow = outputSize_.width; + int oh = outputSize_.height; + effectiveScalerCrop_ = scalerCrop_.boundedTo(sensorCrop_); + + /* Map the scalerCrop to the input pixel space */ + Rectangle localScalerCrop = effectiveScalerCrop_.transformedBetween( + sensorCrop_, Rectangle(inputSize_)); + + Size localCropSizeT = localScalerCrop.size(); + if (!!(transform_ & Transform::Transpose)) + std::swap(localCropSizeT.width, localCropSizeT.height); + + Vector2d size = rotatedRectSize(point2Vec2d({ ow, oh }), rotation_); + + if (mode_ != Crop && mode_ != Fill) { + LOG(Converter, Error) + << "Unknown mode " << mode_ << ". Default to 'Fill'"; + mode_ = Fill; + } + + /* Calculate constraints */ + double scale = scale_; + if (mode_ == Crop) { + /* Scale up if needed */ + scale = std::max(scale, + std::max(size.x() / localCropSizeT.width, + size.y() / localCropSizeT.height)); + effectiveScaleX_ = scale; + effectiveScaleY_ = scale; + + size = size / scale; + + } else if (mode_ == Fill) { + effectiveScaleX_ = size.x() / localCropSizeT.width; + effectiveScaleY_ = size.y() / localCropSizeT.height; + + size.x() /= effectiveScaleX_; + size.y() /= effectiveScaleY_; + } + + /* + * Clamp offset. Due to rounding errors, size might be slightly bigger + * than scaler crop. Clamp the offset to 0 to prevent a crash in the + * next clamp. + */ + double maxoffX, maxoffY; + maxoffX = std::max(0.0, (localCropSizeT.width - size.x())) * 0.5; + maxoffY = std::max(0.0, (localCropSizeT.height - size.y())) * 0.5; + if (!!(transform_ & Transform::Transpose)) + std::swap(maxoffX, maxoffY); + + /* + * Transform the offset from sensor space to local space, apply the + * limit and transform back. + */ + Vector2d offset = point2Vec2d(offset_); + Matrix3x3 m; + + m = makeTransform(effectiveScalerCrop_, localScalerCrop); + offset = transformVector(m, offset); + offset.x() = std::clamp(offset.x(), -maxoffX, maxoffX); + offset.y() = std::clamp(offset.y(), -maxoffY, maxoffY); + m = makeTransform(localScalerCrop, effectiveScalerCrop_); + offset = transformVector(m, offset); + effectiveOffset_.x = offset.x(); + effectiveOffset_.y = offset.y(); +} + +/** + * \fn Dw100VertexMap::setInputSize() + * \brief Set the size of the input data + * \param[in] size The input size + * + * To calculate a proper vertex map, the size of the input images must be set. + */ + +/** + * \fn Dw100VertexMap::setSensorCrop() + * \brief Set the crop rectangle that represents the input data + * \param[in] rect + * + * Set the rectangle that represents the input data in sensor coordinates. This + * must be specified to properly calculate the vertex map. + */ + +/** + * \fn Dw100VertexMap::setScalerCrop() + * \brief Set the requested scaler crop + * \param[in] rect + * + * Set the requested scaler crop. The actually applied scaler crop can be + * queried using \a Dw100VertexMap::effectiveScalerCrop() after calling + * Dw100VertexMap::applyLimits(). + */ + +/** + * \fn Dw100VertexMap::effectiveScalerCrop() + * \brief Get the effective scaler crop + * + * \return The effective scaler crop + */ + +/** + * \fn Dw100VertexMap::setOutputSize() + * \brief Set the output size + * \param[in] size The size of the output images + */ + +/** + * \fn Dw100VertexMap::outputSize() + * \brief Get the output size + * \return The output size + */ + +/** + * \fn Dw100VertexMap::setTransform() + * \brief Sets the transform to apply + * \param[in] transform The transform + */ + +/** + * \fn Dw100VertexMap::transform() + * \brief Get the transform + * \return The transform + */ + +/** + * \fn Dw100VertexMap::setScale() + * \brief Sets the scale to apply + * \param[in] scale The scale + * + * Set the requested scale. The actually applied scale can be queried using \a + * Dw100VertexMap::effectiveScale() after calling \a + * Dw100VertexMap::applyLimits(). + */ + +/** + * \fn Dw100VertexMap::effectiveScale() + * \brief Get the effective scale + * + * Returns the actual scale applied to the input pixels in x and y direction. So + * a value of [2.0, 1.5] means that every input pixel is scaled to cover 2 + * output pixels in x-direction and 1.5 in y-direction. + * + * \return The effective scale + */ + +/** + * \fn Dw100VertexMap::setRotation() + * \brief Sets the rotation to apply + * \param[in] rotation The rotation in degrees + * + * The rotation is in clockwise direction to allow the same transform as + * CameraConfiguration::orientation + */ + +/** + * \fn Dw100VertexMap::rotation() + * \brief Get the rotation + * \return The rotation in degrees + */ + +/** + * \fn Dw100VertexMap::setOffset() + * \brief Sets the offset to apply + * \param[in] offset The offset + * + * Set the requested offset. The actually applied offset can be queried using \a + * Dw100VertexMap::effectiveOffset() after calling \a + * Dw100VertexMap::applyLimits(). + */ + +/** + * \fn Dw100VertexMap::effectiveOffset() + * \brief Get the effective offset + * + * Returns the actual offset applied to the input pixels in ScalerCrop + * coordinates. + * + * \return The effective offset + */ + +/** + * \fn Dw100VertexMap::setMode() + * \brief Sets the scaling mode to apply + * \param[in] mode The mode + */ + +/** + * \fn Dw100VertexMap::mode() + * \brief Get the scaling mode + * \return The scaling mode + */ + +/** + * \brief Get the dw100 vertex map + * + * Calculates the vertex map as a vector of hardware specific entries. + * + * \return The vertex map + */ +std::vector Dw100VertexMap::getVertexMap() +{ + int ow = outputSize_.width; + int oh = outputSize_.height; + int tileCountW = dw100VerticesForLength(ow); + int tileCountH = dw100VerticesForLength(oh); + + applyLimits(); + + /* + * libcamera handles all crop rectangles in sensor space. But the + * dewarper "sees" only the pixels it gets passed. Note that these might + * not cover exactly the max sensor crop, as there might be a crop + * between ISP and dewarper to crop to a format supported by the + * dewarper. effectiveScalerCrop_ is the crop in sensor space that gets + * fed into the dewarper. localScalerCrop is the sensor crop mapped to + * the data that is fed into the dewarper. + */ + Rectangle localScalerCrop = effectiveScalerCrop_.transformedBetween( + sensorCrop_, Rectangle(inputSize_)); + Size localCropSizeT = localScalerCrop.size(); + if (!!(transform_ & Transform::Transpose)) + std::swap(localCropSizeT.width, localCropSizeT.height); + + /* + * The dw100 has a specialty in interpolation that has to be taken into + * account to use in a pixel perfect manner. To explain this, I will + * only use the x direction, the vertical axis behaves the same. + * + * Let's start with a pixel perfect 1:1 mapping of an image with a width + * of 64pixels. The coordinates of the vertex map would then be: + * 0 -- 16 -- 32 -- 48 -- 64 + * Note how the last coordinate lies outside the image (which ends at + * 63) as it is basically the beginning of the next macro block. + * + * if we zoom out a bit we might end up with something like + * -10 -- 0 -- 32 -- 64 -- 74 + * As the dewarper coordinates are unsigned it actually sees + * 0 -- 0 -- 32 -- 64 -- 74 + * Leading to stretched pixels at the beginning and black for everything + * > 63 + * + * Now lets rotate the image by 180 degrees. A trivial rotation would + * end up with: + * + * 64 -- 48 -- 32 -- 16 -- 0 + * + * But as the first column now points to pixel 64 we get a single black + * line. So for a proper 180* rotation, the coordinates need to be + * + * 63 -- 47 -- 31 -- 15 -- -1 + * + * The -1 is clamped to 0 again, leading to a theoretical slight + * interpolation error on the last 16 pixels. + * + * To create this proper transformation there are two things todo: + * + * 1. The rotation centers are offset by -0.5. This evens out for no + * rotation, and leads to a coordinate offset of -1 on 180 degree + * rotations. + * 2. The transformation (flip and transpose) need to act on a size-1 + * to get the same effect. + */ + Vector2d centerS{ { localCropSizeT.width * 0.5 - 0.5, + localCropSizeT.height * 0.5 - 0.5 } }; + Vector2d centerD{ { ow * 0.5 - 0.5, + oh * 0.5 - 0.5 } }; + + LOG(Converter, Debug) + << "Apply vertex map for" + << " inputSize: " << inputSize_ + << " outputSize: " << outputSize_ + << " Transform: " << transformToString(transform_) + << "\n effectiveScalerCrop: " << effectiveScalerCrop_ + << " localCropSizeT: " << localCropSizeT + << " scaleX: " << effectiveScaleX_ + << " scaleY: " << effectiveScaleX_ + << " rotation: " << rotation_ + << " offset: " << effectiveOffset_; + + Matrix3x3 outputToSensor = Matrix3x3::identity(); + /* Move to center of output */ + outputToSensor = makeTranslate(-centerD) * outputToSensor; + outputToSensor = makeRotate(-rotation_) * outputToSensor; + outputToSensor = makeScale(1.0 / effectiveScaleX_, 1.0 / effectiveScaleY_) * outputToSensor; + /* Move to top left of localScalerCropT */ + outputToSensor = makeTranslate(centerS) * outputToSensor; + outputToSensor = makeTransform(-transform_, localCropSizeT.shrunkBy({ 1, 1 })) * + outputToSensor; + /* Transform from "within localScalerCrop" to input reference frame */ + outputToSensor = makeTranslate(localScalerCrop.x, localScalerCrop.y) * outputToSensor; + outputToSensor = makeTransform(localScalerCrop, effectiveScalerCrop_) * outputToSensor; + outputToSensor = makeTranslate(point2Vec2d(effectiveOffset_)) * outputToSensor; + + Matrix3x3 sensorToInput = makeTransform(effectiveScalerCrop_, localScalerCrop); + + /* + * For every output tile, calculate the position of the corners in the + * input image. + */ + std::vector res; + res.reserve(tileCountW * tileCountH); + for (int y = 0; y < tileCountH; y++) { + for (int x = 0; x < tileCountW; x++) { + Vector2d p{ { static_cast(x) * kDw100BlockSize, + static_cast(y) * kDw100BlockSize } }; + p = p.max(0.0).min(Vector2d{ { static_cast(ow), + static_cast(oh) } }); + + p = transformPoint(outputToSensor, p); + + /* + * \todo: Transformations in sensor space to be added + * here. + */ + + p = transformPoint(sensorToInput, p); + + /* Convert to fixed point */ + uint32_t v = static_cast(p.y() * 16) << 16 | + (static_cast(p.x() * 16) & 0xffff); + res.push_back(v); + } + } + + return res; +} + +} /* namespace libcamera */ diff --git a/src/libcamera/converter/meson.build b/src/libcamera/converter/meson.build index af1a80fec683..558d63a1bdd4 100644 --- a/src/libcamera/converter/meson.build +++ b/src/libcamera/converter/meson.build @@ -1,5 +1,6 @@ # SPDX-License-Identifier: CC0-1.0 libcamera_internal_sources += files([ + 'converter_dw100_vertexmap.cpp', 'converter_v4l2_m2m.cpp' ]) From patchwork Tue Nov 25 16:28:28 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 25195 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 16C40C333C for ; Tue, 25 Nov 2025 16:29:46 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CDA4160AA0; Tue, 25 Nov 2025 17:29:45 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="YMnraqZz"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7FA2C60AC8 for ; Tue, 25 Nov 2025 17:29:42 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bae1:340c:573c:570b]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 7610B6AF; Tue, 25 Nov 2025 17:27:33 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1764088053; bh=mtMNb6yc2/K9lvSTs1b9y0wcotfqjFOwFgN75HTWh5A=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=YMnraqZz3UZLz8wox2aQH/B9kxyqqYQ0iuXE+yg5sbVXYlwNkJ4tLnd2fhG9u7RZk tjS85wjCrWVucM3HMm3HQl8GBoQnfHodykXGqOKYEPc5MNawTsr5Fvclx88mPv1xB3 F/G6SHxhQVHBTxOjSn/b5AvvGOOBW3rMa3XY1WUU= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v3 16/29] libcamera: converter: Add dw100 converter module Date: Tue, 25 Nov 2025 17:28:28 +0100 Message-ID: <20251125162851.2301793-17-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> References: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The DW100 Dewarp engine is present on i.MX8MP SoC and possibly others. This patch provides a dedicated converter module that allows easy integration of such a dewarper into a pipeline handler. In this patch only the ScalerCrop control is implemented. Support for additional functionality will be added in later patches. Signed-off-by: Stefan Klug Reviewed-by: Kieran Bingham --- Chnages in v3: - Major rewrite to implement the libcamera controls in the converter class instead of leaving that to the pipeline handler. - Moved V4L2 requests handling into the class instead of leaving that to the pipeline handler Changes in v2: - Drop validateSize() as it is not used anywhere Changes in v0.9 - Use shared_ptr in constructor --- .../internal/converter/converter_dw100.h | 85 ++++ .../libcamera/internal/converter/meson.build | 1 + src/libcamera/converter/converter_dw100.cpp | 413 ++++++++++++++++++ src/libcamera/converter/meson.build | 1 + 4 files changed, 500 insertions(+) create mode 100644 include/libcamera/internal/converter/converter_dw100.h create mode 100644 src/libcamera/converter/converter_dw100.cpp diff --git a/include/libcamera/internal/converter/converter_dw100.h b/include/libcamera/internal/converter/converter_dw100.h new file mode 100644 index 000000000000..8dd21a6228f1 --- /dev/null +++ b/include/libcamera/internal/converter/converter_dw100.h @@ -0,0 +1,85 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (C) 2024, Ideas On Board Oy + * + * i.MX8MP Dewarp Engine integration + */ + +#pragma once + +#include +#include + +#include +#include +#include + +#include "libcamera/internal/converter/converter_dw100_vertexmap.h" +#include "libcamera/internal/converter/converter_v4l2_m2m.h" +#include "libcamera/internal/device_enumerator.h" + +namespace libcamera { + +class MediaDevice; +class Rectangle; +class Stream; + +class ConverterDW100Module +{ +public: + virtual ~ConverterDW100Module() = default; + + static std::unique_ptr createModule(DeviceEnumerator *enumerator); + + int configure(const StreamConfiguration &inputCfg, + const std::vector> + &outputCfg); + bool isConfigured(const Stream *stream) const; + + Size adjustInputSize(const PixelFormat &pixFmt, const Size &size, + Converter::Alignment align = Converter::Alignment::Down); + Size adjustOutputSize(const PixelFormat &pixFmt, const Size &size, + Converter::Alignment align = Converter::Alignment::Down); + + int exportBuffers(const Stream *stream, unsigned int count, + std::vector> *buffers); + int validateOutput(StreamConfiguration *cfg, bool *adjusted, + Converter::Alignment align = Converter::Alignment::Down); + int queueBuffers(FrameBuffer *input, + const std::map &outputs); + + int start(); + void stop(); + + void updateControlInfos(const Stream *stream, ControlInfoMap::Map &infos); + void setControls(const Stream *stream, const ControlList &controls); + void populateMetadata(const Stream *stream, ControlList &meta); + + void setSensorCrop(const Rectangle &rect); + void setTransform(const Stream *stream, const Transform &transform); + + Signal inputBufferReady; + Signal outputBufferReady; + +private: + ConverterDW100Module(std::shared_ptr media); + + int applyControls(const Stream *stream, const V4L2Request *request); + void reinitRequest(V4L2Request *request); + + struct VertexMapInfo { + Dw100VertexMap map; + bool update; + }; + + std::map vertexMaps_; + unsigned int inputBufferCount_; + V4L2M2MConverter converter_; + Rectangle sensorCrop_; + bool running_; + + std::vector> requests_; + std::queue availableRequests_; +}; + +} /* namespace libcamera */ diff --git a/include/libcamera/internal/converter/meson.build b/include/libcamera/internal/converter/meson.build index 9d586293f63a..128c644cb73f 100644 --- a/include/libcamera/internal/converter/meson.build +++ b/include/libcamera/internal/converter/meson.build @@ -1,6 +1,7 @@ # SPDX-License-Identifier: CC0-1.0 libcamera_internal_headers += files([ + 'converter_dw100.h', 'converter_dw100_vertexmap.h', 'converter_v4l2_m2m.h', ]) diff --git a/src/libcamera/converter/converter_dw100.cpp b/src/libcamera/converter/converter_dw100.cpp new file mode 100644 index 000000000000..cba7cc9f709b --- /dev/null +++ b/src/libcamera/converter/converter_dw100.cpp @@ -0,0 +1,413 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (C) 2024, Ideas On Board Oy + * + * i.MX8MP Dewarp Engine integration + */ + +#include "libcamera/internal/converter/converter_dw100.h" + +#include + +#include + +#include +#include + +#include "libcamera/internal/converter.h" +#include "libcamera/internal/converter/converter_v4l2_m2m.h" +#include "libcamera/internal/media_device.h" +#include "libcamera/internal/v4l2_videodevice.h" + +namespace libcamera { + +LOG_DECLARE_CATEGORY(Converter) + +/** + * \class libcamera::ConverterDW100Module + * \brief A converter module for the dw100 dewarper + * + * This class implements a converter module with direct support for libcamera + * controls. Functionality wise it closely resembles the libcamera::Converter + * interface. The main difference is that V4L2 requests are handled internally + * and it has direct support for libcamera controls. + */ + +ConverterDW100Module::ConverterDW100Module(std::shared_ptr media) + : converter_(media), running_(false) +{ + converter_.outputBufferReady.connect(&this->outputBufferReady, &Signal::emit); + converter_.inputBufferReady.connect(&this->inputBufferReady, &Signal::emit); +} + +/** + * \brief Create a ConverterDW100Module + * \param[in] enumerator The enumerator + * + * Static factory function that searches for the dw100 device using the provided + * \a enumerator. If found, a ConverterDW100Module is instantiated and returned. + * + * \return A ConverterDW100Module or null if no converter was found + */ +std::unique_ptr +ConverterDW100Module::createModule(DeviceEnumerator *enumerator) +{ + DeviceMatch dwp("dw100"); + dwp.add("dw100-source"); + dwp.add("dw100-sink"); + + std::shared_ptr dwpMediaDevice = enumerator->search(dwp); + if (!dwpMediaDevice) + return {}; + + std::unique_ptr dwpModule{ new ConverterDW100Module(dwpMediaDevice) }; + if (dwpModule->converter_.isValid()) + return dwpModule; + + LOG(Converter, Warning) + << "Found DW100 dewarper " << dwpMediaDevice->deviceNode() + << " but invalid"; + return {}; +} + +/** + * \copydoc libcamera::V4L2M2MConverter::configure + */ +int ConverterDW100Module::configure(const StreamConfiguration &inputCfg, + const std::vector> + &outputCfgs) +{ + int ret; + + vertexMaps_.clear(); + ret = converter_.configure(inputCfg, outputCfgs); + if (ret) + return ret; + + inputBufferCount_ = inputCfg.bufferCount; + + for (auto &ref : outputCfgs) { + const auto &outputCfg = ref.get(); + auto &info = vertexMaps_[outputCfg.stream()]; + auto &vertexMap = info.map; + vertexMap.setInputSize(inputCfg.size); + vertexMap.setOutputSize(outputCfg.size); + vertexMap.setSensorCrop(sensorCrop_); + info.update = true; + } + + return 0; +} + +/** + * \copydoc libcamera::V4L2M2MConverter::isConfigured + */ +bool ConverterDW100Module::isConfigured(const Stream *stream) const +{ + return vertexMaps_.find(stream) != vertexMaps_.end(); +} + +/** + * \copydoc libcamera::V4L2M2MConverter::adjustInputSize + */ +Size ConverterDW100Module::adjustInputSize(const PixelFormat &pixFmt, + const Size &size, + Converter::Alignment align) +{ + return converter_.adjustInputSize(pixFmt, size, align); +} + +/** + * \copydoc libcamera::V4L2M2MConverter::adjustOutputSize + */ +Size ConverterDW100Module::adjustOutputSize(const PixelFormat &pixFmt, + const Size &size, + Converter::Alignment align) +{ + return converter_.adjustOutputSize(pixFmt, size, align); +} + +/** + * \copydoc libcamera::V4L2M2MConverter::exportBuffers + */ +int ConverterDW100Module::exportBuffers(const Stream *stream, unsigned int count, + std::vector> *buffers) +{ + return converter_.exportBuffers(stream, count, buffers); +} + +/** + * \copydoc libcamera::V4L2M2MConverter::validateOutput + */ +int ConverterDW100Module::validateOutput(StreamConfiguration *cfg, + bool *adjusted, + Converter::Alignment align) +{ + return converter_.validateOutput(cfg, adjusted, align); +} + +/** + * \brief Queue buffers to converter device + * \param[in] input The frame buffer to apply the conversion + * \param[out] outputs The container holding the output stream pointers and + * their respective frame buffer outputs. + * + * This function queues the \a input frame buffer and the output frame buffers + * contained in \a outputs to the device for processing. + * + * Controls are automatically applied to the device before queuing buffers. V4L2 + * requests are used to atomically apply the controls if the kernel supports it. + * + * \return 0 on success or a negative error code otherwise + */ +int ConverterDW100Module::queueBuffers(FrameBuffer *input, + const std::map &outputs) +{ + int ret; + + V4L2Request *request = nullptr; + if (!requests_.empty()) { + /* If we have requests support, there must be one available */ + ASSERT(!availableRequests_.empty()); + request = availableRequests_.front(); + availableRequests_.pop(); + } + + for (auto &[stream, buffer] : outputs) { + ret = applyControls(stream, request); + if (ret) { + reinitRequest(request); + return ret; + } + } + + ret = converter_.queueBuffers(input, outputs, request); + if (ret) { + reinitRequest(request); + return ret; + } + + if (!request) + return 0; + + ret = request->queue(); + if (ret < 0) { + LOG(Converter, Error) << "Failed to queue dewarp request: -" + << strerror(-ret); + /* Push it back into the queue. */ + reinitRequest(request); + } + + return ret; +} + +/** + * \copydoc libcamera::V4L2M2MConverter::start + */ +int ConverterDW100Module::start() +{ + int ret; + + if (converter_.supportsRequests()) { + ret = converter_.allocateRequests(inputBufferCount_, + &requests_); + if (ret < 0) { + LOG(Converter, Error) << "Failed to allocate requests."; + return ret; + } + } + + for (std::unique_ptr &request : requests_) { + request->requestDone.connect(this, &ConverterDW100Module::reinitRequest); + availableRequests_.push(request.get()); + } + + /* + * Apply controls on all streams, to support older kernels without + * request and dynamic vertex map support. + */ + for (auto &[stream, info] : vertexMaps_) + applyControls(stream, nullptr); + + ret = converter_.start(); + if (!ret) { + running_ = true; + return 0; + } + + availableRequests_ = {}; + requests_.clear(); + return ret; +} + +/** + * \copydoc libcamera::V4L2M2MConverter::stop + */ +void ConverterDW100Module::stop() +{ + running_ = false; + converter_.stop(); + availableRequests_ = {}; + requests_.clear(); +} + +/** + * \brief Update the controls + * \param[in] stream The stream + * \param[inout] controls The controls info map to update + * + * Updated the \a controls map with all the controls and limits provided by this + * class. + */ +void ConverterDW100Module::updateControlInfos(const Stream *stream, ControlInfoMap::Map &controls) +{ + ControlValue scalerCropDefault = sensorCrop_; + + if (isConfigured(stream)) { + auto &info = vertexMaps_[stream]; + info.map.applyLimits(); + scalerCropDefault = info.map.effectiveScalerCrop(); + } + + controls[&controls::ScalerCrop] = ControlInfo(Rectangle(sensorCrop_.x, sensorCrop_.y, 1, 1), + sensorCrop_, sensorCrop_); + + if (!converter_.supportsRequests()) + LOG(Converter, Warning) + << "dw100 kernel driver has no requests support." + " Dynamic configuration is not possible."; +} + +/** + * \brief Set libcamera controls + * \param[in] stream The stream to update + * \param[in] controls The controls + * + * Looks up all supported controls in \a controls and sets them on stream \a + * stream. The controls will be applied to the device on the next call to + * queueBuffers(). + */ +void ConverterDW100Module::setControls(const Stream *stream, const ControlList &controls) +{ + if (!isConfigured(stream)) + return; + + auto &info = vertexMaps_[stream]; + auto &vertexMap = info.map; + + const auto &crop = controls.get(controls::ScalerCrop); + if (crop) { + vertexMap.setScalerCrop(*crop); + info.update = true; + } + + if (info.update && running_ && !converter_.supportsRequests()) + LOG(Converter, Error) + << "Dynamically setting dw100 specific controls requires" + " a dw100 kernel driver with requests support"; +} + +/** + * \brief Retrieve updated metadata + * \param[in] stream The stream + * \param[in] meta The metadata list + * + * This function retrieves the metadata for the provided \a stream and writes it + * to \a list. It shall be called after queueBuffers(). + */ +void ConverterDW100Module::populateMetadata(const Stream *stream, ControlList &meta) +{ + if (!isConfigured(stream)) + return; + + auto &vertexMap = vertexMaps_[stream].map; + + meta.set(controls::ScalerCrop, vertexMap.effectiveScalerCrop()); +} + +/** + * \var ConverterDW100Module::inputBufferReady + * \brief A signal emitted when the input frame buffer completes + */ + +/** + * \var ConverterDW100Module::outputBufferReady + * \brief A signal emitted on each frame buffer completion of the output queue + */ + +/** + * \brief Set sensor crop rectangle + * \param[in] rect The crop rectangle + * + * Set the sensor crop rectangle to \a rect. This rectangle describes the area + * covered by the input buffers in sensor coordinates. It is used internally + * to handle the ScalerCrop control and related metadata. + */ +void ConverterDW100Module::setSensorCrop(const Rectangle &rect) +{ + sensorCrop_ = rect; + for (auto &[stream, vertexMap] : vertexMaps_) { + vertexMap.map.setSensorCrop(rect); + vertexMap.update = true; + } +} + +/** + * \brief Set transform + * \param[in] stream The stream + * \param[in] transform The transform + * + * Set the transform that shall be applied by the dewarper on the given stream. + * As orientation is a property of libcamera::CameraConfiguration, the transform + * needs to be set at configure time. + */ +void ConverterDW100Module::setTransform(const Stream *stream, const Transform &transform) +{ + if (!isConfigured(stream)) + return; + + vertexMaps_[stream].map.setTransform(transform); +} + +void ConverterDW100Module::reinitRequest(V4L2Request *request) +{ + if (!request) + return; + + request->reinit(); + availableRequests_.push(request); +} + +/** + * \brief Apply the vertex map for a given stream + * \param[in] stream The stream to update + * \param[in] request An optional request + * + * This function updates the vertex map on the stream \a stream. If \a request + * is provided, the updated happens in that request. + * + * \return 0 on success or a negative error code otherwise + */ +int ConverterDW100Module::applyControls(const Stream *stream, const V4L2Request *request) +{ + if (!isConfigured(stream)) + return -EINVAL; + + auto &info = vertexMaps_[stream]; + if (!info.update) + return 0; + + std::vector map = info.map.getVertexMap(); + auto value = Span(reinterpret_cast(&map[0]), map.size()); + + ControlList ctrls; + ctrls.set(V4L2_CID_DW100_DEWARPING_16x16_VERTEX_MAP, value); + + int ret = converter_.applyControls(stream, ctrls, request); + if (!ret) + info.update = false; + + return ret; +} + +} /* namespace libcamera */ diff --git a/src/libcamera/converter/meson.build b/src/libcamera/converter/meson.build index 558d63a1bdd4..48f27cad5997 100644 --- a/src/libcamera/converter/meson.build +++ b/src/libcamera/converter/meson.build @@ -1,6 +1,7 @@ # SPDX-License-Identifier: CC0-1.0 libcamera_internal_sources += files([ + 'converter_dw100.cpp', 'converter_dw100_vertexmap.cpp', 'converter_v4l2_m2m.cpp' ]) From patchwork Tue Nov 25 16:28:29 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 25196 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 545B5C333D for ; Tue, 25 Nov 2025 16:29:47 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E4C3460AB1; Tue, 25 Nov 2025 17:29:46 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="MJGSHdeM"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D891960AA4 for ; Tue, 25 Nov 2025 17:29:44 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bae1:340c:573c:570b]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id E5C7D6AF; Tue, 25 Nov 2025 17:27:35 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1764088056; bh=Mk3k5f5BpwZwyQZfSvYKSz5AqfgWwA8+1FTcYbeX7D8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=MJGSHdeMUQz5E+K8kUcoklpRRnlYnMPkTax0tOIuElcJp02Owbkfb+DcNy8zguJKa 9Bb3xBdM0wpZSK/c5wdR+UcomWWEhHgNOWYEY4/DAdSnAXsayw8fN5TrnOfAfGsJbJ Xuxi5XlraKoUxfj8ZT47chS6PWShv+HIoQzmkVKw= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v3 17/29] libcamera: rkisp1: Use the dw100 converter module instead of the generic v4l2 converter Date: Tue, 25 Nov 2025 17:28:29 +0100 Message-ID: <20251125162851.2301793-18-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> References: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The dewarper integration into the rkisp1 pipeline is quite complicated. Simplify that by switching to the now available ConverterDW100Module. As there is no other known converter in combination with th rkisp1 ISP this is a safe step to do. This change also paves the way to implement dw100 specific features later. The input crop implemented in the dw100 kernel driver is quite limited in that it doesn't allow arbitrary crop rectangles but only scale factors quantized to the underlying fixed point representation and only aspect ratio preserving crops. The vertex map based implementation allows for pixel perfect crops. The only downside is that ScalerCrop can no longer be set dynamically on older kernels. A corresponding warning is already implemented in the converter module. Signed-off-by: Stefan Klug Reviewed-by: Kieran Bingham --- Changes in v3: - Merge usage of the converter module and ScalerCrop handling into one patch as it is difficult to keep them separate with the new module concept Changes in 0.9: - Code cleanup - Update vertex map before start() to partially support old kernels Changes in 0.7: - Removed double call to applyVertexMap() --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 122 ++++++----------------- 1 file changed, 33 insertions(+), 89 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 138e1d5bf06b..6256a67bc31e 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -36,7 +36,7 @@ #include "libcamera/internal/camera.h" #include "libcamera/internal/camera_sensor.h" #include "libcamera/internal/camera_sensor_properties.h" -#include "libcamera/internal/converter/converter_v4l2_m2m.h" +#include "libcamera/internal/converter/converter_dw100.h" #include "libcamera/internal/delayed_controls.h" #include "libcamera/internal/device_enumerator.h" #include "libcamera/internal/framebuffer.h" @@ -232,10 +232,7 @@ private: RkISP1MainPath mainPath_; RkISP1SelfPath selfPath_; - std::unique_ptr dewarper_; - Rectangle scalerMaxCrop_; - - std::optional activeCrop_; + std::unique_ptr dewarper_; /* Internal buffers used when dewarper is being used */ std::vector> mainPathBuffers_; @@ -940,6 +937,16 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) if (ret) return ret; + /* + * Apply the actual sensor crop, for proper + * dewarp map calculation + */ + Rectangle sensorCrop = outputCrop.transformedBetween( + inputCrop, sensorInfo.analogCrop); + if (data->usesDewarper_) + dewarper_->setSensorCrop(sensorCrop); + data->properties_.set(properties::ScalerCropMaximum, sensorCrop); + std::map streamConfig; std::vector> outputCfgs; @@ -965,13 +972,15 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) return ret; /* - * Calculate the crop rectangle of the data - * flowing into the dewarper in sensor - * coordinates. + * Apply a default scaler crop that keeps the + * aspect ratio. */ - scalerMaxCrop_ = - outputCrop.transformedBetween(inputCrop, - sensorInfo.analogCrop); + Size size = cfg.size; + size = sensorCrop.size().boundedToAspectRatio(size); + + ControlList ctrls; + ctrls.set(controls::ScalerCrop, size.centeredTo(sensorCrop.center())); + dewarper_->setControls(cfg.stream(), ctrls); } ret = mainPath_.configure(ispCfg, format); @@ -1165,6 +1174,9 @@ int PipelineHandlerRkISP1::start(Camera *camera, [[maybe_unused]] const ControlL actions += [&]() { stat_->streamOff(); }; if (data->usesDewarper_) { + if (controls) + dewarper_->setControls(&data->mainPathStream_, *controls); + ret = dewarper_->start(); if (ret) { LOG(RkISP1, Error) << "Failed to start dewarper"; @@ -1315,28 +1327,8 @@ int PipelineHandlerRkISP1::updateControls(RkISP1CameraData *data) { ControlInfoMap::Map controls; - if (data->usesDewarper_) { - std::pair cropLimits; - if (dewarper_->isConfigured(&data->mainPathStream_)) - cropLimits = dewarper_->inputCropBounds(&data->mainPathStream_); - else - cropLimits = dewarper_->inputCropBounds(); - - /* - * ScalerCrop is specified to be in Sensor coordinates. - * So we need to transform the limits to sensor coordinates. - * We can safely assume that the maximum crop limit contains the - * full fov of the dewarper. - */ - Rectangle min = cropLimits.first.transformedBetween(cropLimits.second, - scalerMaxCrop_); - - controls[&controls::ScalerCrop] = ControlInfo(min, - scalerMaxCrop_, - scalerMaxCrop_); - data->properties_.set(properties::ScalerCropMaximum, scalerMaxCrop_); - activeCrop_ = scalerMaxCrop_; - } + if (data->usesDewarper_) + dewarper_->updateControlInfos(&data->mainPathStream_, controls); /* Add the IPA registered controls to list of camera controls. */ for (const auto &ipaControl : data->ipaControls_) @@ -1376,8 +1368,6 @@ int PipelineHandlerRkISP1::createCamera(MediaEntity *sensor) /* Initialize the camera properties. */ data->properties_ = data->sensor_->properties(); - scalerMaxCrop_ = Rectangle(data->sensor_->resolution()); - const CameraSensorProperties::SensorDelays &delays = data->sensor_->sensorDelays(); std::unordered_map params = { { V4L2_CID_ANALOGUE_GAIN, { delays.gainDelay, false } }, @@ -1472,27 +1462,11 @@ bool PipelineHandlerRkISP1::match(DeviceEnumerator *enumerator) stat_->bufferReady.connect(this, &PipelineHandlerRkISP1::statBufferReady); param_->bufferReady.connect(this, &PipelineHandlerRkISP1::paramBufferReady); - /* If dewarper is present, create its instance. */ - DeviceMatch dwp("dw100"); - dwp.add("dw100-source"); - dwp.add("dw100-sink"); - - std::shared_ptr dwpMediaDevice = enumerator->search(dwp); - if (dwpMediaDevice) { - dewarper_ = std::make_unique(dwpMediaDevice); - if (dewarper_->isValid()) { - dewarper_->outputBufferReady.connect( - this, &PipelineHandlerRkISP1::dewarpBufferReady); - - LOG(RkISP1, Info) - << "Found DW100 dewarper " << dewarper_->deviceNode(); - } else { - LOG(RkISP1, Warning) - << "Found DW100 dewarper " << dewarper_->deviceNode() - << " but invalid"; - - dewarper_.reset(); - } + dewarper_ = ConverterDW100Module::createModule(enumerator); + if (dewarper_) { + dewarper_->outputBufferReady.connect( + this, &PipelineHandlerRkISP1::dewarpBufferReady); + LOG(RkISP1, Debug) << "Found DW100 dewarper"; } /* @@ -1603,37 +1577,7 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) return; } - /* Handle scaler crop control. */ - const auto &crop = request->controls().get(controls::ScalerCrop); - if (crop) { - Rectangle rect = crop.value(); - - /* - * ScalerCrop is specified to be in Sensor coordinates. - * So we need to transform it into dewarper coordinates. - * We can safely assume that the maximum crop limit contains the - * full fov of the dewarper. - */ - std::pair cropLimits = - dewarper_->inputCropBounds(&data->mainPathStream_); - - rect = rect.transformedBetween(scalerMaxCrop_, cropLimits.second); - int ret = dewarper_->setInputCrop(&data->mainPathStream_, - &rect); - rect = rect.transformedBetween(cropLimits.second, scalerMaxCrop_); - if (!ret && rect != crop.value()) { - /* - * If the rectangle is changed by setInputCrop on the - * dewarper, log a debug message and cache the actual - * applied rectangle for metadata reporting. - */ - LOG(RkISP1, Debug) - << "Applied rectangle " << rect.toString() - << " differs from requested " << crop.value().toString(); - } - - activeCrop_ = rect; - } + dewarper_->setControls(&data->mainPathStream_, request->controls()); /* * Queue input and output buffers to the dewarper. The output @@ -1642,7 +1586,7 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) */ int ret = dewarper_->queueBuffers(buffer, request->buffers()); if (ret < 0) { - LOG(RkISP1, Error) << "Cannot queue buffers to dewarper: " + LOG(RkISP1, Error) << "Failed to queue buffers to dewarper: -" << strerror(-ret); cancelDewarpRequest(info); @@ -1650,7 +1594,7 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) return; } - request->metadata().set(controls::ScalerCrop, activeCrop_.value()); + dewarper_->populateMetadata(&data->mainPathStream_, request->metadata()); } void PipelineHandlerRkISP1::dewarpBufferReady(FrameBuffer *buffer) From patchwork Tue Nov 25 16:28:30 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 25197 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 89D7CC333C for ; Tue, 25 Nov 2025 16:29:50 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3F61860AA9; Tue, 25 Nov 2025 17:29:50 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="c49g6m8j"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 49F5460AA9 for ; Tue, 25 Nov 2025 17:29:48 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bae1:340c:573c:570b]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 4814813DE; Tue, 25 Nov 2025 17:27:39 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1764088059; bh=HfLvjKiNNLE6G9AXy7i70zO8RwYQ/+PKMZMMvzip0xI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=c49g6m8jUxw+bi79WTL1MYRDjsCev16RQP+Yh7xDKEH17i2SqzzMt5R1G/FMINJKB Qhv5GIGC/mHqEbeDk4JVK4Q0JVv/SBXDLiOFrDLsaHjF6qN3mkmbys3yJhv9qycLON 9euCljBdTGq9BtTvPXSCP9/0TPfcNvmDbgmQVIGc= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v3 18/29] pipeline: rksip1: Move isRaw up in scope Date: Tue, 25 Nov 2025 17:28:30 +0100 Message-ID: <20251125162851.2301793-19-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> References: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In raw mode the number of configurations is actively limited to 1. It is therefore safe to move isRaw up one level to simplify the code and prepare for later use. During that rework it was noticed that the old code actually has a bug in that it reduces the number of configurations to 1 in case a raw config is found, but it doesn't reduce the config vector to that raw config, but the first config. Change that behavior to check the first config and either remove all remaining configs if the first is raw or drop all raw configs if the first is non-raw. Signed-off-by: Stefan Klug Reviewed-by: Kieran Bingham --- Changes in v3: - Added fix for cases where only one stream is requested - Fix adjustment in case of multiple configs - Small code cleanup --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 35 ++++++++++++------------ 1 file changed, 17 insertions(+), 18 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 6256a67bc31e..a11d70849121 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -551,31 +551,30 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() status = Adjusted; /* - * Simultaneous capture of raw and processed streams isn't possible. If - * there is any raw stream, cap the number of streams to one. + * Simultaneous capture of raw and processed streams isn't possible. + * Let the first stream decide on the type. */ - if (config_.size() > 1) { - for (const auto &cfg : config_) { - if (PixelFormatInfo::info(cfg.pixelFormat).colourEncoding == + bool isRaw = (PixelFormatInfo::info(config_[0].pixelFormat).colourEncoding == + PixelFormatInfo::ColourEncodingRAW); + if (isRaw) { + if (config_.size() > 1) { + config_.resize(1); + status = Adjusted; + } + } else { + /* Drop all raw configs. */ + for (auto it = config_.begin(); it != config_.end();) { + if (PixelFormatInfo::info(it->pixelFormat).colourEncoding == PixelFormatInfo::ColourEncodingRAW) { - config_.resize(1); + it = config_.erase(it); status = Adjusted; - break; + continue; } + ++it; } } - bool useDewarper = false; - if (pipe->dewarper_) { - /* - * Platforms with dewarper support, such as i.MX8MP, support - * only a single stream. We can inspect config_[0] only here. - */ - bool isRaw = PixelFormatInfo::info(config_[0].pixelFormat).colourEncoding == - PixelFormatInfo::ColourEncodingRAW; - if (!isRaw) - useDewarper = true; - } + bool useDewarper = (pipe->dewarper_ && !isRaw); /* * If there are more than one stream in the configuration figure out the From patchwork Tue Nov 25 16:28:31 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 25198 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 84295C333C for ; Tue, 25 Nov 2025 16:29:53 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3362760AA9; Tue, 25 Nov 2025 17:29:53 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="YRdzuD74"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5E3EA60A9E for ; Tue, 25 Nov 2025 17:29:51 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bae1:340c:573c:570b]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 64E5813DE; Tue, 25 Nov 2025 17:27:42 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1764088062; bh=Is2AlUQtyGNBusRQNqG1cbaXEUpDTn3nPIWS/o2ObzA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=YRdzuD74A38rblkj2c3rV3fQ7UFJkWo7kjg8vhbwACGeA5tiqimJxLaKK+c+UF318 Kw+QGz0TlJvE3/2z/7/afdvSQuGc/WzTyCkcFF6WiLEn7wu/sIV0aNAA1SY2+lf0o4 uq8rxzIwl6XaRV6u5kvW/F8yr0hoIXtmEzFmwQF4= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Umang Jain , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder Subject: [PATCH v3 19/29] pipeline: rkisp1: Drop rawFormat variable Date: Tue, 25 Nov 2025 17:28:31 +0100 Message-ID: <20251125162851.2301793-20-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> References: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In raw mode we know there is only a single configuration so there is no need to iterate over all configurations to find the format. Drop that. Signed-off-by: Stefan Klug Reviewed-by: Umang Jain Reviewed-by: Barnabás PÅ‘cze Reviewed-by: Paul Elder --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index a11d70849121..2972d6e1cbd1 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -677,21 +677,16 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() } /* Select the sensor format. */ - PixelFormat rawFormat; Size maxSize; for (const StreamConfiguration &cfg : config_) { - const PixelFormatInfo &info = PixelFormatInfo::info(cfg.pixelFormat); - if (info.colourEncoding == PixelFormatInfo::ColourEncodingRAW) - rawFormat = cfg.pixelFormat; - maxSize = std::max(maxSize, cfg.size); } std::vector mbusCodes; - if (rawFormat.isValid()) { - mbusCodes = { rawFormats.at(rawFormat) }; + if (isRaw) { + mbusCodes = { rawFormats.at(config_[0].pixelFormat) }; } else { std::transform(rawFormats.begin(), rawFormats.end(), std::back_inserter(mbusCodes), From patchwork Tue Nov 25 16:28:32 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 25199 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4F038C333C for ; Tue, 25 Nov 2025 16:29:56 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1323060AA9; Tue, 25 Nov 2025 17:29:56 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="iH8MQHEF"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9CCA160A9E for ; Tue, 25 Nov 2025 17:29:54 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bae1:340c:573c:570b]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id B3515FDB; Tue, 25 Nov 2025 17:27:45 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1764088065; bh=SD7cz3jxHVqUE3yKJ+8BYwHqczjKFQHHlcGigHlrdN0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=iH8MQHEFGqw6eFDDs86z/nlfnVAmUXICrN1vNFtH0vx4xKcOtVkhg0BCVVw9RP/KQ bF4AcCm+mcMsw7P7MYYlEFG1RmdvDUL0CVARQP5Joqml29HTTQozui4WGr9cW6rSVf GrSEzlRzIepMMhEDGEkNtJbhG8l2V+Bdnd+FXlXY= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Paul Elder Subject: [PATCH v3 20/29] pipeline: rkisp1: Enable the dewarper based on the tuning file Date: Tue, 25 Nov 2025 17:28:32 +0100 Message-ID: <20251125162851.2301793-21-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> References: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" To do actual lens dewarping, the dewarper will be configured based on the tuning file. As a first step implement the basic loading of the tuning file and enable/disable the dewarper for the given camera based on the existence of the "Dewarp" entry under a new top level element 'modules' in the tuning file. Note: This is an backwards incompatible change in that the dewarper is currently included in the chain unconditionally. Some users may want to not use the dewarper, so it is sensible to make that configurable. Signed-off-by: Stefan Klug Reviewed-by: Paul Elder Reviewed-by: Kieran Bingham --- Changes in v3: - Collected tag - The config is now looked up under the toplevel key 'modules' instead of 'algorithms' - Properly initialize canUseDewarper_ and usesDewarper() so that it still works if there is no dewarper or it is not contained in the tuning file. Changes in v2: - Drop unused params variable - Moved patch a bit earlier so canUseDewarper is available for handling the orientation --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 52 +++++++++++++++++++++++- 1 file changed, 50 insertions(+), 2 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 2972d6e1cbd1..e3dca8b837e8 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -16,6 +16,7 @@ #include #include +#include #include #include @@ -46,6 +47,7 @@ #include "libcamera/internal/pipeline_handler.h" #include "libcamera/internal/v4l2_subdevice.h" #include "libcamera/internal/v4l2_videodevice.h" +#include "libcamera/internal/yaml_parser.h" #include "rkisp1_path.h" @@ -94,7 +96,8 @@ public: RkISP1CameraData(PipelineHandler *pipe, RkISP1MainPath *mainPath, RkISP1SelfPath *selfPath) : Camera::Private(pipe), frame_(0), frameInfo_(pipe), - mainPath_(mainPath), selfPath_(selfPath) + mainPath_(mainPath), selfPath_(selfPath), + canUseDewarper_(false), usesDewarper_(false) { } @@ -122,6 +125,7 @@ public: */ MediaPipeline pipe_; + bool canUseDewarper_; bool usesDewarper_; private: @@ -130,6 +134,7 @@ private: const ControlList &sensorControls); void metadataReady(unsigned int frame, const ControlList &metadata); + int loadTuningFile(const std::string &file); }; class RkISP1CameraConfiguration : public CameraConfiguration @@ -411,6 +416,49 @@ int RkISP1CameraData::loadIPA(unsigned int hwRevision, uint32_t supportedBlocks) return ret; } + ret = loadTuningFile(ipaTuningFile); + if (ret < 0) { + LOG(RkISP1, Error) << "Failed to load tuning file"; + return ret; + } + + return 0; +} + +int RkISP1CameraData::loadTuningFile(const std::string &path) +{ + int ret; + + if (!pipe()->dewarper_) + /* Nothing to do without dewarper */ + return 0; + + LOG(RkISP1, Debug) << "Load tuning file " << path; + + File file(path); + if (!file.open(File::OpenModeFlag::ReadOnly)) { + ret = file.error(); + LOG(RkISP1, Error) + << "Failed to open tuning file " + << path << ": " << strerror(-ret); + return ret; + } + + std::unique_ptr data = YamlParser::parse(file); + if (!data) + return -EINVAL; + + if (!data->contains("modules")) + return 0; + + const auto &modules = (*data)["modules"].asList(); + for (const auto &module : modules) { + if (module.contains("Dewarp")) { + canUseDewarper_ = true; + break; + } + } + return 0; } @@ -574,7 +622,7 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() } } - bool useDewarper = (pipe->dewarper_ && !isRaw); + bool useDewarper = (data_->canUseDewarper_ && !isRaw); /* * If there are more than one stream in the configuration figure out the From patchwork Tue Nov 25 16:28:33 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 25200 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0204EC333C for ; Tue, 25 Nov 2025 16:29:59 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B715A60AA9; Tue, 25 Nov 2025 17:29:59 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="RbZ0nz3r"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0585E60AA0 for ; Tue, 25 Nov 2025 17:29:57 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bae1:340c:573c:570b]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 11EB218CB; Tue, 25 Nov 2025 17:27:48 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1764088068; bh=ixG83Iqpe2OIzb2G9wcrpsacCxtaL9lyfr55i2vN5vw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RbZ0nz3ri+u81sgpb/+VxGHPvrb+NeJx0K2GfwC+W+i9mcsb2cuMbEW5UhB54Ltff pfo6LgvAsPeCLc4BQif+KuUytH3WNIEeqrLjd77KaZjT9Z8m8KgD8ebFXePG3XB+Y1 oHliuWcekwZH0qh7cPXakJW2lPMT7H8HnVxTlR10= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Kieran Bingham , Paul Elder Subject: [PATCH v3 21/29] libcamera: internal: camera_sensor: Add accessor for mountingOrientation_ Date: Tue, 25 Nov 2025 17:28:33 +0100 Message-ID: <20251125162851.2301793-22-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> References: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" To properly handle the orientation in the dewarper, the mounting orientation of the sensor needs to be queryable. Add that. Signed-off-by: Stefan Klug Reviewed-by: Kieran Bingham Reviewed-by: Paul Elder --- Changes in v3: - Collected tag Changes in v2: - Collected tag - Add documentation --- include/libcamera/internal/camera_sensor.h | 1 + src/libcamera/sensor/camera_sensor.cpp | 7 +++++++ src/libcamera/sensor/camera_sensor_legacy.cpp | 1 + src/libcamera/sensor/camera_sensor_raw.cpp | 1 + 4 files changed, 10 insertions(+) diff --git a/include/libcamera/internal/camera_sensor.h b/include/libcamera/internal/camera_sensor.h index f6ef4df170d4..e6b72d22ac21 100644 --- a/include/libcamera/internal/camera_sensor.h +++ b/include/libcamera/internal/camera_sensor.h @@ -73,6 +73,7 @@ public: virtual int sensorInfo(IPACameraSensorInfo *info) const = 0; virtual Transform computeTransform(Orientation *orientation) const = 0; virtual BayerFormat::Order bayerOrder(Transform t) const = 0; + virtual Orientation mountingOrientation() const = 0; virtual const ControlInfoMap &controls() const = 0; virtual ControlList getControls(Span ids) = 0; diff --git a/src/libcamera/sensor/camera_sensor.cpp b/src/libcamera/sensor/camera_sensor.cpp index 4f2fd2690eb5..05390d1e1b60 100644 --- a/src/libcamera/sensor/camera_sensor.cpp +++ b/src/libcamera/sensor/camera_sensor.cpp @@ -331,6 +331,13 @@ int CameraSensor::setEmbeddedDataEnabled(bool enable) * \return The Bayer order produced by the sensor when the Transform is applied */ +/** + * \fn CameraSensor::mountingOrientation() + * \brief Fetch the mounting orientation of the sensor + * + * \return The mounting orientation of the sensor + */ + /** * \fn CameraSensor::controls() * \brief Retrieve the supported V4L2 controls and their information diff --git a/src/libcamera/sensor/camera_sensor_legacy.cpp b/src/libcamera/sensor/camera_sensor_legacy.cpp index f9e685a9acc4..39c34200b0ff 100644 --- a/src/libcamera/sensor/camera_sensor_legacy.cpp +++ b/src/libcamera/sensor/camera_sensor_legacy.cpp @@ -88,6 +88,7 @@ public: int sensorInfo(IPACameraSensorInfo *info) const override; Transform computeTransform(Orientation *orientation) const override; BayerFormat::Order bayerOrder(Transform t) const override; + Orientation mountingOrientation() const override { return mountingOrientation_; } const ControlInfoMap &controls() const override; ControlList getControls(Span ids) override; diff --git a/src/libcamera/sensor/camera_sensor_raw.cpp b/src/libcamera/sensor/camera_sensor_raw.cpp index 8ea4423698cd..759cccafe4a9 100644 --- a/src/libcamera/sensor/camera_sensor_raw.cpp +++ b/src/libcamera/sensor/camera_sensor_raw.cpp @@ -94,6 +94,7 @@ public: int sensorInfo(IPACameraSensorInfo *info) const override; Transform computeTransform(Orientation *orientation) const override; BayerFormat::Order bayerOrder(Transform t) const override; + Orientation mountingOrientation() const override { return mountingOrientation_; } const ControlInfoMap &controls() const override; ControlList getControls(Span ids) override; From patchwork Tue Nov 25 16:28:34 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 25201 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EEE9BC3338 for ; Tue, 25 Nov 2025 16:30:02 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 787A260B33; Tue, 25 Nov 2025 17:30:02 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Sly7K4cR"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E55D060AD0 for ; Tue, 25 Nov 2025 17:29:59 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bae1:340c:573c:570b]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 05586524; Tue, 25 Nov 2025 17:27:50 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1764088071; bh=Whf6J85kiCqddzmN8XRL2gX9/PkNgMqHWeAhj82Lx8U=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Sly7K4cR5Ss17L9PT4HkSWMnH1MP3dS7r/1rQ4n3+BUiXXYtbW4tXITgNGOJEakhY FjKZCPhwqry8HuqTQ0gmRpnWqxw/UtYgodYJyp2HxQD6WMDt2HxvqOh9qqjyj+IXyE aVHwHteN+bgHLnSIo96S9FAi2LKX5xyfunLAaYaE= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v3 22/29] libcamera: Add transpose() function to size Date: Tue, 25 Nov 2025 17:28:34 +0100 Message-ID: <20251125162851.2301793-23-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> References: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a transpose() function to size that applies the Transformation::Transpose operation in the size. This is useful when handling orientation adjustments. Signed-off-by: Stefan Klug Reviewed-by: Kieran Bingham --- Changes in v3: - Added test case --- include/libcamera/geometry.h | 6 ++++++ src/libcamera/geometry.cpp | 10 ++++++++++ test/geometry.cpp | 5 +++++ 3 files changed, 21 insertions(+) diff --git a/include/libcamera/geometry.h b/include/libcamera/geometry.h index d9378efeca8c..572a07e2cf92 100644 --- a/include/libcamera/geometry.h +++ b/include/libcamera/geometry.h @@ -108,6 +108,12 @@ public: return *this; } + Size &transpose() + { + std::swap(width, height); + return *this; + } + [[nodiscard]] constexpr Size alignedDownTo(unsigned int hAlignment, unsigned int vAlignment) const { diff --git a/src/libcamera/geometry.cpp b/src/libcamera/geometry.cpp index de76d0c12a8c..2763967a6e57 100644 --- a/src/libcamera/geometry.cpp +++ b/src/libcamera/geometry.cpp @@ -209,6 +209,16 @@ std::string Size::toString() const * \return A reference to this object */ +/** + * \fn Size::transpose() + * \brief Transpose the size in place + * + * This function swaps width and height of this size. This effectively applies + * the \a Transform::Transpose transformation on this size. + * + * \return A reference to this object + */ + /** * \fn Size::alignedDownTo(unsigned int hAlignment, unsigned int vAlignment) * \brief Align the size down horizontally and vertically diff --git a/test/geometry.cpp b/test/geometry.cpp index 11df043b733b..a50b643b7ee6 100644 --- a/test/geometry.cpp +++ b/test/geometry.cpp @@ -203,6 +203,11 @@ protected: return TestFail; } + if (Size(200, 50).transpose() != Size(50, 200)) { + cout << "Size::transpose() test failed" << endl; + return TestFail; + } + /* Aspect ratio tests */ if (Size(0, 0).boundedToAspectRatio(Size(4, 3)) != Size(0, 0) || Size(1920, 1440).boundedToAspectRatio(Size(16, 9)) != Size(1920, 1080) || From patchwork Tue Nov 25 16:28:35 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 25202 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B950BC333C for ; Tue, 25 Nov 2025 16:30:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5D84D60B33; Tue, 25 Nov 2025 17:30:05 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Uu/gG+yM"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 571C560A9E for ; Tue, 25 Nov 2025 17:30:03 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bae1:340c:573c:570b]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 638636AF; Tue, 25 Nov 2025 17:27:54 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1764088074; bh=wKpcWJscDWlpaAipJQxokq1/WoIZu0thGMePWX1d/xQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Uu/gG+yMmUuHJl7VUljQpB4CrIPKJKP/c2T/v4hZ4n+6Th2UAqnB5cwdL0ACgQJnw rxmire0szVGhaB2MCeWXtCoHqWUC6HOMwGUmvR7wKTo8PNSGAho3sxJslBjhIaii0G RXMOyrYvP58NiuxvrVIqlj/stm0j5awA/DewGFmA= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Paul Elder Subject: [PATCH v3 23/29] libcamera: rkisp1: Handle requested orientation using dewarper Date: Tue, 25 Nov 2025 17:28:35 +0100 Message-ID: <20251125162851.2301793-24-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> References: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When the dewarper is present it can handle arbitrary orientations specified in the requested camera configuration. In that case handle all transformations inside the dewarper (even if the sensor supports some of them) because that makes it easier to handle coordinates for lens dewarping inside the dewarper. This complicates the path selection a bit, as for transformations that include a transpose, the format before the dewarper has swapped width/height. Signed-off-by: Stefan Klug Reviewed-by: Paul Elder Reviewed-by: Kieran Bingham --- Changes in v3: - Collected tag Changes in v2: - Fix path validation for cases where a transpose is involved --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 89 ++++++++++++++++++------ 1 file changed, 67 insertions(+), 22 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index e3dca8b837e8..746eeffab207 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -593,11 +593,6 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() status = Adjusted; } - Orientation requestedOrientation = orientation; - combinedTransform_ = data_->sensor_->computeTransform(&orientation); - if (orientation != requestedOrientation) - status = Adjusted; - /* * Simultaneous capture of raw and processed streams isn't possible. * Let the first stream decide on the type. @@ -622,7 +617,23 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() } } + /* + * If the dewarper supports orientation adjustments, apply that completely + * there. Even if the sensor supports flips, it is beneficial to do that + * in the dewarper so that lens dewarping happens on the unflipped image + */ + bool transposeAfterIsp = false; bool useDewarper = (data_->canUseDewarper_ && !isRaw); + if (useDewarper) { + combinedTransform_ = orientation / data_->sensor_->mountingOrientation(); + if (!!(combinedTransform_ & Transform::Transpose)) + transposeAfterIsp = true; + } else { + Orientation requestedOrientation = orientation; + combinedTransform_ = data_->sensor_->computeTransform(&orientation); + if (orientation != requestedOrientation) + status = Adjusted; + } /* * If there are more than one stream in the configuration figure out the @@ -638,12 +649,18 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() /* * Validate the configuration against the desired path and, if the - * platform supports it, the dewarper. + * platform supports it, the dewarper. While iterating over the + * configurations collect the smallest common sensor format. */ + Size accumulatedSensorSize; auto validateConfig = [&](StreamConfiguration &cfg, RkISP1Path *path, Stream *stream, Status expectedStatus) { StreamConfiguration tryCfg = cfg; + /* Need to validate the path before the transpose */ + if (transposeAfterIsp) + tryCfg.size.transpose(); + Status ret = path->validate(sensor, sensorConfig, &tryCfg); if (ret == Invalid) return false; @@ -652,6 +669,8 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() (expectedStatus == Valid && ret == Adjusted)) return false; + Size sensorSize = tryCfg.size; + if (useDewarper) { /* * The dewarper output is independent of the ISP path. @@ -674,6 +693,8 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() cfg = tryCfg; cfg.setStream(stream); + + accumulatedSensorSize = std::max(accumulatedSensorSize, sensorSize); return true; }; @@ -724,13 +745,6 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() return Invalid; } - /* Select the sensor format. */ - Size maxSize; - - for (const StreamConfiguration &cfg : config_) { - maxSize = std::max(maxSize, cfg.size); - } - std::vector mbusCodes; if (isRaw) { @@ -741,7 +755,7 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() [](const auto &value) { return value.second; }); } - sensorFormat_ = sensor->getFormat(mbusCodes, maxSize, + sensorFormat_ = sensor->getFormat(mbusCodes, accumulatedSensorSize, mainPath->maxResolution()); if (sensorFormat_.size.isNull()) @@ -776,6 +790,22 @@ PipelineHandlerRkISP1::generateConfiguration(Camera *camera, if (roles.empty()) return config; + Transform transform = Transform::Identity; + Size previewSize = kRkISP1PreviewSize; + bool transposeAfterIsp = false; + if (data->canUseDewarper_) { + transform = Orientation::Rotate0 / data->sensor_->mountingOrientation(); + if (!!(transform & Transform::Transpose)) + transposeAfterIsp = true; + } + + /* + * In case of a transpose transform we need to create a path for the + * transposed size. + */ + if (transposeAfterIsp) + previewSize.transpose(); + /* * As the ISP can't output different color spaces for the main and self * path, pick a sensible default color space based on the role of the @@ -804,7 +834,7 @@ PipelineHandlerRkISP1::generateConfiguration(Camera *camera, if (!colorSpace) colorSpace = ColorSpace::Sycc; - size = kRkISP1PreviewSize; + size = previewSize; break; case StreamRole::VideoRecording: @@ -812,7 +842,7 @@ PipelineHandlerRkISP1::generateConfiguration(Camera *camera, if (!colorSpace) colorSpace = ColorSpace::Rec709; - size = kRkISP1PreviewSize; + size = previewSize; break; case StreamRole::Raw: @@ -854,6 +884,9 @@ PipelineHandlerRkISP1::generateConfiguration(Camera *camera, if (!cfg.pixelFormat.isValid()) return nullptr; + if (transposeAfterIsp && role != StreamRole::Raw) + cfg.size.transpose(); + cfg.colorSpace = colorSpace; cfg.bufferCount = kRkISP1MinBufferCount; config->addConfiguration(cfg); @@ -876,6 +909,19 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) if (ret) return ret; + const PixelFormat &streamFormat = config->at(0).pixelFormat; + const PixelFormatInfo &info = PixelFormatInfo::info(streamFormat); + isRaw_ = info.colourEncoding == PixelFormatInfo::ColourEncodingRAW; + data->usesDewarper_ = data->canUseDewarper_ && !isRaw_; + + Transform transform = config->combinedTransform(); + bool transposeAfterIsp = false; + if (data->usesDewarper_) { + if (!!(transform & Transform::Transpose)) + transposeAfterIsp = true; + transform = Transform::Identity; + } + /* * Configure the format on the sensor output and propagate it through * the pipeline. @@ -885,10 +931,10 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) if (config->sensorConfig) ret = sensor->applyConfiguration(*config->sensorConfig, - config->combinedTransform(), + transform, &format); else - ret = sensor->setFormat(&format, config->combinedTransform()); + ret = sensor->setFormat(&format, transform); if (ret < 0) return ret; @@ -915,10 +961,6 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) << " crop " << inputCrop; Rectangle outputCrop = inputCrop; - const PixelFormat &streamFormat = config->at(0).pixelFormat; - const PixelFormatInfo &info = PixelFormatInfo::info(streamFormat); - isRaw_ = info.colourEncoding == PixelFormatInfo::ColourEncodingRAW; - data->usesDewarper_ = dewarper_ && !isRaw_; /* YUYV8_2X8 is required on the ISP source path pad for YUV output. */ if (!isRaw_) @@ -1013,11 +1055,14 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) if (ret) return ret; + dewarper_->setTransform(cfg.stream(), config->combinedTransform()); /* * Apply a default scaler crop that keeps the * aspect ratio. */ Size size = cfg.size; + if (transposeAfterIsp) + size.transpose(); size = sensorCrop.size().boundedToAspectRatio(size); ControlList ctrls; From patchwork Tue Nov 25 16:28:36 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 25203 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C5784C3338 for ; Tue, 25 Nov 2025 16:30:08 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7815460AA9; Tue, 25 Nov 2025 17:30:08 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="UyOYPc9l"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BD4CB60A9E for ; Tue, 25 Nov 2025 17:30:06 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bae1:340c:573c:570b]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id B8ACC13DE; Tue, 25 Nov 2025 17:27:57 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1764088077; bh=kpwkVK/+OFcGiXaPGaOG/FeTA2AsiOHaMDK2yIySh4M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UyOYPc9luX0ZcI3MqVhGqb4fFfAp0noiptR050mGGz8BAz6YlBh01NpgzRZtfO+ZG qTwfnYfLnDnqphl7Ypd7pQdWjV3dHkIDvl73awMmWp3Vm6vzSG+xNI/28DCOfoMBWw blGtqba9acDyOxm4I8/7umtiH1AACpzURohXsfdo= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Paul Elder Subject: [PATCH v3 24/29] libcamera: dw100_vertexmap: Implement parametric dewarping Date: Tue, 25 Nov 2025 17:28:36 +0100 Message-ID: <20251125162851.2301793-25-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> References: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement functions to allow lens dewarping based on the common lens dewarp model used e.g. by OpenCV. See https://docs.opencv.org/4.12.0/d9/d0c/group__calib3d.html for an in depth explanation of the parameters. Signed-off-by: Stefan Klug Reviewed-by: Paul Elder Reviewed-by: Kieran Bingham --- Changes in v3: - Fixed typo in docs - Collected tag Changes in v2: - Adapt to vertex map implementation based on affine transforms - Improved documentation - dropped loadDewarpParams function as it was never used --- .../converter/converter_dw100_vertexmap.h | 16 +++ .../converter/converter_dw100_vertexmap.cpp | 98 +++++++++++++++++-- 2 files changed, 108 insertions(+), 6 deletions(-) diff --git a/include/libcamera/internal/converter/converter_dw100_vertexmap.h b/include/libcamera/internal/converter/converter_dw100_vertexmap.h index 428b3d74d4d2..49ecc81678a0 100644 --- a/include/libcamera/internal/converter/converter_dw100_vertexmap.h +++ b/include/libcamera/internal/converter/converter_dw100_vertexmap.h @@ -10,6 +10,9 @@ #include #include +#include "libcamera/internal/matrix.h" +#include "libcamera/internal/vector.h" + namespace libcamera { class Dw100VertexMap @@ -50,9 +53,17 @@ public: void setMode(const ScaleMode mode) { mode_ = mode; } ScaleMode mode() const { return mode_; } + int setDewarpParams(const Matrix &cm, const Span &coeffs); + bool dewarpParamsValid() { return dewarpParamsValid_; } + + void setLensDewarpEnable(bool enable) { lensDewarpEnable_ = enable; } + bool lensDewarpEnable() { return lensDewarpEnable_; } + std::vector getVertexMap(); private: + Vector dewarpPoint(const Vector &p); + Rectangle scalerCrop_; Rectangle sensorCrop_; Transform transform_ = Transform::Identity; @@ -66,6 +77,11 @@ private: double effectiveScaleY_; Point effectiveOffset_; Rectangle effectiveScalerCrop_; + + Matrix dewarpM_ = Matrix::identity(); + std::array dewarpCoeffs_; + bool lensDewarpEnable_ = true; + bool dewarpParamsValid_ = false; }; } /* namespace libcamera */ diff --git a/src/libcamera/converter/converter_dw100_vertexmap.cpp b/src/libcamera/converter/converter_dw100_vertexmap.cpp index 427d710743b8..b6880ba40a20 100644 --- a/src/libcamera/converter/converter_dw100_vertexmap.cpp +++ b/src/libcamera/converter/converter_dw100_vertexmap.cpp @@ -183,8 +183,6 @@ int dw100VerticesForLength(const int length) * | | | | | Transpose) | | Rotate) | * +-------------+ +-------------+ +------------+ +-----------------+ * - * \todo Lens dewarp is not yet implemented. An identity map is used instead. - * * All parameters are clamped to valid values before creating the vertex map. * * The constrains process works as follows: @@ -544,10 +542,8 @@ std::vector Dw100VertexMap::getVertexMap() p = transformPoint(outputToSensor, p); - /* - * \todo: Transformations in sensor space to be added - * here. - */ + if (dewarpParamsValid_ && lensDewarpEnable_) + p = dewarpPoint(p); p = transformPoint(sensorToInput, p); @@ -561,4 +557,94 @@ std::vector Dw100VertexMap::getVertexMap() return res; } +/** + * \brief Set the dewarp parameters + * \param cm The camera matrix + * \param coeffs The dewarp coefficients + * + * Sets the dewarp parameters according to the commonly used dewarp model. See + * https://docs.opencv.org/4.12.0/d9/d0c/group__calib3d.html for further details + * on the model. The parameter \a coeffs must either hold 4,5,8 or 12 values. + * They represent the parameters k1,k2,p1,p2[,k3[,k4,k5,k6[,s1,s2,s3,s4]]] in + * the model. + * + * \return A negative number on error, 0 otherwise + */ +int Dw100VertexMap::setDewarpParams(const Matrix &cm, + const Span &coeffs) +{ + dewarpM_ = cm; + dewarpCoeffs_.fill(0.0); + + if (coeffs.size() != 4 && coeffs.size() != 5 && + coeffs.size() != 8 && coeffs.size() != 12) { + LOG(Converter, Error) + << "Dewarp 'coefficients' must have 4, 5, 8 or 12 values"; + dewarpParamsValid_ = false; + return -EINVAL; + } + std::copy(coeffs.begin(), coeffs.end(), dewarpCoeffs_.begin()); + + dewarpParamsValid_ = true; + return 0; +} + +/** + * \fn Dw100VertexMap::dewarpParamsValid() + * \brief Returns if the dewarp parameters are valid + * + * \return True if the dewarp parameters are valid, fals otherwise + */ + +/** + * \fn Dw100VertexMap::setLensDewarpEnable() + * \brief Enables or disables lens dewarping + * \param[in] enable Enable or disable lens dewarping + */ + +/** + * \fn Dw100VertexMap::lensDewarpEnable() + * \brief Returns if lens dewarping is enabled + */ + +/** + * \brief Apply dewarp calculation to a point + * \param p The point to dewarp + * + * Applies the dewarp transformation to point \a p according to the commonly + * used dewarp model. See + * https://docs.opencv.org/4.12.0/d9/d0c/group__calib3d.html for further details + * on the model. + * + * \return The dewarped point + */ +Vector2d Dw100VertexMap::dewarpPoint(const Vector2d &p) +{ + double x, y; + double k1 = dewarpCoeffs_[0]; + double k2 = dewarpCoeffs_[1]; + double p1 = dewarpCoeffs_[2]; + double p2 = dewarpCoeffs_[3]; + double k3 = dewarpCoeffs_[4]; + double k4 = dewarpCoeffs_[5]; + double k5 = dewarpCoeffs_[6]; + double k6 = dewarpCoeffs_[7]; + double s1 = dewarpCoeffs_[8]; + double s2 = dewarpCoeffs_[9]; + double s3 = dewarpCoeffs_[10]; + double s4 = dewarpCoeffs_[11]; + + y = (p.y() - dewarpM_[1][2]) / dewarpM_[1][1]; + x = (p.x() - dewarpM_[0][2] - y * dewarpM_[0][1]) / dewarpM_[0][0]; + + double r2 = x * x + y * y; + double d = (1 + k1 * r2 + k2 * r2 * r2 + k3 * r2 * r2 * r2) / + (1 + k4 * r2 + k5 * r2 * r2 + k6 * r2 * r2 * r2); + x = x * d + 2 * p1 * x * y + p2 * (r2 + 2 * x * x) + s1 * r2 + s2 * r2 * r2; + y = y * d + 2 * p2 * x * y + p1 * (r2 + 2 * y * y) + s3 * r2 + s4 * r2 * r2; + + return { { x * dewarpM_[0][0] + y * dewarpM_[0][1] + dewarpM_[0][2], + y * dewarpM_[1][1] + dewarpM_[1][2] } }; +} + } /* namespace libcamera */ From patchwork Tue Nov 25 16:28:37 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 25204 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D6B8CC333C for ; Tue, 25 Nov 2025 16:30:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 825A760AA0; Tue, 25 Nov 2025 17:30:11 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="r2JmcIAu"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DCB6D60BE2 for ; Tue, 25 Nov 2025 17:30:08 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bae1:340c:573c:570b]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id D416518CB; Tue, 25 Nov 2025 17:27:59 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1764088079; bh=ACx7tn6eBPa89EDefqk8pqGvSoIt/yXeJOr5V5kNolg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=r2JmcIAuIDjBKCepPXnYSEObT/6CQD//qM9stFLDDq3shRrh97H0kCwI8ofZTd7ZF ksTtfzk1zMdYbKEcDykPMjsNAxKexL5zrHr4enrs2pZv6/J4oVCI0PC2hIIxlzAcIB cwzAtJJzFlakYmp5CCQKy++VCpMc941h1FBOrRnY= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v3 25/29] pipeline: rkisp1: Load dewarp parameters from tuning file Date: Tue, 25 Nov 2025 17:28:37 +0100 Message-ID: <20251125162851.2301793-26-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> References: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Load the dewarp parameters from the tuning file. Signed-off-by: Stefan Klug Reviewed-by: Kieran Bingham --- Changes in v3: - Moved code into converter module - Added more documentation Changes in v2: - Dropped unused variable --- .../internal/converter/converter_dw100.h | 8 +++ src/libcamera/converter/converter_dw100.cpp | 71 +++++++++++++++++++ src/libcamera/pipeline/rkisp1/rkisp1.cpp | 16 +++-- 3 files changed, 91 insertions(+), 4 deletions(-) diff --git a/include/libcamera/internal/converter/converter_dw100.h b/include/libcamera/internal/converter/converter_dw100.h index 8dd21a6228f1..f0c858689e27 100644 --- a/include/libcamera/internal/converter/converter_dw100.h +++ b/include/libcamera/internal/converter/converter_dw100.h @@ -31,6 +31,8 @@ public: static std::unique_ptr createModule(DeviceEnumerator *enumerator); + int init(const YamlObject ¶ms); + int configure(const StreamConfiguration &inputCfg, const std::vector> &outputCfg); @@ -67,12 +69,18 @@ private: int applyControls(const Stream *stream, const V4L2Request *request); void reinitRequest(V4L2Request *request); + struct DewarpParms { + Matrix cm; + std::vector coeffs; + }; + struct VertexMapInfo { Dw100VertexMap map; bool update; }; std::map vertexMaps_; + std::optional dewarpParams_; unsigned int inputBufferCount_; V4L2M2MConverter converter_; Rectangle sensorCrop_; diff --git a/src/libcamera/converter/converter_dw100.cpp b/src/libcamera/converter/converter_dw100.cpp index cba7cc9f709b..03ffd939dc63 100644 --- a/src/libcamera/converter/converter_dw100.cpp +++ b/src/libcamera/converter/converter_dw100.cpp @@ -70,6 +70,74 @@ ConverterDW100Module::createModule(DeviceEnumerator *enumerator) return {}; } +/** + * \brief Initialize the module with configuration data + * \param[in] params The config parameters + * + * This function shall be called from the pipeline handler to initialize the + * module with the provided parameters. + * + * A typical tuning file entry for the dewarper looks like this: + * \code{.unparsed} + * modules: + * - Dewarp: + * cm: [ + * 1.0, 0.0, 0.0, + * 0.0, 1.0, 0.0, + * 0.0, 0.0, 1.0, + * ] + * coefficients: [ + * 0,0,0,0,0, + * ] + * \endcode + * + * The \a cm and \a coefficients parameters are documented in + * Dw100VertexMap::setDewarpParams() + * + * \sa Dw100VertexMap::setDewarpParams() + * \return 0 if successful, an error code otherwise + */ +int ConverterDW100Module::init(const YamlObject ¶ms) +{ + DewarpParms dp; + + auto &cm = params["cm"]; + auto &coefficients = params["coefficients"]; + + /* If nothing is provided, the dewarper is still functional */ + if (!cm && !coefficients) + return 0; + + if (!cm) { + LOG(Converter, Error) << "Dewarp parameters are missing 'cm' value"; + return -EINVAL; + } + + auto matrix = cm.get>(); + if (!matrix) { + LOG(Converter, Error) << "Failed to load 'cm' value"; + return -EINVAL; + } + + dp.cm = *matrix; + + if (!coefficients) { + LOG(Converter, Error) << "Dewarp parameters are missing 'coefficients' value"; + return -EINVAL; + } + + const auto coeffs = coefficients.getList(); + if (!coeffs) { + LOG(Converter, Error) << "Dewarp parameters 'coefficients' value is not a list"; + return -EINVAL; + } + dp.coeffs = std::move(*coeffs); + + dewarpParams_ = dp; + + return 0; +} + /** * \copydoc libcamera::V4L2M2MConverter::configure */ @@ -93,6 +161,9 @@ int ConverterDW100Module::configure(const StreamConfiguration &inputCfg, vertexMap.setInputSize(inputCfg.size); vertexMap.setOutputSize(outputCfg.size); vertexMap.setSensorCrop(sensorCrop_); + + if (dewarpParams_) + vertexMap.setDewarpParams(dewarpParams_->cm, dewarpParams_->coeffs); info.update = true; } diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 746eeffab207..adce1de35263 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -453,10 +453,18 @@ int RkISP1CameraData::loadTuningFile(const std::string &path) const auto &modules = (*data)["modules"].asList(); for (const auto &module : modules) { - if (module.contains("Dewarp")) { - canUseDewarper_ = true; - break; - } + const auto ¶ms = module["Dewarp"]; + if (!params) + continue; + + ret = pipe()->dewarper_->init(params); + if (ret) + return ret; + + LOG(RkISP1, Info) << "Dw100 dewarper initialized"; + + canUseDewarper_ = true; + return 0; } return 0; From patchwork Tue Nov 25 16:28:38 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 25205 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7811AC3338 for ; Tue, 25 Nov 2025 16:30:13 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 218C360B30; Tue, 25 Nov 2025 17:30:13 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="DojNeDAJ"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DA3B260AD0 for ; Tue, 25 Nov 2025 17:30:11 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bae1:340c:573c:570b]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id E933D6AF; Tue, 25 Nov 2025 17:28:02 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1764088083; bh=Ph1nfg8mvjVX1JdKj+tQ/BpziA69Zqy+s4I6Wt+ey6U=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DojNeDAJ7fnZAIRM3l/nGX+DisYSbW8VR+WIBu7jPd0zHC9jTIjtWOxxWPtE1dnOI L/76HLvu52j79+vOyWoFQC5SqoZG3Gin6zuHXkzYYeKwk19CaVk+VFvN8PrMxkgMLr q8ZVr4qN5T4BAiBgn37Rfzqzsa8So5k7SFZjUEMI= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Paul Elder Subject: [PATCH v3 26/29] libcamera: Add and implement LensDewarpEnable control Date: Tue, 25 Nov 2025 17:28:38 +0100 Message-ID: <20251125162851.2301793-27-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> References: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a LensDewarpEnable control to enable or disable lens dewarping if it is configured. Implement it inside the dw100 converter module. Signed-off-by: Stefan Klug Reviewed-by: Paul Elder Reviewed-by: Kieran Bingham --- Changes in v3: - Moved control handling into converter module - Moved yaml handling into converter module - Collected tag Changes in v2: - Merged control definition and usage into one patch - Improved control description - Fixed actual handling of LensDewarpEnable control that got lost in a previous rebase --- src/libcamera/control_ids_core.yaml | 7 +++++++ src/libcamera/converter/converter_dw100.cpp | 12 ++++++++++++ 2 files changed, 19 insertions(+) diff --git a/src/libcamera/control_ids_core.yaml b/src/libcamera/control_ids_core.yaml index f781865859ac..3bcb475fd102 100644 --- a/src/libcamera/control_ids_core.yaml +++ b/src/libcamera/control_ids_core.yaml @@ -1346,4 +1346,11 @@ controls: reduces the WdrExposureValue until the amount of pixels that are close to saturation is lower than this value. + - LensDewarpEnable: + type: bool + direction: inout + description: | + Enable or disable lens dewarping. This control is only available if lens + dewarp parameters are configured in the tuning file. + ... diff --git a/src/libcamera/converter/converter_dw100.cpp b/src/libcamera/converter/converter_dw100.cpp index 03ffd939dc63..7adad1b502f0 100644 --- a/src/libcamera/converter/converter_dw100.cpp +++ b/src/libcamera/converter/converter_dw100.cpp @@ -343,6 +343,9 @@ void ConverterDW100Module::updateControlInfos(const Stream *stream, ControlInfoM controls[&controls::ScalerCrop] = ControlInfo(Rectangle(sensorCrop_.x, sensorCrop_.y, 1, 1), sensorCrop_, sensorCrop_); + if (dewarpParams_.has_value()) + controls[&controls::LensDewarpEnable] = ControlInfo(false, true, true); + if (!converter_.supportsRequests()) LOG(Converter, Warning) << "dw100 kernel driver has no requests support." @@ -366,6 +369,12 @@ void ConverterDW100Module::setControls(const Stream *stream, const ControlList & auto &info = vertexMaps_[stream]; auto &vertexMap = info.map; + const auto &lensDewarpEnable = controls.get(controls::LensDewarpEnable); + if (lensDewarpEnable) { + vertexMap.setLensDewarpEnable(*lensDewarpEnable); + info.update = true; + } + const auto &crop = controls.get(controls::ScalerCrop); if (crop) { vertexMap.setScalerCrop(*crop); @@ -394,6 +403,9 @@ void ConverterDW100Module::populateMetadata(const Stream *stream, ControlList &m auto &vertexMap = vertexMaps_[stream].map; meta.set(controls::ScalerCrop, vertexMap.effectiveScalerCrop()); + + if (dewarpParams_.has_value()) + meta.set(controls::LensDewarpEnable, vertexMap.lensDewarpEnable()); } /** From patchwork Tue Nov 25 16:28:39 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 25206 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 429DEC3338 for ; Tue, 25 Nov 2025 16:30:16 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id EC44860AD0; Tue, 25 Nov 2025 17:30:15 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="gHnYVq42"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5119560A9E for ; Tue, 25 Nov 2025 17:30:15 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bae1:340c:573c:570b]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 58C6F20E6; Tue, 25 Nov 2025 17:28:06 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1764088086; bh=bhI7RQpAaebY6zxQIfze+sNd3cGZao08ONXsHIuzw0A=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=gHnYVq42Rzz3qDbn87KppTXOq+G7jgtKZng6jl/atIGUXAe4Ca315+uRsBB63+eUv OWcnvGCDXGKXIg4ic4i9IGGM8E/GMlbRys7eKM5oij2+QltPcr3Ad6EkTOCLjLF+qN BCG2BUPo+sF5wa8dTx/cSjzQJKdXJTKhNV+mZJ0Q= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v3 27/29] pipeline: rkisp1: Add warning to unclear format handling Date: Tue, 25 Nov 2025 17:28:39 +0100 Message-ID: <20251125162851.2301793-28-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> References: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" It is not mentioned when the sensorFormat is allowed to have zero size. Add a warning and todo to that case but leave it in place for now. Signed-off-by: Stefan Klug --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index adce1de35263..dc7df22976ff 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -766,8 +766,16 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() sensorFormat_ = sensor->getFormat(mbusCodes, accumulatedSensorSize, mainPath->maxResolution()); - if (sensorFormat_.size.isNull()) + if (sensorFormat_.size.isNull()) { + /* + * \todo When can this happen? Should we return a failure in + * this case? + */ sensorFormat_.size = sensor->resolution(); + LOG(RkISP1, Warning) + << "Failed to select sensor format. Default to " + << sensorFormat_; + } return status; } From patchwork Tue Nov 25 16:28:40 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 25207 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CD921C3338 for ; Tue, 25 Nov 2025 16:30:19 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 797B760C6B; Tue, 25 Nov 2025 17:30:19 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="a3onN/CS"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3958A60A9E for ; Tue, 25 Nov 2025 17:30:18 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bae1:340c:573c:570b]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 4C0DC20E6; Tue, 25 Nov 2025 17:28:09 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1764088089; bh=KaJ8SJFcJP01NIS+AmLdW5zLqpm8qBdeOgGbRh+56xY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=a3onN/CSK42MgwlrPuxrC4Gjo2V81Gs1J3zrUmcP8jHxBX1A9ALsvAmhrRT+JemRy HqeHcIGSAbVgSLeK3r15ATa8xVC9RmFUDyi2W5HUM+t4EaW268IGZexR+Vv5KWADLK eEzX2ENnkrVlPGrcBEua2tP3u/ONAuVxJYE672Kc= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v3 28/29] DNI libcamera: rkisp1: Implement dw100 specific features Date: Tue, 25 Nov 2025 17:28:40 +0100 Message-ID: <20251125162851.2301793-29-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> References: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The dw100 allows more features implemented in the dw100 vertex map. Implement these features for the rkisp1 pipeline. Signed-off-by: Stefan Klug --- Changes in v3: - Move control handling from pipeline handler to the converter class - Moved controls into core namespace Changes in v2: - Marked as DoNotIntegrate because the puplic interface needs further discussions - Added Dw100EffectiveScale metadata --- .../converter/converter_dw100_vertexmap.h | 2 +- src/libcamera/control_ids_core.yaml | 50 +++++++++++++++++++ src/libcamera/converter/converter_dw100.cpp | 36 +++++++++++++ 3 files changed, 87 insertions(+), 1 deletion(-) diff --git a/include/libcamera/internal/converter/converter_dw100_vertexmap.h b/include/libcamera/internal/converter/converter_dw100_vertexmap.h index 49ecc81678a0..f79274a6098a 100644 --- a/include/libcamera/internal/converter/converter_dw100_vertexmap.h +++ b/include/libcamera/internal/converter/converter_dw100_vertexmap.h @@ -42,7 +42,7 @@ public: const Transform &transform() const { return transform_; } void setScale(const float scale) { scale_ = scale; } - float effectiveScale() const { return (effectiveScaleX_ + effectiveScaleY_) * 0.5; } + std::array effectiveScale() const { return { static_cast(effectiveScaleX_), static_cast(effectiveScaleY_) }; } void setRotation(const float rotation) { rotation_ = rotation; } float rotation() const { return rotation_; } diff --git a/src/libcamera/control_ids_core.yaml b/src/libcamera/control_ids_core.yaml index 3bcb475fd102..e03b570282ba 100644 --- a/src/libcamera/control_ids_core.yaml +++ b/src/libcamera/control_ids_core.yaml @@ -1353,4 +1353,54 @@ controls: Enable or disable lens dewarping. This control is only available if lens dewarp parameters are configured in the tuning file. + - Dw100ScaleMode: + type: int32_t + direction: inout + description: | + Scale mode of the dewarper. + enum: + - name: Dw100ScaleModeFill + value: 0 + description: | + Fills the given output size with the largest rectangle possible. + Aspect ratio is not preserved. Dw100Scale is ignored. + - name: Dw100ScaleModeCrop + value: 1 + description: | + Crops to the given output size. The scale factor can be specified + using Dw100Scale. Aspect ratio is preserved. + + - Dw100Scale: + type: float + direction: inout + description: | + Scale factor applied to the image when Dw100ScaleMode is set to Crop. + This value is clamped, so that all pixels have valid input data. + Therefore a value of 0 always provides the largest possible field of + view. Scaling happens centered around the center of the ScalerCrop + + Dw10Offset. + + - Dw100Rotation: + type: float + direction: inout + description: | + Rotates the image by the given angle in degrees in clockwise direction + around the center of ScalerCrop + Dw100Offset + + - Dw100Offset: + type: Point + direction: inout + description: | + Moves the image by the given values in x and y direction in sensor + coordinate space. This is clamped, so that all output pixels contain + valid data. The offset is therefore ignored when the Dw100Scale value is + too small. + + - Dw100EffectiveScale: + type: float + direction: out + description: | + Effective scale + size: [2] + ... diff --git a/src/libcamera/converter/converter_dw100.cpp b/src/libcamera/converter/converter_dw100.cpp index 7adad1b502f0..b39e4ab23cd6 100644 --- a/src/libcamera/converter/converter_dw100.cpp +++ b/src/libcamera/converter/converter_dw100.cpp @@ -346,6 +346,13 @@ void ConverterDW100Module::updateControlInfos(const Stream *stream, ControlInfoM if (dewarpParams_.has_value()) controls[&controls::LensDewarpEnable] = ControlInfo(false, true, true); + controls[&controls::Dw100Scale] = ControlInfo(0.2f, 8.0f, 1.0f); + controls[&controls::Dw100Rotation] = ControlInfo(-180.0f, 180.0f, 0.0f); + controls[&controls::Dw100Offset] = + ControlInfo(Point(-10000, -10000), Point(10000, 10000), Point(0, 0)); + controls[&controls::Dw100ScaleMode] = + ControlInfo(controls::Dw100ScaleModeValues, controls::Dw100ScaleModeFill); + if (!converter_.supportsRequests()) LOG(Converter, Warning) << "dw100 kernel driver has no requests support." @@ -369,6 +376,30 @@ void ConverterDW100Module::setControls(const Stream *stream, const ControlList & auto &info = vertexMaps_[stream]; auto &vertexMap = info.map; + const auto &scale = controls.get(controls::Dw100Scale); + if (scale) { + vertexMap.setScale(*scale); + info.update = true; + } + + const auto &rotation = controls.get(controls::Dw100Rotation); + if (rotation) { + vertexMap.setRotation(*rotation); + info.update = true; + } + + const auto &offset = controls.get(controls::Dw100Offset); + if (offset) { + vertexMap.setOffset(*offset); + info.update = true; + } + + const auto &scaleMode = controls.get(controls::Dw100ScaleMode); + if (scaleMode) { + vertexMap.setMode(static_cast(*scaleMode)); + info.update = true; + } + const auto &lensDewarpEnable = controls.get(controls::LensDewarpEnable); if (lensDewarpEnable) { vertexMap.setLensDewarpEnable(*lensDewarpEnable); @@ -402,6 +433,11 @@ void ConverterDW100Module::populateMetadata(const Stream *stream, ControlList &m auto &vertexMap = vertexMaps_[stream].map; + std::array effectiveScale = vertexMap.effectiveScale(); + meta.set(controls::Dw100EffectiveScale, effectiveScale); + meta.set(controls::Dw100Scale, (effectiveScale[0] + effectiveScale[1]) / 2.0); + meta.set(controls::Dw100Rotation, vertexMap.rotation()); + meta.set(controls::Dw100Offset, vertexMap.effectiveOffset()); meta.set(controls::ScalerCrop, vertexMap.effectiveScalerCrop()); if (dewarpParams_.has_value()) From patchwork Tue Nov 25 16:28:41 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 25208 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B3764C3338 for ; Tue, 25 Nov 2025 16:30:22 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4CFCF60C6B; Tue, 25 Nov 2025 17:30:22 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="r3FHwVVU"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9D6FB60AA0 for ; Tue, 25 Nov 2025 17:30:20 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bae1:340c:573c:570b]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id ABBDC13DE; Tue, 25 Nov 2025 17:28:11 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1764088091; bh=Ej4n1s7zWgywj6UeXPKqrRbpzpaHW3RxX4rYZ0woPQM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=r3FHwVVUBkVLgjbNJW5yyikyoh36+Rf+z33yfp+X/XznzByZGh7NBGMk5ieLJS8cM wqhUkdPh8o/dNUcQGpG6W5c/GQLAIxWYnu0ts4uqSqc314tQl5d3FyhXl7vLQ3HqAV 2PXhgIXvAM+XKbgjeN2EZRsSm4XJePd89c3+WK+I= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v3 29/29] DNI pipeline: rkisp1: Workaround for customer kernels without requests Date: Tue, 25 Nov 2025 17:28:41 +0100 Message-ID: <20251125162851.2301793-30-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> References: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" There are a few kernels out there that have dynamic vertex map support, but no requests support. Allow full dewarp usage on these. Signed-off-by: Stefan Klug --- src/libcamera/converter/converter_dw100.cpp | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/src/libcamera/converter/converter_dw100.cpp b/src/libcamera/converter/converter_dw100.cpp index b39e4ab23cd6..22a15a368e38 100644 --- a/src/libcamera/converter/converter_dw100.cpp +++ b/src/libcamera/converter/converter_dw100.cpp @@ -33,6 +33,17 @@ LOG_DECLARE_CATEGORY(Converter) * and it has direct support for libcamera controls. */ +namespace { + +/* + * This flag allows to use dynamic dewarp maps to support pan, zoom, rotate when + * the kernel driver doesn't support requests. Only needed for legacy customer + * kernels. + */ +static constexpr bool kAllowDynamicDewarpMapsWithoutRequests = true; + +} /* namespace */ + ConverterDW100Module::ConverterDW100Module(std::shared_ptr media) : converter_(media), running_(false) { @@ -353,7 +364,7 @@ void ConverterDW100Module::updateControlInfos(const Stream *stream, ControlInfoM controls[&controls::Dw100ScaleMode] = ControlInfo(controls::Dw100ScaleModeValues, controls::Dw100ScaleModeFill); - if (!converter_.supportsRequests()) + if (!converter_.supportsRequests() && !kAllowDynamicDewarpMapsWithoutRequests) LOG(Converter, Warning) << "dw100 kernel driver has no requests support." " Dynamic configuration is not possible.";