From patchwork Tue Nov 25 16:28:12 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 25179 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0B887C333C for ; Tue, 25 Nov 2025 16:28:58 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0DF2860A9E; Tue, 25 Nov 2025 17:28:57 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="H6w9iVWA"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1894C609D8 for ; Tue, 25 Nov 2025 17:28:56 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bae1:340c:573c:570b]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id BFA43524; Tue, 25 Nov 2025 17:26:46 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1764088006; bh=QbIUjPjETjDnPJYgCfyizhf9a3girFicclscWh4I5G4=; h=From:To:Cc:Subject:Date:From; b=H6w9iVWABgijF9YMrPSSSCQpFCdf9KsJYE+X2xeC4dLIGuN8DolgKdYenNc/SmIv2 T0OrX8DZOTfGTUuhjCyu5y/Rg1dlyYcxgpdrvDBKfRyOYGk4XVH9wd3R1hbAxbe4W1 SfSDmCii+iBJrI41jjDsCTlPgtMxaaLS7e/QlRtg= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v3 00/29] Full dewarper support on imx8mp Date: Tue, 25 Nov 2025 17:28:12 +0100 Message-ID: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.51.0 MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Hi all, The imx8mp has a very flexible dewarper that can be used to implement cropping, digital zoom, pan, rotate, flip and actual lens dewarping. This series implements all these features for libcamera. Control wise, the functionality is split into multiple aspects. ScalerCrop is now implemented using the vertex map, because that provides pixel perfect control for arbitrary rectangles which is an improvement over the current solution that only supports a limited set of scaling factors. On top of that a few specific Dw100 specific controls are added to implement the other functions. Lens dewarp is based on calibration data in the tuning file and can be enabled/disabled at runtime. As prerequisite it requires an updated kernel driver that supports programming the vertex map at runtime. The corresponding implementation is well underway, but not yet upstream. You can find the current version of the kernel driver here: https://git.uk.ideasonboard.com/sklug/linux/src/branch/sklug/v6.16/topic/dw100-v2 Changes in v3: In v3 all the dewarper specifics like controls handling and request handling that were entangled within the rkisp1 pipeline handler got moved into a new ConverterDW100Module class. This way, the code got more modular and easier to use in a different pipeline. Aside from this big rewrite, a few bugs were fixed and a lot of review comments addressed. Please see the individual logs for details. Changes in v2: In the discussions around v2 it got clear that the extended controls need further discussions and can not be merged on mainline as is. I moved them to the end of the series and marked them with DNI. The rest of the series is valuable in itself as it brings pixel perfect ScalerCrop and lens dewarping support without changing much on the controls side. The vertexmap code was rewritten to use affine transforms and matrices insted of manual transformation. This makes the code way cleaner and was on my todo list anyways. In the orientation handling a bug was fixed if the transformation involved a transpose. This bug was visible by a camera configuration that validated successfully, but the camera failed to configure. Also a lot of documentation improvements were done and the series passes CI successfully: https://gitlab.freedesktop.org/camera/libcamera/-/pipelines/1533035 For smaller fixes please see the changelogs on the individual patches. Patch summary: Patches 1-2 replaces plain pointers to media devices by shared pointers. Patches 3-5 contain more plumbing and add support for V4L2 requests in libcamera. Patches 7-27 contain the actual implementation and I guess it is best to read the individual patches. Patch 28-29 shall not be mainlined and is only there to keep compatibility with prior versions of the kernel driver that didn't use V4L2 requests and to show the extended controls that are possible with the dw100. Best regards, Stefan Kieran Bingham (1): libcamera: pipeline: utilise shared MediaDevice pointers Stefan Klug (26): libcamera: converter: Utilise shared MediaDevice pointers libcamera: Add support for V4L2 requests libcamera: converter: Add V4L2 request support libcamera: converter_v4l2_m2m: Add suport for V4L2 requests libcamera: converter_v4l2_m2m: Always set stride libcamera: rkisp1: Properly cancel buffers in dewarp case libcamera: converter_v4l2_m2m: Add debug logging for formats libcamera: rkisp1: Move useDewarper_ flag into RkISP1CameraData libcamera: rkisp1: Scale down in dewarper instead of resizer libcamera: rkisp1: Allow upscaling when the dewarper is present pipeline: rkisp1: Fix number of buffers imported into the dewarper libcamera: converter: Add dw100 vertex map class libcamera: converter: Add dw100 converter module libcamera: rkisp1: Use the dw100 converter module instead of the generic v4l2 converter pipeline: rksip1: Move isRaw up in scope pipeline: rkisp1: Drop rawFormat variable pipeline: rkisp1: Enable the dewarper based on the tuning file libcamera: internal: camera_sensor: Add accessor for mountingOrientation_ libcamera: Add transpose() function to size libcamera: rkisp1: Handle requested orientation using dewarper libcamera: dw100_vertexmap: Implement parametric dewarping pipeline: rkisp1: Load dewarp parameters from tuning file libcamera: Add and implement LensDewarpEnable control pipeline: rkisp1: Add warning to unclear format handling DNI libcamera: rkisp1: Implement dw100 specific features DNI pipeline: rkisp1: Workaround for customer kernels without requests Umang Jain (1): libcamera: converter_v4l2_m2m: Add helper to apply controls Xavier Roumegue (1): include: linux: Update headers for Dw100 dewarper engine include/libcamera/geometry.h | 6 + include/libcamera/internal/camera_sensor.h | 1 + include/libcamera/internal/converter.h | 13 +- .../internal/converter/converter_dw100.h | 93 +++ .../converter/converter_dw100_vertexmap.h | 87 +++ .../internal/converter/converter_v4l2_m2m.h | 20 +- .../libcamera/internal/converter/meson.build | 2 + include/libcamera/internal/media_device.h | 8 + include/libcamera/internal/meson.build | 1 + include/libcamera/internal/pipeline_handler.h | 8 +- include/libcamera/internal/v4l2_device.h | 5 +- include/libcamera/internal/v4l2_request.h | 44 ++ include/libcamera/internal/v4l2_videodevice.h | 3 +- include/linux/dw100.h | 14 + src/libcamera/control_ids_core.yaml | 57 ++ src/libcamera/converter.cpp | 8 +- src/libcamera/converter/converter_dw100.cpp | 543 +++++++++++++++ .../converter/converter_dw100_vertexmap.cpp | 650 ++++++++++++++++++ .../converter/converter_v4l2_m2m.cpp | 87 ++- src/libcamera/converter/meson.build | 2 + src/libcamera/geometry.cpp | 10 + src/libcamera/media_device.cpp | 47 ++ src/libcamera/meson.build | 1 + src/libcamera/pipeline/imx8-isi/imx8-isi.cpp | 8 +- src/libcamera/pipeline/ipu3/cio2.cpp | 4 +- src/libcamera/pipeline/ipu3/cio2.h | 2 +- src/libcamera/pipeline/ipu3/imgu.cpp | 15 +- src/libcamera/pipeline/ipu3/imgu.h | 4 +- src/libcamera/pipeline/ipu3/ipu3.cpp | 4 +- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 16 +- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 451 +++++++----- src/libcamera/pipeline/rkisp1/rkisp1_path.cpp | 6 +- src/libcamera/pipeline/rkisp1/rkisp1_path.h | 2 +- .../pipeline/rpi/common/pipeline_base.cpp | 6 +- .../pipeline/rpi/common/pipeline_base.h | 9 +- src/libcamera/pipeline/rpi/pisp/pisp.cpp | 10 +- src/libcamera/pipeline/rpi/vc4/vc4.cpp | 13 +- src/libcamera/pipeline/simple/simple.cpp | 15 +- src/libcamera/pipeline/uvcvideo/uvcvideo.cpp | 6 +- src/libcamera/pipeline/vimc/vimc.cpp | 14 +- src/libcamera/pipeline_handler.cpp | 14 +- src/libcamera/sensor/camera_sensor.cpp | 7 + src/libcamera/sensor/camera_sensor_legacy.cpp | 1 + src/libcamera/sensor/camera_sensor_raw.cpp | 1 + src/libcamera/v4l2_device.cpp | 30 +- src/libcamera/v4l2_request.cpp | 128 ++++ src/libcamera/v4l2_videodevice.cpp | 10 +- test/geometry.cpp | 5 + test/libtest/buffer_source.cpp | 3 +- 49 files changed, 2219 insertions(+), 275 deletions(-) create mode 100644 include/libcamera/internal/converter/converter_dw100.h create mode 100644 include/libcamera/internal/converter/converter_dw100_vertexmap.h create mode 100644 include/libcamera/internal/v4l2_request.h create mode 100644 include/linux/dw100.h create mode 100644 src/libcamera/converter/converter_dw100.cpp create mode 100644 src/libcamera/converter/converter_dw100_vertexmap.cpp create mode 100644 src/libcamera/v4l2_request.cpp