From patchwork Thu Oct 23 14:48:01 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24728 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0E97EBE080 for ; Thu, 23 Oct 2025 14:48:49 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D1171606DE; Thu, 23 Oct 2025 16:48:47 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="SAr8IBJd"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 99B26606DE for ; Thu, 23 Oct 2025 16:48:46 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7328:357b:4ce1:72b6]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 556701127; Thu, 23 Oct 2025 16:47:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761230821; bh=bYx8QUiPTB5WIy7Awqh6DjrL4IR7Qv2IpYJdmc6QadY=; h=From:To:Cc:Subject:Date:From; b=SAr8IBJd2G3ShtfaqkPZGJTwxCfDEOwPWaMMQLIaYSjTmne3pX374FyztxqTAdDo8 9x1+6Z11/q8xP3Wp9xbTa6o1364RWQss9eq/KiXtw6kH7bauFhA1kQyxF+jYw7hueo ZSd5xM6IM+Gm3zksuKh1wdkdvNcE8iWPdAw3VarM= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 00/35] Full dewarper support on imx8mp Date: Thu, 23 Oct 2025 16:48:01 +0200 Message-ID: <20251023144841.403689-1-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Hi all, the imx8mp has a very flexible dewarper that can be used to implement cropping, digital zoom, pan, rotate, flip and actual lens dewarping. This series implements all these features for libcamera. Control wise, the functionality is split into multiple aspects. ScalerCrop is now implemented using the vertex map, because that provides pixel perfect control for arbitrary rectangles which is an improvement over the current solution that only supports a limited set of scaling factors. On top of that a few specific Dw100 specific controls are added to implement the other functions. Lens dewarp is based on calibration data in the tuning file and can be enabled/disabled at runtime. As prerequisite it requires an updated kernel driver that supports programming the vertex map at runtime. The corresponding implementation is well underway, but not yet upstream. You can find the current version of the kernel driver here: https://git.uk.ideasonboard.com/sklug/linux/src/branch/sklug/v6.16/topic/dw100-v2 Changes in v2: In the discussions around v2 it got clear that the extended controls need further discussions and can not be merged on mainline as is. I moved them to the end of the series and marked them with DNI. The rest of the series is valuable in itself as it brings pixel perfect ScalerCrop and lens dewarping support without changing much on the controls side. The vertexmap code was rewritten to use affine transforms and matrices insted of manual transformation. This makes the code way cleaner and was on my todo list anyways. In the orientation handling a bug was fixed if the transformation involved a transpose. This bug was visible by a camera configuration that validated successfully, but the camera failed to configure. Also a lot of documentation improvements were done and the series passes CI successfully: https://gitlab.freedesktop.org/camera/libcamera/-/pipelines/1533035 For smaller fixes please see the changelogs on the individual patches. Patch summary: Patches 1-3 replace plain pointers to media devices by shared pointers. The patches were already posted here: https://patchwork.libcamera.org/project/libcamera/list/?series=5113 and review should continue there. Patches 4-9 contain more plumbing and add support for V4L2 requests in libcamera. Patches 10-33 contain the actual implementation and I guess it is best to read the individual patches. Patch 34-35 shall not be mainlined and is only there to keep compatibility with prior versions of the kernel driver that didn't use V4L2 requests and to show the extended controls that are possible with the dw100. Best regards, Stefan Kieran Bingham (3): libcamera: v4l2: Support fromEntityName with shared_ptr libcamera: pipeline: utilise shared MediaDevice pointers libcamera: v4l2: Remove fromEntityName(MediaDevice*) Stefan Klug (29): libcamera: converter: Utilise shared MediaDevice pointers libcamera: Add support for V4L2 requests libcamera: converter: Add V4L2 request support libcamera: converter_v4l2_m2m: Add suport for V4L2 requests libcamera: converter_v4l2_m2m: Always set stride pipeline: rkisp1: Use V4L2 requests for the dewarper libcamera: rkisp1: Properly cancel buffers in dewarp case libcamera: converter_v4l2_m2m: Add debug logging for formats libcamera: rkisp1: Move useDewarper_ flag into RkISP1CameraData libcamera: rkisp1: Scale down in dewarper instead of resizer libcamera: rkisp1: Allow upscaling when the dewarper is present libcamera: converter: converter_v4l2_m2m: Add makeStream() function libcamera: converter_m2m: Make some parts protected libcamera: converter: Add dw100 vertex map class libcamera: converter_dw100: Use vertex map libcamera: rkisp1: Use the dw100 class instead of the generic v4l2 converter libcamera: rkisp1: Use vertex map to implement ScalerCrop pipeline: rksip1: Move isRaw up in scope pipeline: rkisp1: Drop rawFormat variable pipeline: rkisp1: Enable the dewarper based on the tuning file libcamera: internal: camera_sensor: Add accessor for mountingOrientation_ libcamera: Add transpose() function to size libcamera: rkisp1: Handle requested orientation using dewarper libcamera: dw100_vertexmap: Implement parametric dewarping pipeline: rkisp1: Load dewarp parameters from tuning file libcamera: Add and implement LensDewarpEnable control pipeline: rkisp1: Add warning to unclear format handling DNI libcamera: rkisp1: Implement dw100 specific features DNI pipeline: rkisp1: Workaround for customer kernels without requests Umang Jain (2): libcamera: converter_v4l2_m2m: Add helper to apply controls libcamera: converter: Add dw100 converter class Xavier Roumegue (1): include: linux: Update headers for Dw100 dewarper engine include/libcamera/geometry.h | 6 + include/libcamera/internal/camera_sensor.h | 1 + include/libcamera/internal/converter.h | 13 +- .../internal/converter/converter_dw100.h | 43 ++ .../converter/converter_dw100_vertexmap.h | 92 +++ .../internal/converter/converter_v4l2_m2m.h | 29 +- .../libcamera/internal/converter/meson.build | 2 + include/libcamera/internal/media_device.h | 7 + include/libcamera/internal/meson.build | 1 + include/libcamera/internal/pipeline_handler.h | 8 +- include/libcamera/internal/v4l2_device.h | 5 +- include/libcamera/internal/v4l2_request.h | 49 ++ include/libcamera/internal/v4l2_subdevice.h | 2 +- include/libcamera/internal/v4l2_videodevice.h | 5 +- include/linux/dw100.h | 14 + src/ipa/libipa/module.h | 4 + src/libcamera/control_ids_core.yaml | 7 + src/libcamera/control_ids_draft.yaml | 45 ++ src/libcamera/converter.cpp | 8 +- src/libcamera/converter/converter_dw100.cpp | 111 +++ .../converter/converter_dw100_vertexmap.cpp | 652 ++++++++++++++++++ .../converter/converter_v4l2_m2m.cpp | 202 +++++- src/libcamera/converter/meson.build | 2 + src/libcamera/geometry.cpp | 10 + src/libcamera/media_device.cpp | 47 ++ src/libcamera/meson.build | 1 + src/libcamera/pipeline/imx8-isi/imx8-isi.cpp | 2 +- src/libcamera/pipeline/ipu3/cio2.cpp | 2 +- src/libcamera/pipeline/ipu3/cio2.h | 2 +- src/libcamera/pipeline/ipu3/imgu.cpp | 3 +- src/libcamera/pipeline/ipu3/imgu.h | 4 +- src/libcamera/pipeline/ipu3/ipu3.cpp | 4 +- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 2 +- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 564 +++++++++++---- src/libcamera/pipeline/rkisp1/rkisp1_path.cpp | 2 +- src/libcamera/pipeline/rkisp1/rkisp1_path.h | 2 +- .../pipeline/rpi/common/pipeline_base.cpp | 6 +- .../pipeline/rpi/common/pipeline_base.h | 9 +- src/libcamera/pipeline/rpi/pisp/pisp.cpp | 10 +- src/libcamera/pipeline/rpi/vc4/vc4.cpp | 13 +- src/libcamera/pipeline/simple/simple.cpp | 15 +- src/libcamera/pipeline/uvcvideo/uvcvideo.cpp | 6 +- src/libcamera/pipeline/vimc/vimc.cpp | 6 +- src/libcamera/pipeline_handler.cpp | 14 +- src/libcamera/sensor/camera_sensor.cpp | 7 + src/libcamera/sensor/camera_sensor_legacy.cpp | 1 + src/libcamera/sensor/camera_sensor_raw.cpp | 1 + src/libcamera/v4l2_device.cpp | 28 +- src/libcamera/v4l2_request.cpp | 128 ++++ src/libcamera/v4l2_subdevice.cpp | 2 +- src/libcamera/v4l2_videodevice.cpp | 12 +- test/delayed_controls.cpp | 2 +- test/libtest/buffer_source.cpp | 2 +- 53 files changed, 1992 insertions(+), 223 deletions(-) create mode 100644 include/libcamera/internal/converter/converter_dw100.h create mode 100644 include/libcamera/internal/converter/converter_dw100_vertexmap.h create mode 100644 include/libcamera/internal/v4l2_request.h create mode 100644 include/linux/dw100.h create mode 100644 src/libcamera/converter/converter_dw100.cpp create mode 100644 src/libcamera/converter/converter_dw100_vertexmap.cpp create mode 100644 src/libcamera/v4l2_request.cpp