From patchwork Tue Sep 30 12:26:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24497 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A2E1EC328C for ; Tue, 30 Sep 2025 12:29:48 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 73FDD6B5F0; Tue, 30 Sep 2025 14:29:47 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="UQs0lQLR"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5332469367 for ; Tue, 30 Sep 2025 14:29:45 +0200 (CEST) Received: from ideasonboard.com (unknown [94.31.94.171]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id E50A1220; Tue, 30 Sep 2025 14:28:16 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1759235297; bh=xuy9oppEIGd/AUxS7BACUQVZ6LpiJlvRcmmCs7s6KRM=; h=From:To:Cc:Subject:Date:From; b=UQs0lQLR7idbh8mmy3tMrxM17V8K0mNReNMY1TCBY/4HYNR+JjbKOPdG5qcLFoRNH J+8t7wawAySYEtH+Cro7P/6zk/vTZgyT9g38iTh4xTDOFV1e8RFhvuwXk6bhgP08Jt h6pCQ3JVlV3PBzQQGQbZpnroQ3PFTBLpFk/jmEoU= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 00/33] Full dewarper support on imx8mp Date: Tue, 30 Sep 2025 14:26:21 +0200 Message-ID: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Hi all, the imx8mp has a very flexible dewarper that can be used to implement cropping, digital zoom, pan, rotate, flip and actual lens dewarping. This series implements all these features for libcamera. Control wise, the functionality is split into multiple aspects. ScalerCrop is now implemented using the vertex map, because that provides pixel perfect control for arbitrary rectangles which is an improvement over the current solution that only supports a limited set of scaling factors. On top of that a few specific Dw100 specific controls are added to implement the other functions. Lens dewarp is based on calibration data in the tuning file and can be enabled/disabled at runtime. As prerequisite it requires an updated kernel driver that supports programming the vertex map at runtime. The corresponding implementation is well underway, but not yet upstream. You can find the current version of the kernel driver here: https://git.uk.ideasonboard.com/sklug/linux/src/branch/sklug/v6.16/topic/dw100-v2 Now to the series. Patches 2-4 replace plain pointers to media devices by shared pointers. The patches were already posted here: https://patchwork.libcamera.org/project/libcamera/list/?series=5113 and review should continue there. Patches 5-10 contain more plumbing and add support for V4L2 requests in libcamera. Patches 11-32 contain the actual implementation and I guess it is best to read the individual patches. Patch 33 shall not be mainlined and is only there to keep compatibility with prior versions of the kernel driver that didn't use V4L2 requests. As always there are things to improve but functionality wise it is well tested. So I think it is a good time to get more eyes on the code. Best regards, Stefan Kieran Bingham (4): libcamera: rkisp1: Only connect delayed controls at start/stop libcamera: v4l2: Support fromEntityName with shared_ptr libcamera: pipeline: utilise shared MediaDevice pointers libcamera: v4l2: Remove fromEntityName(MediaDevice*) Stefan Klug (26): libcamera: converter: Utilise shared MediaDevice pointers libcamera: Add support for V4L2 requests libcamera: converter: Add V4L2 request support libcamera: converter_v4l2_m2m: Add suport for V4L2 requests libcamera: converter_v4l2_m2m: Always set stride pipeline: rkisp1: Use V4L2 requests for the dewarper libcamera: rkisp1: Properly cancel buffers in dewarp case libcamera: converter_v4l2_m2m: Add debug logging for formats libcamera: rkisp1: Move useDewarper_ flag into RkISP1CameraData libcamera: rkisp1: Scale down in dewarper instead of resizer libcamera: rkisp1: Allow upscaling when the dewarper is present libcamera: converter: converter_v4l2_m2m: Add makeStream() function libcamera: converter: Add dw100 vertex map class libcamera: converter_dw100: Use vertex map libcamera: rkisp1: Use the dw100 class instead of the generic v4l2 converter libcamera: rkisp1: Implement dw100 specific features libcamera: rkisp1: Use vertex map to implement ScalerCrop libcamera: internal: camera_sensor: Add accessor for mountingOrientation_ libcamera: Add transpose() function to size libcamera: rkisp1: Handle requested orientation using dewarper libcamera: dw100_vertexmap: Implement parametric dewarping pipeline: rkisp1: Enable the dewarper based on the tuning file pipeline: rkisp1: Load dewarp parameters from tuning file libcamera: Add LensDewarpEnable control libcamera: rkisp1: Implement LensDewarpEnable control DNI pipeline: rkisp1: Workaround for customer kernels without requests Umang Jain (2): libcamera: converter_v4l2_m2m: Add helper to apply controls libcamera: converter: Add dw100 converter class Xavier Roumegue (1): include: linux: Update headers for Dw100 dewarper engine include/libcamera/geometry.h | 6 + include/libcamera/internal/camera_sensor.h | 1 + include/libcamera/internal/converter.h | 13 +- .../internal/converter/converter_dw100.h | 45 ++ .../converter/converter_dw100_vertexmap.h | 97 ++++ .../internal/converter/converter_v4l2_m2m.h | 28 +- .../libcamera/internal/converter/meson.build | 2 + include/libcamera/internal/media_device.h | 7 + include/libcamera/internal/meson.build | 1 + include/libcamera/internal/pipeline_handler.h | 8 +- include/libcamera/internal/v4l2_device.h | 5 +- include/libcamera/internal/v4l2_request.h | 49 ++ include/libcamera/internal/v4l2_subdevice.h | 2 +- include/libcamera/internal/v4l2_videodevice.h | 5 +- include/linux/dw100.h | 14 + src/ipa/libipa/module.h | 4 + src/libcamera/control_ids_core.yaml | 6 + src/libcamera/control_ids_draft.yaml | 39 +- src/libcamera/converter.cpp | 8 +- src/libcamera/converter/converter_dw100.cpp | 99 ++++ .../converter/converter_dw100_vertexmap.cpp | 449 +++++++++++++++ .../converter/converter_v4l2_m2m.cpp | 80 ++- src/libcamera/converter/meson.build | 2 + src/libcamera/geometry.cpp | 10 + src/libcamera/media_device.cpp | 47 ++ src/libcamera/meson.build | 1 + src/libcamera/pipeline/imx8-isi/imx8-isi.cpp | 2 +- src/libcamera/pipeline/ipu3/cio2.cpp | 2 +- src/libcamera/pipeline/ipu3/cio2.h | 2 +- src/libcamera/pipeline/ipu3/imgu.cpp | 3 +- src/libcamera/pipeline/ipu3/imgu.h | 4 +- src/libcamera/pipeline/ipu3/ipu3.cpp | 4 +- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 2 +- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 523 ++++++++++++++---- src/libcamera/pipeline/rkisp1/rkisp1_path.cpp | 2 +- src/libcamera/pipeline/rkisp1/rkisp1_path.h | 2 +- .../pipeline/rpi/common/pipeline_base.cpp | 6 +- .../pipeline/rpi/common/pipeline_base.h | 9 +- src/libcamera/pipeline/rpi/pisp/pisp.cpp | 10 +- src/libcamera/pipeline/rpi/vc4/vc4.cpp | 13 +- src/libcamera/pipeline/simple/simple.cpp | 15 +- src/libcamera/pipeline/uvcvideo/uvcvideo.cpp | 6 +- src/libcamera/pipeline/vimc/vimc.cpp | 6 +- src/libcamera/pipeline_handler.cpp | 14 +- src/libcamera/sensor/camera_sensor_legacy.cpp | 1 + src/libcamera/sensor/camera_sensor_raw.cpp | 1 + src/libcamera/v4l2_device.cpp | 28 +- src/libcamera/v4l2_request.cpp | 107 ++++ src/libcamera/v4l2_subdevice.cpp | 2 +- src/libcamera/v4l2_videodevice.cpp | 12 +- test/delayed_controls.cpp | 2 +- test/libtest/buffer_source.cpp | 2 +- 52 files changed, 1606 insertions(+), 202 deletions(-) create mode 100644 include/libcamera/internal/converter/converter_dw100.h create mode 100644 include/libcamera/internal/converter/converter_dw100_vertexmap.h create mode 100644 include/libcamera/internal/v4l2_request.h create mode 100644 include/linux/dw100.h create mode 100644 src/libcamera/converter/converter_dw100.cpp create mode 100644 src/libcamera/converter/converter_dw100_vertexmap.cpp create mode 100644 src/libcamera/v4l2_request.cpp