{"id":24728,"url":"https://patchwork.libcamera.org/api/1.1/covers/24728/?format=json","web_url":"https://patchwork.libcamera.org/cover/24728/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20251023144841.403689-1-stefan.klug@ideasonboard.com>","date":"2025-10-23T14:48:01","name":"[v2,00/35] Full dewarper support on imx8mp","submitter":{"id":184,"url":"https://patchwork.libcamera.org/api/1.1/people/184/?format=json","name":"Stefan Klug","email":"stefan.klug@ideasonboard.com"},"mbox":"https://patchwork.libcamera.org/cover/24728/mbox/","series":[{"id":5520,"url":"https://patchwork.libcamera.org/api/1.1/series/5520/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=5520","date":"2025-10-23T14:48:01","name":"Full dewarper support on imx8mp","version":2,"mbox":"https://patchwork.libcamera.org/series/5520/mbox/"}],"comments":"https://patchwork.libcamera.org/api/covers/24728/comments/","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 0E97EBE080\n\tfor ;\n\tThu, 23 Oct 2025 14:48:49 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id D1171606DE;\n\tThu, 23 Oct 2025 16:48:47 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 99B26606DE\n\tfor ;\n\tThu, 23 Oct 2025 16:48:46 +0200 (CEST)","from ideasonboard.com (unknown\n\t[IPv6:2a00:6020:448c:6c00:7328:357b:4ce1:72b6])\n\tby perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 556701127; \n\tThu, 23 Oct 2025 16:47:01 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"SAr8IBJd\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1761230821;\n\tbh=bYx8QUiPTB5WIy7Awqh6DjrL4IR7Qv2IpYJdmc6QadY=;\n\th=From:To:Cc:Subject:Date:From;\n\tb=SAr8IBJd2G3ShtfaqkPZGJTwxCfDEOwPWaMMQLIaYSjTmne3pX374FyztxqTAdDo8\n\t9x1+6Z11/q8xP3Wp9xbTa6o1364RWQss9eq/KiXtw6kH7bauFhA1kQyxF+jYw7hueo\n\tZSd5xM6IM+Gm3zksuKh1wdkdvNcE8iWPdAw3VarM=","From":"Stefan Klug ","To":"libcamera-devel@lists.libcamera.org","Cc":"Stefan Klug ","Subject":"[PATCH v2 00/35] Full dewarper support on imx8mp","Date":"Thu, 23 Oct 2025 16:48:01 +0200","Message-ID":"<20251023144841.403689-1-stefan.klug@ideasonboard.com>","X-Mailer":"git-send-email 2.48.1","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Hi all,\n\nthe imx8mp has a very flexible dewarper that can be used to implement\ncropping, digital zoom, pan, rotate, flip and actual lens dewarping. This\nseries implements all these features for libcamera.\n\nControl wise, the functionality is split into multiple aspects.\nScalerCrop is now implemented using the vertex map, because that\nprovides pixel perfect control for arbitrary rectangles which is an\nimprovement over the current solution that only supports a limited set\nof scaling factors. On top of that a few specific Dw100 specific\ncontrols are added to implement the other functions. Lens dewarp is\nbased on calibration data in the tuning file and can be enabled/disabled\nat runtime.\n\nAs prerequisite it requires an updated kernel driver that supports\nprogramming the vertex map at runtime. The corresponding implementation\nis well underway, but not yet upstream.\n\nYou can find the current version of the kernel driver here:\nhttps://git.uk.ideasonboard.com/sklug/linux/src/branch/sklug/v6.16/topic/dw100-v2\n\nChanges in v2:\n\nIn the discussions around v2 it got clear that the extended controls\nneed further discussions and can not be merged on mainline as is. I\nmoved them to the end of the series and marked them with DNI. The rest\nof the series is valuable in itself as it brings pixel perfect ScalerCrop\nand lens dewarping support without changing much on the controls side.\n\nThe vertexmap code was rewritten to use affine transforms and matrices\ninsted of manual transformation. This makes the code way cleaner and was\non my todo list anyways.\n\nIn the orientation handling a bug was fixed if the transformation\ninvolved a transpose. This bug was visible by a camera configuration\nthat validated successfully, but the camera failed to configure.\n\nAlso a lot of documentation improvements were done and the series passes\nCI successfully: https://gitlab.freedesktop.org/camera/libcamera/-/pipelines/1533035\n\nFor smaller fixes please see the changelogs on the individual patches.\n\n\nPatch summary:\n\nPatches 1-3 replace plain pointers to media devices\nby shared pointers. The patches were already posted here:\nhttps://patchwork.libcamera.org/project/libcamera/list/?series=5113 and\nreview should continue there.\n\nPatches 4-9 contain more plumbing and add support for V4L2 requests in\nlibcamera.\n\nPatches 10-33 contain the actual implementation and I guess it is best\nto read the individual patches.\n\nPatch 34-35 shall not be mainlined and is only there to keep\ncompatibility with prior versions of the kernel driver that didn't use\nV4L2 requests and to show the extended controls that are possible with\nthe dw100.\n\nBest regards,\nStefan\n\nKieran Bingham (3):\n libcamera: v4l2: Support fromEntityName with shared_ptr\n libcamera: pipeline: utilise shared MediaDevice pointers\n libcamera: v4l2: Remove fromEntityName(MediaDevice*)\n\nStefan Klug (29):\n libcamera: converter: Utilise shared MediaDevice pointers\n libcamera: Add support for V4L2 requests\n libcamera: converter: Add V4L2 request support\n libcamera: converter_v4l2_m2m: Add suport for V4L2 requests\n libcamera: converter_v4l2_m2m: Always set stride\n pipeline: rkisp1: Use V4L2 requests for the dewarper\n libcamera: rkisp1: Properly cancel buffers in dewarp case\n libcamera: converter_v4l2_m2m: Add debug logging for formats\n libcamera: rkisp1: Move useDewarper_ flag into RkISP1CameraData\n libcamera: rkisp1: Scale down in dewarper instead of resizer\n libcamera: rkisp1: Allow upscaling when the dewarper is present\n libcamera: converter: converter_v4l2_m2m: Add makeStream() function\n libcamera: converter_m2m: Make some parts protected\n libcamera: converter: Add dw100 vertex map class\n libcamera: converter_dw100: Use vertex map\n libcamera: rkisp1: Use the dw100 class instead of the generic v4l2\n converter\n libcamera: rkisp1: Use vertex map to implement ScalerCrop\n pipeline: rksip1: Move isRaw up in scope\n pipeline: rkisp1: Drop rawFormat variable\n pipeline: rkisp1: Enable the dewarper based on the tuning file\n libcamera: internal: camera_sensor: Add accessor for\n mountingOrientation_\n libcamera: Add transpose() function to size\n libcamera: rkisp1: Handle requested orientation using dewarper\n libcamera: dw100_vertexmap: Implement parametric dewarping\n pipeline: rkisp1: Load dewarp parameters from tuning file\n libcamera: Add and implement LensDewarpEnable control\n pipeline: rkisp1: Add warning to unclear format handling\n DNI libcamera: rkisp1: Implement dw100 specific features\n DNI pipeline: rkisp1: Workaround for customer kernels without requests\n\nUmang Jain (2):\n libcamera: converter_v4l2_m2m: Add helper to apply controls\n libcamera: converter: Add dw100 converter class\n\nXavier Roumegue (1):\n include: linux: Update headers for Dw100 dewarper engine\n\n include/libcamera/geometry.h | 6 +\n include/libcamera/internal/camera_sensor.h | 1 +\n include/libcamera/internal/converter.h | 13 +-\n .../internal/converter/converter_dw100.h | 43 ++\n .../converter/converter_dw100_vertexmap.h | 92 +++\n .../internal/converter/converter_v4l2_m2m.h | 29 +-\n .../libcamera/internal/converter/meson.build | 2 +\n include/libcamera/internal/media_device.h | 7 +\n include/libcamera/internal/meson.build | 1 +\n include/libcamera/internal/pipeline_handler.h | 8 +-\n include/libcamera/internal/v4l2_device.h | 5 +-\n include/libcamera/internal/v4l2_request.h | 49 ++\n include/libcamera/internal/v4l2_subdevice.h | 2 +-\n include/libcamera/internal/v4l2_videodevice.h | 5 +-\n include/linux/dw100.h | 14 +\n src/ipa/libipa/module.h | 4 +\n src/libcamera/control_ids_core.yaml | 7 +\n src/libcamera/control_ids_draft.yaml | 45 ++\n src/libcamera/converter.cpp | 8 +-\n src/libcamera/converter/converter_dw100.cpp | 111 +++\n .../converter/converter_dw100_vertexmap.cpp | 652 ++++++++++++++++++\n .../converter/converter_v4l2_m2m.cpp | 202 +++++-\n src/libcamera/converter/meson.build | 2 +\n src/libcamera/geometry.cpp | 10 +\n src/libcamera/media_device.cpp | 47 ++\n src/libcamera/meson.build | 1 +\n src/libcamera/pipeline/imx8-isi/imx8-isi.cpp | 2 +-\n src/libcamera/pipeline/ipu3/cio2.cpp | 2 +-\n src/libcamera/pipeline/ipu3/cio2.h | 2 +-\n src/libcamera/pipeline/ipu3/imgu.cpp | 3 +-\n src/libcamera/pipeline/ipu3/imgu.h | 4 +-\n src/libcamera/pipeline/ipu3/ipu3.cpp | 4 +-\n src/libcamera/pipeline/mali-c55/mali-c55.cpp | 2 +-\n src/libcamera/pipeline/rkisp1/rkisp1.cpp | 564 +++++++++++----\n src/libcamera/pipeline/rkisp1/rkisp1_path.cpp | 2 +-\n src/libcamera/pipeline/rkisp1/rkisp1_path.h | 2 +-\n .../pipeline/rpi/common/pipeline_base.cpp | 6 +-\n .../pipeline/rpi/common/pipeline_base.h | 9 +-\n src/libcamera/pipeline/rpi/pisp/pisp.cpp | 10 +-\n src/libcamera/pipeline/rpi/vc4/vc4.cpp | 13 +-\n src/libcamera/pipeline/simple/simple.cpp | 15 +-\n src/libcamera/pipeline/uvcvideo/uvcvideo.cpp | 6 +-\n src/libcamera/pipeline/vimc/vimc.cpp | 6 +-\n src/libcamera/pipeline_handler.cpp | 14 +-\n src/libcamera/sensor/camera_sensor.cpp | 7 +\n src/libcamera/sensor/camera_sensor_legacy.cpp | 1 +\n src/libcamera/sensor/camera_sensor_raw.cpp | 1 +\n src/libcamera/v4l2_device.cpp | 28 +-\n src/libcamera/v4l2_request.cpp | 128 ++++\n src/libcamera/v4l2_subdevice.cpp | 2 +-\n src/libcamera/v4l2_videodevice.cpp | 12 +-\n test/delayed_controls.cpp | 2 +-\n test/libtest/buffer_source.cpp | 2 +-\n 53 files changed, 1992 insertions(+), 223 deletions(-)\n create mode 100644 include/libcamera/internal/converter/converter_dw100.h\n create mode 100644 include/libcamera/internal/converter/converter_dw100_vertexmap.h\n create mode 100644 include/libcamera/internal/v4l2_request.h\n create mode 100644 include/linux/dw100.h\n create mode 100644 src/libcamera/converter/converter_dw100.cpp\n create mode 100644 src/libcamera/converter/converter_dw100_vertexmap.cpp\n create mode 100644 src/libcamera/v4l2_request.cpp"}