{"id":25179,"url":"https://patchwork.libcamera.org/api/1.1/covers/25179/?format=json","web_url":"https://patchwork.libcamera.org/cover/25179/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20251125162851.2301793-1-stefan.klug@ideasonboard.com>","date":"2025-11-25T16:28:12","name":"[v3,00/29] Full dewarper support on imx8mp","submitter":{"id":184,"url":"https://patchwork.libcamera.org/api/1.1/people/184/?format=json","name":"Stefan Klug","email":"stefan.klug@ideasonboard.com"},"mbox":"https://patchwork.libcamera.org/cover/25179/mbox/","series":[{"id":5613,"url":"https://patchwork.libcamera.org/api/1.1/series/5613/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=5613","date":"2025-11-25T16:28:12","name":"Full dewarper support on imx8mp","version":3,"mbox":"https://patchwork.libcamera.org/series/5613/mbox/"}],"comments":"https://patchwork.libcamera.org/api/covers/25179/comments/","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 0B887C333C\n\tfor ;\n\tTue, 25 Nov 2025 16:28:58 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 0DF2860A9E;\n\tTue, 25 Nov 2025 17:28:57 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 1894C609D8\n\tfor ;\n\tTue, 25 Nov 2025 17:28:56 +0100 (CET)","from ideasonboard.com (unknown\n\t[IPv6:2a00:6020:448c:6c00:bae1:340c:573c:570b])\n\tby perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id BFA43524;\n\tTue, 25 Nov 2025 17:26:46 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"H6w9iVWA\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1764088006;\n\tbh=QbIUjPjETjDnPJYgCfyizhf9a3girFicclscWh4I5G4=;\n\th=From:To:Cc:Subject:Date:From;\n\tb=H6w9iVWABgijF9YMrPSSSCQpFCdf9KsJYE+X2xeC4dLIGuN8DolgKdYenNc/SmIv2\n\tT0OrX8DZOTfGTUuhjCyu5y/Rg1dlyYcxgpdrvDBKfRyOYGk4XVH9wd3R1hbAxbe4W1\n\tSfSDmCii+iBJrI41jjDsCTlPgtMxaaLS7e/QlRtg=","From":"Stefan Klug ","To":"libcamera-devel@lists.libcamera.org","Cc":"Stefan Klug ","Subject":"[PATCH v3 00/29] Full dewarper support on imx8mp","Date":"Tue, 25 Nov 2025 17:28:12 +0100","Message-ID":"<20251125162851.2301793-1-stefan.klug@ideasonboard.com>","X-Mailer":"git-send-email 2.51.0","MIME-Version":"1.0","Content-Type":"text/plain; charset=UTF-8","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Hi all,\n\nThe imx8mp has a very flexible dewarper that can be used to implement\ncropping, digital zoom, pan, rotate, flip and actual lens dewarping. This\nseries implements all these features for libcamera.\n\nControl wise, the functionality is split into multiple aspects.\nScalerCrop is now implemented using the vertex map, because that\nprovides pixel perfect control for arbitrary rectangles which is an\nimprovement over the current solution that only supports a limited set\nof scaling factors. On top of that a few specific Dw100 specific\ncontrols are added to implement the other functions. Lens dewarp is\nbased on calibration data in the tuning file and can be enabled/disabled\nat runtime.\n\nAs prerequisite it requires an updated kernel driver that supports\nprogramming the vertex map at runtime. The corresponding implementation\nis well underway, but not yet upstream.\n\nYou can find the current version of the kernel driver here:\nhttps://git.uk.ideasonboard.com/sklug/linux/src/branch/sklug/v6.16/topic/dw100-v2\n\nChanges in v3:\n\nIn v3 all the dewarper specifics like controls handling and request handling\nthat were entangled within the rkisp1 pipeline handler got moved into a new\nConverterDW100Module class. This way, the code got more modular and easier to\nuse in a different pipeline. Aside from this big rewrite, a few bugs were fixed\nand a lot of review comments addressed. Please see the individual logs for\ndetails.\n\nChanges in v2:\n\nIn the discussions around v2 it got clear that the extended controls\nneed further discussions and can not be merged on mainline as is. I\nmoved them to the end of the series and marked them with DNI. The rest\nof the series is valuable in itself as it brings pixel perfect ScalerCrop\nand lens dewarping support without changing much on the controls side.\n\nThe vertexmap code was rewritten to use affine transforms and matrices\ninsted of manual transformation. This makes the code way cleaner and was\non my todo list anyways.\n\nIn the orientation handling a bug was fixed if the transformation\ninvolved a transpose. This bug was visible by a camera configuration\nthat validated successfully, but the camera failed to configure.\n\nAlso a lot of documentation improvements were done and the series passes\nCI successfully: https://gitlab.freedesktop.org/camera/libcamera/-/pipelines/1533035\n\nFor smaller fixes please see the changelogs on the individual patches.\n\n\nPatch summary:\n\nPatches 1-2 replaces plain pointers to media devices\nby shared pointers.\n\nPatches 3-5 contain more plumbing and add support for V4L2 requests in\nlibcamera.\n\nPatches 7-27 contain the actual implementation and I guess it is best\nto read the individual patches.\n\nPatch 28-29 shall not be mainlined and is only there to keep\ncompatibility with prior versions of the kernel driver that didn't use\nV4L2 requests and to show the extended controls that are possible with\nthe dw100.\n\nBest regards,\nStefan\n\nKieran Bingham (1):\n libcamera: pipeline: utilise shared MediaDevice pointers\n\nStefan Klug (26):\n libcamera: converter: Utilise shared MediaDevice pointers\n libcamera: Add support for V4L2 requests\n libcamera: converter: Add V4L2 request support\n libcamera: converter_v4l2_m2m: Add suport for V4L2 requests\n libcamera: converter_v4l2_m2m: Always set stride\n libcamera: rkisp1: Properly cancel buffers in dewarp case\n libcamera: converter_v4l2_m2m: Add debug logging for formats\n libcamera: rkisp1: Move useDewarper_ flag into RkISP1CameraData\n libcamera: rkisp1: Scale down in dewarper instead of resizer\n libcamera: rkisp1: Allow upscaling when the dewarper is present\n pipeline: rkisp1: Fix number of buffers imported into the dewarper\n libcamera: converter: Add dw100 vertex map class\n libcamera: converter: Add dw100 converter module\n libcamera: rkisp1: Use the dw100 converter module instead of the\n generic v4l2 converter\n pipeline: rksip1: Move isRaw up in scope\n pipeline: rkisp1: Drop rawFormat variable\n pipeline: rkisp1: Enable the dewarper based on the tuning file\n libcamera: internal: camera_sensor: Add accessor for\n mountingOrientation_\n libcamera: Add transpose() function to size\n libcamera: rkisp1: Handle requested orientation using dewarper\n libcamera: dw100_vertexmap: Implement parametric dewarping\n pipeline: rkisp1: Load dewarp parameters from tuning file\n libcamera: Add and implement LensDewarpEnable control\n pipeline: rkisp1: Add warning to unclear format handling\n DNI libcamera: rkisp1: Implement dw100 specific features\n DNI pipeline: rkisp1: Workaround for customer kernels without requests\n\nUmang Jain (1):\n libcamera: converter_v4l2_m2m: Add helper to apply controls\n\nXavier Roumegue (1):\n include: linux: Update headers for Dw100 dewarper engine\n\n include/libcamera/geometry.h | 6 +\n include/libcamera/internal/camera_sensor.h | 1 +\n include/libcamera/internal/converter.h | 13 +-\n .../internal/converter/converter_dw100.h | 93 +++\n .../converter/converter_dw100_vertexmap.h | 87 +++\n .../internal/converter/converter_v4l2_m2m.h | 20 +-\n .../libcamera/internal/converter/meson.build | 2 +\n include/libcamera/internal/media_device.h | 8 +\n include/libcamera/internal/meson.build | 1 +\n include/libcamera/internal/pipeline_handler.h | 8 +-\n include/libcamera/internal/v4l2_device.h | 5 +-\n include/libcamera/internal/v4l2_request.h | 44 ++\n include/libcamera/internal/v4l2_videodevice.h | 3 +-\n include/linux/dw100.h | 14 +\n src/libcamera/control_ids_core.yaml | 57 ++\n src/libcamera/converter.cpp | 8 +-\n src/libcamera/converter/converter_dw100.cpp | 543 +++++++++++++++\n .../converter/converter_dw100_vertexmap.cpp | 650 ++++++++++++++++++\n .../converter/converter_v4l2_m2m.cpp | 87 ++-\n src/libcamera/converter/meson.build | 2 +\n src/libcamera/geometry.cpp | 10 +\n src/libcamera/media_device.cpp | 47 ++\n src/libcamera/meson.build | 1 +\n src/libcamera/pipeline/imx8-isi/imx8-isi.cpp | 8 +-\n src/libcamera/pipeline/ipu3/cio2.cpp | 4 +-\n src/libcamera/pipeline/ipu3/cio2.h | 2 +-\n src/libcamera/pipeline/ipu3/imgu.cpp | 15 +-\n src/libcamera/pipeline/ipu3/imgu.h | 4 +-\n src/libcamera/pipeline/ipu3/ipu3.cpp | 4 +-\n src/libcamera/pipeline/mali-c55/mali-c55.cpp | 16 +-\n src/libcamera/pipeline/rkisp1/rkisp1.cpp | 451 +++++++-----\n src/libcamera/pipeline/rkisp1/rkisp1_path.cpp | 6 +-\n src/libcamera/pipeline/rkisp1/rkisp1_path.h | 2 +-\n .../pipeline/rpi/common/pipeline_base.cpp | 6 +-\n .../pipeline/rpi/common/pipeline_base.h | 9 +-\n src/libcamera/pipeline/rpi/pisp/pisp.cpp | 10 +-\n src/libcamera/pipeline/rpi/vc4/vc4.cpp | 13 +-\n src/libcamera/pipeline/simple/simple.cpp | 15 +-\n src/libcamera/pipeline/uvcvideo/uvcvideo.cpp | 6 +-\n src/libcamera/pipeline/vimc/vimc.cpp | 14 +-\n src/libcamera/pipeline_handler.cpp | 14 +-\n src/libcamera/sensor/camera_sensor.cpp | 7 +\n src/libcamera/sensor/camera_sensor_legacy.cpp | 1 +\n src/libcamera/sensor/camera_sensor_raw.cpp | 1 +\n src/libcamera/v4l2_device.cpp | 30 +-\n src/libcamera/v4l2_request.cpp | 128 ++++\n src/libcamera/v4l2_videodevice.cpp | 10 +-\n test/geometry.cpp | 5 +\n test/libtest/buffer_source.cpp | 3 +-\n 49 files changed, 2219 insertions(+), 275 deletions(-)\n create mode 100644 include/libcamera/internal/converter/converter_dw100.h\n create mode 100644 include/libcamera/internal/converter/converter_dw100_vertexmap.h\n create mode 100644 include/libcamera/internal/v4l2_request.h\n create mode 100644 include/linux/dw100.h\n create mode 100644 src/libcamera/converter/converter_dw100.cpp\n create mode 100644 src/libcamera/converter/converter_dw100_vertexmap.cpp\n create mode 100644 src/libcamera/v4l2_request.cpp"}