{"id":19591,"url":"https://patchwork.libcamera.org/api/covers/19591/?format=json","web_url":"https://patchwork.libcamera.org/cover/19591/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20240301212121.9072-1-laurent.pinchart@ideasonboard.com>","date":"2024-03-01T21:20:49","name":"[PATCH/RFC,00/32] libcamera: Support the upstream Unicam driver","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/?format=json","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"mbox":"https://patchwork.libcamera.org/cover/19591/mbox/","series":[{"id":4197,"url":"https://patchwork.libcamera.org/api/series/4197/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=4197","date":"2024-03-01T21:20:49","name":"libcamera: Support the upstream Unicam driver","version":1,"mbox":"https://patchwork.libcamera.org/series/4197/mbox/"}],"comments":"https://patchwork.libcamera.org/api/covers/19591/comments/","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 26BE0BD160\n\tfor ;\n\tFri, 1 Mar 2024 21:21:22 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 5F7506286F;\n\tFri, 1 Mar 2024 22:21:21 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 564536285F\n\tfor ;\n\tFri, 1 Mar 2024 22:21:19 +0100 (CET)","from pendragon.ideasonboard.com (89-27-53-110.bb.dnainternet.fi\n\t[89.27.53.110])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id ADC2C673;\n\tFri, 1 Mar 2024 22:21:04 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"hR6cdFwC\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1709328064;\n\tbh=csbfQ56eQ4d4ma8G6b14BNrZXULzSzSc15AvxwypVCo=;\n\th=From:To:Cc:Subject:Date:From;\n\tb=hR6cdFwCpL8w30nLzNvC3XhYfcaOGRvXNqyQuIlwdGiLFtLL6o29HgikV7W+o7akM\n\tFwoqUNS68qBqGfEvLWWtL6yMoSoP9d8rYPBc1gKQKOSTI2f6xh3E4lSnwWfpCJlXD/\n\tEzww4uNZE2Bu1mIXRUT91grFnb4k+09ZNi0VhX90=","From":"Laurent Pinchart ","To":"libcamera-devel@lists.libcamera.org","Subject":"[PATCH/RFC 00/32] libcamera: Support the upstream Unicam driver","Date":"Fri, 1 Mar 2024 23:20:49 +0200","Message-ID":"<20240301212121.9072-1-laurent.pinchart@ideasonboard.com>","X-Mailer":"git-send-email 2.43.0","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Sakari Ailus ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Hello everybody,\n\nThis large(-ish) patch series is the first complete implementation of\nsupport for the Unicam upstream kernel driver for libcamera.\n\nOr, it would be, if the Unicam driver was upstream. While I have also\ncompleted a version of the Unicam driver for the upstream kernel,\nthere's a bit of a chicken-and-egg issue here as I would like to\ncross-reference the cover letters. To break the loop, I'm posting this\nseries first, and will reply with a link to the Unicam patches ([1]).\n\nSupport for Raspberry Pi 4 in libcamera currently relies on downstream\ndrivers that live in the Raspberry Pi kernel tree, for Unicam (the CSI-2\nreceiver) and the ISP. The Unicam driver, in particular, includes a few\nworkarounds to implement support for sensor embedded data, to compensate\nfor features that are or were until recently missing in the V4L2\nin-kernel and userspace APIs. The new Unicam submission for the kernel\nreworks the driver extensively to use the recently merged V4L2 streams\nAPI, as well as the under development V4L2 generic metadata and internal\npads APIs ([2]).\n\nThe series starts with nine patches that clean up, rework and improve\nthe V4L2Subdevice class (01/32 to 09/32). They have been posted to the\nlist before, and mostly reviewed. Patches 10/32 is where the fun starts,\nas it pulls the APIs from [2] in the kernel headers. The next three\npatches, 11/32 to 13/32, then update the V4L2Subdevice and\nV4L2VideoDevice classes to support those APIs.\n\nThe next ten patches rework the CameraSensor class to support sensor\ndrivers that use the new APIs. Patches 14/32 to to 21/32 contain various\nreworks, including moving the camera sensor support to the new\nsrc/libcamera/sensor/ directory, and turning CameraSensor into an\nabstract base class with multiple implementations, using a factory\npattern to pick the right match. Patches 22/32 introduces a new\nCameraSensor subclass for the new APIs, and patch 23/32 then extends the\nCameraSensor interface to support embedded data.\n\nWith all the building blocks in place, the next seven patches focus on\nthe Raspberry Pi VC4 pipeline handler to use the new APIs, when\ninteracting with the sensor (24/32), or with the mainline Unicam driver\n(27/32 to 30/32). Once done, patch 31/32 drops the\nV4L2_META_FMT_SENSOR_DATA format from the videodev2.h kernel header, as\nthe format is not used anymore and has never been present in the\nupstream kernel.\n\nThe last patch, 32/32, is a hack that unconditionally enables embedded\ndata for the IMX219 sensor, as that is the platform I've been using to\ndevelop and test this series.\n\nWorthy of a note, this series breaks compatibility with the downstream\nRaspberry Pi kernel. I have decided not to preserve backward\ncompatibiliy to ease development, and to make the new code easier to\nreview. We still need to discuss how to integrate the various components\n(libcamera and kernel space) in a way that will not generate lots of\nanger among users. This may lead to some temporary changes to the code\nto help with the transition.\n\nFor convenience, the patches can be found at [3].\n\n[1] TBD due to lack of a time machine\n[2] https://lore.kernel.org/linux-media/20231106122539.1268265-1-sakari.ailus@linux.intel.com\n[3] https://git.libcamera.org/libcamera/pinchartl/libcamera.git/log/?h=rpi/streams/next\n\nJacopo Mondi (1):\n libcamera: camera_sensor: Introduce CameraSensorFactory\n\nLaurent Pinchart (31):\n libcamera: v4l2_subdevice: Rename V4L2SubdeviceFormatInfo\n libcamera: v4l2_subdevice: Add code member to MediaBusFormatInfo\n libcamera: v4l2_subdevice: Expose media bus format info as internal\n API\n libcamera: v4l2_subdevice: Extend MediaBusFormatInfo with metadata\n formats\n libcamera: v4l2_subdevice: Drop V4L2SubdeviceFormat::bitsPerPixel()\n libcamera: v4l2_subdevice: Rename V4L2SubdeviceFormat::mbus_code to\n code\n libcamera: v4l2_subdevice: Add stream support to get/set functions\n libcamera: v4l2_subdevice: Replace Routing::toString() with\n operator<<()\n libcamera: v4l2_subdevice: Add V4L2Subdevice::Route structure\n [DNI] include: linux: Update kernel headers to metadata API\n libcamera: v4l2_subdevice: Update to the new kernel routing API\n libcamera: v4l2_subdevice: Add new metadata formats\n libcamera: v4l2_videodevice: Update to the new kernel metadata API\n libcamera: camera_sensor: Move related classes to subdirectory\n libcamera: camera_sensor: Drop updateControlInfo() function\n libcamera: camera_sensor: Reorder functions\n libcamera: camera_sensor: Test for read-only HBLANK with READ_ONLY\n flag\n libcamera: camera_sensor: Expose the Bayer order\n libcamera: camera_sensor: Create abstract base class\n libcamera: camera_sensor: Sort factories by priority\n libcamera: Add CameraSensor implementation for raw V4L2 sensors\n libcamera: camera_sensor: Add support for embedded data\n pipeline: raspberrypi: common: Configure sensor embedded data\n pipeline: raspberrypi: vc4: Use operator<<(V4L2VideoFormat)\n pipeline: raspberrypi: vc4: Reorganize platformConfigure()\n pipeline: raspberrypi: vc4: Use the CameraSensor embedded data API\n pipeline: raspberrypi: vc4: Unconditionally create embedded data\n stream\n pipeline: raspberrypi: vc4: Configure format on Unicam subdev\n pipeline: raspberrypi: vc4: Fix configuration of the embedded data\n node\n include: linux: Drop V4L2_META_FMT_SENSOR_DATA\n [HACK]: ipa: rpi: cam_helper_imx219: Enable embedded data\n\n Documentation/Doxyfile.in | 2 +\n include/libcamera/internal/camera_sensor.h | 162 +--\n include/libcamera/internal/v4l2_subdevice.h | 103 +-\n include/linux/README | 2 +-\n include/linux/media-bus-format.h | 13 +\n include/linux/media.h | 1 +\n include/linux/v4l2-controls.h | 16 +-\n include/linux/v4l2-mediabus.h | 18 +-\n include/linux/v4l2-subdev.h | 23 +-\n include/linux/videodev2.h | 30 +-\n src/ipa/rpi/cam_helper/cam_helper_imx219.cpp | 2 +-\n src/libcamera/meson.build | 3 +-\n src/libcamera/pipeline/imx8-isi/imx8-isi.cpp | 19 +-\n src/libcamera/pipeline/ipu3/cio2.cpp | 15 +-\n src/libcamera/pipeline/ipu3/imgu.cpp | 4 +-\n src/libcamera/pipeline/rkisp1/rkisp1.cpp | 9 +-\n src/libcamera/pipeline/rkisp1/rkisp1_path.cpp | 2 +-\n .../pipeline/rpi/common/pipeline_base.cpp | 85 +-\n .../pipeline/rpi/common/pipeline_base.h | 6 +-\n src/libcamera/pipeline/rpi/vc4/vc4.cpp | 174 ++-\n src/libcamera/pipeline/simple/simple.cpp | 29 +-\n src/libcamera/pipeline/vimc/vimc.cpp | 11 +-\n src/libcamera/sensor/camera_sensor.cpp | 557 ++++++++\n .../camera_sensor_legacy.cpp} | 717 ++++------\n .../{ => sensor}/camera_sensor_properties.cpp | 0\n src/libcamera/sensor/camera_sensor_raw.cpp | 1135 +++++++++++++++\n src/libcamera/sensor/meson.build | 8 +\n src/libcamera/v4l2_subdevice.cpp | 1239 ++++++++++++++---\n src/libcamera/v4l2_videodevice.cpp | 31 +-\n test/camera-sensor.cpp | 9 +-\n .../v4l2_videodevice_test.cpp | 7 +-\n test/v4l2_videodevice/v4l2_videodevice_test.h | 2 +-\n 32 files changed, 3471 insertions(+), 963 deletions(-)\n create mode 100644 src/libcamera/sensor/camera_sensor.cpp\n rename src/libcamera/{camera_sensor.cpp => sensor/camera_sensor_legacy.cpp} (60%)\n rename src/libcamera/{ => sensor}/camera_sensor_properties.cpp (100%)\n create mode 100644 src/libcamera/sensor/camera_sensor_raw.cpp\n create mode 100644 src/libcamera/sensor/meson.build\n\n\nbase-commit: c64446c226d4e629884d2f5b148a01969e8ee84a"}