From patchwork Fri Oct 23 10:58:08 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jai Luthra X-Patchwork-Id: 28570 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 28653C324C for ; Fri, 23 Oct 2026 10:58:46 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3115168FA9; Fri, 23 Oct 2026 12:58:45 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="B2E+1KWq"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2191368F50 for ; Fri, 23 Oct 2026 12:58:43 +0200 (CEST) Received: from mail.ideasonboard.com (unknown [IPv6:2401:4900:1c30:59f6:be2b:2877:6bc4:392]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 9F46A1E4; Fri, 23 Oct 2026 12:56:33 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1792752994; bh=l6ynmBJ0usd7IWZh63aFR4WozkWokpESrJN9gceXBII=; h=From:Subject:Date:To:Cc:From; b=B2E+1KWqjyUIU1uoWY9/OJVWROxoovEqR/nuYtNZj43v8Nq9wVc4K0p2j7fRe4EvZ OBZXuAp9/rglBz7exM7PA7KLKWslWT2uo+po9KNwS9P1rhGUYSLum8bQa+cfyF7fHt rW9ZCKlG2qnnmxAgoauN8NHZTJ89EFk170IpIq0s= From: Jai Luthra Subject: [WIP RFC PATCH 00/26] Support the new common raw sensor model Date: Fri, 23 Oct 2026 16:28:08 +0530 Message-Id: <20261023-b4-raw-model-v1-0-eadb9948901b@ideasonboard.com> MIME-Version: 1.0 X-B4-Tracking: v=1; b=H4sIAAAAAAAC/6tWKk4tykwtVrJSqFYqSi3LLM7MzwNyDHUUlJIzE vPSU3UzU4B8JSMDIzNDAyNj3SQT3aLEct3c/JTUHF3j1FRDEwPLJEPTZCMloJaCotS0zAqwcdG xtbUASYhKAl4AAAA= X-Change-ID: 20261023-b4-raw-model-3ee1409b15c2 To: libcamera-devel@lists.libcamera.org Cc: Laurent Pinchart , Kieran Bingham , Sakari Ailus , Naushir Patuck , Jai Luthra X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=6616; i=jai.luthra@ideasonboard.com; h=from:subject:message-id; bh=l6ynmBJ0usd7IWZh63aFR4WozkWokpESrJN9gceXBII=; b=owEBbQKS/ZANAwAKAUPekfkkmnFFAcsmYgBq2z3CsfFmkU+bEb9byb3gEigmY3KdIFlbpH/Kz mqI4DN6IZSJAjMEAAEKAB0WIQRN4NgY5dV16NRar8VD3pH5JJpxRQUCats9wgAKCRBD3pH5JJpx RQZMEADOiuTn8QpISozIV44WtjdobxXCdVu3PBc4JqTgdNK9llPdQOSmYMnAcqHqqMBax8q7qJN /rJLvQYp7xFTwhQYMNBoZmI9w+Two0F5eD/rXGaymCNB/awTONaaPmnLLZB0cjVp0M6Dg8wedRj hNOH52u4nXD7Y8wclJ4l55WOpbHMe3zS5aostaL1t9Hujzb8ufwuU4JfUyskZbQDHHYa0TLsRf4 cMhIifhixcjM/lszPpzfOwXZF/Uv/t0SUG2uOlWdpyL/xZaAVT/RRLZBK8OiqIVIgtG3tCOkNcM W1WlsO7VrQ71eNZGUdM3AuH0kSdmvpXNzDA+7mG9PwY3ZHLSqLtYMVIgw62D6DAQrlZb53kzAqF ryUJbshg6SRRI2N88ZeDptHadi988gmlpbLoDiVoEmuLYWSPCBUxQv+Yt7J2etfqJiEWxsTgRnu SKPY7vNaBUNMBHxvAAJlGjd/S0mGSw7YYbPwl+JhFyWsY6LrS+Zo8IkmWiKkwQbXwDYL6KQ1EGf e9cIG6ttKidTUIfAW5PIcflGGXJx7S9Ld87793DtEGop1OLr1rzjTZNPtWWAeKH7A4wW8mF/Ic7 1Cr/BFN/oylFqj8m079zdWHbXpyKoXZlNhB/iEBWby4AKi2DgOGLL6puYrKJJdaMJvtcBMuTWPb 9buaQJpmaG+ue0g== X-Developer-Key: i=jai.luthra@ideasonboard.com; a=openpgp; fpr=4DE0D818E5D575E8D45AAFC543DE91F9249A7145 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" NOTE: This is a WIP series that doesn't compile for all platforms :-) Hi all, This series introduces support for the V4L2 Common Raw Sensor Model [1]. I'm posting this as an RFC/WIP for early feedback. PATCH 1-6: Baseline, adding headers for the new model, fixing minor issues PATCH 7-11: Updates to CameraSensorRaw and enabling the capability bit for the new model PATCH 12-17: Split out ColorFilterArray from BayerFormat, drop bayer-specific PixelFormats, switch to generic ones PATCH 18-26: Migrate three pipelines (vc4, pisp, mali-c55) and an application (dng_writer) to the new architecture There are of course many more pipeline handlers and applications that can be converted, but each requires carefully going through it to do it cleanly, so I've stopped at two for now. I've also completely dropped the legacy bayer-order specific PixelFormats, which would break all applications. This is only temporary. Once the migration is complete, the plan is to add them back as deprecated formats, so applications have some notice to switch to the newer model. This series depends on my earlier series for migrating the Raspberry Pi 4 pipeline and IPA to work with mainline kernel [2]. I have also prepared a kernel branch [3] and libcamera branch [4] you can use for testing this. The kernel branch has support for the raw sensor model, including 3 sensors (IMX678, IMX708, IMX219) that fully support the raw sensor model, and updates to the raspberry pi drivers to work with the new formats. I've only tested this on Raspberry Pi 5. The Mali-C55 and dng writer changes are only compile-tested. Lastly, there are quite a few features of the new raw sensor model that are not yet supported, like binning, skipping and new frame rate controls. I'm still working on those :-) [1]: https://lore.kernel.org/all/20260409201501.975242-1-sakari.ailus@linux.intel.com/ [2]: https://patchwork.libcamera.org/cover/28300/ [3]: https://gitlab.freedesktop.org/linux-media/users/jluthra/-/commits/mainline-metadata [4]: https://gitlab.freedesktop.org/jluthra/libcamera/-/commits/mainline-metadata Signed-off-by: Jai Luthra --- Jai Luthra (26): include: linux: Update uAPI headers to v7.3 + Common Raw Sensor Model libcamera: v4l2_subdevice: Add mediaBusInfo for generic raw formats libcamera: v4l2_pixelformat: Update to latest version of generic META formats pipeline: rpi: vc4: Map subdev metadata formats to V4L2 fourcc include: uapi: videodev2: Add formats for unpacked RAW include: uapi: videodev2: Add RAW device-specific bayer formats libcamera: Drop MediaBusFormatInfo::Type::EmbeddedData libcamera: camera_sensor_raw: Read the new CFA_PATTERN control libcamera: camera_sensor_raw: Enumerate formats using sink pads libcamera: camera_sensor_raw: Check for CONFIG_MODEL libcamera: Support raw sensor model libcamera: bayer_format: Introduce a ColorFilterArray class include: uapi: drm_fourcc: Add 14-bit R format libcamera: formats: Switch to generic layout-agnostic formats HACK: libcamera: formats: Add a modifier for RAW colorspace libcamera: v4l2_pixelformat: Add toNonGeneric() helper libcamera: bayer_format: Strip out ColorFilterArray from BayerFormat libcamera: stream: Add bayerOrder for raw formats libcamera: camera_sensor: Add helpers for isRaw() and isMono() pipeline: rpi: Use CameraSensor::isMono() helper pipeline: rpi: Split bayerOrder from PixelFormat pipeline: rpi: Prefer generic raw formats pipeline: rpi: pisp: Migrate to RAW formats pipeline: rpi: vc4: Migrate to generic RAW formats pipeline: mali-c55: Use generic RAW formats apps: dng_writer: Split cfa pattern handling include/libcamera/color_filter_array.h | 58 +++ include/libcamera/internal/bayer_format.h | 18 +- include/libcamera/internal/camera_sensor.h | 6 +- .../libcamera/internal/software_isp/swstats_cpu.h | 2 +- include/libcamera/internal/v4l2_pixelformat.h | 3 + include/libcamera/internal/v4l2_subdevice.h | 2 +- include/libcamera/meson.build | 1 + include/libcamera/stream.h | 17 + include/linux/drm_fourcc.h | 7 + include/linux/media-bus-format.h | 12 +- include/linux/v4l2-controls.h | 58 ++- include/linux/v4l2-subdev.h | 15 +- include/linux/videodev2.h | 35 +- src/apps/common/dng_writer.cpp | 189 ++------- src/ipa/mali-c55/algorithms/agc.cpp | 14 +- src/ipa/mali-c55/algorithms/agc.h | 5 +- src/ipa/mali-c55/ipa_context.h | 5 +- src/ipa/mali-c55/mali-c55.cpp | 4 +- src/libcamera/bayer_format.cpp | 303 ++++---------- src/libcamera/color_filter_array.cpp | 334 ++++++++++++++++ src/libcamera/formats.cpp | 440 +++++---------------- src/libcamera/formats.yaml | 149 ++----- src/libcamera/meson.build | 1 + src/libcamera/pipeline/mali-c55/mali-c55.cpp | 165 ++++---- .../pipeline/rpi/common/pipeline_base.cpp | 38 +- src/libcamera/pipeline/rpi/pisp/pisp.cpp | 86 ++-- src/libcamera/pipeline/rpi/vc4/vc4.cpp | 56 +-- src/libcamera/pipeline/simple/simple.cpp | 3 +- src/libcamera/sensor/camera_sensor_legacy.cpp | 45 ++- src/libcamera/sensor/camera_sensor_raw.cpp | 124 +++--- src/libcamera/software_isp/debayer.cpp | 8 +- src/libcamera/software_isp/debayer.h | 2 +- src/libcamera/software_isp/debayer_cpu.cpp | 58 +-- src/libcamera/software_isp/debayer_cpu.h | 2 +- src/libcamera/software_isp/debayer_egl.cpp | 6 +- src/libcamera/software_isp/swstats_cpu.cpp | 26 +- src/libcamera/v4l2_pixelformat.cpp | 221 +++++++++-- src/libcamera/v4l2_subdevice.cpp | 203 ++++++++-- test/bayer-format.cpp | 40 +- 39 files changed, 1495 insertions(+), 1266 deletions(-) --- base-commit: 01772f1c0c8a48a77dc99a47f5f50afa51c58f1f change-id: 20261023-b4-raw-model-3ee1409b15c2 Best regards,