From patchwork Tue Aug 25 13:08:16 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: John Cronin X-Patchwork-Id: 28088 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0CF8CC0F1B for ; Tue, 25 Aug 2026 13:08:32 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B6193683F6; Tue, 25 Aug 2026 15:08:31 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=stromback-com.20251104.gappssmtp.com header.i=@stromback-com.20251104.gappssmtp.com header.b="qz3WcB1s"; dkim-atps=neutral Received: from mail-ua1-x929.google.com (mail-ua1-x929.google.com [IPv6:2607:f8b0:4864:20::929]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 34E9F683F2 for ; Tue, 25 Aug 2026 15:08:28 +0200 (CEST) Received: by mail-ua1-x929.google.com with SMTP id a1e0cc1a2514c-9667ea2fc22so1086053241.2 for ; Tue, 25 Aug 2026 06:08:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=stromback-com.20251104.gappssmtp.com; s=20251104; t=1787663307; x=1788268107; darn=lists.libcamera.org; h=content-transfer-encoding:content-type:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to:content-type; bh=E/9Y+mvu27L+XwBt23SkPNELypgtxkq66aY+AdP71cc=; b=qz3WcB1sZznaWsU89HhuyRCDDDxb5onlHqWDm9R0QaRLWH2vp3AF0YmUI9l5U16W0F KHclbneqjcXjbXHl0ZsFln7u0XO05pBTzUsQ5tkdC3PCsnZcoJ6UZ+Y7zgz8o7qwmfr3 RmZ+CyncNC2Oh5k/IeLBykLjmE/YL95GkB5rt0rfuSv8dKoLgCFNfoyGPZnq4H5h/ew/ X/WMVI/iyG/SFbb9A3WCvvrnD2h9K8bSRP49L56pdA3tVNbfaHVDGIxd2U0y4cVOaFQu pKW7WcI8tB11B7EIA3T6vhn0uzIKxGZ4Q3mF5R0oCVAdwwqHk4dQrY0zrdLPj2qmP5Nx nYtw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1787663307; x=1788268107; h=content-transfer-encoding:content-type:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=E/9Y+mvu27L+XwBt23SkPNELypgtxkq66aY+AdP71cc=; b=DQrcTF/X4b3JTrvFcOP+nuMiu/3IRjYtSTAx+7EO1zAKiemrhO+mpYZTu6iR5Z98ZT S3LXzdtqVznSN3/6bXC/3HVnMc3qnFhuBzlY42tF99GDeX9B+DGV+2RaqQItqGaIp8v4 rYjUfgwPjNmNRKTCLyHNbhfVPfSG6pW0RKJj+gW4xZvD+K8TcxsFpLXwoj2DpVwTwiaW MyG0d5dF6/JBdfZskD3AJ9kWdMgEu9eXkUaL6FrX0TPowWeFrBApWKoZPqipw7fOp/b8 p4lcX+saCztN5HaBoJVmbDiuyp88etj/hKuWfQW1pw/S4/HuoGHR1zZcGzbpHsmnWz8J jH7w== X-Gm-Message-State: AFuF++lXFaDCm0Q2Zssfv5q0wHsgrHB7O0gQv60/a0AbAYx/so30WPU5 1Yh4MhXIrsvbQR1NA2LgP1ucHrRCudtbstGRQ1t67+6tTfkaZ13Mt0NKFHwEYo09khe84EZqxQR NM52tk6zwO3mDPlw= X-Gm-Gg: AR+sD12v2HCBcgFTNUjsJFls9KPvxCJ7RJQYaySgid0mdfHJy6j5n+Uso7FJwRe6LnM XEUagXB8uzgGPPArKfR/ZcF8gg4MtNDBPYqKqL/m1KmldCs/VQevRHOpGx1E6nrNsYRTdc1+cqy o9BUQZKy/pLKoI9MkXNIT0L7Y3dD1OrzylKAeV6I7pLFM1CcAAmQYRqB/nd/VNZoDHF2q6BXZQz b3nVTH2BVkx6rnB339FO/rZfTifJmAyOHHH8mrcZD34iAL1Xill13dJLL3T5pUVzfpZkzSSExX7 dPidVUSkvDXkK+yvsIh11Qfvsc96sfPKvDuE0wi3JN8wWlN4nx1NHZtTD9Xi9CrwusDfyV8G1sF uec06Z5oF7f9hoXI+bImAwCJ/OJ0DGI5aZmOiO7FGs/f14CotVaWCQgE/drZbp1E6TeLaJ6PIgE jcG7XvpwJawfUzjeIQDrKSIWvqKVxEEsEDmNFtBmq/cCYHgFTSKmMKDI2jAeVPcrPbOx9FzMarQ My/cymrGjImH0Lc6gHk6F6JtaaX/c5IhRIJ5Xk5Lq4+RKpOKUx9yIdboWICHGhOh4bbR4Y7cOnH ku2993FD5qJQjLWEw9Fquq8ka93Gqz/soWPJQVRHD+lgVG9Fr36R5wzMbMQayYKmRvaZZeTjTzi EmhowwPgtyToiMD9N5+lFtCjaS0fGIsscC1TB1mxfjrSFq+YZ2IUTtym6tnpB6dUxI/ZDZfaUnw == X-Received: by 2002:a05:6122:2802:b0:5bf:a181:d46d with SMTP id 71dfb90a1353d-5c642d8391bmr2216615e0c.3.1787663304890; Tue, 25 Aug 2026 06:08:24 -0700 (PDT) Received: from customer.mmmiflx1.isp.starlink.com ([66.9.164.90]) by smtp.gmail.com with ESMTPSA id 71dfb90a1353d-5c623f1c717sm5982652e0c.16.2026.08.25.06.08.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Aug 2026 06:08:24 -0700 (PDT) From: John Cronin X-Google-Original-From: John Cronin To: libcamera-devel@lists.libcamera.org Cc: laurent.pinchart@ideasonboard.com, jacopo.mondi@ideasonboard.com, barnabas.pocze@ideasonboard.com, stefan.klug@ideasonboard.com, John Cronin Subject: [PATCH v3 2/2] Documentation: Add Camera Sensor Helper guide Date: Tue, 25 Aug 2026 09:08:16 -0400 Message-ID: <20260825130816.2305405-3-john.cronin@opcenter.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260825130816.2305405-1-john.cronin@opcenter.com> References: <20260825130816.2305405-1-john.cronin@opcenter.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Document how to add a CameraSensorHelper, measure analogue gain and black level with utils/measure-analogue-gain.py (run from the source tree), and what to put in camera_sensor_properties versus the helper. Describe the bring-up (v4l2-ctl codes) versus verification (cam --script linear AnalogueGain) split, and that the fitter only targets AnalogueGainLinear { m0=0, c0=k, m1=-1, c1=k }. Signed-off-by: John Cronin --- Documentation/guides/camera-sensor-helper.rst | 174 ++++++++++++++++++ Documentation/index.rst | 1 + Documentation/meson.build | 1 + 3 files changed, 176 insertions(+) create mode 100644 Documentation/guides/camera-sensor-helper.rst diff --git a/Documentation/guides/camera-sensor-helper.rst b/Documentation/guides/camera-sensor-helper.rst new file mode 100644 index 0000000..13d8e38 --- /dev/null +++ b/Documentation/guides/camera-sensor-helper.rst @@ -0,0 +1,174 @@ +.. SPDX-License-Identifier: CC-BY-SA-4.0 + +Camera Sensor Helper Guide +========================== + +This guide explains how to add a ``CameraSensorHelper`` for a new sensor and how +to measure the analogue gain model and black level when a public datasheet is +not available. + +Background +---------- + +The software ISP and other IPA modules need to know how the sensor maps +analogue gain control codes to linear gain factors, and which digital number +corresponds to optical black. That information lives in +``src/ipa/libipa/camera_sensor_helper.cpp`` as a small helper class registered +with ``REGISTER_CAMERA_SENSOR_HELPER``. + +Static metadata such as unit cell size and test pattern mode maps belong in +``src/libcamera/sensor/camera_sensor_properties.cpp``. Sensor control application +delays should only be listed there when they have been measured or documented; +leave ``sensorDelays`` empty to use libcamera defaults. + +Adding a helper +--------------- + +1. Confirm the kernel driver name (for example ``imx471``) and the V4L2 control + ranges for exposure and analogue gain (``v4l2-ctl -d /dev/v4l-subdevX + --list-ctrls``). +2. Prefer values from a datasheet when available. Document the source in the + commit message and, if short, a brief code comment (for example + ``/* From datasheet: 64 at 10bits. */``). +3. If no datasheet is available, measure gain response and black level as + described below. Put the measurement summary in the **commit message**; keep + in-code comments short. +4. Register the helper under the same string the kernel uses for the subdev + name (without bus address suffixes). + +Common Sony sensors program a linear code ``c`` such that: + +.. math:: + + G = \frac{k}{k - c} + +with ``k = 1024`` for many IMX models. The maximum V4L2 code may still be lower +than ``k - 1`` (for example codes ``0..800``), which yields a modest maximum +gain even when the model is correct. + +Measuring analogue gain +----------------------- + +The ``utils/measure-analogue-gain.py`` helper automates a fixed-exposure gain +sweep using the libcamera ``cam`` tool and ``v4l2-ctl``. It is a developer +tool intended to be run from a libcamera source tree (it is not installed +with the package). + +Dependencies: + +* ``cam`` (libcamera tools), or set ``LIBCAMERA_CAM`` to a wrapper/command +* ``v4l2-ctl`` from v4l-utils +* Python 3.10+ + +Bring-up vs verification: + +* **New helper (no ``CameraSensorHelper`` yet):** set V4L2 *codes* with + ``v4l2-ctl``. ``cam`` ``AnalogueGain`` is a linear factor and is not + available until a helper loads. +* **After a helper exists:** emit or use a ``cam --script`` YAML that steps + linear ``AnalogueGain`` (see ``utils/measure-analogue-gain-cam.yaml`` and + ``--emit-cam-script``). + +The fitter only targets the Sony-style linear sub-model:: + + AnalogueGainLinear { m0 = 0, c0 = k, m1 = -1, c1 = k } + +which is :math:`G = k / (k - c)`. Other ``AnalogueGainLinear`` families and +``AnalogueGainExp`` are out of scope for this helper. + +High-level procedure: + +1. Capture **raw** Bayer frames (``cam --stream role=raw,...``), not processed + RGB. Soft ISP AGC is avoided so the V4L2 codes you set stay put. +2. Lock exposure (and digital gain if present) on the sensor subdev. +3. Step ``V4L2_CID_ANALOGUE_GAIN`` across the driver range. +4. Compute the mean of active-area samples (useful bit depth, for example + 10-bit values carried in 16-bit words). +5. Black-subtract using a low percentile at minimum gain, or a dark frame. +6. Fit measured brightness ratios to ``G = k/(k-code)`` and compare with the + model you intend to hard-code (often ``k = 1024``). + +Terminology used by the tool: + +* **p1** — first percentile of the sample histogram (near-black floor) +* **p50** — median +* **signal** — ``mean - black`` after black subtraction +* **ratio** — ``signal(code) / signal(0)`` + +Example (Sony IMX471 on an IPU7 laptop, 1928×1088, stride 3904): + +.. code-block:: shell + + ./utils/measure-analogue-gain.py \ + --sensor-name imx471 \ + --width 1928 --height 1088 --stride 3904 --bit-depth 10 \ + --exposure 200 --digital-gain 256 \ + --gains 0,50,100,150,200,300,400,500,600,700,800 \ + --model-k 1024 \ + --out /tmp/gain-measure + +The tool writes ``results.csv``, ``results.json``, and a ``cam-script.yaml`` +replay file (linear ``AnalogueGain`` for the same codes). Relative error +versus the reference model and a best-fit ``k`` for ``G=k/(k-code)`` are +included. + +To emit only the ``cam --script`` YAML (no capture):: + + ./utils/measure-analogue-gain.py --emit-cam-script /tmp/cam.yaml \ + --gains 0,200,400,800 --model-k 1024 --exposure-us 10000 + + cam --camera 1 --stream role=raw,width=1928,height=1088 \ + --script /tmp/cam.yaml --capture=12 --file=/tmp/gain-#.bin + +Choose an exposure short enough that the highest gain code does not saturate +(watch the reported ``p95`` / ``max`` columns). If the scene is too dark, +increase exposure carefully and re-run. + +Measuring black level +--------------------- + +Prefer a **covered lens** dark frame at minimum analogue gain. If that is not +practical, a very short exposure at minimum gain is a useful approximation. + +.. code-block:: shell + + # Cover the lens if possible, then: + ./utils/measure-analogue-gain.py --sensor-name imx471 --dark \ + --width 1928 --height 1088 --stride 3904 --bit-depth 10 \ + --out /tmp/gain-measure + +For a 10-bit sensor pedestal of ``B`` DN, the 16-bit black level used by +helpers is typically ``B << 6`` (for example ``64`` → ``4096``). + +Do not invent a pedestal solely because another sensor in the same vendor +family uses that value; measure when the datasheet is missing. + +Sensor geometry and properties +------------------------------ + +``camera_sensor_properties.cpp`` entries should list: + +* ``unitCellSize`` in nanometres when known +* ``testPatternModes`` mapped to the modes the kernel driver registers +* ``sensorDelays`` only when verified + +Frame sizes and crop rectangles still come from the V4L2 subdev format and +selection API; properties do not replace a complete kernel driver. + +Submitting the result +--------------------- + +* Use ``git format-patch`` and ``git send-email`` so maintainers can ``git am`` + the series without MUA line wrapping (see :doc:`/contributing`). +* Split helper registration and static properties into separate commits when + both change. +* Put measurement methodology and tables in the commit message (and this guide + when adding or improving the tool), not large duplicated comments in the + helper constructor. + +Related reading +--------------- + +* :doc:`/sensor_driver_requirements` +* :doc:`/camera-sensor-model` +* :doc:`/guides/ipa` diff --git a/Documentation/index.rst b/Documentation/index.rst index e40cd0c..0712c27 100644 --- a/Documentation/index.rst +++ b/Documentation/index.rst @@ -22,6 +22,7 @@ Architecture Pipeline Handler Writer's Guide IPA Writer's guide + Camera Sensor Helper Guide Tracing guide Camera Sensor Model SoftwareISP Benchmarking diff --git a/Documentation/meson.build b/Documentation/meson.build index a156bd0..3e14498 100644 --- a/Documentation/meson.build +++ b/Documentation/meson.build @@ -158,6 +158,7 @@ if sphinx.found() 'design/ae.rst', 'feature_requirements.rst', 'guides/application-developer.rst', + 'guides/camera-sensor-helper.rst', 'guides/ipa.rst', 'guides/pipeline-handler.rst', 'guides/tracing.rst',