From patchwork Sat Aug 8 12:59:14 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: John Cronin X-Patchwork-Id: 27671 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0C8F8C3303 for ; Sat, 8 Aug 2026 12:59:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 708FC68193; Sat, 8 Aug 2026 14:59:51 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=stromback-com.20251104.gappssmtp.com header.i=@stromback-com.20251104.gappssmtp.com header.b="LaDn3YZJ"; dkim-atps=neutral Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 038CD68164 for ; Sat, 8 Aug 2026 14:59:50 +0200 (CEST) Received: by mail-wr1-x436.google.com with SMTP id ffacd0b85a97d-480033bdcf4so170234f8f.2 for ; Sat, 08 Aug 2026 05:59:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=stromback-com.20251104.gappssmtp.com; s=20251104; t=1786193989; x=1786798789; darn=lists.libcamera.org; h=content-transfer-encoding:content-type:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to:content-type; bh=BrfVOufEJ6NCldU8kh3tbSsTZscsYgQz3AZ4p7FUy2A=; b=LaDn3YZJqKu5jBtFRaGYwgtArdPXWDRE6Sc3VIEw/S2hUDjbVzaaQC6JHfe8AOf7QD z9WKQqVmdEvakvb9faRgDlpxaA3jFgOfgBRYkIcGe2lSOHj5NN+T/uJj5c5NUE/QxnU5 i7q+buptROKW5gkjLHCPnrhLfVHXaac1RwMLA9dytvxkhskI7JyMfSavten/Ys6Reh+H QAGlyi1IMteL8ZmccyZ16DJ4zPMs916AhHLy0YqiZSvebNfBaexi7dD+e+64cQR4wIhO hw4uSmAua1FFHB8u7nZleCW2n2NfJkvYQs4ESQ3PqpJVFu7hAn5aeVFCyjNMLPU7mMNp EIbQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1786193989; x=1786798789; h=content-transfer-encoding:content-type:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=BrfVOufEJ6NCldU8kh3tbSsTZscsYgQz3AZ4p7FUy2A=; b=L+kWLy2jwbgpwl0BnvxqrI0QuS7o8A3RvXHcg0ep0pRT6SQ8cjLvlU6FZHA4O/SUkn q3GQSAfufS9fWKZMiGqnItFCDFNA399L58R83X0Se7qSFiBLc3H5X582hR1Lige7fBFp UwtNJiek28UWC3oHquq9zSKiWT5eiIfAK2ae+jNbFXUVIKPYlR8KR3lbuQPaD0OeRFgp Ls+fj3cC97OyJpbDuNoNXs0274+LTtD4k1FJX0x1fnQSiZpaJGynmJOSsf1BlTEyooZh vRBiAxFUOkgYASLkSLE/F8IKCRgMjKXK+lMPPpF06hlY2jqntteMGYsq6bYKtCputRAQ AZ9w== X-Gm-Message-State: AOJu0YxB/S1n6Q34q2WBLAh3DUGF6fvxg4CYdJf/ybh/xa2X5WhUCD8Q Zsy1l9ZNcGZC9JCP15eKZVk9fRo4jawTDY2ajdOq2eEVWC8cIwELDykUkTPLaHIjZUhJj4nx7Nq qQC/gBKAcpg== X-Gm-Gg: AR+sD139FViPdsxBelNfRlqMdEfSkja6mTWUEEoZ+S9qUP5AbxfcWA13rM2lsNNvVJE wOzafwJa06wdIxTojkYSkX6MGDIffpMENGR5EhFdvRpUdkryFDmKSGPPGbNZW9xFEJDRkJKlKt/ wGc8no1J/EO6N6DKjSZOhuIlAug/DocA959STOyrKb2+JLySweVbrPYzJGNeJHU/mqJSJkD5bVg G1JuQkwwEQ53RUUOzCy9j7VH/0ce5mXzwhorcXmChm2sojkRpT8BpFOO1ha7Qm5hBJvLpNGNsqz WuAnowQV3+auQokV1GFmDzAx1FEFwS8z3FJNRfHmBYJt3v8IX/dgs6JMmUhIOrIxPIRL6OJfizv FjG1dOgaXQhouy0UU3ZRqPKUfUsMiQwQI1shW1+EqVI/mDd1hu2xIhDVBu77KleClNbnJdx5GSs OCxjxMh3AjMPSQ5/KLMngfwPmaVshNm3q7BTdwdxKPM6AS53LWsOD+sgS0gyRa7wwIvk1zgPYrz Grw7YWDQhjiycV/dKEb0tMqMvixmDye0DFisXyoeVPCwDqbWEboUO2mzlvhZTT0bSrIKumxO8UL HVwZKXnEHz7O9gJmtkF9h3Y6c4GW6J2QGspJqA48TW+12FNfqdkOUUkxvoVCeLls1qyJW7oINdz vdthOCZO/I7AMxSZwFjkco4ahL6kfdoUtKHmNXRqssacl3gjPrS7K3R2+fa9C9TEO3vC6QGCtcl DlsVkS/GY8ySc4vUzukM4= X-Received: by 2002:a05:6000:29d8:b0:47f:95e8:2ebf with SMTP id ffacd0b85a97d-47fec51b693mr33720085f8f.12.1786193989116; Sat, 08 Aug 2026 05:59:49 -0700 (PDT) Received: from fedora.tail5f8cb3.ts.net (host-79-58-5-82.business.telecomitalia.it. [79.58.5.82]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-480021e7b3fsm16161616f8f.20.2026.08.08.05.59.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 08 Aug 2026 05:59:48 -0700 (PDT) From: John Cronin X-Google-Original-From: John Cronin To: libcamera-devel@lists.libcamera.org Cc: jacopo.mondi@ideasonboard.com, John Cronin Subject: [PATCH 2/2] Documentation: Add Camera Sensor Helper guide Date: Sat, 8 Aug 2026 08:59:14 -0400 Message-ID: <20260808125915.974559-3-john.cronin@opcenter.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260808125915.974559-1-john.cronin@opcenter.com> References: <20260808125915.974559-1-john.cronin@opcenter.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Document how to add a CameraSensorHelper, measure analogue gain and black level with utils/measure-analogue-gain.py, and what to put in camera_sensor_properties versus the helper. Signed-off-by: John Cronin --- Documentation/guides/camera-sensor-helper.rst | 146 ++++++++++++++++++ Documentation/index.rst | 1 + Documentation/meson.build | 1 + 3 files changed, 148 insertions(+) create mode 100644 Documentation/guides/camera-sensor-helper.rst diff --git a/Documentation/guides/camera-sensor-helper.rst b/Documentation/guides/camera-sensor-helper.rst new file mode 100644 index 0000000..21e3219 --- /dev/null +++ b/Documentation/guides/camera-sensor-helper.rst @@ -0,0 +1,146 @@ +.. SPDX-License-Identifier: CC-BY-SA-4.0 + +Camera Sensor Helper Guide +========================== + +This guide explains how to add a ``CameraSensorHelper`` for a new sensor and how +to measure the analogue gain model and black level when a public datasheet is +not available. + +Background +---------- + +The software ISP and other IPA modules need to know how the sensor maps +analogue gain control codes to linear gain factors, and which digital number +corresponds to optical black. That information lives in +``src/ipa/libipa/camera_sensor_helper.cpp`` as a small helper class registered +with ``REGISTER_CAMERA_SENSOR_HELPER``. + +Static metadata such as unit cell size and test pattern mode maps belong in +``src/libcamera/sensor/camera_sensor_properties.cpp``. Sensor control application +delays should only be listed there when they have been measured or documented; +leave ``sensorDelays`` empty to use libcamera defaults. + +Adding a helper +--------------- + +1. Confirm the kernel driver name (for example ``imx471``) and the V4L2 control + ranges for exposure and analogue gain (``v4l2-ctl -d /dev/v4l-subdevX + --list-ctrls``). +2. Prefer values from a datasheet when available. Document the source in the + commit message and, if short, a brief code comment (for example + ``/* From datasheet: 64 at 10bits. */``). +3. If no datasheet is available, measure gain response and black level as + described below. Put the measurement summary in the **commit message**; keep + in-code comments short. +4. Register the helper under the same string the kernel uses for the subdev + name (without bus address suffixes). + +Common Sony sensors program a linear code ``c`` such that: + +.. math:: + + G = \frac{k}{k - c} + +with ``k = 1024`` for many IMX models. The maximum V4L2 code may still be lower +than ``k - 1`` (for example codes ``0..800``), which yields a modest maximum +gain even when the model is correct. + +Measuring analogue gain +----------------------- + +The ``utils/measure-analogue-gain.py`` helper automates a fixed-exposure gain +sweep using the libcamera ``cam`` tool and ``v4l2-ctl``. + +Dependencies: + +* ``cam`` (libcamera tools), or set ``LIBCAMERA_CAM`` to a wrapper/command +* ``v4l2-ctl`` from v4l-utils +* Python 3.10+ + +High-level procedure: + +1. Capture **raw** Bayer frames (``cam --stream role=raw,...``), not processed + RGB. Soft ISP AGC is avoided so the V4L2 codes you set stay put. +2. Lock exposure (and digital gain if present) on the sensor subdev. +3. Step ``V4L2_CID_ANALOGUE_GAIN`` across the driver range. +4. Compute the mean of active-area samples (useful bit depth, for example + 10-bit values carried in 16-bit words). +5. Black-subtract using a low percentile at minimum gain, or a dark frame. +6. Fit measured brightness ratios to ``G = k/(k-code)`` and compare with the + model you intend to hard-code (often ``k = 1024``). + +Terminology used by the tool: + +* **p1** — first percentile of the sample histogram (near-black floor) +* **p50** — median +* **signal** — ``mean - black`` after black subtraction +* **ratio** — ``signal(code) / signal(0)`` + +Example (Sony IMX471 on an IPU7 laptop, 1928×1088, stride 3904): + +.. code-block:: shell + + ./utils/measure-analogue-gain.py \ + --sensor-name imx471 \ + --width 1928 --height 1088 --stride 3904 --bit-depth 10 \ + --exposure 200 --digital-gain 256 \ + --gains 0,50,100,150,200,300,400,500,600,700,800 \ + --model-k 1024 \ + --out /tmp/gain-measure + +The tool writes ``results.csv`` and ``results.json`` including relative error +versus the reference model and a best-fit ``k`` for ``G=k/(k-code)``. + +Choose an exposure short enough that the highest gain code does not saturate +(watch the reported ``p95`` / ``max`` columns). If the scene is too dark, +increase exposure carefully and re-run. + +Measuring black level +--------------------- + +Prefer a **covered lens** dark frame at minimum analogue gain. If that is not +practical, a very short exposure at minimum gain is a useful approximation. + +.. code-block:: shell + + # Cover the lens if possible, then: + ./utils/measure-analogue-gain.py --sensor-name imx471 --dark \ + --width 1928 --height 1088 --stride 3904 --bit-depth 10 \ + --out /tmp/gain-measure + +For a 10-bit sensor pedestal of ``B`` DN, the 16-bit black level used by +helpers is typically ``B << 6`` (for example ``64`` → ``4096``). + +Do not invent a pedestal solely because another sensor in the same vendor +family uses that value; measure when the datasheet is missing. + +Sensor geometry and properties +------------------------------ + +``camera_sensor_properties.cpp`` entries should list: + +* ``unitCellSize`` in nanometres when known +* ``testPatternModes`` mapped to the modes the kernel driver registers +* ``sensorDelays`` only when verified + +Frame sizes and crop rectangles still come from the V4L2 subdev format and +selection API; properties do not replace a complete kernel driver. + +Submitting the result +--------------------- + +* Use ``git format-patch`` and ``git send-email`` so maintainers can ``git am`` + the series without MUA line wrapping (see :doc:`/contributing`). +* Split helper registration and static properties into separate commits when + both change. +* Put measurement methodology and tables in the commit message (and this guide + when adding or improving the tool), not large duplicated comments in the + helper constructor. + +Related reading +--------------- + +* :doc:`/sensor_driver_requirements` +* :doc:`/camera-sensor-model` +* :doc:`/guides/ipa` diff --git a/Documentation/index.rst b/Documentation/index.rst index e40cd0c..0712c27 100644 --- a/Documentation/index.rst +++ b/Documentation/index.rst @@ -22,6 +22,7 @@ Architecture Pipeline Handler Writer's Guide IPA Writer's guide + Camera Sensor Helper Guide Tracing guide Camera Sensor Model SoftwareISP Benchmarking diff --git a/Documentation/meson.build b/Documentation/meson.build index a156bd0..3e14498 100644 --- a/Documentation/meson.build +++ b/Documentation/meson.build @@ -158,6 +158,7 @@ if sphinx.found() 'design/ae.rst', 'feature_requirements.rst', 'guides/application-developer.rst', + 'guides/camera-sensor-helper.rst', 'guides/ipa.rst', 'guides/pipeline-handler.rst', 'guides/tracing.rst',