From patchwork Mon Aug 10 17:46:11 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: John Cronin X-Patchwork-Id: 27733 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 93015BE080 for ; Mon, 10 Aug 2026 17:46:21 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 55E0B68209; Mon, 10 Aug 2026 19:46:21 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=stromback-com.20251104.gappssmtp.com header.i=@stromback-com.20251104.gappssmtp.com header.b="XO0AIBUr"; dkim-atps=neutral Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 73FBF681FC for ; Mon, 10 Aug 2026 19:46:16 +0200 (CEST) Received: by mail-wr1-x431.google.com with SMTP id ffacd0b85a97d-4799b3f7c83so1384638f8f.2 for ; Mon, 10 Aug 2026 10:46:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=stromback-com.20251104.gappssmtp.com; s=20251104; t=1786383976; x=1786988776; darn=lists.libcamera.org; h=content-transfer-encoding:content-type:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to:content-type; bh=29flHvpEQFTHVL8xaiOE7lIwnai795l7bDKFZdaQ4d8=; b=XO0AIBUrnVQSyl2PrSGxVUEBe27s1Vakxg07OLUrKfdZfVW7HE31hHKtk42G3gIve2 OGTrxMhurMuOrBKcW0d9mdZdX/flveiuiJ/OPjpTjp1pNljQU1kTpEVt5tEn0EacWP+u LVAgScWxUfz4JRb+XqHizvlBFBZwokrlvRgd9yr+XtmWQe3/DiZpmQDveC3EiNJxEYMB UlLuY77e+DxAfWYH85rrIQtedSbnh6BCVHIChBpDrbgqrfVfjVx60ZbiQumweYJi6Izr AwXfFsCMyD5H4A0qg4G8F6mbGx0rlXS0SL9uyX8EYOOz0y9NmPu4Rd8AC283PiJIrxon cuOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1786383976; x=1786988776; h=content-transfer-encoding:content-type:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=29flHvpEQFTHVL8xaiOE7lIwnai795l7bDKFZdaQ4d8=; b=l+25e0hyEddn7BgEifg+20sDf7aoWa6BdZL9aB1fYxIWqFQwAIWlnzxMI9xnGhC5qJ MwZNXD0uH8NTDE9/IlblknaaqPbVSBowmFbpqdViuSnmC1Zki+RxCAKyoB0Pyf1z7NjF XzrSrlY5JX+h2Wcue9vTgzKlrFWFsaZ7UJSck0STt6YcL89Mts0BMuYye4Lncbqup5iQ NIHED33LACHWRXIuDFb4h2p1s8mM3sCjNwEC5M7cVutPa8+4MYB7EurPN2pRvPg3MksT SBcuaAobw1j5+ZgIOnkjLfKge2RxVSs/oHkS+CgQdgJ4Oqo4b5lWlPwCmtEiX0ftR0Be K+Zg== X-Gm-Message-State: AOJu0YxgmEI5kgX2a62/fP9s62RuyBIBRTyjuAn2POTKPHs4j7snzUM+ jPxzLesXikSz+yUtlqWaGYK1GF/fEPUr7JKTHrlM4CMvkCYFijX+VsLtTIPlyPoOcoZesWLS9GW Ht3x0iZ/ym9IM X-Gm-Gg: AR+sD12I6eGmnFgu6/Kh0c62ovZg3vaNkK5LgmBq41tUCP1Hp2ru0rFyqMPKFn8E+ao yNChujjZBcG6RyH9gntUYRvnB7UaA34FLHZ2YCgsyuPBk7jIMft56irpJYLQMckSXNWXIhl/Y+n oOfmItALcHyWeAsi0zjc2YZtIOiGSupOfcSjsoBFhRiHYfbp3KwNtAFb93p8OIiSxqKv+24Ev52 Ka+L8xU+kQ5zCKN94t46cAqUIrUZeqM9RqUgXMcdFCjG9kunuVmJlHYgWzYXiZbvAZ0NnRH4QIW tYBucCPr98Ozp6iC79K50HL051YaDqkh9mfp7k4pnx6fgZiWlZoXCVARZC4h9Uy0FOf8zjcXa6A NauGI+8pHOa54Cv9xTHpzHVgQ+J1BMHnjDvdAaWAyb40gJje1+6bvbNGoITTd2rDf4mjeA5l5Sv UojOx5MAHY8iUuSry5ywf3MVdSxN5ReJGvtL51CJASQVu9EfZ6TdIVxtEA9iTAjk8lrbnKZuZNz fvDDtuD94fzawsDItzMRvgVzZcyX44vucYLcuKqrAfAUDd/GmAttgfAA9UKANSfYUlT7mrOfbpi CAL7CTqM8kSrwUZsPyTxlbqHZ/7N3IjsKHhBA4PBSD0ITWCqrgrcllsUjoCdWTyl0Rewn7nKvOz f7GXCnapFwhLIzSOy8muspCdErIDjmHraKtDhU+lQAmseULDZ/k/BjIienkvWXJD5nTU9UQ== X-Received: by 2002:a05:6000:2f89:b0:47f:c62e:9cca with SMTP id ffacd0b85a97d-4814564d767mr5833197f8f.22.1786383975994; Mon, 10 Aug 2026 10:46:15 -0700 (PDT) Received: from fedora.tail5f8cb3.ts.net (251-61.dsl.iskon.hr. [89.164.251.61]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-480021506desm35040583f8f.10.2026.08.10.10.46.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Aug 2026 10:46:15 -0700 (PDT) From: John Cronin X-Google-Original-From: John Cronin To: libcamera-devel@lists.libcamera.org Cc: laurent.pinchart@ideasonboard.com, jacopo.mondi@ideasonboard.com, barnabas.pocze@ideasonboard.com, stefan.klug@ideasonboard.com, John Cronin Subject: [PATCH v2 2/2] Documentation: Add Camera Sensor Helper guide Date: Mon, 10 Aug 2026 13:46:11 -0400 Message-ID: <20260810174611.2472046-3-john.cronin@opcenter.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260810174611.2472046-1-john.cronin@opcenter.com> References: <20260810174611.2472046-1-john.cronin@opcenter.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Document how to add a CameraSensorHelper, measure analogue gain and black level with utils/measure-analogue-gain.py (run from the source tree), and what to put in camera_sensor_properties versus the helper. Signed-off-by: John Cronin --- Documentation/guides/camera-sensor-helper.rst | 148 ++++++++++++++++++ Documentation/index.rst | 1 + Documentation/meson.build | 1 + 3 files changed, 150 insertions(+) create mode 100644 Documentation/guides/camera-sensor-helper.rst diff --git a/Documentation/guides/camera-sensor-helper.rst b/Documentation/guides/camera-sensor-helper.rst new file mode 100644 index 0000000..6e62624 --- /dev/null +++ b/Documentation/guides/camera-sensor-helper.rst @@ -0,0 +1,148 @@ +.. SPDX-License-Identifier: CC-BY-SA-4.0 + +Camera Sensor Helper Guide +========================== + +This guide explains how to add a ``CameraSensorHelper`` for a new sensor and how +to measure the analogue gain model and black level when a public datasheet is +not available. + +Background +---------- + +The software ISP and other IPA modules need to know how the sensor maps +analogue gain control codes to linear gain factors, and which digital number +corresponds to optical black. That information lives in +``src/ipa/libipa/camera_sensor_helper.cpp`` as a small helper class registered +with ``REGISTER_CAMERA_SENSOR_HELPER``. + +Static metadata such as unit cell size and test pattern mode maps belong in +``src/libcamera/sensor/camera_sensor_properties.cpp``. Sensor control application +delays should only be listed there when they have been measured or documented; +leave ``sensorDelays`` empty to use libcamera defaults. + +Adding a helper +--------------- + +1. Confirm the kernel driver name (for example ``imx471``) and the V4L2 control + ranges for exposure and analogue gain (``v4l2-ctl -d /dev/v4l-subdevX + --list-ctrls``). +2. Prefer values from a datasheet when available. Document the source in the + commit message and, if short, a brief code comment (for example + ``/* From datasheet: 64 at 10bits. */``). +3. If no datasheet is available, measure gain response and black level as + described below. Put the measurement summary in the **commit message**; keep + in-code comments short. +4. Register the helper under the same string the kernel uses for the subdev + name (without bus address suffixes). + +Common Sony sensors program a linear code ``c`` such that: + +.. math:: + + G = \frac{k}{k - c} + +with ``k = 1024`` for many IMX models. The maximum V4L2 code may still be lower +than ``k - 1`` (for example codes ``0..800``), which yields a modest maximum +gain even when the model is correct. + +Measuring analogue gain +----------------------- + +The ``utils/measure-analogue-gain.py`` helper automates a fixed-exposure gain +sweep using the libcamera ``cam`` tool and ``v4l2-ctl``. It is a developer +tool intended to be run from a libcamera source tree (it is not installed +with the package). + +Dependencies: + +* ``cam`` (libcamera tools), or set ``LIBCAMERA_CAM`` to a wrapper/command +* ``v4l2-ctl`` from v4l-utils +* Python 3.10+ + +High-level procedure: + +1. Capture **raw** Bayer frames (``cam --stream role=raw,...``), not processed + RGB. Soft ISP AGC is avoided so the V4L2 codes you set stay put. +2. Lock exposure (and digital gain if present) on the sensor subdev. +3. Step ``V4L2_CID_ANALOGUE_GAIN`` across the driver range. +4. Compute the mean of active-area samples (useful bit depth, for example + 10-bit values carried in 16-bit words). +5. Black-subtract using a low percentile at minimum gain, or a dark frame. +6. Fit measured brightness ratios to ``G = k/(k-code)`` and compare with the + model you intend to hard-code (often ``k = 1024``). + +Terminology used by the tool: + +* **p1** — first percentile of the sample histogram (near-black floor) +* **p50** — median +* **signal** — ``mean - black`` after black subtraction +* **ratio** — ``signal(code) / signal(0)`` + +Example (Sony IMX471 on an IPU7 laptop, 1928×1088, stride 3904): + +.. code-block:: shell + + ./utils/measure-analogue-gain.py \ + --sensor-name imx471 \ + --width 1928 --height 1088 --stride 3904 --bit-depth 10 \ + --exposure 200 --digital-gain 256 \ + --gains 0,50,100,150,200,300,400,500,600,700,800 \ + --model-k 1024 \ + --out /tmp/gain-measure + +The tool writes ``results.csv`` and ``results.json`` including relative error +versus the reference model and a best-fit ``k`` for ``G=k/(k-code)``. + +Choose an exposure short enough that the highest gain code does not saturate +(watch the reported ``p95`` / ``max`` columns). If the scene is too dark, +increase exposure carefully and re-run. + +Measuring black level +--------------------- + +Prefer a **covered lens** dark frame at minimum analogue gain. If that is not +practical, a very short exposure at minimum gain is a useful approximation. + +.. code-block:: shell + + # Cover the lens if possible, then: + ./utils/measure-analogue-gain.py --sensor-name imx471 --dark \ + --width 1928 --height 1088 --stride 3904 --bit-depth 10 \ + --out /tmp/gain-measure + +For a 10-bit sensor pedestal of ``B`` DN, the 16-bit black level used by +helpers is typically ``B << 6`` (for example ``64`` → ``4096``). + +Do not invent a pedestal solely because another sensor in the same vendor +family uses that value; measure when the datasheet is missing. + +Sensor geometry and properties +------------------------------ + +``camera_sensor_properties.cpp`` entries should list: + +* ``unitCellSize`` in nanometres when known +* ``testPatternModes`` mapped to the modes the kernel driver registers +* ``sensorDelays`` only when verified + +Frame sizes and crop rectangles still come from the V4L2 subdev format and +selection API; properties do not replace a complete kernel driver. + +Submitting the result +--------------------- + +* Use ``git format-patch`` and ``git send-email`` so maintainers can ``git am`` + the series without MUA line wrapping (see :doc:`/contributing`). +* Split helper registration and static properties into separate commits when + both change. +* Put measurement methodology and tables in the commit message (and this guide + when adding or improving the tool), not large duplicated comments in the + helper constructor. + +Related reading +--------------- + +* :doc:`/sensor_driver_requirements` +* :doc:`/camera-sensor-model` +* :doc:`/guides/ipa` diff --git a/Documentation/index.rst b/Documentation/index.rst index e40cd0c..0712c27 100644 --- a/Documentation/index.rst +++ b/Documentation/index.rst @@ -22,6 +22,7 @@ Architecture Pipeline Handler Writer's Guide IPA Writer's guide + Camera Sensor Helper Guide Tracing guide Camera Sensor Model SoftwareISP Benchmarking diff --git a/Documentation/meson.build b/Documentation/meson.build index a156bd0..3e14498 100644 --- a/Documentation/meson.build +++ b/Documentation/meson.build @@ -158,6 +158,7 @@ if sphinx.found() 'design/ae.rst', 'feature_requirements.rst', 'guides/application-developer.rst', + 'guides/camera-sensor-helper.rst', 'guides/ipa.rst', 'guides/pipeline-handler.rst', 'guides/tracing.rst',