[{"id":29106,"web_url":"https://patchwork.libcamera.org/comment/29106/","msgid":"<20240327184640.GM4721@pendragon.ideasonboard.com>","date":"2024-03-27T18:46:40","subject":"Re: [PATCH v6 16/18] libcamera: Add \"Software ISP benchmarking\"\n\tdocumentation","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Milan and Hans,\n\nThank you for the patch.\n\nOn Tue, Mar 19, 2024 at 01:36:03PM +0100, Milan Zamazal wrote:\n> From: Hans de Goede \n> \n> Add a \"Software ISP benchmarking\" documentation section which describes\n> the performance/power consumption measurements used during\n> the Software ISP's development.\n\nDoesn't your text editor offer automated text reflows to the line length\nlimit ? :-)\n\n> Reviewed-by: Milan Zamazal \n> Signed-off-by: Hans de Goede \n> Reviewed-by: Stefan Klug \n> ---\n> Documentation/index.rst | 1 +\n> Documentation/meson.build | 1 +\n> Documentation/software-isp-benchmarking.rst | 82 +++++++++++++++++++++\n> 3 files changed, 84 insertions(+)\n> create mode 100644 Documentation/software-isp-benchmarking.rst\n> \n> diff --git a/Documentation/index.rst b/Documentation/index.rst\n> index 63fac72d..5442ae75 100644\n> --- a/Documentation/index.rst\n> +++ b/Documentation/index.rst\n> @@ -24,3 +24,4 @@\n> Lens driver requirements \n> Python Bindings \n> Camera Sensor Model \n> + SoftwareISP Benchmarking \n> diff --git a/Documentation/meson.build b/Documentation/meson.build\n> index 7a58fec8..3872e0a8 100644\n> --- a/Documentation/meson.build\n> +++ b/Documentation/meson.build\n> @@ -80,6 +80,7 @@ if sphinx.found()\n> 'lens_driver_requirements.rst',\n> 'python-bindings.rst',\n> 'sensor_driver_requirements.rst',\n> + 'software-isp-benchmarking.rst',\n> '../README.rst',\n> ]\n> \n> diff --git a/Documentation/software-isp-benchmarking.rst b/Documentation/software-isp-benchmarking.rst\n> new file mode 100644\n> index 00000000..a9a5edbe\n> --- /dev/null\n> +++ b/Documentation/software-isp-benchmarking.rst\n> @@ -0,0 +1,82 @@\n> +.. SPDX-License-Identifier: CC-BY-SA-4.0\n> +\n> +.. _software-isp-benchmarking:\n> +\n> +Software ISP benchmarking\n> +=========================\n> +\n\nThe same comment applies through this file actually.\n\n> +The Software ISP is particularly sensitive to performance regressions\n> +therefore it is a good idea to always benchmark the Software ISP\n> +before and after making changes to it and ensure that there are\n> +no performance regressions.\n\nThat will be interesting to add to CI.\n\nReviewed-by: Laurent Pinchart \n\n> +\n> +DebayerCpu class builtin benchmark\n> +----------------------------------\n> +\n> +The DebayerCpu class has a builtin benchmark. This benchmark\n> +measures the time spent on processing (collecting statistics\n> +and debayering) only, it does not measure the time spent on\n> +capturing or outputting the frames.\n> +\n> +The builtin benchmark always runs. So this can be used by simply\n> +running \"cam\" or \"qcam\" with a pipeline using the Software ISP.\n> +\n> +When it runs it will skip measuring the first 30 frames to\n> +allow the caches and the CPU temperature (turbo-ing) to warm-up\n> +and then it measures 30 fps and shows the total and per frame\n> +processing time using an info level log message:\n> +\n> +.. code-block:: text\n> +\n> + INFO Debayer debayer_cpu.cpp:907 Processed 30 frames in 244317us, 8143 us/frame\n> +\n> +To get stable measurements it is advised to disable any other processes which\n> +may cause significant CPU usage (e.g. disable wifi, bluetooth and browsers).\n> +When possible it is also advisable to disable CPU turbo-ing and\n> +frequency-scaling.\n> +\n> +For example when benchmarking on a Lenovo ThinkPad X1 Yoga Gen 8, with\n> +the charger plugged in, the CPU can be fixed to run at 2 GHz using:\n> +\n> +.. code-block:: shell\n> +\n> + sudo x86_energy_perf_policy --turbo-enable 0\n> + sudo cpupower frequency-set -d 2GHz -u 2GHz\n> +\n> +with these settings the builtin bench reports a processing time of ~7.8ms/frame\n> +on this laptop for FHD SGRBG10 (unpacked) bayer data.\n> +\n> +Measuring power consumption\n> +---------------------------\n> +\n> +Since the Software ISP is often used on mobile devices it is also\n> +important to measure power consumption and ensure that that does\n> +not regress.\n> +\n> +For example to measure power consumption on a Lenovo ThinkPad X1 Yoga Gen 8\n> +it needs to be running on battery and it should be configured with its\n> +platform-profile (/sys/firmware/acpi/platform_profile) set to balanced and\n> +with its default turbo and frequency-scaling behavior to match real world usage.\n> +\n> +Then start qcam to capture a FHD picture at 30 fps and position the qcam window\n> +so that it is fully visible. After this run the following command to monitor\n> +the power consumption:\n> +\n> +.. code-block:: shell\n> +\n> + watch -n 10 cat /sys/class/power_supply/BAT0/power_now /sys/class/hwmon/hwmon6/fan?_input\n> +\n> +Note this not only measures the power consumption in µW it also monitors\n> +the speed of this laptop's 2 fans. This is important because depending on\n> +the ambient temperature the 2 fans may spin up while testing and this\n> +will cause an additional power consumption of approx. 0.5 W messing up\n> +the measurement.\n> +\n> +After starting qcam + the watch command let the laptop sit without using\n> +it for 2 minutes for the readings to stabilize. Then check that the fans\n> +have not turned on and manually take a couple of consecutive power readings\n> +and average these.\n> +\n> +On the example Lenovo ThinkPad X1 Yoga Gen 8 laptop this results in\n> +a measured power consumption of approx. 13 W while running qcam versus\n> +approx. 4-5 W while setting idle with its OLED panel on.","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 00927C0DA4\n\tfor ;\n\tWed, 27 Mar 2024 18:46:51 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 9953963331;\n\tWed, 27 Mar 2024 19:46:50 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 3ABC761C35\n\tfor ;\n\tWed, 27 Mar 2024 19:46:49 +0100 (CET)","from pendragon.ideasonboard.com (81-175-209-231.bb.dnainternet.fi\n\t[81.175.209.231])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 2F533675;\n\tWed, 27 Mar 2024 19:46:16 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"D1Jkhw+8\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1711565176;\n\tbh=msokziJ2gTNx6TfFdTSan0hsAgNk7qJ1n0eeLc5e1VM=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=D1Jkhw+8pdFl9EiS8seGWxfTM2YVLPRd57xArVuOkp8/lbS4B48lceaTYGUonm4TN\n\tMlAztO8mWxOyE4dmQTiFFVr7kIvLFA50LUb+Yh4IVkZ1+dEaU8L8i4XjPqACYCUuVI\n\tiBSQ+QczH3OszAtpHotq/u2SqX0h0kC/TJ+4dlDM=","Date":"Wed, 27 Mar 2024 20:46:40 +0200","From":"Laurent Pinchart ","To":"Milan Zamazal ","Cc":"libcamera-devel@lists.libcamera.org, Hans de Goede ,\n\tAndrey Konovalov ,\n\tBryan O'Donoghue ,\n\tMaxime Ripard , Pavel Machek ,\n\tKieran Bingham ,\n\tStefan Klug ","Subject":"Re: [PATCH v6 16/18] libcamera: Add \"Software ISP benchmarking\"\n\tdocumentation","Message-ID":"<20240327184640.GM4721@pendragon.ideasonboard.com>","References":"<20240319123622.675599-1-mzamazal@redhat.com>\n\t<20240319123622.675599-17-mzamazal@redhat.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","Content-Transfer-Encoding":"8bit","In-Reply-To":"<20240319123622.675599-17-mzamazal@redhat.com>","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]