[{"id":28837,"web_url":"https://patchwork.libcamera.org/comment/28837/","msgid":"<87frx5anmf.fsf@redhat.com>","date":"2024-03-04T17:54:48","subject":"Re: [PATCH v4 16/18] libcamera: Add \"Software ISP benchmarking\"\n\tdocumentation","submitter":{"id":177,"url":"https://patchwork.libcamera.org/api/people/177/","name":"Milan Zamazal","email":"mzamazal@redhat.com"},"content":"Hans de Goede writes:\n\n> Add a \"Software ISP benchmarking\" documentation section which describes\n> the performance/power consumption measurements used during\n> the Software ISP's development.\n>\n> Signed-off-by: Hans de Goede \n\nMy comments have been addressed:\n\nReviewed-by: Milan Zamazal \n\nIt still needs to be reviewed by a native speaker. (And I still wonder whether\nbetter methods could be utilized with a modern hardware; but in any case, the\ndescribed method fulfills its purpose.)\n\n> ---\n> Documentation/index.rst | 1 +\n> Documentation/meson.build | 1 +\n> Documentation/software-isp-benchmarking.rst | 82 +++++++++++++++++++++\n> 3 files changed, 84 insertions(+)\n> create mode 100644 Documentation/software-isp-benchmarking.rst\n>\n> diff --git a/Documentation/index.rst b/Documentation/index.rst\n> index 63fac72d..5442ae75 100644\n> --- a/Documentation/index.rst\n> +++ b/Documentation/index.rst\n> @@ -24,3 +24,4 @@\n> Lens driver requirements \n> Python Bindings \n> Camera Sensor Model \n> + SoftwareISP Benchmarking \n> diff --git a/Documentation/meson.build b/Documentation/meson.build\n> index 7a58fec8..3872e0a8 100644\n> --- a/Documentation/meson.build\n> +++ b/Documentation/meson.build\n> @@ -80,6 +80,7 @@ if sphinx.found()\n> 'lens_driver_requirements.rst',\n> 'python-bindings.rst',\n> 'sensor_driver_requirements.rst',\n> + 'software-isp-benchmarking.rst',\n> '../README.rst',\n> ]\n> \n> diff --git a/Documentation/software-isp-benchmarking.rst b/Documentation/software-isp-benchmarking.rst\n> new file mode 100644\n> index 00000000..d94595d5\n> --- /dev/null\n> +++ b/Documentation/software-isp-benchmarking.rst\n> @@ -0,0 +1,82 @@\n> +.. SPDX-License-Identifier: CC-BY-SA-4.0\n> +\n> +.. _software-isp-benchmarking:\n> +\n> +Software ISP benchmarking\n> +=========================\n> +\n> +The Software ISP is particularly sensitive to performance regressions\n> +therefore it is a good idea to always benchmark the Software ISP\n> +before and after making changes to it and ensure that there are\n> +no performance regressions.\n> +\n> +DebayerCpu class builtin benchmark\n> +----------------------------------\n> +\n> +The DebayerCpu class has a builtin benchmark. This benchmark\n> +measures the time spent on processing (collecting statistics\n> +and debayering) only, it does not measure the time spent on\n> +capturing or outputting the frames.\n> +\n> +The builtin benchmark always runs. So this can be used by simply\n> +running \"cam\" or \"qcam\" with a pipeline using the Software ISP.\n> +\n> +When it runs it will skip measuring the first 30 frames to\n> +allow the caches and the CPU temperature (turbo-ing) to warm-up\n> +and then it measures 30 fps and shows the total and per frame\n> +processing time using an info level log message:\n> +\n> +.. code-block::\n> +\n> + INFO Debayer debayer_cpu.cpp:907 Processed 30 frames in 244317us, 8143 us/frame\n> +\n> +To get stable measurements it is advised to disable any other processes which\n> +may cause significant CPU usage (e.g. disable wifi, bluetooth and browsers).\n> +When possible it is also advisable to disable CPU turbo-ing and\n> +frequency-scaling.\n> +\n> +For example when benchmarking on a Lenovo ThinkPad X1 Yoga Gen 8, with\n> +the charger plugged in, the CPU can be fixed to run at 2 GHz using:\n> +\n> +.. code-block::\n> +\n> + sudo x86_energy_perf_policy --turbo-enable 0\n> + sudo cpupower frequency-set -d 2GHz -u 2GHz\n> +\n> +with these settings the builtin bench reports a processing time of ~7.8ms/frame\n> +on this laptop for FHD SGRBG10 (unpacked) bayer data.\n> +\n> +Measuring power consumption\n> +---------------------------\n> +\n> +Since the Software ISP is often used on mobile devices it is also\n> +important to measure power consumption and ensure that that does\n> +not regress.\n> +\n> +For example to measure power consumption on a Lenovo ThinkPad X1 Yoga Gen 8\n> +it needs to be running on battery and it should be configured with its\n> +platform-profile (/sys/firmware/acpi/platform_profile) set to balanced and\n> +with its default turbo and frequency-scaling behavior to match real world usage.\n> +\n> +Then start qcam to capture a FHD picture at 30 fps and position the qcam window\n> +so that it is fully visible. After this run the following command to monitor\n> +the power consumption:\n> +\n> +.. code-block::\n> +\n> + watch -n 10 cat /sys/class/power_supply/BAT0/power_now /sys/class/hwmon/hwmon6/fan?_input\n> +\n> +Note this not only measures the power consumption in µW it also monitors\n> +the speed of this laptop's 2 fans. This is important because depending on\n> +the ambient temperature the 2 fans may spin up while testing and this\n> +will cause an additional power consumption of approx. 0.5 W messing up\n> +the measurement.\n> +\n> +After starting qcam + the watch command let the laptop sit without using\n> +it for 2 minutes for the readings to stabilize. Then check that the fans\n> +have not turned on and manually take a couple of consecutive power readings\n> +and avarage these.\n> +\n> +On the example Lenovo ThinkPad X1 Yoga Gen 8 laptop this results in\n> +a measured power consumption of approx. 13 W while running qcam versus\n> +approx. 4-5 W while setting idle with its OLED panel on.","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id CDBFCBD160\n\tfor ;\n\tMon, 4 Mar 2024 17:54:56 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 2239862872;\n\tMon, 4 Mar 2024 18:54:56 +0100 (CET)","from us-smtp-delivery-124.mimecast.com\n\t(us-smtp-delivery-124.mimecast.com [170.10.129.124])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 58F78627FC\n\tfor ;\n\tMon, 4 Mar 2024 18:54:54 +0100 (CET)","from mail-wm1-f71.google.com (mail-wm1-f71.google.com\n\t[209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS\n\t(version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id\n\tus-mta-541-OsIv_5FMNuy0J00YALtftA-1; Mon, 04 Mar 2024 12:54:52 -0500","by mail-wm1-f71.google.com with SMTP id\n\t5b1f17b1804b1-412c9e3c9b9so15833875e9.0\n\tfor ;\n\tMon, 04 Mar 2024 09:54:51 -0800 (PST)","from nuthatch (nat-pool-brq-t.redhat.com. [213.175.37.10])\n\tby smtp.gmail.com with ESMTPSA id\n\tu22-20020a05600c139600b0040fdf5e6d40sm15841365wmf.20.2024.03.04.09.54.49\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tMon, 04 Mar 2024 09:54:49 -0800 (PST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=redhat.com header.i=@redhat.com\n\theader.b=\"DytR1TXq\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;\n\ts=mimecast20190719; t=1709574893;\n\th=from:from:reply-to:subject:subject:date:date:message-id:message-id:\n\tto:to:cc:cc:mime-version:mime-version:content-type:content-type:\n\tcontent-transfer-encoding:content-transfer-encoding:\n\tin-reply-to:in-reply-to:references:references;\n\tbh=bphF4tfdam2YpMiQon6+7xeCg1BA4OhcPX05KQR5Zpk=;\n\tb=DytR1TXqq3+mS97CBpC9bPA2VseMv0uBI8VmFmWF1orbGrMHK+EIf5y5tKS2p9hd/4azwu\n\tt75DEugF9v72Cx9dgLQAA8t7/uDZPWGV6bsMhHO81fitCBg3JPnZVB3hvu41hdPOLi98NF\n\tgb4EMmU4Bgv9O0mx0Lqber0YXfKK/zA=","X-MC-Unique":"OsIv_5FMNuy0J00YALtftA-1","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20230601; t=1709574890; x=1710179690;\n\th=content-transfer-encoding:mime-version:user-agent:message-id:date\n\t:references:in-reply-to:subject:cc:to:from:x-gm-message-state:from\n\t:to:cc:subject:date:message-id:reply-to;\n\tbh=mIY7/ZIvUXwJ55HUDkl9h0QH1SsiMZx40nNQc4+ORL0=;\n\tb=AkyGyv2vrvmJfTgvMRq9EowImxYqbkXIE4qrM93MNuhptdqU01I7D7N6fnxyPhabyH\n\tuU5YflCRHr3C9NgHceb/5Mx6+h6sp/EALndldOqjiaNduWUC0Zfxsfpvfv9J/K0aFlX1\n\t29wwnTlPLkedH+I54EgGj9izoTIul36aWEpR1thbZS5H+lAIXKahHZf3POjGCKUhVEpM\n\tjYaqd/MfgesKbGPzxdzWQSNXNN8KaUDoJjGQ5f7VIit+sYwr1+lWN3OtTnWXpR6kFZAR\n\tn0scktnE2TCej1PVP20pDCTQdjh6IB7CKrbbh/2C9vdmrHbG4obJkihsADs39HKPUwQU\n\t1qBw==","X-Gm-Message-State":"AOJu0YwD7BsXxOd4QZWCFNkFXRCraUc0Vt0DAMunXCWicKStx1YRY8fb\n\tH06YzYFaQboCTAjeGqIGSb1DK72RNus7+aDI0adjJoOvMoyhDmpRh6vE9MbJDiRt6N/ILBbCGdr\n\t1uBzH/VKayWVVLpBc5NAn/t2iErLqmEPMuRL/BjDemF6BX3P9RUOD1WYczduUMNWRa8s0qCKuJR\n\tVm8hk=","X-Received":["by 2002:a05:600c:3ca9:b0:412:e444:133a with SMTP id\n\tbg41-20020a05600c3ca900b00412e444133amr175636wmb.20.1709574890165; \n\tMon, 04 Mar 2024 09:54:50 -0800 (PST)","by 2002:a05:600c:3ca9:b0:412:e444:133a with SMTP id\n\tbg41-20020a05600c3ca900b00412e444133amr175624wmb.20.1709574889821; \n\tMon, 04 Mar 2024 09:54:49 -0800 (PST)"],"X-Google-Smtp-Source":"AGHT+IFNOob+QT8Yxw+oYHlKL+UjmFRBVqXgggr8235onKVKUfaxqz/eFh40wm9hmwuTseHiF0505Q==","From":"Milan Zamazal ","To":"Hans de Goede ","Subject":"Re: [PATCH v4 16/18] libcamera: Add \"Software ISP benchmarking\"\n\tdocumentation","In-Reply-To":"<20240229183654.7206-17-hdegoede@redhat.com> (Hans de Goede's\n\tmessage of \"Thu, 29 Feb 2024 19:36:27 +0100\")","References":"<20240229183654.7206-1-hdegoede@redhat.com>\n\t<20240229183654.7206-17-hdegoede@redhat.com>","Date":"Mon, 04 Mar 2024 18:54:48 +0100","Message-ID":"<87frx5anmf.fsf@redhat.com>","User-Agent":"Gnus/5.13 (Gnus v5.13)","MIME-Version":"1.0","X-Mimecast-Spam-Score":"0","X-Mimecast-Originator":"redhat.com","Content-Type":"text/plain; charset=utf-8","Content-Transfer-Encoding":"quoted-printable","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Bryan O'Donoghue ,\n\tlibcamera-devel@lists.libcamera.org, Maxime Ripard , \n\tPavel Machek ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":28838,"web_url":"https://patchwork.libcamera.org/comment/28838/","msgid":"","date":"2024-03-04T19:08:18","subject":"Re: [PATCH v4 16/18] libcamera: Add \"Software ISP benchmarking\"\n\tdocumentation","submitter":{"id":49,"url":"https://patchwork.libcamera.org/api/people/49/","name":"Pavel Machek","email":"pavel@ucw.cz"},"content":"On Mon 2024-03-04 18:54:48, Milan Zamazal wrote:\n> Hans de Goede writes:\n> \n> > Add a \"Software ISP benchmarking\" documentation section which describes\n> > the performance/power consumption measurements used during\n> > the Software ISP's development.\n> >\n> > Signed-off-by: Hans de Goede \n> \n> My comments have been addressed:\n> \n> Reviewed-by: Milan Zamazal \n> \n> It still needs to be reviewed by a native speaker. (And I still wonder whether\n> better methods could be utilized with a modern hardware; but in any case, the\n> described method fulfills its purpose.)\n\nWell, if english is not perfect, it can be still fixed after merge\n;-).\n\nBest regards,\n\t\t\t\t\t\t\t\tPavel","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 8F8FDBD160\n\tfor ;\n\tMon, 4 Mar 2024 19:08:22 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id C197462867;\n\tMon, 4 Mar 2024 20:08:21 +0100 (CET)","from jabberwock.ucw.cz (jabberwock.ucw.cz [46.255.230.98])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id CD317627FC\n\tfor ;\n\tMon, 4 Mar 2024 20:08:19 +0100 (CET)","by jabberwock.ucw.cz (Postfix, from userid 1017)\n\tid 2BC121C007D; Mon, 4 Mar 2024 20:08:19 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ucw.cz header.i=@ucw.cz header.b=\"FIhRIYDe\";\n\tdkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=ucw.cz; s=gen1;\n\tt=1709579299;\n\th=from:from:reply-to:subject:subject:date:date:message-id:message-id:\n\tto:to:cc:cc:mime-version:mime-version:content-type:content-type:\n\tin-reply-to:in-reply-to:references:references;\n\tbh=nnTgn9xC3ffUH1hxsV5ERL9AMdAU+NXFLU3j+itDPKU=;\n\tb=FIhRIYDeJU5Th+w+bCQ+HF8K6OO7kfs43dUXuXUF8Jtjmy4IUwvwquSBMcK+aTL7jltiXt\n\t/TNKXLPYVJCI53fi6vklCPSokE42JUnakG4Baes/WnhGqGP3R94vAV3DT7Nt5Th3HC2C+u\n\tGowae5OyX5TT174QWMxaRcqQcRcQQ30=","Date":"Mon, 4 Mar 2024 20:08:18 +0100","From":"Pavel Machek ","To":"Milan Zamazal ","Subject":"Re: [PATCH v4 16/18] libcamera: Add \"Software ISP benchmarking\"\n\tdocumentation","Message-ID":"","References":"<20240229183654.7206-1-hdegoede@redhat.com>\n\t<20240229183654.7206-17-hdegoede@redhat.com>\n\t<87frx5anmf.fsf@redhat.com>","MIME-Version":"1.0","Content-Type":"multipart/signed; micalg=pgp-sha1;\n\tprotocol=\"application/pgp-signature\"; boundary=\"Q9g55JpaXqGYH5ia\"","Content-Disposition":"inline","In-Reply-To":"<87frx5anmf.fsf@redhat.com>","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org, Maxime Ripard , \n\tBryan O'Donoghue ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]