From patchwork Fri Aug 27 12:07:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 13544 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9806FBD87D for ; Fri, 27 Aug 2021 12:07:14 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id F0A7268928; Fri, 27 Aug 2021 14:07:13 +0200 (CEST) Received: from relay7-d.mail.gandi.net (relay7-d.mail.gandi.net [217.70.183.200]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EB5E860256 for ; Fri, 27 Aug 2021 14:07:12 +0200 (CEST) Received: (Authenticated sender: jacopo@jmondi.org) by relay7-d.mail.gandi.net (Postfix) with ESMTPSA id 28BEF2000A; Fri, 27 Aug 2021 12:07:11 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Fri, 27 Aug 2021 14:07:41 +0200 Message-Id: <20210827120757.110615-1-jacopo@jmondi.org> X-Mailer: git-send-email 2.32.0 MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 00/16] IPU3 control info update and HAL frame durations X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" As the patch subject suggests, this series is composed of two main parts, which concur in correctly handling the frame durations reported to the Android HAL. The series is based on Umang's https://patchwork.libcamera.org/project/libcamera/list/?series=2328 which I would really like to see integrated soon and has been tested with https://patchwork.libcamera.org/project/libcamera/list/?series=2406 which fixes a few CTS tests for devices that do not support manual sensor mode. The series starts by allowing the IPU3 pipeline handler to select the optimal sensor size to feed the ImgU with. Umang's series is a requirement for this patch, otherwise the ImgU configuration fails. Patches 2 and 3 update the Camera's ControlInfoMap in response to a Camera::configure() call, by updating the controls limits in the ph and the IPA. From patch 4 on the Android HAL parts begins by collecting per-stream frame durations. As the frame durations are updated as part of the Camera's ControlInfoMap limits update it is necessary to configure the camera during the HAL startup. A few cleanup patches follows, including the correct handling of STALL durations Patch 10 introduces a filtering criteria for YUV streams: only streams that can produce 30 FPS can be registered for preview. I was not able to find this clearly specified in documentation but the requirement has been confirmed by the cros camera team and in the Intel HAL implementation. After two more cleanup patches the way ANDROID_CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES gets populated is changed to comply with the documentation. Note that this does not align with the Intel HAL implementation which registers 8 values instead of 4. The last part of the series is there mostly for comments. The idea is that streams capable of running at more than 30 FPS should be limited to such frame rate, as it has proven to be the optimal one for the platform, and including more frame rates (in example ov5670 is capable of 60FPS at smaller resolutions) would require a different handling of ANDROID_CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES which I've not been able to fully clarify yet. As a confirmation, the Intel HAL limits all its streams to 30 FPS as well for IQ optimization reasons. The series contains one patch to limit the reported duration to 30 FPS and please CTS without actually limiting the frame rate. The patch is then reverted and the same mechanism is implemented in the IPA, this time actually setting the right vblank to maintain the 30 FPS frame rate (confirmed inspecting timestamps reported by 'cam'). With both solutions applied I got several 0 failures CTS runs. With the HAL-only solution the flaky recording tests failed once, with the IPA solution I got not failures in 3 CTS runs. Thanks j Jacopo Mondi (16): libcamera: ipu3: Use the optimal sensor size libcamera: ipu3: Split controls init/update ipa: ipu3: Update camera controls in configure() android: capabilities: Collect per-stream frame durations android: capabilities: Initialize camera state when building properties android: capabilties: Assume controls::FrameDurationLimits is supported android: capabilities: Use per-configuration durations android: capabilties: Correctly populate STALL durations android: capabilities: Collect absolute max frame durations android: Filter preview streams on FPS android: capabilities: Print output stream list android: Populate streams and duration in the same loop android: capabilties: Fix ANDROID_CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES android: capabilities: Cap frame rate to 30 FPS Revert "android: capabilities: Cap frame rate to 30 FPS" ipa: ipu3: Cap frame duration to 30 FPS include/libcamera/ipa/ipu3.mojom | 3 +- src/android/camera_capabilities.cpp | 199 +++++++++++++++++---------- src/android/camera_capabilities.h | 3 + src/ipa/ipu3/ipu3.cpp | 119 +++++++++++----- src/libcamera/pipeline/ipu3/ipu3.cpp | 75 ++++++---- 5 files changed, 267 insertions(+), 132 deletions(-) Reviewed-by: Paul Elder Reviewed-by: Paul Elder Reviewed-by: Paul Elder Reviewed-by: Umang Jain Reviewed-by: Umang Jain Reviewed-by: Umang Jain Reviewed-by: Paul Elder Reviewed-by: Umang Jain --- 2.32.0