From patchwork Tue Mar 28 08:55:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 18483 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 76B45BD160 for ; Tue, 28 Mar 2023 08:55:31 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 226BE6274E; Tue, 28 Mar 2023 10:55:31 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1679993731; bh=euIoSd/qb9gVyuY6+xi1UwFmAN+QrbRRNSE9tkA/JRc=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=MopqDpixQB5+ehst+faWJOZS/vNY3P+puI9sDkO/+nW7fRzLtMSypAVpbtSCVb6zl 50iji1/hebxZl0Fzasdq0XhJxOJI4teAHijHtG9AiAZQM42luk1ffK1wYoMY5AxkTS iiRauLcthXqOMsHgd3w6pDrIf9ANVZow6IlsJbt7+3hhZ7zpExboI9FH+zBY3JX36Y SRyiwPagO/eLMJJs9jTX4IKcnPqsEFIs/DoAa3eGzBvWF3Afwj3+KjjNEd5cJxOIWI Y9oA5B2Z4yqRGEqg8irmW9qjGMyhbjGM2olKtl4DRkoKmR3vF2swgQy5K7eBKJrwdZ XCVV5/wckuyHQ== Received: from mail-io1-xd33.google.com (mail-io1-xd33.google.com [IPv6:2607:f8b0:4864:20::d33]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 19457626E2 for ; Tue, 28 Mar 2023 10:55:27 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="HrYUK83u"; dkim-atps=neutral Received: by mail-io1-xd33.google.com with SMTP id o12so5070643iow.6 for ; Tue, 28 Mar 2023 01:55:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1679993726; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6z1WOhXY6J1PXKp0uYL/HuVRwLxh8DoooxywOcHa0iY=; b=HrYUK83uzWSXLNWecpTgnndUm/1sdY4cPD/LgAsA2jiJhiarqQ3ACQuZkTFg6bgRS6 hmWdeHTTRGHbO52QUP1hiFmxhEZJz37roAc0bmG1sfLNxBhi/C+WfTeThJGw5xtXpVLN 6O3A7UReZi3SVMCeBEzPVbNlxZwOeWHY6L/iVlnekMVoXEXgt/eRtCSK6irXRVD8BIP6 KG45mM/opfkjEALiplahL434VHL02SML+AmBVzzYXXb1a9Ceo5CjQAy8ejkFwZag0cmE HsN370lJ09eWSG+O+CfBK1dmQOgemRcR6ugnrb+C4XI0i0l7/0OlxddedHm7h+o/FBLS DzVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679993726; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6z1WOhXY6J1PXKp0uYL/HuVRwLxh8DoooxywOcHa0iY=; b=f5WdeBXqsUoQc8uSZtR7HEAbBFqUnNRlobPA2FPaa6OJrH9tR3shSCMjSLRC2D9gkr 6w14vQZInmTzLvzGOVSxdfUtT8NMqauTpKOXJzS3dmKLFdsJpp1Bm1iMdm9mb9P7030r +uAa3dc17sRixKmyiQzc4Nw9GMeCt6bzmYSUb0D0JKFkj1MeY/j+Z7SlaSkdSHrMXJAc jQk8Ik8fJVIUdLQ+WGmmovw9QXpvb7WWQiTxUGPgNyhwrim87kc8SSe7uDCcT47fTDe6 mY0F0oyX9otUrDxrI7A60wWUUYzfr5sPmZP2Y+gYXDarIscso6XIB3yvniy0a2+lhEJn XB7g== X-Gm-Message-State: AAQBX9c6WX58Pm/8iFPgAubXfr9dnhF7Qbh73CjyWleWsvLhrRYjCOAz S95rraTCf6B7Z115b9mLNYvEG+sOLv1KSzG7XLVz8w== X-Google-Smtp-Source: AKy350YjizxQWngljcwvJ+qrDLT93GLQHACV88ckgDMTd7lm+3eiTQhv7LVRBSwbhzLAz7xWf5JXsw== X-Received: by 2002:a6b:c90d:0:b0:75c:90f1:4622 with SMTP id z13-20020a6bc90d000000b0075c90f14622mr1372113iof.0.1679993725711; Tue, 28 Mar 2023 01:55:25 -0700 (PDT) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:2bce:64d6:1a5c:49a2]) by smtp.gmail.com with ESMTPSA id g1-20020a028501000000b0040380d7c768sm9887414jai.106.2023.03.28.01.55.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Mar 2023 01:55:25 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Tue, 28 Mar 2023 09:55:20 +0100 Message-Id: <20230328085521.7409-2-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230328085521.7409-1-david.plowman@raspberrypi.com> References: <20230328085521.7409-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 1/2] libcamera: controls: Add controls for AEC/AGC flicker avoidance X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: David Plowman via libcamera-devel From: David Plowman Reply-To: David Plowman Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Flicker is the term used to describe brightness banding or oscillation of images caused typically by artificial lighting driven by a 50 or 60Hz mains supply. We add three controls intended to be used by AEC/AGC algorithms: AeFlickerMode to enable flicker avoidance. AeFlickerCustom to set custom flicker periods. AeFlickerDetected to report any flicker that is currently detected. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- src/libcamera/control_ids.yaml | 84 +++++++++++++++++++++++++++------- 1 file changed, 67 insertions(+), 17 deletions(-) diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml index adea5f90..b472050c 100644 --- a/src/libcamera/control_ids.yaml +++ b/src/libcamera/control_ids.yaml @@ -156,6 +156,73 @@ controls: control of which features should be automatically adjusted shouldn't better be handled through a separate AE mode control. + - AeFlickerMode: + type: int32_t + description: | + Set the flicker mode, which determines whether, and how, the AGC/AEC + algorithm attempts to hide flicker effects caused by the duty cycle of + artificial lighting. + + Although implementation dependent, many algorithms for "flicker + avoidance" work by restricting this exposure time to integer multiples + of the cycle period, wherever possible. + + Implementations may not support all of the flicker modes listed below. + + enum: + - name: FlickerOff + value: 0 + description: No flicker avoidance is performed. + - name: FlickerFreq50Hz + value: 1 + description: 50Hz flicker avoidance. + Suppress flicker effects caused by lighting running with a 100Hz + period (such as that produced by 50Hz mains electricity). + - name: FlickerFreq60Hz + value: 2 + description: 60Hz flicker avoidance. + Suppress flicker effects caused by lighting running with a 120Hz + period (such as that produced by 60Hz mains electricity). + - name: FlickerCustom + value: 3 + description: Custom flicker avoidance. + Suppress flicker effects caused by lighting running with a period + specified by the AeFlickerCustom control. + \sa AeFlickerCustom + - name: FlickerAuto + value: 4 + description: Automatic flicker period detection and avoidance. + The system will automatically determine the most likely value of + flicker period, and avoid flicker of this frequency. + + - AeFlickerCustom: + type: int32_t + description: Custom flicker period in microseconds. + This value sets the current flicker period to avoid. It is used when + AeFlickerMode is set to FlickerCustom. + + If this control is not available, then the setting of custom flicker + periods is not supported. + + \sa AeFlickerMode + + - AeFlickerDetected: + type: int32_t + description: Flicker period detected in microseconds. + The value reported here indicates the currently detected flicker + period, or zero if no flicker at all is detected. + + So in the case of 50Hz mains flicker, the value would be 10000 + (corresponding to 100Hz), or 8333 (120Hz) for 60Hz mains flicker. But + note that it is not required that any form of scene flicker is detected + at all, so depending on the implementation, this metadata may just + report zero or be entirely absent. + + AeFlickerDetected may also report other values for non-standard flicker + periods. + + \sa AeFlickerMode + - Brightness: type: float description: | @@ -843,23 +910,6 @@ controls: value: 1 description: The lens shading map mode is available. - - SceneFlicker: - type: int32_t - draft: true - description: | - Control to report the detected scene light frequency. Currently - identical to ANDROID_STATISTICS_SCENE_FLICKER. - enum: - - name: SceneFickerOff - value: 0 - description: No flickering detected. - - name: SceneFicker50Hz - value: 1 - description: 50Hz flickering detected. - - name: SceneFicker60Hz - value: 2 - description: 60Hz flickering detected. - - PipelineDepth: type: int32_t draft: true From patchwork Tue Mar 28 08:55:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 18484 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 56FE8C326C for ; Tue, 28 Mar 2023 08:55:33 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 774F762742; Tue, 28 Mar 2023 10:55:32 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1679993732; bh=2r/yf1zzoBFA6VOfia/avCOBqkRGNngouNN9mTUgU0Y=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=Lyf5FRp2qI9J8D0Epp00t9xiOFKeIWQmqFtdcSPaZJ417+/tkmFKX6fSZWTLQu9Ai GrmCy4EmQG6xQ+5r+9eLfUZT15oLowJRSm4YS0bqRmgHRqZwUIS8bAblNHD3F84wVD Iuh8xVouZ9DXvSIX0GrEDRTqQhml9RhwxyQkM1yprXm1eqRC6HqB7BwnhhJXecN072 gZ5cEQMiKMtTKFKah7FsfdHmVtqsilIvfcYB3SYl2qNZGC7ywRRnACSX5DgLGoW5kW IqyRSTBIH7N4xbfA29lx3e636CrzBzXNvVL3I/f1nROPjLQA7Zj5EMDA0P2DA3CrYN 3DUHAtSnDt/0g== Received: from mail-il1-x134.google.com (mail-il1-x134.google.com [IPv6:2607:f8b0:4864:20::134]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 079C862742 for ; Tue, 28 Mar 2023 10:55:28 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="Bo29vH9s"; dkim-atps=neutral Received: by mail-il1-x134.google.com with SMTP id h7so6023401ila.5 for ; Tue, 28 Mar 2023 01:55:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1679993726; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MqMbt9OzNQyhbcVo1jTzGTjETY6G4gi1d15/I+rRNFs=; b=Bo29vH9s5f6NRxUT61OKKFPMx3zj+U4QQIGzKUWp7xkdn9ScmSLEt5O1pKMK/8KFXE LzGrscYXf3CB8VA8bQbnnJkNl0RonSnGOaeGl9EEBat31oquVS5ZeYZFQfNge4y4KQG6 XO+IoRzbi52XVADG/WXwk2TFCW/9kWi8XanuSEW6FWkLS+7J6oKAZmM87/2LC+G7w5/E FdfH6FsCQ6h56mOsjNyRnRxGsb3THPHbMqmcQqu6vwfyYccjfH2XpJ9otNH2Fo40gSHR AZfd3vuRzLAJGjLJaxtzqCkpatUNh9w/54NXEhcLCU7/wGOFubYsqEDD1Zc5cFbO+RnQ UolQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679993726; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MqMbt9OzNQyhbcVo1jTzGTjETY6G4gi1d15/I+rRNFs=; b=GUKIPNzV648E45LtsQYK7G6c39XYg9XMCwvUl2AaglP21pPFIu10KrsrU4b2pPS3ld mSrwRqHVCyNMsmH/8si87kzmadqQAlXvLHIRS0UPFwwj2yIuWUNJ+4gf2E70YQnqfD9a seOWD2gnC0ITBlueVpUkCRcEXfFMCl0NgAg9gbOHaiOO92C+uaA9ERj+o1MaF0NqTPzj bMyMchU3mS0fvLVQd2PV/Yesghj3Ggs36Ai+h6vMzV29cYkSBs9G8zGH48xTkF+2yxK3 EqXNlXeRA7cXiaLm/IDI/DEYBn97hiPpIiGNybC26cbMglQxythjnSuSVD9GRzfUr6sp +8sg== X-Gm-Message-State: AAQBX9esj9P13KgOHDEbORe4BMKLpA9l1N+cesNDcB+Lufu2z3YtaX33 5YLEm4GGReqy9Gjufvor5M3kWv6sl0vaFx/BRIUXkA== X-Google-Smtp-Source: AKy350aD/J5HvZKcCZHrWpkeQ7PJicJH7FNA87eA84/JSh1sxmk1pjK6uNjJzpI3PYzYm9NTC/RdMw== X-Received: by 2002:a05:6e02:4cb:b0:323:f57c:806d with SMTP id f11-20020a056e0204cb00b00323f57c806dmr6602296ils.3.1679993726642; Tue, 28 Mar 2023 01:55:26 -0700 (PDT) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:2bce:64d6:1a5c:49a2]) by smtp.gmail.com with ESMTPSA id g1-20020a028501000000b0040380d7c768sm9887414jai.106.2023.03.28.01.55.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Mar 2023 01:55:26 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Tue, 28 Mar 2023 09:55:21 +0100 Message-Id: <20230328085521.7409-3-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230328085521.7409-1-david.plowman@raspberrypi.com> References: <20230328085521.7409-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 2/2] ipa: raspberrypi: Handle AEC/AGC flicker controls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: David Plowman via libcamera-devel From: David Plowman Reply-To: David Plowman Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" We handle the flicker modes by passing the correct period to the AEC/AGC algorithm which already contains the necessary code. The "Auto" mode, as well as reporting the detected flicker period via the "AeFlickerDetected" metadata, are unsupported for now. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck Tested-by: Maarten Lankhorst --- src/ipa/raspberrypi/raspberrypi.cpp | 80 +++++++++++++++++++++++++++++ 1 file changed, 80 insertions(+) diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 13757955..d485b851 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -91,6 +91,8 @@ static const ControlInfoMap::Map ipaControls{ { &controls::AeConstraintMode, ControlInfo(controls::AeConstraintModeValues) }, { &controls::AeExposureMode, ControlInfo(controls::AeExposureModeValues) }, { &controls::ExposureValue, ControlInfo(-8.0f, 8.0f, 0.0f) }, + { &controls::AeFlickerMode, ControlInfo(controls::AeFlickerModeValues) }, + { &controls::AeFlickerCustom, ControlInfo(100, 1000000) }, { &controls::AwbEnable, ControlInfo(false, true) }, { &controls::ColourGains, ControlInfo(0.0f, 32.0f) }, { &controls::AwbMode, ControlInfo(controls::AwbModeValues) }, @@ -230,6 +232,12 @@ private: /* Track the frame length times over FrameLengthsQueueSize frames. */ std::deque frameLengths_; Duration lastTimeout_; + + /* The current state of flicker avoidance. */ + struct FlickerState { + int32_t mode; + Duration customPeriod; + } flickerState_; }; int IPARPi::init(const IPASettings &settings, bool lensPresent, IPAInitResult *result) @@ -962,6 +970,78 @@ void IPARPi::queueRequest(const ControlList &controls) break; } + case controls::AE_FLICKER_MODE: { + RPiController::AgcAlgorithm *agc = dynamic_cast( + controller_.getAlgorithm("agc")); + if (!agc) { + LOG(IPARPI, Warning) + << "Could not set AeFlickerMode - no AGC algorithm"; + break; + } + + int32_t mode = ctrl.second.get(); + bool modeValid = true; + + switch (mode) { + case controls::FlickerOff: { + agc->setFlickerPeriod(0us); + + break; + } + + case controls::FlickerFreq50Hz: { + agc->setFlickerPeriod(10000 * 1.0us); + + break; + } + + case controls::FlickerFreq60Hz: { + agc->setFlickerPeriod(8333.333 * 1.0us); + + break; + } + + case controls::FlickerCustom: { + agc->setFlickerPeriod(flickerState_.customPeriod); + + break; + } + + default: + LOG(IPARPI, Error) << "Flicker mode " << mode << " is not supported"; + modeValid = false; + + break; + } + + if (modeValid) + flickerState_.mode = mode; + + break; + } + + case controls::AE_FLICKER_CUSTOM: { + RPiController::AgcAlgorithm *agc = dynamic_cast( + controller_.getAlgorithm("agc")); + if (!agc) { + LOG(IPARPI, Warning) + << "Could not set AeFlickerCustom - no AGC algorithm"; + break; + } + + uint32_t customPeriod = ctrl.second.get(); + flickerState_.customPeriod = customPeriod * 1.0us; + + /* + * We note that it makes no difference if the mode gets set to "custom" + * first, and the period updated after, or vice versa. + */ + if (flickerState_.mode == controls::FlickerCustom) + agc->setFlickerPeriod(flickerState_.customPeriod); + + break; + } + case controls::AWB_ENABLE: { RPiController::AwbAlgorithm *awb = dynamic_cast( controller_.getAlgorithm("awb"));