From patchwork Sat Aug 1 08:01:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 9125 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7DAFCBD878 for ; Sat, 1 Aug 2020 08:01:59 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1965561F41; Sat, 1 Aug 2020 10:01:59 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="A4UNMKW2"; dkim-atps=neutral Received: from mail-wm1-x342.google.com (mail-wm1-x342.google.com [IPv6:2a00:1450:4864:20::342]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2F62160394 for ; Sat, 1 Aug 2020 10:01:57 +0200 (CEST) Received: by mail-wm1-x342.google.com with SMTP id c19so493448wmd.1 for ; Sat, 01 Aug 2020 01:01:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=YsLkAYx4rUDXYF5WdVBATIg6lWA6O/bC19Vhs9ldyOk=; b=A4UNMKW2vZnDIe9HhJxFwjiEfbw7vHTvBrOaUc1AD0jSQPgQ0q2JTtrwcLHntnpPpD 54ZSe62mTR+ytC8NN5y0FYozLrd+0OD08jiMUbiPf4ZSSxdwD/lLIABoEadXCrVpba7c mkCYfHqVkgarz7dEWRpI0jUEnCraw6BjvroT82dVgc3w1k/iL0M+vhXejxPKYI93fT6V roOhZxg4SPnpfDy1Rw01FDrQ6RUpAGn4wezWkcY90SV/HDorJbpzlclQIwUFrY4ThPmS q7XJB0pDKa/9FGuxyalISM+R6xnITrWj9eK7ZbMHCaRERPxibMjE40EzQ1fnTXUyRxd2 5nSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=YsLkAYx4rUDXYF5WdVBATIg6lWA6O/bC19Vhs9ldyOk=; b=s/ZRZmd8zZYSJa5aiVH6seMOqDc6wDEynafkF715X5dogYZ79abqu5fhSF1NT+GpJZ B98vZ34su0VT11L87zw6WVnKZ6naeO8WbxsCtnuihqXQ9F7dMP9luSOWEpSmwscgeJvw PG/5+mIYMtauRvX4eZiXHGLRWESQMiGC1ROx54cFG7HsOYZ9R1Sh1uSteOiaTRA0APOu kRaR5plMenKfUXJ6QenB8IyeyR67vIQwrYe2MlTPQ2iFS1xZCybRtHT0F6/YrXHzdCFY JusRZ8IzBZGtPraVmX/6/z2AkYr4Ym1wmyMfZqKjIKnTQjXeMZtDyc42TnW27cvgFivZ Hw7g== X-Gm-Message-State: AOAM531mOSRFXVdtom2iK+VJJrmOYNgtGHP1JP3VtadiTIC/FttQj74e rfbY4cVgvKwYichP0nO/L2U7Z5GQOxPK+Q== X-Google-Smtp-Source: ABdhPJy/hrMnXn9+LdartVHUYWzDG/btUshPe+H43HXTGh1+igT5oDMhD2BjQPc6cB7P4kkL1bITKQ== X-Received: by 2002:a1c:3245:: with SMTP id y66mr6669328wmy.64.1596268916697; Sat, 01 Aug 2020 01:01:56 -0700 (PDT) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id l67sm17385253wml.13.2020.08.01.01.01.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 01 Aug 2020 01:01:56 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Sat, 1 Aug 2020 09:01:48 +0100 Message-Id: <20200801080151.4282-2-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200801080151.4282-1-david.plowman@raspberrypi.com> References: <20200801080151.4282-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 1/4] libcamera: ipa: raspberrypi: ALSC: Camera mode does not need to be atomic X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In the libcamera framework, SwitchMode (which overwrites the camera_mode) cannot run concurrently with Prepare (which uses it). Signed-off-by: David Plowman Reviewed-by: Laurent Pinchart Reviewed-by: Naushir Patuck --- src/ipa/raspberrypi/controller/rpi/alsc.hpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ipa/raspberrypi/controller/rpi/alsc.hpp b/src/ipa/raspberrypi/controller/rpi/alsc.hpp index 3806257..e895913 100644 --- a/src/ipa/raspberrypi/controller/rpi/alsc.hpp +++ b/src/ipa/raspberrypi/controller/rpi/alsc.hpp @@ -59,7 +59,7 @@ private: // configuration is read-only, and available to both threads AlscConfig config_; bool first_time_; - std::atomic camera_mode_; + CameraMode camera_mode_; std::thread async_thread_; void asyncFunc(); // asynchronous thread function std::mutex mutex_; From patchwork Sat Aug 1 08:01:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 9126 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 951FCBD878 for ; Sat, 1 Aug 2020 08:02:00 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6130261F44; Sat, 1 Aug 2020 10:02:00 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="WVw+3uUV"; dkim-atps=neutral Received: from mail-wm1-x343.google.com (mail-wm1-x343.google.com [IPv6:2a00:1450:4864:20::343]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1CA3C61F34 for ; Sat, 1 Aug 2020 10:01:58 +0200 (CEST) Received: by mail-wm1-x343.google.com with SMTP id k20so11099567wmi.5 for ; Sat, 01 Aug 2020 01:01:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=6/gEh2ZY2aznNHMFS+s1Y6SPKqzLhNzjjy68vEp8MrE=; b=WVw+3uUVKutxQI+D+AsQL2kiIpKNMo5vJCoOMJVn1GlUxqnzjdCGNcRanIXcPBqVzd iYy7b7zP3Mi646Wc/WMALivG64r9M6OY1ryN6ssnafVUZcgz/Z+gfRdMQJjQcWZk9EBJ g7NYcqI4WS9eyJ09bnxf4pG6afLDvzASjJBxMEorHHtuncCQjNcBbuUZJSgkXcaiye7s uZTWi+CsD+dSKyLU/aKmRIMj8Ekyw7aUAvTGNmCQR+GSewkN4bj1uDLXhYz9XMN8SmHf kj6rq+AI7Cx4vO9TbdYkXIVVTgvSE2L/XFgNNhunNvmp90WbQbZycQ29pjYXXPavMAzK atcA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6/gEh2ZY2aznNHMFS+s1Y6SPKqzLhNzjjy68vEp8MrE=; b=IPRwhDU6ixN371aq9w9Y6gzv3+MVxx4fj2Aq7XDk86COTFUrX4E/6UTI3H4JSAjWjW HLdMGaoSQrW7kgC4Iz1M7zxLOTt9bfAAwdZlM2PqwGIEG7/KzXwba8zT7hx3MsAu9zbW HrfpZN4XszoNkrfC//L3V391IWHcrGiCb/+08CNopZ+mrzAd0WTCePDQUTJKgkz7Pyid yvbz/oTqFsszv/ZfDX9Ph8+rH3RWr+sFUYo10vP4fHCdD6/AHFJmaojmAniZQx9w3cYu 6w6AnU+PkmlPXFFYwGS/0x/bEuqgRrX1iK55Z34cJ0RNmq2GuW0Jt6mi5nymRSTe4slr oZPQ== X-Gm-Message-State: AOAM5323pA+GT7/HjBrr1kwONNI+RGNS0eabAWt7unfdkbEOsbA7oynn pGQW1rdUXerHAvxiA9gxWrKTnHuDADCi8w== X-Google-Smtp-Source: ABdhPJyao8QKPSeG6WvLL5I8zIGnj3zDeTLALzfn1XrqRn8R1s9icXl/SBMho2m2EA4BYwDmMTLNcA== X-Received: by 2002:a1c:3285:: with SMTP id y127mr7127980wmy.104.1596268917523; Sat, 01 Aug 2020 01:01:57 -0700 (PDT) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id l67sm17385253wml.13.2020.08.01.01.01.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 01 Aug 2020 01:01:57 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Sat, 1 Aug 2020 09:01:49 +0100 Message-Id: <20200801080151.4282-3-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200801080151.4282-1-david.plowman@raspberrypi.com> References: <20200801080151.4282-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 2/4] libcamera: ipa: raspberrypi: ALSC: Improve locking in a few places X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Fix up a few locations where we call notify_one() with the lock held. In particular, restartAsync does not need to be called with the lock held for its entire duration. Signed-off-by: David Plowman Reviewed-by: Laurent Pinchart Reviewed-by: Naushir Patuck Reported-by: Coverity CID=298183 --- src/ipa/raspberrypi/controller/rpi/alsc.cpp | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/alsc.cpp b/src/ipa/raspberrypi/controller/rpi/alsc.cpp index 76e2f04..12359dc 100644 --- a/src/ipa/raspberrypi/controller/rpi/alsc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/alsc.cpp @@ -32,8 +32,8 @@ Alsc::~Alsc() { std::lock_guard lock(mutex_); async_abort_ = true; - async_signal_.notify_one(); } + async_signal_.notify_one(); async_thread_.join(); } @@ -268,8 +268,11 @@ void Alsc::restartAsync(StatisticsPtr &stats, Metadata *image_metadata) frame_phase_ = 0; // copy the camera mode so it won't change during the calculations async_camera_mode_ = camera_mode_; - async_start_ = true; async_started_ = true; + { + std::lock_guard lock(mutex_); + async_start_ = true; + } async_signal_.notify_one(); } @@ -315,7 +318,6 @@ void Alsc::Process(StatisticsPtr &stats, Metadata *image_metadata) RPI_LOG("Alsc: frame_phase " << frame_phase_); if (frame_phase_ >= (int)config_.frame_period || frame_count2_ < (int)config_.startup_frames) { - std::unique_lock lock(mutex_); if (async_started_ == false) { RPI_LOG("ALSC thread starting"); restartAsync(stats, image_metadata); @@ -339,8 +341,8 @@ void Alsc::asyncFunc() { std::lock_guard lock(mutex_); async_finished_ = true; - sync_signal_.notify_one(); } + sync_signal_.notify_one(); } } From patchwork Sat Aug 1 08:01:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 9127 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 06A23BD879 for ; Sat, 1 Aug 2020 08:02:01 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B0CF661F5D; Sat, 1 Aug 2020 10:02:00 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="bivLCrY2"; dkim-atps=neutral Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3AE2561F4C for ; Sat, 1 Aug 2020 10:01:59 +0200 (CEST) Received: by mail-wm1-x32e.google.com with SMTP id 184so11136223wmb.0 for ; Sat, 01 Aug 2020 01:01:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=BkhnOjtcYZUZR4IAtisTMj71R1n33rN9aiwswXHSvXE=; b=bivLCrY2IP9kfcnBB6Vgk8rOMrxSqcRhlMHbMwbZiouoD7xdQGn3qO0zjD/qbsBRMQ 3CigZhgzB18UNrGf23ZVHB8cKcxWiLPu8sXsnq9d3ln0ZIIL6ps2ccs6gyS7CrDgTRH7 QJZ/CfIbeosVOz7BxMj7WOaHqt8JCpvQqc9MKhXEvpxfKFgC2An7iCDwchD2g83ws5Ej v8Ttkw9sVOc8hREkaz8sIidbCqO5XwG2ME1v6DPRrkEyjYl6v1CAsUniJZ5KAibdn8sN iJbX/g8AdRonIOJ+q1E7zYmjwaf/FDONgm+jM4CDbSXw8VG+4KQvcsmQl6aOsmDS5vA6 PaXQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BkhnOjtcYZUZR4IAtisTMj71R1n33rN9aiwswXHSvXE=; b=L/9K1uPpFFlZnCwX4PsrT/PLO7l93neL8e1vSd7MC2bfDNe7Hk+Os2F03UpdBOSO0e /dqAmOXRHCD944KImjOWIYUwlhEe8WaPgkEfgfH8tMU/CWn17l3pQbIBAGFg7nFgGa47 nruNVNy19hVZWzyERC/khlFgM0luPWAEFdjmguhSqMUy52pspFGvnyHq3w7fjyyFX+99 HLr0f1ulZzHr499GnVl+pICv8H2GHi22XSW+FrULVhVRQc9JNWJ/5EoLDA+zXuvmJ/vT o3C4St0nJr19/LSH1k7JgW7w1slE2Bk6hM4/zmdmWuMKGGfQkKyeNiN2Luxiz6Ok+k7i u/nA== X-Gm-Message-State: AOAM532zshcnhraLOlk3HJ3IrHkQDt0/OR9DA07Vrla6IehxgKYMHWzd XuqEIWMQ9Ypmvv24oNUzN8PYJ1svS6Vdgg== X-Google-Smtp-Source: ABdhPJxQ/DfCwrABv2F7Xce1dQ1YUmiHS/z+/q1AMdFIBuTkIKN+rueIaPIIyDhPhXzM4jpjisZuBA== X-Received: by 2002:a1c:7918:: with SMTP id l24mr7009530wme.158.1596268918432; Sat, 01 Aug 2020 01:01:58 -0700 (PDT) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id l67sm17385253wml.13.2020.08.01.01.01.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 01 Aug 2020 01:01:57 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Sat, 1 Aug 2020 09:01:50 +0100 Message-Id: <20200801080151.4282-4-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200801080151.4282-1-david.plowman@raspberrypi.com> References: <20200801080151.4282-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 3/4] libcamera: ipa: raspberrypi: ALSC: Resample luminance table X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This fixes a bug where the luminance correction table was not being resampled according to the camera mode, in the same way as the colour tables. This could be noticeable if any camera modes crop aggressively. This resampling can be done "up front" in the SwitchMode, as we have only a single fixed luminance table. In order to protect the recalculation of the table from the asynchronous thread (which reads it) I've elected to wait for that thread to go idle (though I doubt it would have mattered much). As a by-product of stopping the thread, it no longer needs its own copy of the camera mode (async_camera_mode_). Signed-off-by: David Plowman Reviewed-by: Laurent Pinchart Reviewed-by: Naushir Patuck --- src/ipa/raspberrypi/controller/rpi/alsc.cpp | 39 ++++++++++++++------- src/ipa/raspberrypi/controller/rpi/alsc.hpp | 3 +- 2 files changed, 28 insertions(+), 14 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/alsc.cpp b/src/ipa/raspberrypi/controller/rpi/alsc.cpp index 12359dc..4df9934 100644 --- a/src/ipa/raspberrypi/controller/rpi/alsc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/alsc.cpp @@ -173,19 +173,34 @@ void Alsc::Initialise() lambda_r_[i] = lambda_b_[i] = 1.0; } +void Alsc::waitForAysncThread() +{ + if (async_started_) { + async_started_ = false; + std::unique_lock lock(mutex_); + sync_signal_.wait(lock, [&] { + return async_finished_; + }); + async_finished_ = false; + } +} + void Alsc::SwitchMode(CameraMode const &camera_mode, Metadata *metadata) { (void)metadata; + // Ensure the other thread isn't running while we do this. + waitForAysncThread(); + // There's a bit of a question what we should do if the "crop" of the - // camera mode has changed. Any calculation currently in flight would - // not be useful to the new mode, so arguably we should abort it, and - // generate a new table (like the "first_time" code already here). When - // the crop doesn't change, we can presumably just leave things - // alone. For now, I think we'll just wait and see. When the crop does - // change, any effects should be transient, and if they're not transient - // enough, we'll revisit the question then. + // camera mode has changed. Should we effectively reset the algorithm + // and start over? camera_mode_ = camera_mode; + + // We must resample the luminance table like we do the others, but it's + // fixed so we can simply do it up front here. + resample_cal_table(config_.luminance_lut, camera_mode_, luminance_table_); + if (first_time_) { // On the first time, arrange for something sensible in the // initial tables. Construct the tables for some default colour @@ -201,7 +216,7 @@ void Alsc::SwitchMode(CameraMode const &camera_mode, Metadata *metadata) compensate_lambdas_for_cal(cal_table_b, lambda_b_, async_lambda_b_); add_luminance_to_tables(sync_results_, async_lambda_r_, 1.0, - async_lambda_b_, config_.luminance_lut, + async_lambda_b_, luminance_table_, config_.luminance_strength); memcpy(prev_sync_results_, sync_results_, sizeof(prev_sync_results_)); @@ -266,8 +281,6 @@ void Alsc::restartAsync(StatisticsPtr &stats, Metadata *image_metadata) } copy_stats(statistics_, stats, alsc_status); frame_phase_ = 0; - // copy the camera mode so it won't change during the calculations - async_camera_mode_ = camera_mode_; async_started_ = true; { std::lock_guard lock(mutex_); @@ -672,9 +685,9 @@ void Alsc::doAlsc() // Fetch the new calibrations (if any) for this CT. Resample them in // case the camera mode is not full-frame. get_cal_table(ct_, config_.calibrations_Cr, cal_table_tmp); - resample_cal_table(cal_table_tmp, async_camera_mode_, cal_table_r); + resample_cal_table(cal_table_tmp, camera_mode_, cal_table_r); get_cal_table(ct_, config_.calibrations_Cb, cal_table_tmp); - resample_cal_table(cal_table_tmp, async_camera_mode_, cal_table_b); + resample_cal_table(cal_table_tmp, camera_mode_, cal_table_b); // You could print out the cal tables for this image here, if you're // tuning the algorithm... (void)print_cal_table; @@ -697,7 +710,7 @@ void Alsc::doAlsc() compensate_lambdas_for_cal(cal_table_b, lambda_b_, async_lambda_b_); // Fold in the luminance table at the appropriate strength. add_luminance_to_tables(async_results_, async_lambda_r_, 1.0, - async_lambda_b_, config_.luminance_lut, + async_lambda_b_, luminance_table_, config_.luminance_strength); } diff --git a/src/ipa/raspberrypi/controller/rpi/alsc.hpp b/src/ipa/raspberrypi/controller/rpi/alsc.hpp index e895913..95572af 100644 --- a/src/ipa/raspberrypi/controller/rpi/alsc.hpp +++ b/src/ipa/raspberrypi/controller/rpi/alsc.hpp @@ -60,10 +60,10 @@ private: AlscConfig config_; bool first_time_; CameraMode camera_mode_; + double luminance_table_[ALSC_CELLS_X * ALSC_CELLS_Y]; std::thread async_thread_; void asyncFunc(); // asynchronous thread function std::mutex mutex_; - CameraMode async_camera_mode_; // condvar for async thread to wait on std::condition_variable async_signal_; // condvar for synchronous thread to wait on @@ -86,6 +86,7 @@ private: int frame_count2_; double sync_results_[3][ALSC_CELLS_Y][ALSC_CELLS_X]; double prev_sync_results_[3][ALSC_CELLS_Y][ALSC_CELLS_X]; + void waitForAysncThread(); // The following are for the asynchronous thread to use, though the main // thread can set/reset them if the async thread is known to be idle: void restartAsync(StatisticsPtr &stats, Metadata *image_metadata); From patchwork Sat Aug 1 08:01:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 9128 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4EA1CBD878 for ; Sat, 1 Aug 2020 08:02:02 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 188FC61F2E; Sat, 1 Aug 2020 10:02:02 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="PGv3l83p"; dkim-atps=neutral Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2771B61F24 for ; Sat, 1 Aug 2020 10:02:00 +0200 (CEST) Received: by mail-wr1-x432.google.com with SMTP id r4so26872084wrx.9 for ; Sat, 01 Aug 2020 01:02:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=WY3rwcoThIxMWhmWUhlt0lQ29+IsYSrjXgY7k1TXiQM=; b=PGv3l83p3htpdtCnzfeKmPHEF5uQX3uO5syDZ+V3IDqlmzs8m+qHKa0ukwzkfuY1IH 05kEzX+Adyr65kbeGxXoKByH2vxUy/mo7tHJHgEgVdr9efRWbP16H2yD+WvXJLf+f+KU SjRs04Fdp5FewFFhdwOKM8BUylAA+6rgDQERaqXZLUZQTtjBKncj3CLfow+d3vPPBuE0 cV+xBWeQJFFieSRRFHSKRO691QuTtlLF9dtxHbuvKoyq7FPp+tN5KICcOOg2W1szSmb9 d9jmVxK/5NAfEhuvLADV+ISkCXD2iKGD5AaOKawS915ZnbJd+etnRzNohqa5WZp9BWfX xYhg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=WY3rwcoThIxMWhmWUhlt0lQ29+IsYSrjXgY7k1TXiQM=; b=L/Mjbgb9ecqvaBHj2RWKGLMhF0LCL5N1chPF3BU44kWuHIr+Ok9Cx613fOMXUK8LCk xzsRQDGz6DOuvvOKOVJFScfriGwcUXhcgYDgqpLGkSUY7YSfNNLgl7Qnzig1AObyp1mm Bynn10BzLjasjUAH7hHiI6ygMnaMAsaKfiidf9iAv5+ie9zXrqsxeX0jJLYMXuSXat1I fFL3ciTwKBSgVsgR7X2z0T2TZMF4zO4vI9w8F7eg/rO9InJz0z3ZC3oPYZTEd7RpEiFm c6OC4x5GJf7NvVxFk+iEqSsWwwaAzob+4MYn7qqreo/7Yt8iVmC74igoOmVJik/qwp/g 0Q0w== X-Gm-Message-State: AOAM530l6Smk4b9Cra1Wl3F+7cHfHVvMZE6wtHOdmkiaU22ZW/1gt4B3 9cxqxf0mW9CefKlljXEH311OtpPs5dO7ww== X-Google-Smtp-Source: ABdhPJyLPqtXnAb1Uw67vzlD72l+MBw48t5x0FU1NJJBme0lx9gNx9PIIle4FxJU3WiDiUQmPX2aVw== X-Received: by 2002:adf:8282:: with SMTP id 2mr6454337wrc.76.1596268919406; Sat, 01 Aug 2020 01:01:59 -0700 (PDT) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id l67sm17385253wml.13.2020.08.01.01.01.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 01 Aug 2020 01:01:58 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Sat, 1 Aug 2020 09:01:51 +0100 Message-Id: <20200801080151.4282-5-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200801080151.4282-1-david.plowman@raspberrypi.com> References: <20200801080151.4282-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 4/4] libcamera: ipa: raspberrypi: ALSC: Improve behaviour when camera mode changes X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Now that we stop the asynchronous thread on a SwitchMode, we would do better to regenerate all the tables if the new camera mode crops in a significantly different way to the old one. A few minor tweaks make sense along with this: * Reset the lambda values when we reset everything. It wouldn't make sense to re-start with the old mode's values. * Use the last recorded colour temperature to generate new tables rather than any default value. * Set the frame "phase" counter to ensure the adaptive procedure will run asap. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck Reviewed-by: Naushir Patuck --- src/ipa/raspberrypi/controller/rpi/alsc.cpp | 54 ++++++++++++++------- 1 file changed, 37 insertions(+), 17 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/alsc.cpp b/src/ipa/raspberrypi/controller/rpi/alsc.cpp index 4df9934..5e55954 100644 --- a/src/ipa/raspberrypi/controller/rpi/alsc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/alsc.cpp @@ -166,11 +166,8 @@ void Alsc::Initialise() RPI_LOG("Alsc"); frame_count2_ = frame_count_ = frame_phase_ = 0; first_time_ = true; - // Initialise the lambdas. Each call to Process then restarts from the - // previous results. Also initialise the previous frame tables to the - // same harmless values. - for (int i = 0; i < XY; i++) - lambda_r_[i] = lambda_b_[i] = 1.0; + ct_ = config_.default_ct; + // The lambdas are initialised in the SwitchMode. } void Alsc::waitForAysncThread() @@ -185,31 +182,53 @@ void Alsc::waitForAysncThread() } } +static bool compare_modes(CameraMode const &cm0, CameraMode const &cm1) +{ + // Return true if the modes crop from the sensor significantly differently. + int left_diff = fabs(cm0.crop_x - cm1.crop_x); + int top_diff = fabs(cm0.crop_y - cm1.crop_y); + int right_diff = fabs(cm0.crop_x + cm0.scale_x * cm0.width - + cm1.crop_x - cm1.scale_x * cm1.width); + int bottom_diff = fabs(cm0.crop_y + cm0.scale_y * cm0.height - + cm1.crop_y - cm1.scale_y * cm1.height); + // These thresholds are a rather arbitrary amount chosen to trigger + // when carrying on with the previously calculated tables might be + // worse than regenerating them (but without the adaptive algorithm). + int threshold_x = cm0.sensor_width >> 4; + int threshold_y = cm0.sensor_height >> 4; + return left_diff > threshold_x || right_diff > threshold_x || + top_diff > threshold_y || bottom_diff > threshold_y; +} + void Alsc::SwitchMode(CameraMode const &camera_mode, Metadata *metadata) { (void)metadata; + // We're going to start over with the tables if there's any "significant" + // change. + bool reset_tables = first_time_ || compare_modes(camera_mode_, camera_mode); + // Ensure the other thread isn't running while we do this. waitForAysncThread(); - // There's a bit of a question what we should do if the "crop" of the - // camera mode has changed. Should we effectively reset the algorithm - // and start over? camera_mode_ = camera_mode; // We must resample the luminance table like we do the others, but it's // fixed so we can simply do it up front here. resample_cal_table(config_.luminance_lut, camera_mode_, luminance_table_); - if (first_time_) { - // On the first time, arrange for something sensible in the - // initial tables. Construct the tables for some default colour - // temperature. This echoes the code in doAlsc, without the - // adaptive algorithm. + if (reset_tables) { + // Upon every "table reset", arrange for something sensible to be + // generated. Construct the tables for the previous recorded colour + // temperature. In order to start over from scratch we initialise + // the lambdas, but the rest of this code then echoes the code in + // doAlsc, without the adaptive algorithm. + for (int i = 0; i < XY; i++) + lambda_r_[i] = lambda_b_[i] = 1.0; double cal_table_r[XY], cal_table_b[XY], cal_table_tmp[XY]; - get_cal_table(4000, config_.calibrations_Cr, cal_table_tmp); + get_cal_table(ct_, config_.calibrations_Cr, cal_table_tmp); resample_cal_table(cal_table_tmp, camera_mode_, cal_table_r); - get_cal_table(4000, config_.calibrations_Cb, cal_table_tmp); + get_cal_table(ct_, config_.calibrations_Cb, cal_table_tmp); resample_cal_table(cal_table_tmp, camera_mode_, cal_table_b); compensate_lambdas_for_cal(cal_table_r, lambda_r_, async_lambda_r_); @@ -220,6 +239,7 @@ void Alsc::SwitchMode(CameraMode const &camera_mode, Metadata *metadata) config_.luminance_strength); memcpy(prev_sync_results_, sync_results_, sizeof(prev_sync_results_)); + frame_phase_ = config_.frame_period; // run the algo again asap first_time_ = false; } } @@ -265,8 +285,8 @@ void Alsc::restartAsync(StatisticsPtr &stats, Metadata *image_metadata) { RPI_LOG("Starting ALSC thread"); // Get the current colour temperature. It's all we need from the - // metadata. - ct_ = get_ct(image_metadata, config_.default_ct); + // metadata. Default to the last CT value (which could be the default). + ct_ = get_ct(image_metadata, ct_); // We have to copy the statistics here, dividing out our best guess of // the LSC table that the pipeline applied to them. AlscStatus alsc_status;