From patchwork Fri Jul 31 14:07:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 9092 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 18690BD86F for ; Fri, 31 Jul 2020 14:08:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D824A61ED2; Fri, 31 Jul 2020 16:08:10 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="ArUzHbxB"; dkim-atps=neutral Received: from mail-wr1-x442.google.com (mail-wr1-x442.google.com [IPv6:2a00:1450:4864:20::442]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7A38C61EA4 for ; Fri, 31 Jul 2020 16:08:08 +0200 (CEST) Received: by mail-wr1-x442.google.com with SMTP id z18so24567021wrm.12 for ; Fri, 31 Jul 2020 07:08:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=YsLkAYx4rUDXYF5WdVBATIg6lWA6O/bC19Vhs9ldyOk=; b=ArUzHbxB6oQpKl+h8z4DulkUc3SOFVXGCUIBG2bVMkXn7kaaFIB5jeJswU22YGgr9z fT0zcyl9U1DK/kah1wYn+LF6c8+xa8Sc+k854dYaAiXtQw04r/HZOUe7y/Y+ApQkiXYO gkvqFaBT0eH5PdoXYMN3KhRN72m7i1OlFZoL8uoTtZuczxy4rm12OmDrOk2Ymh+YhEzc tBMevtvljMlQ478IqH9r5tDYSIZHc4sK624mJU4RJiG6ahH1ZU2vh9Qf2h9solkO+wLg FDdK/lrpM/381A0sz7cTe6n8m0fQmsxvrl8o6xQV773JUNWtGSne115fOVUMeVi0pppa AzYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=YsLkAYx4rUDXYF5WdVBATIg6lWA6O/bC19Vhs9ldyOk=; b=bkvOLtO8eUL50eMzDyuYF9nzbEUzDMZW/s9EONbcI/xUOrjlbd0EeMdSGtSl7EQnAl aSz5apMhuoGLENtT+QP3+5GCnbC3zgghI90CII551wa43OxSI6dSKJojqb8VHG4K0Lxc paUNEvE4gfFq8+ec6ptLynspFB7ZGSlqqDrRiqMbNpXH2cHfqWdKVDkAkd1jO/vicVPM Z5BzcBUC6NoFBgZCZ1qTIN2lu64pT8Vloov50vn4qYbvupzck1fiICTrGGKrSbfytjQr uRsBGjLMv8axgxt7G2F4KvzDJlcYslO5sHf2atzuP7tT9hWOYVnodzgSMEXP0lP7kaOH lTxw== X-Gm-Message-State: AOAM532hsqI9rttkebB+4kGD5zWZMpZcrjYXJe8FD2bGCDUBc/2OTLAL qDIZ193UausUmS6BWyXF+2bK7P2Si39s0A== X-Google-Smtp-Source: ABdhPJyu2ywM2sC4OSYY5uctd3pHWL0SI/1GRgaHVmJupifdzPTg319VOkc2U9v84mTGhpVQTAvdOw== X-Received: by 2002:a5d:4a41:: with SMTP id v1mr4032378wrs.371.1596204487646; Fri, 31 Jul 2020 07:08:07 -0700 (PDT) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id g126sm12815084wme.16.2020.07.31.07.08.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 31 Jul 2020 07:08:07 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Fri, 31 Jul 2020 15:07:58 +0100 Message-Id: <20200731140801.13253-2-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200731140801.13253-1-david.plowman@raspberrypi.com> References: <20200731140801.13253-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 1/4] libcamera: ipa: raspberrypi: ALSC: Camera mode does not need to be atomic X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In the libcamera framework, SwitchMode (which overwrites the camera_mode) cannot run concurrently with Prepare (which uses it). Signed-off-by: David Plowman Reviewed-by: Laurent Pinchart --- src/ipa/raspberrypi/controller/rpi/alsc.hpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ipa/raspberrypi/controller/rpi/alsc.hpp b/src/ipa/raspberrypi/controller/rpi/alsc.hpp index 3806257..e895913 100644 --- a/src/ipa/raspberrypi/controller/rpi/alsc.hpp +++ b/src/ipa/raspberrypi/controller/rpi/alsc.hpp @@ -59,7 +59,7 @@ private: // configuration is read-only, and available to both threads AlscConfig config_; bool first_time_; - std::atomic camera_mode_; + CameraMode camera_mode_; std::thread async_thread_; void asyncFunc(); // asynchronous thread function std::mutex mutex_; From patchwork Fri Jul 31 14:07:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 9093 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 54472BD86F for ; Fri, 31 Jul 2020 14:08:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 61F2A61DCF; Fri, 31 Jul 2020 16:08:11 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="Lx111BbB"; dkim-atps=neutral Received: from mail-wm1-x342.google.com (mail-wm1-x342.google.com [IPv6:2a00:1450:4864:20::342]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 434B061DCF for ; Fri, 31 Jul 2020 16:08:09 +0200 (CEST) Received: by mail-wm1-x342.google.com with SMTP id q76so8148412wme.4 for ; Fri, 31 Jul 2020 07:08:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ZqilBsLzMKmrddlWmS1RYYMDXLW+mCOVb0l3nhtKIeU=; b=Lx111BbBtf8qRqWTyaeYITHlokM6pliFuuNahUllbWTZov0iS0geMNURRpAm/q4qMD bcunSkfooCmEjrvVjou7md/fja5XsBZiWcQExLuog4BkpBijB3dVG/KSVS7moLIABn5i fnnRzhANKrW/8Slr4vYZHDf0XF7ZuRUcZmw/evOmUUhPd/jcXYuEuJwe1oDV44Tw+vK5 Mo75cn3Ii1Ds9GxHlc2D3jcRQJB7+DtJfU6Mtg2rBIryIhTW86hA8+i82vq271jtTsIN BSDgWHZNj5IB15uLcCBhKVgNFCbpD1gtCOgQQqjmcxE7Dpg27L0v9XkYuBn+kyX/8aCI rlow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ZqilBsLzMKmrddlWmS1RYYMDXLW+mCOVb0l3nhtKIeU=; b=cVFabsKgxBP9jEU43XagAgXjZ0lzCRU1lEs1toVX8+7lhY4PbXQQIaX6phmGmUIEDW MbW0/x9t+PC5NG2IlCX1Z/ms/s2+AsUMfy4oPzFRSsWO6tnIMGq2ugdVoBAYA3YeWKI2 ooHnBqaKAuAIK/3gmdfxH+PZ/NvASayQAuTnHOv3SiHTJ0coBXgf8L5DGTxHgsXtVxX+ 6U60yF3qt/DUIszqu/qO6eEQgZIS0BUxXwich/QRWqUUCCk2n7qWos9QyLa5NVhxM5s5 /PzErJE3dLQo5xbZZuxmTjVZO1syqlJvM8EGYbjTvXlaryCt3yiCeKbbetxqlKNvI1A8 DDxQ== X-Gm-Message-State: AOAM533h4q4oQhfBZ/9SLUYxjmwdThWv7r0TQSQz2YSdEOIzwWzrt0Bw c5LNhDuhfcl+ZFlWzSpFkJb9dDk9j6yb+A== X-Google-Smtp-Source: ABdhPJz0mkurWtkGrIjMfJI/a6zySzd+mb5ktcPSRgr8Cgju9qsILQPLI8K2HnBNbLTxIxm6VV0Tlw== X-Received: by 2002:a7b:c40a:: with SMTP id k10mr3849860wmi.127.1596204488580; Fri, 31 Jul 2020 07:08:08 -0700 (PDT) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id g126sm12815084wme.16.2020.07.31.07.08.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 31 Jul 2020 07:08:08 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Fri, 31 Jul 2020 15:07:59 +0100 Message-Id: <20200731140801.13253-3-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200731140801.13253-1-david.plowman@raspberrypi.com> References: <20200731140801.13253-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 2/4] libcamera: ipa: raspberrypi: ALSC: Improve locking in a few places X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Fix up a few locations where we call notify_one() with the lock held. In particular, restartAsync does not need to be called with the lock held for its entire duration. Signed-off-by: David Plowman Reviewed-by: Laurent Pinchart --- src/ipa/raspberrypi/controller/rpi/alsc.cpp | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/alsc.cpp b/src/ipa/raspberrypi/controller/rpi/alsc.cpp index 76e2f04..12359dc 100644 --- a/src/ipa/raspberrypi/controller/rpi/alsc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/alsc.cpp @@ -32,8 +32,8 @@ Alsc::~Alsc() { std::lock_guard lock(mutex_); async_abort_ = true; - async_signal_.notify_one(); } + async_signal_.notify_one(); async_thread_.join(); } @@ -268,8 +268,11 @@ void Alsc::restartAsync(StatisticsPtr &stats, Metadata *image_metadata) frame_phase_ = 0; // copy the camera mode so it won't change during the calculations async_camera_mode_ = camera_mode_; - async_start_ = true; async_started_ = true; + { + std::lock_guard lock(mutex_); + async_start_ = true; + } async_signal_.notify_one(); } @@ -315,7 +318,6 @@ void Alsc::Process(StatisticsPtr &stats, Metadata *image_metadata) RPI_LOG("Alsc: frame_phase " << frame_phase_); if (frame_phase_ >= (int)config_.frame_period || frame_count2_ < (int)config_.startup_frames) { - std::unique_lock lock(mutex_); if (async_started_ == false) { RPI_LOG("ALSC thread starting"); restartAsync(stats, image_metadata); @@ -339,8 +341,8 @@ void Alsc::asyncFunc() { std::lock_guard lock(mutex_); async_finished_ = true; - sync_signal_.notify_one(); } + sync_signal_.notify_one(); } } From patchwork Fri Jul 31 14:08:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 9094 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 916BFBD86F for ; Fri, 31 Jul 2020 14:08:13 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4CDB961EC4; Fri, 31 Jul 2020 16:08:13 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="Im42hNYC"; dkim-atps=neutral Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4349B61A14 for ; Fri, 31 Jul 2020 16:08:10 +0200 (CEST) Received: by mail-wr1-x429.google.com with SMTP id r4so25164349wrx.9 for ; Fri, 31 Jul 2020 07:08:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=37mPhLT6nTLQDR37FqVWz+4BsnzAlxE/N76++4smj1U=; b=Im42hNYCG3MOnm8gjfaVhz54I3W3sxhbeYV8NQ4GQYoejJH8qxfNyhGZ1byAbhJt6i /nkPy0sIn5KhGEdp+ZH09VmAun/CM4ODx+4cai4o11qzglcvoz03XJhclvxPN40W3a9U 5sltp6YDWzDkFbUz8U2BaNiYJZnwgtRYLtJW/6/3GKzHtlBbx4Hrqy9iUdP5YycCchiQ bpKu0BqWV5vuf9yPM3pIF5a3XoikqY5RG10S7qlnDVJLP+Yqr3KRKXJt2rRLLEXatamu Z+vKqj8DJaO+smDSDn6qPzFRdkr9rvvY32e7Z3ILDH+VEjUabh0H+0JEmYnFI7FHEMKW eCGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=37mPhLT6nTLQDR37FqVWz+4BsnzAlxE/N76++4smj1U=; b=pa6kOf8aobNsFDETJTB3Zy8FQBUdEwbv55BcemF0oQLx1Ou5NyDBYoSW6qnFVHmAOc RSDf8/RrGR257WdFDEEieHUodCZu6UnEwFJCv+gA6rwUeckGgWf61gmYLewf3v1SRgHw N70+w6e0x60pyUUa95C5NYzmq9iSfR06+qMgZfrdLxz6ziQ1XD6MkN7wUV7NFJS9h7vj qQCHi2iVXbIhTmpswEXRVw2/QGlLLuwpGnPyMvS2vE232OOwxZMW3JjJT2HiDD5M2bd0 mvEMeiSXAGcsz1nLh7U0jiEazxZcFVr9dx1UZPeaJOvgGo+F1Vsl3hmgmVy+Ad72CoCr 780g== X-Gm-Message-State: AOAM532ZnKuGAKEXhRmF1TwLn2BSoiYSrNDEsQ2sJaTMq5mhXt4DwHLZ TOTmuQnrPb6xHvp8uBhbckdKh0T6su6SNw== X-Google-Smtp-Source: ABdhPJy+FtZ3qiZdiGx2ArS4JZjewVCwnIwSOlBvGDd5BzZjoiR2cGDdYuCKzeissoSHyaic2q0DNA== X-Received: by 2002:a5d:6910:: with SMTP id t16mr3954928wru.178.1596204489530; Fri, 31 Jul 2020 07:08:09 -0700 (PDT) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id g126sm12815084wme.16.2020.07.31.07.08.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 31 Jul 2020 07:08:09 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Fri, 31 Jul 2020 15:08:00 +0100 Message-Id: <20200731140801.13253-4-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200731140801.13253-1-david.plowman@raspberrypi.com> References: <20200731140801.13253-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 3/4] libcamera: ipa: raspberrypi: ALSC: Resample luminance table X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This fixes a bug where the luminance correction table was not being resampled according to the camera mode, in the same way as the colour tables. This could be noticeable if any camera modes crop aggressively. This resampling can be done "up front" in the SwitchMode, as we have only a single fixed luminance table. In order to protect the recalculation of the table from the asynchronous thread (which reads it) I've elected to wait for that thread to go idle (though I doubt it would have mattered much). As a by-product of stopping the thread, it no longer needs its own copy of the camera mode (async_camera_mode_). Signed-off-by: David Plowman Reviewed-by: Laurent Pinchart --- src/ipa/raspberrypi/controller/rpi/alsc.cpp | 39 ++++++++++++++------- src/ipa/raspberrypi/controller/rpi/alsc.hpp | 3 +- 2 files changed, 28 insertions(+), 14 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/alsc.cpp b/src/ipa/raspberrypi/controller/rpi/alsc.cpp index 12359dc..4df9934 100644 --- a/src/ipa/raspberrypi/controller/rpi/alsc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/alsc.cpp @@ -173,19 +173,34 @@ void Alsc::Initialise() lambda_r_[i] = lambda_b_[i] = 1.0; } +void Alsc::waitForAysncThread() +{ + if (async_started_) { + async_started_ = false; + std::unique_lock lock(mutex_); + sync_signal_.wait(lock, [&] { + return async_finished_; + }); + async_finished_ = false; + } +} + void Alsc::SwitchMode(CameraMode const &camera_mode, Metadata *metadata) { (void)metadata; + // Ensure the other thread isn't running while we do this. + waitForAysncThread(); + // There's a bit of a question what we should do if the "crop" of the - // camera mode has changed. Any calculation currently in flight would - // not be useful to the new mode, so arguably we should abort it, and - // generate a new table (like the "first_time" code already here). When - // the crop doesn't change, we can presumably just leave things - // alone. For now, I think we'll just wait and see. When the crop does - // change, any effects should be transient, and if they're not transient - // enough, we'll revisit the question then. + // camera mode has changed. Should we effectively reset the algorithm + // and start over? camera_mode_ = camera_mode; + + // We must resample the luminance table like we do the others, but it's + // fixed so we can simply do it up front here. + resample_cal_table(config_.luminance_lut, camera_mode_, luminance_table_); + if (first_time_) { // On the first time, arrange for something sensible in the // initial tables. Construct the tables for some default colour @@ -201,7 +216,7 @@ void Alsc::SwitchMode(CameraMode const &camera_mode, Metadata *metadata) compensate_lambdas_for_cal(cal_table_b, lambda_b_, async_lambda_b_); add_luminance_to_tables(sync_results_, async_lambda_r_, 1.0, - async_lambda_b_, config_.luminance_lut, + async_lambda_b_, luminance_table_, config_.luminance_strength); memcpy(prev_sync_results_, sync_results_, sizeof(prev_sync_results_)); @@ -266,8 +281,6 @@ void Alsc::restartAsync(StatisticsPtr &stats, Metadata *image_metadata) } copy_stats(statistics_, stats, alsc_status); frame_phase_ = 0; - // copy the camera mode so it won't change during the calculations - async_camera_mode_ = camera_mode_; async_started_ = true; { std::lock_guard lock(mutex_); @@ -672,9 +685,9 @@ void Alsc::doAlsc() // Fetch the new calibrations (if any) for this CT. Resample them in // case the camera mode is not full-frame. get_cal_table(ct_, config_.calibrations_Cr, cal_table_tmp); - resample_cal_table(cal_table_tmp, async_camera_mode_, cal_table_r); + resample_cal_table(cal_table_tmp, camera_mode_, cal_table_r); get_cal_table(ct_, config_.calibrations_Cb, cal_table_tmp); - resample_cal_table(cal_table_tmp, async_camera_mode_, cal_table_b); + resample_cal_table(cal_table_tmp, camera_mode_, cal_table_b); // You could print out the cal tables for this image here, if you're // tuning the algorithm... (void)print_cal_table; @@ -697,7 +710,7 @@ void Alsc::doAlsc() compensate_lambdas_for_cal(cal_table_b, lambda_b_, async_lambda_b_); // Fold in the luminance table at the appropriate strength. add_luminance_to_tables(async_results_, async_lambda_r_, 1.0, - async_lambda_b_, config_.luminance_lut, + async_lambda_b_, luminance_table_, config_.luminance_strength); } diff --git a/src/ipa/raspberrypi/controller/rpi/alsc.hpp b/src/ipa/raspberrypi/controller/rpi/alsc.hpp index e895913..95572af 100644 --- a/src/ipa/raspberrypi/controller/rpi/alsc.hpp +++ b/src/ipa/raspberrypi/controller/rpi/alsc.hpp @@ -60,10 +60,10 @@ private: AlscConfig config_; bool first_time_; CameraMode camera_mode_; + double luminance_table_[ALSC_CELLS_X * ALSC_CELLS_Y]; std::thread async_thread_; void asyncFunc(); // asynchronous thread function std::mutex mutex_; - CameraMode async_camera_mode_; // condvar for async thread to wait on std::condition_variable async_signal_; // condvar for synchronous thread to wait on @@ -86,6 +86,7 @@ private: int frame_count2_; double sync_results_[3][ALSC_CELLS_Y][ALSC_CELLS_X]; double prev_sync_results_[3][ALSC_CELLS_Y][ALSC_CELLS_X]; + void waitForAysncThread(); // The following are for the asynchronous thread to use, though the main // thread can set/reset them if the async thread is known to be idle: void restartAsync(StatisticsPtr &stats, Metadata *image_metadata); From patchwork Fri Jul 31 14:08:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 9095 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 110BCBD86F for ; Fri, 31 Jul 2020 14:08:14 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9C62A61F0C; Fri, 31 Jul 2020 16:08:13 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="TkFZ5n7+"; dkim-atps=neutral Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2FF9C61EEF for ; Fri, 31 Jul 2020 16:08:11 +0200 (CEST) Received: by mail-wr1-x431.google.com with SMTP id r2so23045750wrs.8 for ; Fri, 31 Jul 2020 07:08:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=vr8MqjIF3etQv7i3lRrEg6lcaH4p38YZ64pRJ90aOUs=; b=TkFZ5n7+JBmCGZdkZQRtu5N4ZGxym2oEVPXKzNctA69vku+q4HPXi9ift4CrswEVzk 3mZjVfKK2U+U3KDEKlM3ozLAhVx8UnHZX5JvrxlhXxtLhXHY+worHsaSSvH5qButLY3n MYSe7q9QE94VZEdQGfKkG8/czMxxRWL1aR6C74B+WHaHFnroh4jrQqRIATNq9NituVp4 8wqydZjIxv/LWMaDPwYPk2WDb6v2B0fZEuwWD+kc4un0pzWMSSQ75M7YaseJbxsup7Lg lDtjWDba3GtykSFJhw45EZ3mpU7rKRLXbzTm75oll2ai8i4kkOBF43ASiwcrSmsduASe h26A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=vr8MqjIF3etQv7i3lRrEg6lcaH4p38YZ64pRJ90aOUs=; b=JQ60dOEk054yRPZlsQcTTaZmi0kIWLBJTvMn/42ABQswFDutvbNecClrnsO0PxytED YH9/45fB6TL/m9+TK6ki7l91Ny4P61VeFP0TvfKYyxQbE8d9lQA+s76+urCtpErs9osi sLURbnnDtFGJeAP5UKZXTkCMw2u5d9UsCIO4sG5h0cP6mNzH1eb8mcgeU1JljqlrMfHr T2lK8qocXE2An/IVcES8nokcW9W4aR1KGNdH+Yg/3rQwXDMqmmXjcEFrchq2jqXseszv YFqwWK/eFbydU8WyxIRIsyAkMSx960lW1ZkiBtOavnxrLyMAD981aEeasvf9Whqj74gu N2vw== X-Gm-Message-State: AOAM532R3PBUwh2385TSKu6q254u6zn5Qwr/0OG1AdldT73Eh3gFeeO/ yvesniwiASPruguEGh9savp6A1S9NQAGWg== X-Google-Smtp-Source: ABdhPJy28Ttwdw7mO260IjWq6AW+rJtMiO+Mj5Li6EJ3EEUiPEcUBUSanEHk+htVsL3D78u+uyG4IQ== X-Received: by 2002:a5d:5446:: with SMTP id w6mr3611926wrv.127.1596204490513; Fri, 31 Jul 2020 07:08:10 -0700 (PDT) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id g126sm12815084wme.16.2020.07.31.07.08.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 31 Jul 2020 07:08:09 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Fri, 31 Jul 2020 15:08:01 +0100 Message-Id: <20200731140801.13253-5-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200731140801.13253-1-david.plowman@raspberrypi.com> References: <20200731140801.13253-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 4/4] libcamera: ipa: raspberrypi: ALSC: Improve behaviour when camera mode changes X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Now that we stop the asynchronous thread on a SwitchMode, we would do better to regenerate all the tables if the new camera mode crops in a significantly different way to the old one. A few minor tweaks make sense along with this: * Reset the lambda values when we reset everything. It wouldn't make sense to re-start with the old mode's values. * Use the last recorded colour temperature to generate new tables rather than any default value. * Set the frame "phase" counter to ensure the adaptive procedure will run asap. Signed-off-by: David Plowman Reviewed-by: Laurent Pinchart --- src/ipa/raspberrypi/controller/rpi/alsc.cpp | 54 ++++++++++++++------- 1 file changed, 37 insertions(+), 17 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/alsc.cpp b/src/ipa/raspberrypi/controller/rpi/alsc.cpp index 4df9934..6e781bd 100644 --- a/src/ipa/raspberrypi/controller/rpi/alsc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/alsc.cpp @@ -166,11 +166,8 @@ void Alsc::Initialise() RPI_LOG("Alsc"); frame_count2_ = frame_count_ = frame_phase_ = 0; first_time_ = true; - // Initialise the lambdas. Each call to Process then restarts from the - // previous results. Also initialise the previous frame tables to the - // same harmless values. - for (int i = 0; i < XY; i++) - lambda_r_[i] = lambda_b_[i] = 1.0; + ct_ = config_.default_ct; + // The lambdas are initialised in the SwitchMode. } void Alsc::waitForAysncThread() @@ -185,31 +182,53 @@ void Alsc::waitForAysncThread() } } +static bool compare_modes(CameraMode const &cm0, CameraMode const &cm1) +{ + // Return true if the modes crop from the sensor significantly differently. + int left_diff = std::abs(cm0.crop_x - cm1.crop_x); + int top_diff = std::abs(cm0.crop_y - cm1.crop_y); + int right_diff = std::abs(cm0.crop_x + cm0.scale_x * cm0.width - + cm1.crop_x - cm1.scale_x * cm1.width); + int bottom_diff = std::abs(cm0.crop_y + cm0.scale_y * cm0.height - + cm1.crop_y - cm1.scale_y * cm1.height); + // These thresholds are a rather arbitrary amount chosen to trigger + // when carrying on with the previously calculated tables might be + // worse than regenerating them (but without the adaptive algorithm). + int threshold_x = cm0.sensor_width >> 4; + int threshold_y = cm0.sensor_height >> 4; + return left_diff > threshold_x || right_diff > threshold_x || + top_diff > threshold_y || bottom_diff > threshold_y; +} + void Alsc::SwitchMode(CameraMode const &camera_mode, Metadata *metadata) { (void)metadata; + // We're going to start over with the tables if there's any "significant" + // change. + bool reset_tables = first_time_ || compare_modes(camera_mode_, camera_mode); + // Ensure the other thread isn't running while we do this. waitForAysncThread(); - // There's a bit of a question what we should do if the "crop" of the - // camera mode has changed. Should we effectively reset the algorithm - // and start over? camera_mode_ = camera_mode; // We must resample the luminance table like we do the others, but it's // fixed so we can simply do it up front here. resample_cal_table(config_.luminance_lut, camera_mode_, luminance_table_); - if (first_time_) { - // On the first time, arrange for something sensible in the - // initial tables. Construct the tables for some default colour - // temperature. This echoes the code in doAlsc, without the - // adaptive algorithm. + if (reset_tables) { + // Upon every "table reset", arrange for something sensible to be + // generated. Construct the tables for the previous recorded colour + // temperature. In order to start over from scratch we initialise + // the lambdas, but the rest of this code then echoes the code in + // doAlsc, without the adaptive algorithm. + for (int i = 0; i < XY; i++) + lambda_r_[i] = lambda_b_[i] = 1.0; double cal_table_r[XY], cal_table_b[XY], cal_table_tmp[XY]; - get_cal_table(4000, config_.calibrations_Cr, cal_table_tmp); + get_cal_table(ct_, config_.calibrations_Cr, cal_table_tmp); resample_cal_table(cal_table_tmp, camera_mode_, cal_table_r); - get_cal_table(4000, config_.calibrations_Cb, cal_table_tmp); + get_cal_table(ct_, config_.calibrations_Cb, cal_table_tmp); resample_cal_table(cal_table_tmp, camera_mode_, cal_table_b); compensate_lambdas_for_cal(cal_table_r, lambda_r_, async_lambda_r_); @@ -220,6 +239,7 @@ void Alsc::SwitchMode(CameraMode const &camera_mode, Metadata *metadata) config_.luminance_strength); memcpy(prev_sync_results_, sync_results_, sizeof(prev_sync_results_)); + frame_phase_ = config_.frame_period; // run the algo again asap first_time_ = false; } } @@ -265,8 +285,8 @@ void Alsc::restartAsync(StatisticsPtr &stats, Metadata *image_metadata) { RPI_LOG("Starting ALSC thread"); // Get the current colour temperature. It's all we need from the - // metadata. - ct_ = get_ct(image_metadata, config_.default_ct); + // metadata. Default to the last CT value (which could be the default). + ct_ = get_ct(image_metadata, ct_); // We have to copy the statistics here, dividing out our best guess of // the LSC table that the pipeline applied to them. AlscStatus alsc_status;