From patchwork Mon Jul 6 08:56:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 8645 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 13EBEBD792 for ; Mon, 6 Jul 2020 08:56:40 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8B1B760C59; Mon, 6 Jul 2020 10:56:39 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="F0pF90f6"; dkim-atps=neutral Received: from mail-wm1-x343.google.com (mail-wm1-x343.google.com [IPv6:2a00:1450:4864:20::343]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CEE77603B1 for ; Mon, 6 Jul 2020 10:56:37 +0200 (CEST) Received: by mail-wm1-x343.google.com with SMTP id f139so41030059wmf.5 for ; Mon, 06 Jul 2020 01:56:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=n32uuMBO84Te9GK7nFDuckXfnJke7rImJZsKTjKeGHo=; b=F0pF90f62UV/HMZ1BFzIQw9YGSyfhO3RStkuJu+emd0g69HlWWfyDHnhfVm538/6zH RPfhQA+G70DfGwwRDdppfYxyRGydn0IfuP8ER0RPuLzRyaM71wIj5rEDmPjuZxLI5UFs D0vfS8wLfNBTYz56E42ZvH/Rldt8X5nxUjf8MM89qi7iey1VNA4HTbPYn0/bRPU7+CSK gke4VfyefoNfTmHmMyB4RtdWimZKQcdpD0yYDpvRFDb8l2GKbCNgcTn6OuDbELwpMJrq cx/QKPHRaRpmR49eyBIk/fPj8LsCgbWftytWc9nNm61XDi7FFSRRI5wrJ0IXewVzv/+M bNdA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=n32uuMBO84Te9GK7nFDuckXfnJke7rImJZsKTjKeGHo=; b=a4m0FcxqIWnZ5D3HUVQ1qmXPYPYrVTe7KhggbuPNd+Go74uOSzQLivQ0Hucce7Ivam Iy5zW+pdNGqAHimzDuNKeJKnS0HcLXl5HzCBuSMXW2SawC3kcOrre1Dnd4e6aTjmYcaK hlq/Qyssup4Uzunz781wpLbkoQSD8VNilBfSxCYp5+5gCGQxym06zaMYwg9dMeWiapv4 9xLzekuKXD8ub+GZL7jHJeY3aww0fUbK9pxhjsdn/1TVkhWcMxRB+qwzHDJp8BZ7fAvS S29GJ0nNmawBGQ7Omk/UfnmezBNst4UjiZUeHr9FMQ4p7Spq4LZE10Fq+OzcMLCaVkQu Oi+w== X-Gm-Message-State: AOAM5311CsyBzASgBHsZXu89xyxTDfHnPHL82ozQdwWFK1c9uv0dDZ51 z870QVMVzWG/Sp88BTC5MFeFE15HJoY= X-Google-Smtp-Source: ABdhPJxO+fpRFEhFBsfWJUcZH/VsYfQqiGEHTMEsk7zz93zoRYVCnwmQ4PCCycE6EebvJkm2OU36zw== X-Received: by 2002:a1c:1bc6:: with SMTP id b189mr46471606wmb.166.1594025797276; Mon, 06 Jul 2020 01:56:37 -0700 (PDT) Received: from naushir-VirtualBox.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id k185sm22600282wmk.47.2020.07.06.01.56.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Jul 2020 01:56:36 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Mon, 6 Jul 2020 09:56:23 +0100 Message-Id: <20200706085625.1625432-2-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200706085625.1625432-1-naush@raspberrypi.com> References: <20200706085625.1625432-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 1/3] libcamera: controls: Add focus Figure of Merit (FoM) control X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Provide a control to allow the IPA to return a FoM to indicate how in-focus a scene is. Note, this is not to be used as a means to implement a focus algorithm by the application, rather an indication of how in-focus a scene is. Signed-off-by: Naushir Patuck Reviewed-by: Laurent Pinchart --- src/libcamera/control_ids.yaml | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml index 8c3e4c72..988b5016 100644 --- a/src/libcamera/control_ids.yaml +++ b/src/libcamera/control_ids.yaml @@ -251,4 +251,15 @@ controls: higher than anyone could reasonably want. Negative values are not allowed. Note also that sharpening is not applied to raw streams. + + - FocusFoM: + type: int32_t + description: | + Reports a Figure of Merit (FoM) to indicate how in-focus the frame is. + A larger FocusFoM value indicates a more in-focus frame. This control + depends on the IPA to gather ISP statistics from the defined focus + region, and combine them in a suitable way to generate a FocusFoM value. + In this respect, it is not necessarily aimed at providing a way to + implement a focus algorithm by the application, rather an indication of + how in-focus a frame is. ... From patchwork Mon Jul 6 08:56:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 8646 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5B1E8BD794 for ; Mon, 6 Jul 2020 08:56:40 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CC18260E14; Mon, 6 Jul 2020 10:56:39 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="EHRyymkp"; dkim-atps=neutral Received: from mail-wm1-x342.google.com (mail-wm1-x342.google.com [IPv6:2a00:1450:4864:20::342]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E076860E01 for ; Mon, 6 Jul 2020 10:56:38 +0200 (CEST) Received: by mail-wm1-x342.google.com with SMTP id g10so15202787wmc.1 for ; Mon, 06 Jul 2020 01:56:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=2SGLNdyFdsUGjI3R6sQATmjr6zk1FJpyPnlDLESP9oY=; b=EHRyymkpY8P7vQPkO4qUNwUsop6F9Myo9Qx5lmuPzwezOLqw+3v9F8FnvE9Gj70qh2 XUSNOh5FCXB70oZDMShMzwLd5P5KUTf5WhMOsRoFBTVbR/+7gPuexhhwLYukJ2ux3dSJ /ujWwhSppTbdZU0XW5DzpmSisxXgneMKuUX8Xxw+IipfYFH6iMwUHGcEcSlVgFZ79PuK a0HeUKaAZ/npL2HUDPqkK26F0xDeSwXmLrvtwGlYazVRCJBQoma9T4Anq33lvR4MTcgT GCKMi/LkRzhivXJ1sGqWpGLM8Wz+xPeiL27aU4iToa+6EavPfkWH0NrlC7aXVBmJcW3u OufA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2SGLNdyFdsUGjI3R6sQATmjr6zk1FJpyPnlDLESP9oY=; b=iMiCOVI1+ZK7et6uWpKmVNabtduMDnhybjJqXeiFF4YB/j0iN5FhnRzYEsnvFQRJ1F I0ZsENdhxCgXy0lJL6Fv5mgGfguEs/9q7/8e9+LYoiDyjY5eBsxrlvtmFzNX6xvfeLpB kE5uQD/MfltUJmByiq/Rbv8eJEnFXC06bl6yg5dmm2eirEOytxear8SetNDQAXbXMsej heQKNgA+D7XHwDVg3AUagG9figUQsFnbJHSXLjJFbZr5VkXGHzK/qfB/IXR2r/7SH5s2 8Gs8noWUD/H6BGJ0KSaK1SqxEEfxNUyY+ohWlFYeZqudGLiF8cNfmT3hwRNgqm6q6AOR f8Tg== X-Gm-Message-State: AOAM532p/eGjToK3ViCrR2m3kMvHMv+gxDMFNjORUkDLA6CAaiwNYq3O YttHUEZnbH1vtxCDq5Dc+KALG1s6/5k= X-Google-Smtp-Source: ABdhPJy8aI0Jyv49cstyUm5W35HzLVmXQGxWY4uGd9w9S+7l19kbEwOlLmCyhyDMPUd0hk3o4YO+KA== X-Received: by 2002:a1c:a4c6:: with SMTP id n189mr47012939wme.173.1594025798350; Mon, 06 Jul 2020 01:56:38 -0700 (PDT) Received: from naushir-VirtualBox.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id k185sm22600282wmk.47.2020.07.06.01.56.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Jul 2020 01:56:37 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Mon, 6 Jul 2020 09:56:24 +0100 Message-Id: <20200706085625.1625432-3-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200706085625.1625432-1-naush@raspberrypi.com> References: <20200706085625.1625432-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 2/3] libcamera: ipa: raspberrypi: Populate libcamera metadata after controller process X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" For control algorithms like focus, this is needed to return out the the current frame focus statistics. For other algorithms, there is no functional change. Signed-off-By: Naushir Patuck Reviewed-by: David Plowman Reviewed-by: Laurent Pinchart --- src/ipa/raspberrypi/raspberrypi.cpp | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index bc89ab58..62730198 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -318,6 +318,8 @@ void IPARPi::processEvent(const IPAOperationData &event) if (frame_count_ > mistrust_count_) processStats(bufferId); + reportMetadata(); + IPAOperationData op; op.operation = RPI_IPA_ACTION_STATS_METADATA_COMPLETE; op.data = { bufferId & RPiIpaMask::ID }; @@ -336,7 +338,6 @@ void IPARPi::processEvent(const IPAOperationData &event) * they are "unreliable". */ prepareISP(embeddedbufferId); - reportMetadata(); /* Ready to push the input buffer into the ISP. */ IPAOperationData op; From patchwork Mon Jul 6 08:56:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 8647 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5E32DBD792 for ; Mon, 6 Jul 2020 08:56:42 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2ACB660E05; Mon, 6 Jul 2020 10:56:42 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="V2fBFJaY"; dkim-atps=neutral Received: from mail-wm1-x343.google.com (mail-wm1-x343.google.com [IPv6:2a00:1450:4864:20::343]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DA0EF60E35 for ; Mon, 6 Jul 2020 10:56:39 +0200 (CEST) Received: by mail-wm1-x343.google.com with SMTP id j18so38381333wmi.3 for ; Mon, 06 Jul 2020 01:56:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=qSV2uqoWNA3gnKhA2JBI5GAxVQ3yH469eK933yQUNrg=; b=V2fBFJaYqFlYeaj1+8IRDuS6D0cC8rFTKpQCWvSZdYd+FXC1vOHzdgEukEhrYHQLhG aym36yCzhxn411c5I0Dlbeu51zP8DCO6P/t82jeku2RrM2c+tYy33YNop687wbLplR0+ 4X/AF9Os7xIoJiKCSl8KUIYJI4WtWbD6CEaZ6JdkM70miL2E/hzCBNWKEiuYjVzgVHLr eWTrY/VoZ9RqT6hW7FbSolbEVXHmd5Ub04ACXJHe0LgB0K/VxAYInjQqpuFwCC6ukGWG iYc+kKmRgOHJJo7ShGqeDe7SES1vnZqTu5/pO4z3eUD8HuMCMFrjY/q5J0+G98EVRB6Z hJYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=qSV2uqoWNA3gnKhA2JBI5GAxVQ3yH469eK933yQUNrg=; b=EZ/CDwOEHn+8+ReGjAn/ny0tVuPt4DCiqOYcQK0eSf0r1rmQG05bPt9biDUS+Db+V9 Z+bj5/+A3fx7N6vdSkyMX2yKwWXcALy6acc8Al/pgu/ZPosYB2hPb+d5S3vjtsOh3stL Qg3hWvdgkPphEjQ+WvZgw4GbBMkDTg12CB3mliq1HJvo/dJ4tjFgutQ6iyg94Pk4rnqR JP7rOsmaY/Og6GwOGodWJMJDZBLcUrHxHx7clhuugPiHwn6laYSXE/WYRafCYTTwdfc4 iPMxj45AoZfID/hcXt/55IlFb5iHr793C4QMXzRgAObYbsml9H4HvMvCEeAfvZXc6JM6 PeZg== X-Gm-Message-State: AOAM533qClQBUTrgqGJ1ApFecpWZDFbZXDV3rTiXc27hQtdflH0H8dxE g1H19WLm+Kh7OeLg+9x0J1paR5zqMDk= X-Google-Smtp-Source: ABdhPJzRKg3lWI6bQ9BnhkChyyMuoQojc7MKmejiz2mT/RhFQHU363ubuCUbAe0XxsibZkquAWEZBA== X-Received: by 2002:a1c:9a06:: with SMTP id c6mr9579150wme.23.1594025799335; Mon, 06 Jul 2020 01:56:39 -0700 (PDT) Received: from naushir-VirtualBox.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id k185sm22600282wmk.47.2020.07.06.01.56.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Jul 2020 01:56:38 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Mon, 6 Jul 2020 09:56:25 +0100 Message-Id: <20200706085625.1625432-4-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200706085625.1625432-1-naush@raspberrypi.com> References: <20200706085625.1625432-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 3/3] libcamera: ipa: raspberrypi: Populate focus FoM from the ISP statistics X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Switch FocusStatus::num to unsigned int for convenience. Fill in libcamera::controls::FocusFoM with the average of the middle two regions (across a 4x3 grid) FoM statistics from the ISP. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Reviewed-by: Laurent Pinchart --- src/ipa/raspberrypi/controller/focus_status.h | 2 +- src/ipa/raspberrypi/raspberrypi.cpp | 12 ++++++++++++ 2 files changed, 13 insertions(+), 1 deletion(-) diff --git a/src/ipa/raspberrypi/controller/focus_status.h b/src/ipa/raspberrypi/controller/focus_status.h index 3ad88777..ace2fe2c 100644 --- a/src/ipa/raspberrypi/controller/focus_status.h +++ b/src/ipa/raspberrypi/controller/focus_status.h @@ -17,7 +17,7 @@ extern "C" { #endif struct FocusStatus { - int num; + unsigned int num; uint32_t focus_measures[FOCUS_REGIONS]; }; diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 62730198..b1f27861 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -42,6 +42,7 @@ #include "contrast_status.h" #include "controller.hpp" #include "dpc_status.h" +#include "focus_status.h" #include "geq_status.h" #include "lux_status.h" #include "metadata.hpp" @@ -405,6 +406,17 @@ void IPARPi::reportMetadata() static_cast(blackLevelStatus->black_level_g), static_cast(blackLevelStatus->black_level_g), static_cast(blackLevelStatus->black_level_b) }); + + FocusStatus *focusStatus = rpiMetadata_.GetLocked("focus.status"); + if (focusStatus && focusStatus->num == 12) { + /* + * We get a 4x3 grid of regions by default. Calculate the average + * FoM over the central two positions to give an overall scene FoM. + * This can change later if it is not deemed suitable. + */ + int32_t focusFoM = (focusStatus->focus_measures[5] + focusStatus->focus_measures[6]) / 2; + libcameraMetadata_.set(controls::FocusFoM, focusFoM); + } } /*