From patchwork Fri Jun 19 09:27:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 4099 Return-Path: Received: from mail-wm1-x341.google.com (mail-wm1-x341.google.com [IPv6:2a00:1450:4864:20::341]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1145A609A5 for ; Fri, 19 Jun 2020 11:27:52 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="cPi1Gktm"; dkim-atps=neutral Received: by mail-wm1-x341.google.com with SMTP id g10so7797256wmh.4 for ; Fri, 19 Jun 2020 02:27:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=p53m5urB+gdoT9u3aVX/6ufi992DRz37k9liETGXFCU=; b=cPi1Gktm5HtULLFDoRotl+KvwdsHAgJ3P1PGLNHLC07vTptkbfzcxkNohAysWPvjA4 egr2mo2pe5J269aNTDz557OyUj4Hi+89+c2tczMx4sMD+AKmpVqhTz3eU9AyrgN5Y6Le pkNQOJBrwTDvrLdJexUQvZH/tfPpjNEMFjvKtueZ6NnxzAglXF/6FtMncqaLCAbQGuKr BKSyLAmqqmR5XQoRLFIE3vph1yKSShyzyZHtjybSlaXgnbfJZ5oIKq+5aAExdMwbmDxG 4O2vKdRhmtecpJXlaHuYANkaaMtre54IMJBPXibaKOqHXsHRhgz7PULphkBPGqHVO2np eqsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=p53m5urB+gdoT9u3aVX/6ufi992DRz37k9liETGXFCU=; b=qNsOT4v/QDHWLI02dEhvSbjbdpJPfxIUsSgkIneFzH8/9/8Ftn/SlNEaWtE7Dy4SEV ZXOfTmfMD0kqMVebYImcA5TfUVE+enojegCmd06oWZxICxn5cftZYv4l5wP7aoGwrOtp cecaHYCCrWs3QxH4VHRdwb8UUIGs3gi5iuhvwk1tRhNsrXnyBDbWkl9tm1NalCf7s1O1 xQghqpc/e5yOW3Lu3hL7IYRenZEZM0rPGT61iHpRAbkSiK9U9Hn8Qo/ynBZdWYzwJw/Z iZnX1owjq4PbpH9yaouOmCEUBoJIKHXEYzkSRkj/4BTMkBHyJPVGb1uk6KDuR5Kxih7j b9OQ== X-Gm-Message-State: AOAM532X/jJXLGXQlz3+Zz27sbJJJYBvClt3Xv71m0FHa4XnjMlzNdyb uUgadBMdhaZp9nx9LLeV7y2ENm0u1q0= X-Google-Smtp-Source: ABdhPJwcoa2waiZEh0kL4uEp5bKUFZclOgo4zF7OmH/w1Ryoe/7vYgSA1A/61vJTWhH+mA9LLCAaIg== X-Received: by 2002:a1c:bc0a:: with SMTP id m10mr2688067wmf.173.1592558871513; Fri, 19 Jun 2020 02:27:51 -0700 (PDT) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id n65sm5156154wmf.47.2020.06.19.02.27.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 19 Jun 2020 02:27:51 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Fri, 19 Jun 2020 10:27:23 +0100 Message-Id: <20200619092725.19109-2-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200619092725.19109-1-david.plowman@raspberrypi.com> References: <20200619092725.19109-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 1/3] libcamera: add a sharpness strength control X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Jun 2020 09:27:52 -0000 The control is a single float value with minimum, default and maximum values. Please read the description for more details. Signed-off-by: David Plowman --- src/libcamera/control_ids.yaml | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml index 77ebc3f..1bc1b10 100644 --- a/src/libcamera/control_ids.yaml +++ b/src/libcamera/control_ids.yaml @@ -239,4 +239,15 @@ controls: pixel range (as if pixels ranged from 0 to 65535). The SensorBlackLevels control can only be returned in metadata. size: [4] + + - Sharpness: + type: float + description: | + The strength of the sharpening to be applied. The minimum value + means minimal (or preferably no) sharpening, the maximum should + signify extremely high levels of sharpening (higher than anyone could + reasonably want), and the default value should give a "reasonable" + level, suitable for many use cases. We recommand that the amount + of sharpening applied should be "approximately" proportional to this + parameter. ... From patchwork Fri Jun 19 09:27:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 4100 Return-Path: Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 98524609B3 for ; Fri, 19 Jun 2020 11:27:53 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="aQKA7vm0"; dkim-atps=neutral Received: by mail-wr1-x42b.google.com with SMTP id q11so8997824wrp.3 for ; Fri, 19 Jun 2020 02:27:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3QCDBkaxFXVXaBWcvlJm6aIO+K2xVTiA1JY72i6arQ8=; b=aQKA7vm0D8C1i3DTs3IAo5x5ER8wyE/snCIqoRvJZLrDp/qiDfyVc4hJiPn3kU19M1 kgb+tExfzWk/ynqKeeZ/FHr66aCARpuMIQsF7iAItZMrJ2cf6efRZg8928lXhmskP4qB s5vXhAL1VGFJCEA2/wk8BNox6o7Y4CmtrnDwrugiByVC83kq4+Azd7NFXXJuidu8GghR +1qGR5P8I3LXrZK1thgom7LAyEPtKsGlRa8ofHmXHZgMoSKKrMDc3xbXmU/RZQ6BIGyn a6VX1SHiHm2SNsHogfUucVUhaabpAbyZgIlCTns2h71NVGsfqBfGzBO4DBNLdf3QUrem B3Bw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3QCDBkaxFXVXaBWcvlJm6aIO+K2xVTiA1JY72i6arQ8=; b=ZI/QuoORfL4qLml0TT4oPQyq8pDwLbco6hbEHg5qWzy1L8Ee9nCTbMwqVEO6thcK1p 8D1E2wadweFRY4zQAcef61kYqCUuSOs4YTZXBMrtmKZIPdUqaZaNz1ScFlA7NdoPuoNy tYhwHPB4o+GDAoFfdrd64iX5K1vB+y8PaIDhZK15goh0ygF1XpEamuzD/ixjol6JIFmh 06U64TA5a2rUNAqnY+/cnQCA1Ysy4+5PhaRu5WYQkYHzBwVbAZXxgH6cZk9SOw7oiqWY gc1SZccHgfTD5ol1KmaD2G/PRb6yCt4+Pe/WdN0/bXK2jiDCVp63+5EJBCuEExzVYXaL gZmw== X-Gm-Message-State: AOAM533WzkMuefpJiNJVM0YFnqoXPyXourzbSuq4DWg/XSdp/BKrx31q 8tGOER7xMmZdWt5fzJtPAe+SAh6lJ2c= X-Google-Smtp-Source: ABdhPJxZwExjevM8zKMtt6JI9M1QTQS8xjMnPXDFbRQJTdowrM+F/TT3jyJR9w8g5pKG4FS28A+IlA== X-Received: by 2002:a5d:490f:: with SMTP id x15mr2998614wrq.259.1592558872976; Fri, 19 Jun 2020 02:27:52 -0700 (PDT) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id n65sm5156154wmf.47.2020.06.19.02.27.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 19 Jun 2020 02:27:52 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Fri, 19 Jun 2020 10:27:24 +0100 Message-Id: <20200619092725.19109-3-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200619092725.19109-1-david.plowman@raspberrypi.com> References: <20200619092725.19109-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 2/3] libcamera: raspberrypi: add sharpness strength control to Raspberry Pi IPAs X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Jun 2020 09:27:53 -0000 The sharpness control is, loosely speaking, a gain applied to the amount of sharpening added to an image. We also report the sharpness setting used back to the caller in metadata. Signed-off-by: David Plowman --- .../raspberrypi/controller/rpi/sharpen.cpp | 23 +++++++++++++++---- .../raspberrypi/controller/rpi/sharpen.hpp | 6 +++-- .../controller/sharpen_algorithm.hpp | 21 +++++++++++++++++ .../raspberrypi/controller/sharpen_status.h | 2 ++ 4 files changed, 46 insertions(+), 6 deletions(-) create mode 100644 src/ipa/raspberrypi/controller/sharpen_algorithm.hpp diff --git a/src/ipa/raspberrypi/controller/rpi/sharpen.cpp b/src/ipa/raspberrypi/controller/rpi/sharpen.cpp index 1f07bb6..45fe3fe 100644 --- a/src/ipa/raspberrypi/controller/rpi/sharpen.cpp +++ b/src/ipa/raspberrypi/controller/rpi/sharpen.cpp @@ -17,7 +17,7 @@ using namespace RPi; #define NAME "rpi.sharpen" Sharpen::Sharpen(Controller *controller) - : Algorithm(controller) + : SharpenAlgorithm(controller), user_strength_(1.0) { } @@ -40,15 +40,30 @@ void Sharpen::Read(boost::property_tree::ptree const ¶ms) limit_ = params.get("limit", 1.0); } +void Sharpen::SetStrength(double strength) +{ + // Note that this method is how an application sets the overall + // sharpening "strength". We call this the "user strength" field + // as there already is a strength_ field - being an internal gain + // parameter that gets passed to the ISP control code. + user_strength_ = strength; +} + void Sharpen::Prepare(Metadata *image_metadata) { double mode_factor = mode_factor_; + // The user_strength affects the algorithm's internal gain directly, but + // we adjust the limit and threshold more cautiously, hence the sqrt. + double user_strength = user_strength_; + double user_strength_sqrt = sqrt(user_strength); struct SharpenStatus status; // Binned modes seem to need the sharpening toned down with this // pipeline. - status.threshold = threshold_ * mode_factor; - status.strength = strength_ / mode_factor; - status.limit = limit_ / mode_factor; + status.threshold = threshold_ * mode_factor / user_strength_sqrt; + status.strength = (strength_ / mode_factor) * user_strength; + status.limit = (limit_ / mode_factor) * user_strength_sqrt; + // Finally, report any application-supplied parameters that were used. + status.user_strength = user_strength; image_metadata->Set("sharpen.status", status); } diff --git a/src/ipa/raspberrypi/controller/rpi/sharpen.hpp b/src/ipa/raspberrypi/controller/rpi/sharpen.hpp index 3b0d680..5b419b4 100644 --- a/src/ipa/raspberrypi/controller/rpi/sharpen.hpp +++ b/src/ipa/raspberrypi/controller/rpi/sharpen.hpp @@ -6,20 +6,21 @@ */ #pragma once -#include "../algorithm.hpp" +#include "../sharpen_algorithm.hpp" #include "../sharpen_status.h" // This is our implementation of the "sharpen algorithm". namespace RPi { -class Sharpen : public Algorithm +class Sharpen : public SharpenAlgorithm { public: Sharpen(Controller *controller); char const *Name() const override; void SwitchMode(CameraMode const &camera_mode) override; void Read(boost::property_tree::ptree const ¶ms) override; + void SetStrength(double strength) override; void Prepare(Metadata *image_metadata) override; private: @@ -27,6 +28,7 @@ private: double strength_; double limit_; std::atomic mode_factor_; + std::atomic user_strength_; }; } // namespace RPi diff --git a/src/ipa/raspberrypi/controller/sharpen_algorithm.hpp b/src/ipa/raspberrypi/controller/sharpen_algorithm.hpp new file mode 100644 index 0000000..3b27a74 --- /dev/null +++ b/src/ipa/raspberrypi/controller/sharpen_algorithm.hpp @@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: BSD-2-Clause */ +/* + * Copyright (C) 2020, Raspberry Pi (Trading) Limited + * + * sharpen_algorithm.hpp - sharpness control algorithm interface + */ +#pragma once + +#include "algorithm.hpp" + +namespace RPi { + +class SharpenAlgorithm : public Algorithm +{ +public: + SharpenAlgorithm(Controller *controller) : Algorithm(controller) {} + // A sharpness control algorithm must provide the following: + virtual void SetStrength(double strength) = 0; +}; + +} // namespace RPi diff --git a/src/ipa/raspberrypi/controller/sharpen_status.h b/src/ipa/raspberrypi/controller/sharpen_status.h index 6de80f4..7501b19 100644 --- a/src/ipa/raspberrypi/controller/sharpen_status.h +++ b/src/ipa/raspberrypi/controller/sharpen_status.h @@ -19,6 +19,8 @@ struct SharpenStatus { double strength; // upper limit of the allowed sharpening response double limit; + // The sharpening strength requested by the user or application. + double user_strength; }; #ifdef __cplusplus From patchwork Fri Jun 19 09:27:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 4101 Return-Path: Received: from mail-wm1-x343.google.com (mail-wm1-x343.google.com [IPv6:2a00:1450:4864:20::343]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 03BCA609C5 for ; Fri, 19 Jun 2020 11:27:55 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="k8F9rQuJ"; dkim-atps=neutral Received: by mail-wm1-x343.google.com with SMTP id r9so7802070wmh.2 for ; Fri, 19 Jun 2020 02:27:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=al417ZoyvhZTJN/nkNKL4b92jhJkUbp6nCYhmxuDiBU=; b=k8F9rQuJMrQzQCWDeOUrP8Izs1G8Nvi3UW9Nxrhyrf/5Ni/5X4oqvz+ixqLzZgjzLE lXd5IQiQaeguxDTvNo9VhJ+dAQfL/gwAlkiOcA68afTMm33qmXnzGlahuHjCBjWaswqN xeoB8QDcaqSDcL2VVKYFfscvSdV0Bdswds+Dk+AixQ+ios43PKDj51tnQ23Z8zgHEjdY l11nw8CEdvIEPD/30opATqoIPtQd5Xfx6pC+yn70ott6cRxUGJzMf+Xgv5eYROiJ48W8 gGZK6ArP/Qc3W3fhU6slgmmOwJlmnPA8ZANApPw3kT6LAqLHrj5MY8LS9fKH+kCXsUuJ HurQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=al417ZoyvhZTJN/nkNKL4b92jhJkUbp6nCYhmxuDiBU=; b=N6l3FRzmplEQW+Z63WHBgZDavTZvzqWXMNRZUZ0CVkgOGUuuhx0VfB/9gPq7deJlSq ZOdXLdOifE5jCGw7GvFy3HIePU39PGh0H2YWXvARgpwTf6cepr2O5jOZGwCHRnyeFMDv tiNeqEH2djLo3TZq41W0pZlCARd7DPwwpvSZpFjq3+qYdqVO4rTdHwefdSN8RoX7MoM4 2TTIg//UpF3/GMgs2rTJIl9CgAVaWK91PEn9GveRbBIB3424zYH3lQ3hE8jGYm/cTEgI k4CECq3uZIRG0vWmlDpLovbVVJoASdeYeauEKBNzW6crb0FCjhDo5CYMhkwemuZPaq41 nFdA== X-Gm-Message-State: AOAM530NqypvYXnEhLKbB4ctB4fNm1A85z8j1kjHYKawRerYYhezw+3v 8o6B/HaLUfs3z8I8wz2hulT6kESldec= X-Google-Smtp-Source: ABdhPJyJUNGanz5IlgbPck55QxtYl6BY3qD0FWkwz5iX0RJW0iUeV7BQs7yJyUPYMgP8xADYsTMUzg== X-Received: by 2002:a1c:cc07:: with SMTP id h7mr2734941wmb.179.1592558874387; Fri, 19 Jun 2020 02:27:54 -0700 (PDT) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id n65sm5156154wmf.47.2020.06.19.02.27.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 19 Jun 2020 02:27:54 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Fri, 19 Jun 2020 10:27:25 +0100 Message-Id: <20200619092725.19109-4-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200619092725.19109-1-david.plowman@raspberrypi.com> References: <20200619092725.19109-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 3/3] libcamera: raspberrypi: plumb the libcamera sharpness control through to the Raspberry Pi implementation X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Jun 2020 09:27:55 -0000 This simply wires up the libcamera sharpness control in the Raspberry Pi IPAs so that it controls the strength of the Raspberry Pi sharpness control algorithm. Signed-off-by: David Plowman Reviewed-by: Laurent Pinchart --- include/libcamera/ipa/raspberrypi.h | 1 + src/ipa/raspberrypi/raspberrypi.cpp | 12 ++++++++++++ 2 files changed, 13 insertions(+) diff --git a/include/libcamera/ipa/raspberrypi.h b/include/libcamera/ipa/raspberrypi.h index c109469..a18ce9a 100644 --- a/include/libcamera/ipa/raspberrypi.h +++ b/include/libcamera/ipa/raspberrypi.h @@ -51,6 +51,7 @@ static const ControlInfoMap RPiControls = { { &controls::Brightness, ControlInfo(-1.0f, 1.0f) }, { &controls::Contrast, ControlInfo(0.0f, 32.0f) }, { &controls::Saturation, ControlInfo(0.0f, 32.0f) }, + { &controls::Sharpness, ControlInfo(0.0f, 16.0f, 1.0f) }, }; } /* namespace libcamera */ diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 9669f21..2f2550e 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -47,6 +47,7 @@ #include "metadata.hpp" #include "noise_status.h" #include "sdn_status.h" +#include "sharpen_algorithm.hpp" #include "sharpen_status.h" namespace libcamera { @@ -631,6 +632,17 @@ void IPARPi::queueRequest(const ControlList &controls) break; } + case controls::SHARPNESS: { + RPi::SharpenAlgorithm *sharpen = dynamic_cast( + controller_.GetAlgorithm("sharpen")); + ASSERT(sharpen); + + sharpen->SetStrength(ctrl.second.get()); + libcameraMetadata_.set(controls::Sharpness, + ctrl.second.get()); + break; + } + default: LOG(IPARPI, Warning) << "Ctrl " << controls::controls.at(ctrl.first)->name()