From patchwork Thu Dec 1 14:51:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 17930 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8344EBE08B for ; Thu, 1 Dec 2022 14:52:43 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 33BE963342; Thu, 1 Dec 2022 15:52:43 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1669906363; bh=9QHTIxtxNjyrescww7dX9bQpgbhBsBAAA0Admp9949M=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=kxKrFHrNEosLxakuVHjRMH2UIpZc8oZOwbYGIB/KAuUXvrXUYUDp6TdXg2Uz9aDpG DJ2pzH41LW/cIEl/whfQdeS7DdB2lB577IkSkMug5AhLRzMUWTMpRjfz0E/l93XnIj kbycrJqDpPepuYrG1oSQLXSn91YlAh+cWEQc1sAoW3cjsvq/XHyPCkYX9m9/sHaOiA MKExYjSmhDENYBDL+bJuSTxwtc+PNow3hh43kuCD9J3bkulGYECQnFVxR54icaFXbo cI4gmtupNcnGy4YHJkCNsKLe7TX0AbcR9bi+roIB3c/hF49nQPzkv/3wnVqi9Pe/jw liZiPhLQSCE1Q== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E44EA63335 for ; Thu, 1 Dec 2022 15:52:41 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="XF1rQDeo"; dkim-atps=neutral Received: from umang.jainideasonboard.com (unknown [210.186.188.117]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id B2A252D9; Thu, 1 Dec 2022 15:52:40 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1669906361; bh=9QHTIxtxNjyrescww7dX9bQpgbhBsBAAA0Admp9949M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XF1rQDeoX3d3rWMtINiaFWWm6vBnQKvwElBGMAMzr5l1eyGv4Kf6Lt4oF5rotNFoY xKZZVHXCeTsA1g1P6axSlwm/Co5FPz873kzp9YoazVtea2x8HTCEdUsQbfI3M361iU pmedrJicpvQ9NcgFsCeaSySbSDlF/GMzFqEoAqo0= To: libcamera-devel@lists.libcamera.org Date: Thu, 1 Dec 2022 22:51:29 +0800 Message-Id: <20221201145133.479794-2-umang.jain@ideasonboard.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221201145133.479794-1-umang.jain@ideasonboard.com> References: <20221201145133.479794-1-umang.jain@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 1/5] ipa: raspberrypi: Introduce an autofocus algorithm X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Umang Jain via libcamera-devel From: Umang Jain Reply-To: Umang Jain Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jean-Michel Hautbois Now that the ancillary links are plumbed and we can set the lens position, implement a contrast-based algorithm for RPi. This algorithm is adapted from the one proposed for the IPU3 IPA. It is currently taking all the regions and tries to make the focus on the global scene in a first attempt. Signed-off-by: Jean-Michel Hautbois --- src/ipa/raspberrypi/controller/af_algorithm.h | 33 +++ src/ipa/raspberrypi/controller/af_status.h | 21 ++ src/ipa/raspberrypi/controller/iob/af.cpp | 212 ++++++++++++++++++ src/ipa/raspberrypi/controller/iob/af.h | 62 +++++ src/ipa/raspberrypi/meson.build | 1 + 5 files changed, 329 insertions(+) create mode 100644 src/ipa/raspberrypi/controller/af_algorithm.h create mode 100644 src/ipa/raspberrypi/controller/af_status.h create mode 100644 src/ipa/raspberrypi/controller/iob/af.cpp create mode 100644 src/ipa/raspberrypi/controller/iob/af.h diff --git a/src/ipa/raspberrypi/controller/af_algorithm.h b/src/ipa/raspberrypi/controller/af_algorithm.h new file mode 100644 index 00000000..e88683a0 --- /dev/null +++ b/src/ipa/raspberrypi/controller/af_algorithm.h @@ -0,0 +1,33 @@ +/* SPDX-License-Identifier: BSD-2-Clause */ +/* + * Copyright (C) 2022, Raspberry Pi (Trading) Limited + * + * af_algorithm.h - autofocus control algorithm interface + */ +#pragma once + +#include + +#include "algorithm.h" + +namespace RPiController { + +class AfAlgorithm : public Algorithm +{ +public: + AfAlgorithm(Controller *controller) : Algorithm(controller) {} + /* An af algorithm must provide the following: */ + virtual void setMode(const uint32_t &mode) = 0; + /* start a cycle (in auto mode) */ + virtual void trigger() = 0; + /* cancel a cycle (in auto mode) */ + virtual void cancel() = 0; + /* set AF windows */ + virtual void setWindows(const libcamera::Rectangle &afWindows) = 0; + /* set AF range */ + virtual void setRange(const uint32_t &range) = 0; + /* set AF speed */ + virtual void setSpeed(const uint32_t &speed) = 0; +}; + +} /* namespace RPiController */ diff --git a/src/ipa/raspberrypi/controller/af_status.h b/src/ipa/raspberrypi/controller/af_status.h new file mode 100644 index 00000000..f8cb1301 --- /dev/null +++ b/src/ipa/raspberrypi/controller/af_status.h @@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: BSD-2-Clause */ +/* + * Copyright (C) 2022, Raspberry Pi (Trading) Limited + * Copyright (C) 2022, Ideas On Board + * + * af_status.h - autofocus measurement status + */ +#pragma once + +#include + +/* + * The focus algorithm should post the following structure into the image's + * "af.status" metadata. + */ + +struct AfStatus { + uint32_t lensPosition; + uint32_t state; + libcamera::Rectangle windows; +}; diff --git a/src/ipa/raspberrypi/controller/iob/af.cpp b/src/ipa/raspberrypi/controller/iob/af.cpp new file mode 100644 index 00000000..0100bbaf --- /dev/null +++ b/src/ipa/raspberrypi/controller/iob/af.cpp @@ -0,0 +1,212 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Red Hat + * Copyright (C) 2022, Ideas On Board + * + * af.cpp - automatic contrast-based focus algorithm + */ + +#include "af.h" + +#include +#include + +#include + +using namespace RPiController; +using namespace libcamera; + +LOG_DEFINE_CATEGORY(IoBAf) + +#define NAME "iob.af" + +/* + * Maximum focus steps of the VCM control + * \todo should be obtained from the VCM driver + */ +static constexpr uint32_t kMaxFocusSteps = 1023; + +/* Minimum focus step for searching appropriate focus */ +static constexpr uint32_t kCoarseSearchStep = 30; +static constexpr uint32_t kFineSearchStep = 1; + +/* Max ratio of variance change, 0.0 < kMaxChange < 1.0 */ +static constexpr double kMaxChange = 0.5; + +/* Fine scan range 0 < kFineRange < 1 */ +static constexpr double kFineRange = 0.05; + +Af::Af(Controller *controller) + : AfAlgorithm(controller), focus_(0), bestFocus_(0), + currentContrast_(0.0), previousContrast_(0.0), maxContrast_(0.0), + maxStep_(0), coarseCompleted_(false), fineCompleted_(false), + mode_(0) +{ +} + +char const *Af::name() const +{ + return NAME; +} + +void Af::setMode([[maybe_unused]] const uint32_t &mode) +{ + mode_ = mode; +} + +void Af::trigger() +{ +} + +void Af::cancel() +{ +} + +void Af::setWindows([[maybe_unused]] const libcamera::Rectangle &afWindows) +{ +} + +void Af::setRange([[maybe_unused]] const uint32_t &range) +{ +} + +void Af::setSpeed([[maybe_unused]] const uint32_t &speed) +{ +} + +void Af::initialise() +{ + status_.lensPosition = 0.0; + maxContrast_ = 0.0; + status_.state = 1; +} + +void Af::prepare(Metadata *imageMetadata) +{ + imageMetadata->set("af.status", status_); +} + +void Af::afCoarseScan() +{ + if (coarseCompleted_) + return; + + if (afScan(kCoarseSearchStep)) { + coarseCompleted_ = true; + maxContrast_ = 0; + focus_ = status_.lensPosition - (status_.lensPosition * kFineRange); + status_.lensPosition = focus_; + previousContrast_ = 0; + maxStep_ = std::clamp(focus_ + static_cast((focus_ * kFineRange)), + 0U, kMaxFocusSteps); + } +} + +void Af::afFineScan() +{ + if (!coarseCompleted_) + return; + + if (afScan(kFineSearchStep)) { + LOG(IoBAf, Debug) << "AF found the best focus position !"; + status_.state = 2; + fineCompleted_ = true; + } +} + +bool Af::afScan(uint32_t minSteps) +{ + if (focus_ > maxStep_) { + /* If the max step is reached, move lens to the position. */ + status_.lensPosition = bestFocus_; + return true; + } else { + /* + * Find the maximum of the variance by estimating its + * derivative. If the direction changes, it means we have passed + * a maximum one step before. + */ + if ((currentContrast_ - maxContrast_) >= -(maxContrast_ * 0.1)) { + /* + * Positive and zero derivative: + * The variance is still increasing. The focus could be + * increased for the next comparison. Also, the max + * variance and previous focus value are updated. + */ + bestFocus_ = focus_; + focus_ += minSteps; + maxContrast_ = currentContrast_; + status_.lensPosition = focus_; + } else { + /* + * Negative derivative: + * The variance starts to decrease which means the maximum + * variance is found. Set focus step to previous good one + * then return immediately. + */ + status_.lensPosition = bestFocus_; + return true; + } + } + + previousContrast_ = currentContrast_; + LOG(IoBAf, Debug) << " Previous step is " + << bestFocus_ + << " Current step is " + << focus_; + return false; +} + +void Af::afReset() +{ + LOG(IoBAf, Debug) << "Reset AF parameters"; + status_.lensPosition = 0; + focus_ = 0; + status_.state = 0; + previousContrast_ = 0.0; + coarseCompleted_ = false; + fineCompleted_ = false; + maxStep_ = kMaxFocusSteps; + maxContrast_ = 0.0; +} + +bool Af::afIsOutOfFocus() +{ + const uint32_t diff_var = std::abs(currentContrast_ - + maxContrast_); + const double var_ratio = diff_var / maxContrast_; + LOG(IoBAf, Debug) << "Variance change rate: " + << var_ratio + << " Current VCM step: " + << status_.lensPosition; + if (var_ratio > kMaxChange) + return true; + else + return false; +} + +void Af::process(StatisticsPtr &stats, [[maybe_unused]] Metadata *imageMetadata) +{ + unsigned int i; + currentContrast_ = 0.0; + + /* Use the second filter results only, and cache those. */ + for (i = 0; i < FOCUS_REGIONS; i++) + currentContrast_ += stats->focus_stats[i].contrast_val[1][1] + / stats->focus_stats[i].contrast_val_num[1][1]; + + if (status_.state != 2) { + afCoarseScan(); + afFineScan(); + } else { + if (afIsOutOfFocus()) + afReset(); + } +} + +/* Register algorithm with the system. */ +static Algorithm *Create(Controller *controller) +{ + return new Af(controller); +} +static RegisterAlgorithm reg(NAME, &Create); diff --git a/src/ipa/raspberrypi/controller/iob/af.h b/src/ipa/raspberrypi/controller/iob/af.h new file mode 100644 index 00000000..c7ed6109 --- /dev/null +++ b/src/ipa/raspberrypi/controller/iob/af.h @@ -0,0 +1,62 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Red Hat + * Copyright (C) 2022, Ideas On Board + * + * af.h - automatic contrast-based focus algorithm + */ +#pragma once + +#include + +#include "../af_algorithm.h" +#include "../af_status.h" +#include "../metadata.h" + +namespace RPiController { + +class Af : public AfAlgorithm +{ +public: + Af(Controller *controller); + char const *name() const override; + void initialise() override; + void prepare(Metadata *image_metadata) override; + void process(StatisticsPtr &stats, Metadata *image_metadata) override; + void setMode(const uint32_t &mode) override; + void trigger() override; + void cancel() override; + void setWindows(const libcamera::Rectangle &afWindows) override; + void setRange(const uint32_t &range) override; + void setSpeed(const uint32_t &speed) override; +private: + bool afNeedIgnoreFrame(); + void afCoarseScan(); + void afFineScan(); + bool afScan(uint32_t minSteps); + void afReset(); + bool afIsOutOfFocus(); + + AfStatus status_; + + /* VCM step configuration. It is the current setting of the VCM step. */ + uint32_t focus_; + /* The best VCM step. It is a local optimum VCM step during scanning. */ + uint32_t bestFocus_; + + /* Current AF statistic contrast. */ + double currentContrast_; + /* It is used to determine the derivative during scanning */ + double previousContrast_; + double maxContrast_; + /* The designated maximum range of focus scanning. */ + uint32_t maxStep_; + /* If the coarse scan completes, it is set to true. */ + bool coarseCompleted_; + /* If the fine scan completes, it is set to true. */ + bool fineCompleted_; + + uint32_t mode_; +}; + +} /* namespace RPiController */ diff --git a/src/ipa/raspberrypi/meson.build b/src/ipa/raspberrypi/meson.build index 517d815b..10ba9608 100644 --- a/src/ipa/raspberrypi/meson.build +++ b/src/ipa/raspberrypi/meson.build @@ -27,6 +27,7 @@ rpi_ipa_sources = files([ 'controller/controller.cpp', 'controller/histogram.cpp', 'controller/algorithm.cpp', + 'controller/iob/af.cpp', 'controller/rpi/alsc.cpp', 'controller/rpi/awb.cpp', 'controller/rpi/sharpen.cpp', From patchwork Thu Dec 1 14:51:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 17931 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E054ABE08B for ; Thu, 1 Dec 2022 14:52:45 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9C2D76333F; Thu, 1 Dec 2022 15:52:45 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1669906365; bh=sRr+TVmkOtBZrmMpxPYURb0t1dQR+ieK7x5VMfVjpD0=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=cDtZvMxuQaQMEIr91CrqlMV/7JY2HSuxIfRTTxpzsGzCOQ0qTrHAtEnAOe1y6M5ry umJJY01RaCnXSlEg7QbVpkG8YCC/Mz6syScv+wkae/BFDup29OfOn8mmLyijgf2Yu7 7lb9oou/9pAAdAewAmzdMYFvHpjavsIZnUyutZrgAWapzWuPfJx3iCE+Tqwbma5eqy 6txBCnFZGIm7bFr72+jgwTXF/YW09JuXmMk7uNm2kz7Yb/tM8q0HCPjmXniJi33h8H aujHaUKgxf6frJV1y+KbZENvtA0K+tHEjkmgVp3N2Mz0pOBKVCz8nfQFcTVf4nEgOA 0zmjz2c6oLVqQ== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CF67163336 for ; Thu, 1 Dec 2022 15:52:43 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ZRPkYmI6"; dkim-atps=neutral Received: from umang.jainideasonboard.com (unknown [210.186.188.117]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id BDCE06D0; Thu, 1 Dec 2022 15:52:42 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1669906363; bh=sRr+TVmkOtBZrmMpxPYURb0t1dQR+ieK7x5VMfVjpD0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ZRPkYmI6mgmhn5Zz4BOj9YpJkDYxaJmlH9MMdG17A9cFPQar/42cjyeJRWP1obO7/ 11MXaqSYKUZP3z24kfz6PstcJinzdYkHJkM7icwQhS7t15UcCogvfbcX5vLV7qpoh8 exqrjJGG5ha+ZraTI3xWVNR0qlI7KZm2GHxskSoI= To: libcamera-devel@lists.libcamera.org Date: Thu, 1 Dec 2022 22:51:30 +0800 Message-Id: <20221201145133.479794-3-umang.jain@ideasonboard.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221201145133.479794-1-umang.jain@ideasonboard.com> References: <20221201145133.479794-1-umang.jain@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 2/5] libcamera: raspberrypi: Control the lens from pipeline X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Umang Jain via libcamera-devel From: Umang Jain Reply-To: Umang Jain Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jean-Michel Hautbois The lens focus is controled by a VCM, which is linked to the sensor using the ancillary links. Pass the control to the config info structure and make it possible to update by the IPA. Signed-off-by: Jean-Michel Hautbois --- include/libcamera/ipa/raspberrypi.mojom | 1 + .../pipeline/raspberrypi/raspberrypi.cpp | 17 +++++++++++++++++ 2 files changed, 18 insertions(+) diff --git a/include/libcamera/ipa/raspberrypi.mojom b/include/libcamera/ipa/raspberrypi.mojom index d53644fe..e01f0b2b 100644 --- a/include/libcamera/ipa/raspberrypi.mojom +++ b/include/libcamera/ipa/raspberrypi.mojom @@ -130,4 +130,5 @@ interface IPARPiEventInterface { embeddedComplete(uint32 bufferId); setIspControls(libcamera.ControlList controls); setDelayedControls(libcamera.ControlList controls); + setLensControls(libcamera.ControlList controls); }; diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 0e0b7194..911d5b55 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -32,6 +32,7 @@ #include "libcamera/internal/bayer_format.h" #include "libcamera/internal/camera.h" +#include "libcamera/internal/camera_lens.h" #include "libcamera/internal/camera_sensor.h" #include "libcamera/internal/delayed_controls.h" #include "libcamera/internal/device_enumerator.h" @@ -209,6 +210,7 @@ public: void setDelayedControls(const ControlList &controls); void setSensorControls(ControlList &controls); void unicamTimeout(); + void setLensControls(const ControlList &controls); /* bufferComplete signal handlers. */ void unicamBufferDequeue(FrameBuffer *buffer); @@ -1551,6 +1553,7 @@ int RPiCameraData::loadIPA(ipa::RPi::IPAInitResult *result) ipa_->embeddedComplete.connect(this, &RPiCameraData::embeddedComplete); ipa_->setIspControls.connect(this, &RPiCameraData::setIspControls); ipa_->setDelayedControls.connect(this, &RPiCameraData::setDelayedControls); + ipa_->setLensControls.connect(this, &RPiCameraData::setLensControls); /* * The configuration (tuning file) is made from the sensor name unless @@ -1591,6 +1594,10 @@ int RPiCameraData::configureIPA(const CameraConfiguration *config, ipa::RPi::IPA entityControls.emplace(0, sensor_->controls()); entityControls.emplace(1, isp_[Isp::Input].dev()->controls()); + CameraLens *lens = sensor_->focusLens(); + if (lens) + entityControls.emplace(2, lens->controls()); + /* Always send the user transform to the IPA. */ ipaConfig.transform = static_cast(config->transform); @@ -1807,6 +1814,16 @@ void RPiCameraData::setDelayedControls(const ControlList &controls) handleState(); } +void RPiCameraData::setLensControls(const ControlList &ctrls) +{ + CameraLens *lens = sensor_->focusLens(); + if (!lens) + return; + + /* \todo Should we keep track of the latest value applied ? */ + lens->setFocusPosition(ctrls.get(V4L2_CID_FOCUS_ABSOLUTE).get()); +} + void RPiCameraData::setSensorControls(ControlList &controls) { /* From patchwork Thu Dec 1 14:51:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 17932 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2E549BE08B for ; Thu, 1 Dec 2022 14:52:48 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 001366333F; Thu, 1 Dec 2022 15:52:47 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1669906368; bh=/fb3lBrt+3qWxFcKDt5/FgOcDcBWLu/Xq10njz/52Bw=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=jFzlvI/6Ln8/yufVYuG7v1qRT9bwngJwcUAT+XCcOzUx9lAWbSjgzEAESpY0nP1F8 B4k009BDPG72mNSS78ozuVfy/+aLKRsPhrtEY3/7/lHMkFPVqCLtUoba9IlyfqMZgA CPUY6b9e3UETEu7bdyeHS7xz3WenXUUdbnT1EUPPrqrBEGKpoFPOLpMLKE3Gnzuyow +/XOmLa/i0KuhyTqxa0CToYRPhBw7ZMwq15W07LH2jc6LyGKzd7TC7puCKHnSfUs9z xz4WZAfuifTsQyxmZsMc5sU4YXfwxp/+hByxw3SnFAANhfSREzeCr3aQ4/Sxnmgz2i +5RF4d+S1f+kA== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BB0A663345 for ; Thu, 1 Dec 2022 15:52:45 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="sBHAgOHA"; dkim-atps=neutral Received: from umang.jainideasonboard.com (unknown [210.186.188.117]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id B09FC6D0; Thu, 1 Dec 2022 15:52:44 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1669906365; bh=/fb3lBrt+3qWxFcKDt5/FgOcDcBWLu/Xq10njz/52Bw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sBHAgOHALLz4XhNo/aHX/dWbhswV9AyuSYKPkZBs2IfX/FmpUYVHcovzYNK1yhHmY FFQm/LZTVrw+hbfIZqhzn0qFq11sRVJRNX4Q4jqDQ7XnPfCRcsRiw2Ja7lCw0XkfNe Mq+YV1JosMsB+/HrbUQiYCX9ZofepO8bYys9PJiY= To: libcamera-devel@lists.libcamera.org Date: Thu, 1 Dec 2022 22:51:31 +0800 Message-Id: <20221201145133.479794-4-umang.jain@ideasonboard.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221201145133.479794-1-umang.jain@ideasonboard.com> References: <20221201145133.479794-1-umang.jain@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 3/5] ipa: raspberrypi: Control the lens position X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Umang Jain via libcamera-devel From: Umang Jain Reply-To: Umang Jain Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jean-Michel Hautbois Now that the ancillary links are configured, we can use the CameraLens class and control the VCM through the IPA. Signed-off-by: Jean-Michel Hautbois --- src/ipa/raspberrypi/raspberrypi.cpp | 30 ++++++++++++++++++++++++++++- 1 file changed, 29 insertions(+), 1 deletion(-) diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 4e10c57d..3d1ac8fc 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -30,6 +30,7 @@ #include "libcamera/internal/mapped_framebuffer.h" +#include "af_status.h" #include "agc_algorithm.h" #include "agc_status.h" #include "alsc_status.h" @@ -131,6 +132,7 @@ private: void setMode(const IPACameraSensorInfo &sensorInfo); bool validateSensorControls(); bool validateIspControls(); + bool validateLensControls(); void queueRequest(const ControlList &controls); void returnEmbeddedBuffer(unsigned int bufferId); void prepareISP(const ISPConfig &data); @@ -155,6 +157,7 @@ private: ControlInfoMap sensorCtrls_; ControlInfoMap ispCtrls_; + ControlInfoMap lensCtrls_; ControlList libcameraMetadata_; /* Camera sensor params. */ @@ -381,7 +384,7 @@ int IPARPi::configure(const IPACameraSensorInfo &sensorInfo, const IPAConfig &ipaConfig, ControlList *controls, IPAConfigResult *result) { - if (entityControls.size() != 2) { + if (entityControls.size() < 2) { LOG(IPARPI, Error) << "No ISP or sensor controls found."; return -1; } @@ -389,6 +392,14 @@ int IPARPi::configure(const IPACameraSensorInfo &sensorInfo, sensorCtrls_ = entityControls.at(0); ispCtrls_ = entityControls.at(1); + /* Lens may not be present, don't make it an hard assumption. */ + auto lensControl = entityControls.find(2); + if (lensControl != entityControls.end()) { + lensCtrls_ = lensControl->second; + if (!validateLensControls()) + LOG(IPARPI, Error) << "Lens control validation failed."; + } + if (!validateSensorControls()) { LOG(IPARPI, Error) << "Sensor control validation failed."; return -1; @@ -650,6 +661,23 @@ bool IPARPi::validateIspControls() return true; } +bool IPARPi::validateLensControls() +{ + static const uint32_t ctrls[] = { + V4L2_CID_FOCUS_ABSOLUTE, + }; + + for (auto c : ctrls) { + if (lensCtrls_.find(c) == lensCtrls_.end()) { + LOG(IPARPI, Error) << "Unable to find lens control " + << utils::hex(c); + return false; + } + } + + return true; +} + /* * Converting between enums (used in the libcamera API) and the names that * we use to identify different modes. Unfortunately, the conversion tables From patchwork Thu Dec 1 14:51:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 17933 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 82CB5BE08B for ; Thu, 1 Dec 2022 14:52:49 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 50F9063348; Thu, 1 Dec 2022 15:52:49 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1669906369; bh=aU2q0UOwCJY4ufftiSrikse4mSAYAiaPIuAJdUjOF4M=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=jE9V/tkTOf4HF+o7XTrK2cmwMP02vnydymkUVNFXH52lLy2V8IAZ06TQwy1+QzyRF u+WBmxTylSE8GHye/75D4WO+2nZ0XkugGhDPn88orUvAC4t+vwimkcoDV6MP1dGP5H vNs/7ppGpEYt+lnovrMgJ8KJT3U/L+v2oimUlqI/nBfKt28BE1W0/+4yE63jNVU1nu UsBpsAfr34DHENOi5zL1lsojjJqiK/PiXEgjndRcR6GUVRc2Tq3WzU6rxzR34bMUmt 5VKsZmOPmkwb0IhuM53gt0fjYUzBrmRky24DK1jZQZcJKhw8QMZKwSClr4cUMETG2w ta5uuZKRjHoPg== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DD5D863335 for ; Thu, 1 Dec 2022 15:52:47 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="cNAFj9TI"; dkim-atps=neutral Received: from umang.jainideasonboard.com (unknown [210.186.188.117]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id A41306D0; Thu, 1 Dec 2022 15:52:46 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1669906367; bh=aU2q0UOwCJY4ufftiSrikse4mSAYAiaPIuAJdUjOF4M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cNAFj9TIAOCBlblwlMMOETBIWLfxBTLdr6qmodkY8ZYml+RbHIvkMRYAzZrGc1P4T OfQft+ZyqVyQoe1zoPEWCFLDQ6e2ltkzeVi7rzAjmyCqLZHoGEvwi1PGVi+Tuq7BH5 bT4v0xrDzcQ2XGk4HjENw+WJwl9ewW5PSkpLhkpA= To: libcamera-devel@lists.libcamera.org Date: Thu, 1 Dec 2022 22:51:32 +0800 Message-Id: <20221201145133.479794-5-umang.jain@ideasonboard.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221201145133.479794-1-umang.jain@ideasonboard.com> References: <20221201145133.479794-1-umang.jain@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 4/5] ipa: raspberrypi: Send the AF status back X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Umang Jain via libcamera-devel From: Umang Jain Reply-To: Umang Jain Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jean-Michel Hautbois The user can read the status for each frame with the AfState control value. Populate the AF status into the metadata when it is available. Signed-off-by: Jean-Michel Hautbois Signed-off-by: Umang Jain --- src/ipa/raspberrypi/raspberrypi.cpp | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 3d1ac8fc..19669600 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -91,7 +91,8 @@ static const ControlInfoMap::Map ipaControls{ { &controls::ColourCorrectionMatrix, ControlInfo(-16.0f, 16.0f) }, { &controls::ScalerCrop, ControlInfo(Rectangle{}, Rectangle(65535, 65535, 65535, 65535), Rectangle{}) }, { &controls::FrameDurationLimits, ControlInfo(INT64_C(33333), INT64_C(120000)) }, - { &controls::draft::NoiseReductionMode, ControlInfo(controls::draft::NoiseReductionModeValues) } + { &controls::draft::NoiseReductionMode, ControlInfo(controls::draft::NoiseReductionModeValues) }, + { &controls::AfState, ControlInfo(controls::AfStateValues) }, }; LOG_DEFINE_CATEGORY(IPARPI) @@ -611,6 +612,10 @@ void IPARPi::reportMetadata() m[i] = ccmStatus->matrix[i]; libcameraMetadata_.set(controls::ColourCorrectionMatrix, m); } + + AfStatus *afStatus = rpiMetadata_.getLocked("af.status"); + if (afStatus) + libcameraMetadata_.set(controls::AfState, afStatus->state); } bool IPARPi::validateSensorControls() @@ -1170,6 +1175,14 @@ void IPARPi::processStats(unsigned int bufferId) setDelayedControls.emit(ctrls); } + + struct AfStatus afStatus; + if (rpiMetadata_.get("af.status", afStatus) == 0) { + ControlList lensCtrls(lensCtrls_); + lensCtrls.set(V4L2_CID_FOCUS_ABSOLUTE, + static_cast(afStatus.lensPosition)); + setLensControls.emit(lensCtrls); + } } void IPARPi::applyAWB(const struct AwbStatus *awbStatus, ControlList &ctrls) From patchwork Thu Dec 1 14:51:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 17934 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EAF24BE08B for ; Thu, 1 Dec 2022 14:52:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id ACD496333B; Thu, 1 Dec 2022 15:52:51 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1669906371; bh=DOkCaajna8TVtuBiKUoY5s04E2HxZKI5jwHTPraCCDk=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=TlD8YACAb609EQ2Qv32ZCVinPptryESkyVywzQ8Kva9Z/VxUnVotub5MpUaL0k14K zTin+m8mWJWl3vwzNI1nlGLdNI8yvVQeNOuijDrQTX68cE25cJjwirkniefXcYViJg R18zYk6Bxf4mR2NVdo0j5QmMVP2UWqG/VycqnPbVRHKdSKRZs9AgihI46bw0UBXXIx eBqvQvo8hfFehoZqAt1YvZUbOza58sMmyOziwbC+uE7DFozLa5gAibj+NwhFs1d/Ki t4C9c64xfw2WbSX5eOeaU9kDKfaBBLGXQf2H884RUp6zIYyCMqDVmDHXU8Wt9gZHAW PgMwQQ4a62auA== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id F24CE63336 for ; Thu, 1 Dec 2022 15:52:49 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="iydDQzMW"; dkim-atps=neutral Received: from umang.jainideasonboard.com (unknown [210.186.188.117]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E0F806D0; Thu, 1 Dec 2022 15:52:48 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1669906369; bh=DOkCaajna8TVtuBiKUoY5s04E2HxZKI5jwHTPraCCDk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=iydDQzMWizPYscVkNNPM4tqaVjcjzjdG1bICOnhkACutPFabC1LEwEFcY1iPi4Z2O Jvmi9cwfCAkwtkTAutxh3GgIm96SsYZDUZdi/FOzQ+1WWH7Rs95X1CtLfyt56OG4if QYVr+UvTzVWMglBQVKIn87UINIiz+2sXnhseI3BA= To: libcamera-devel@lists.libcamera.org Date: Thu, 1 Dec 2022 22:51:33 +0800 Message-Id: <20221201145133.479794-6-umang.jain@ideasonboard.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221201145133.479794-1-umang.jain@ideasonboard.com> References: <20221201145133.479794-1-umang.jain@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 5/5] ipa: raspberrypi: imx519: Instantiate auto-focus algorithm X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Umang Jain via libcamera-devel From: Umang Jain Reply-To: Umang Jain Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Instantiate the auto-focus algorithm "iob.af" for IMX519 sensor from the tuning data file. Signed-off-by: Umang Jain --- src/ipa/raspberrypi/data/imx519.json | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/src/ipa/raspberrypi/data/imx519.json b/src/ipa/raspberrypi/data/imx519.json index 8ccfd3a6..7921cde7 100644 --- a/src/ipa/raspberrypi/data/imx519.json +++ b/src/ipa/raspberrypi/data/imx519.json @@ -408,6 +408,9 @@ }, { "rpi.sharpen": { } + }, + { + "iob.af": { } } ] -} \ No newline at end of file +}