From patchwork Tue Nov 15 09:07:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 17793 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A93B0BE08B for ; Tue, 15 Nov 2022 09:08:04 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6939863088; Tue, 15 Nov 2022 10:08:04 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1668503284; bh=DByEoimg1lvh8ErW1sv4VviuI8HSuOZoLi0E4p+TQ7w=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=WmOolv35PZwvDTjRnS+lDSj/mW7UsJ6sD8PJF3A+dl85UbZ8BA738R0fT/q0xIQ7J iHbUmtEdIQaJxPAB0Eu0x6yQk0MrTwYh1ELYnMy4h/cQyLdIuk/WE9RIny+66HYhI2 pbPxrdVG9j8IqUhxYRugTfGuz5eUE9i13dr1N9I+nSW914BDobP5vj3miXibStapSy U1swO06bg7mkV2Uxewt3TatFUoSAaA2fo8GyoOkHe5i+hJRkQryujERQG/1gzAk1if SpydOD8GTtq+E8RLmkO9XOhRobT8GNqMOy8F+AApePKEStF7iQWY3j8c388hXxBd/z EjKHjUCdarzVA== Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E5C0263089 for ; Tue, 15 Nov 2022 10:08:01 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="tQsaDZRQ"; dkim-atps=neutral Received: by mail-wm1-x32f.google.com with SMTP id ay14-20020a05600c1e0e00b003cf6ab34b61so12613467wmb.2 for ; Tue, 15 Nov 2022 01:08:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=XUEQUArpIfN6duNUMuOAcBmxj99qc641z9vVt3gWhPQ=; b=tQsaDZRQkPLzjwh9PERIJnFmq3D6177SeqrAXeOna2R9L8Of5n7wvlPTJgKLzVFq4/ tfLJQD/mhkeLM+BoRGxtOEryOrhCqrQQ8HadZYuVk1gEeKa2gY2/n555HBgR3O7UHn9I +PjARn1qY4+Fl7cIuyI1GniV6Gk2EYpn+fS3Wucnk0tfQ+091sbu3dojykdvMgfX4qHM O0xq5zmZC0bRdB9ocLKY1IA230dmNN/eC0kM/atNJZN1h48Lrpt8vNDrhtv8THVuV/gr ynY8KEAZQPrMNxiI3FFWsarl67X/Uf2mr08Iv1Jo9cI0ZptCIblKbGb1HWZ9dkSrlZAd aK0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XUEQUArpIfN6duNUMuOAcBmxj99qc641z9vVt3gWhPQ=; b=hCILExgdQ6cQ+wtrHlbcFf9FtcHsYa9jd8zfAe+brDRLwpRWhzpUIpiepN2DtcnXY9 gufKgp0aZwbBLeAIxLdTu4uw3enRGt3lRC03vyB4gr8MDmIDJ8B3+xKxfY6/b7LV5DQB fcZ+dFbJkZ1Fvaj10PQFNuCHCqxXbPe2zWQc+vtaD/1uRF5yrs/6zYRUeTXsKow+Z5ls 2Pw8K1Ng2uYLdWbrtuh2cAihtfr+1tUYYlNmrE1fk7FnsjrhlrWj4GLCpP9DJup4RANX jOa7tuOgKRAdDOH5fQwehivH/mNX6Qu2ECGTj0OfYgOPMrr1FSH+7c5intk4e7wP2C84 43ZQ== X-Gm-Message-State: ANoB5pn2PCx77VL9BjUULicqkRvSrQd91ea6APqO/HtqsvS26pKvaK50 fUnPtBnGK+aK9SqK7AVFRY4YC/idSbcLdg== X-Google-Smtp-Source: AA0mqf7B0OB0FfOkzvZiW6WnLqhgpFqDCxcTSbc0/MVrj3L2mqS3xZLgyE0tuafzb1PslRmG9HaOJA== X-Received: by 2002:a7b:c010:0:b0:3cf:eaf5:77c6 with SMTP id c16-20020a7bc010000000b003cfeaf577c6mr677213wmb.56.1668503280934; Tue, 15 Nov 2022 01:08:00 -0800 (PST) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id p25-20020a05600c1d9900b003c64c186206sm15143696wms.16.2022.11.15.01.08.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Nov 2022 01:08:00 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Tue, 15 Nov 2022 09:07:48 +0000 Message-Id: <20221115090755.2921-2-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221115090755.2921-1-naush@raspberrypi.com> References: <20221115090755.2921-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v6 1/8] pipeline: raspberrypi: Fork DelayedControls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Fork the libcamera::DelayedControls implementation in the RPi:: namespace, with the intention of updating the API for Raspberry Pi specific features. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Reviewed-by: Kieran Bingham --- .../pipeline/raspberrypi/delayed_controls.cpp | 291 ++++++++++++++++++ .../pipeline/raspberrypi/delayed_controls.h | 84 +++++ .../pipeline/raspberrypi/meson.build | 1 + 3 files changed, 376 insertions(+) create mode 100644 src/libcamera/pipeline/raspberrypi/delayed_controls.cpp create mode 100644 src/libcamera/pipeline/raspberrypi/delayed_controls.h diff --git a/src/libcamera/pipeline/raspberrypi/delayed_controls.cpp b/src/libcamera/pipeline/raspberrypi/delayed_controls.cpp new file mode 100644 index 000000000000..867e3866cc46 --- /dev/null +++ b/src/libcamera/pipeline/raspberrypi/delayed_controls.cpp @@ -0,0 +1,291 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Raspberry Pi Ltd + * + * delayed_controls.cpp - Helper to deal with controls that take effect with a delay + * + * Note: This has been forked from the libcamera core implementation. + */ + +#include "delayed_controls.h" + +#include + +#include + +#include "libcamera/internal/v4l2_device.h" + +/** + * \file delayed_controls.h + * \brief Helper to deal with controls that take effect with a delay + */ + +namespace libcamera { + +LOG_DEFINE_CATEGORY(RPiDelayedControls) + +namespace RPi { + +/** + * \class DelayedControls + * \brief Helper to deal with controls that take effect with a delay + * + * Some sensor controls take effect with a delay as the sensor needs time to + * adjust, for example exposure and analog gain. This is a helper class to deal + * with such controls and the intended users are pipeline handlers. + * + * The idea is to extend the concept of the buffer depth of a pipeline the + * application needs to maintain to also cover controls. Just as with buffer + * depth if the application keeps the number of requests queued above the + * control depth the controls are guaranteed to take effect for the correct + * request. The control depth is determined by the control with the greatest + * delay. + */ + +/** + * \struct DelayedControls::ControlParams + * \brief Parameters associated with controls handled by the \a DelayedControls + * helper class + * + * \var ControlParams::delay + * \brief Frame delay from setting the control on a sensor device to when it is + * consumed during framing. + * + * \var ControlParams::priorityWrite + * \brief Flag to indicate that this control must be applied ahead of, and + * separately from the other controls. + * + * Typically set for the \a V4L2_CID_VBLANK control so that the device driver + * does not reject \a V4L2_CID_EXPOSURE control values that may be outside of + * the existing vertical blanking specified bounds, but are within the new + * blanking bounds. + */ + +/** + * \brief Construct a DelayedControls instance + * \param[in] device The V4L2 device the controls have to be applied to + * \param[in] controlParams Map of the numerical V4L2 control ids to their + * associated control parameters. + * + * The control parameters comprise of delays (in frames) and a priority write + * flag. If this flag is set, the relevant control is written separately from, + * and ahead of the rest of the batched controls. + * + * Only controls specified in \a controlParams are handled. If it's desired to + * mix delayed controls and controls that take effect immediately the immediate + * controls must be listed in the \a controlParams map with a delay value of 0. + */ +DelayedControls::DelayedControls(V4L2Device *device, + const std::unordered_map &controlParams) + : device_(device), maxDelay_(0) +{ + const ControlInfoMap &controls = device_->controls(); + + /* + * Create a map of control ids to delays for controls exposed by the + * device. + */ + for (auto const ¶m : controlParams) { + auto it = controls.find(param.first); + if (it == controls.end()) { + LOG(RPiDelayedControls, Error) + << "Delay request for control id " + << utils::hex(param.first) + << " but control is not exposed by device " + << device_->deviceNode(); + continue; + } + + const ControlId *id = it->first; + + controlParams_[id] = param.second; + + LOG(RPiDelayedControls, Debug) + << "Set a delay of " << controlParams_[id].delay + << " and priority write flag " << controlParams_[id].priorityWrite + << " for " << id->name(); + + maxDelay_ = std::max(maxDelay_, controlParams_[id].delay); + } + + reset(); +} + +/** + * \brief Reset state machine + * + * Resets the state machine to a starting position based on control values + * retrieved from the device. + */ +void DelayedControls::reset() +{ + queueCount_ = 1; + writeCount_ = 0; + + /* Retrieve control as reported by the device. */ + std::vector ids; + for (auto const ¶m : controlParams_) + ids.push_back(param.first->id()); + + ControlList controls = device_->getControls(ids); + + /* Seed the control queue with the controls reported by the device. */ + values_.clear(); + for (const auto &ctrl : controls) { + const ControlId *id = device_->controls().idmap().at(ctrl.first); + /* + * Do not mark this control value as updated, it does not need + * to be written to to device on startup. + */ + values_[id][0] = Info(ctrl.second, false); + } +} + +/** + * \brief Push a set of controls on the queue + * \param[in] controls List of controls to add to the device queue + * + * Push a set of controls to the control queue. This increases the control queue + * depth by one. + * + * \returns true if \a controls are accepted, or false otherwise + */ +bool DelayedControls::push(const ControlList &controls) +{ + /* Copy state from previous frame. */ + for (auto &ctrl : values_) { + Info &info = ctrl.second[queueCount_]; + info = values_[ctrl.first][queueCount_ - 1]; + info.updated = false; + } + + /* Update with new controls. */ + const ControlIdMap &idmap = device_->controls().idmap(); + for (const auto &control : controls) { + const auto &it = idmap.find(control.first); + if (it == idmap.end()) { + LOG(RPiDelayedControls, Warning) + << "Unknown control " << control.first; + return false; + } + + const ControlId *id = it->second; + + if (controlParams_.find(id) == controlParams_.end()) + return false; + + Info &info = values_[id][queueCount_]; + + info = Info(control.second); + + LOG(RPiDelayedControls, Debug) + << "Queuing " << id->name() + << " to " << info.toString() + << " at index " << queueCount_; + } + + queueCount_++; + + return true; +} + +/** + * \brief Read back controls in effect at a sequence number + * \param[in] sequence The sequence number to get controls for + * + * Read back what controls where in effect at a specific sequence number. The + * history is a ring buffer of 16 entries where new and old values coexist. It's + * the callers responsibility to not read too old sequence numbers that have been + * pushed out of the history. + * + * Historic values are evicted by pushing new values onto the queue using + * push(). The max history from the current sequence number that yields valid + * values are thus 16 minus number of controls pushed. + * + * \return The controls at \a sequence number + */ +ControlList DelayedControls::get(uint32_t sequence) +{ + unsigned int index = std::max(0, sequence - maxDelay_); + + ControlList out(device_->controls()); + for (const auto &ctrl : values_) { + const ControlId *id = ctrl.first; + const Info &info = ctrl.second[index]; + + out.set(id->id(), info); + + LOG(RPiDelayedControls, Debug) + << "Reading " << id->name() + << " to " << info.toString() + << " at index " << index; + } + + return out; +} + +/** + * \brief Inform DelayedControls of the start of a new frame + * \param[in] sequence Sequence number of the frame that started + * + * Inform the state machine that a new frame has started and of its sequence + * number. Any user of these helpers is responsible to inform the helper about + * the start of any frame. This can be connected with ease to the start of a + * exposure (SOE) V4L2 event. + */ +void DelayedControls::applyControls(uint32_t sequence) +{ + LOG(RPiDelayedControls, Debug) << "frame " << sequence << " started"; + + /* + * Create control list peeking ahead in the value queue to ensure + * values are set in time to satisfy the sensor delay. + */ + ControlList out(device_->controls()); + for (auto &ctrl : values_) { + const ControlId *id = ctrl.first; + unsigned int delayDiff = maxDelay_ - controlParams_[id].delay; + unsigned int index = std::max(0, writeCount_ - delayDiff); + Info &info = ctrl.second[index]; + + if (info.updated) { + if (controlParams_[id].priorityWrite) { + /* + * This control must be written now, it could + * affect validity of the other controls. + */ + ControlList priority(device_->controls()); + priority.set(id->id(), info); + device_->setControls(&priority); + } else { + /* + * Batch up the list of controls and write them + * at the end of the function. + */ + out.set(id->id(), info); + } + + LOG(RPiDelayedControls, Debug) + << "Setting " << id->name() + << " to " << info.toString() + << " at index " << index; + + /* Done with this update, so mark as completed. */ + info.updated = false; + } + } + + writeCount_ = sequence + 1; + + while (writeCount_ > queueCount_) { + LOG(RPiDelayedControls, Debug) + << "Queue is empty, auto queue no-op."; + push({}); + } + + device_->setControls(&out); +} + +} /* namespace RPi */ + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/raspberrypi/delayed_controls.h b/src/libcamera/pipeline/raspberrypi/delayed_controls.h new file mode 100644 index 000000000000..f7f246482968 --- /dev/null +++ b/src/libcamera/pipeline/raspberrypi/delayed_controls.h @@ -0,0 +1,84 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Raspberry Pi Ltd + * + * delayed_controls.h - Helper to deal with controls that take effect with a delay + * + * Note: This has been forked from the libcamera core implementation. + */ + +#pragma once + +#include +#include + +#include + +namespace libcamera { + +class V4L2Device; + +namespace RPi { + +class DelayedControls +{ +public: + struct ControlParams { + unsigned int delay; + bool priorityWrite; + }; + + DelayedControls(V4L2Device *device, + const std::unordered_map &controlParams); + + void reset(); + + bool push(const ControlList &controls); + ControlList get(uint32_t sequence); + + void applyControls(uint32_t sequence); + +private: + class Info : public ControlValue + { + public: + Info() + : updated(false) + { + } + + Info(const ControlValue &v, bool updated_ = true) + : ControlValue(v), updated(updated_) + { + } + + bool updated; + }; + + static constexpr int listSize = 16; + class ControlRingBuffer : public std::array + { + public: + Info &operator[](unsigned int index) + { + return std::array::operator[](index % listSize); + } + + const Info &operator[](unsigned int index) const + { + return std::array::operator[](index % listSize); + } + }; + + V4L2Device *device_; + std::unordered_map controlParams_; + unsigned int maxDelay_; + + uint32_t queueCount_; + uint32_t writeCount_; + std::unordered_map values_; +}; + +} /* namespace RPi */ + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/raspberrypi/meson.build b/src/libcamera/pipeline/raspberrypi/meson.build index f1a2f5ee72c2..6064a3f00122 100644 --- a/src/libcamera/pipeline/raspberrypi/meson.build +++ b/src/libcamera/pipeline/raspberrypi/meson.build @@ -1,6 +1,7 @@ # SPDX-License-Identifier: CC0-1.0 libcamera_sources += files([ + 'delayed_controls.cpp', 'dma_heaps.cpp', 'raspberrypi.cpp', 'rpi_stream.cpp', From patchwork Tue Nov 15 09:07:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 17794 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id F1E3EBE08B for ; Tue, 15 Nov 2022 09:08:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CD43663095; Tue, 15 Nov 2022 10:08:04 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1668503284; bh=aMyrgkzUOkwUYQTup5cASIlz0rdgebhBWZ2ofkFbw4g=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=SYoz3pKuX9mg0TVvuO/ZP7RnYyYJYVBivJZjeSvcQ+8tXBPT96LJXBUwzLR52Lef5 xuKxRJ6Ab50LFq8wnRutSN6OLbFwtvnFZ8+tcn79H1OrB3iX5F0X7JW+153+HrX4Ak RIpv6HYr+ca/zCNKFOhHjG/iNuSGB7fBkw8+J8aeIPG9ZlaQOtTHtI7arCpxQ0xmO4 OSMvYN00dagUA43SFIhAZl46LAkG9AXzCVK0o/12ua4cb1AK+gHlx7oto13bvpHj8k bveG6VL+WOIyy/BbPDlr4UWqyETpjTLkNnAyHQpPdiNuBQPmAUaYhTRYNrhpfrZsz5 1pg+XKz4j7ubw== Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1844063079 for ; Tue, 15 Nov 2022 10:08:02 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="qBnZbjlY"; dkim-atps=neutral Received: by mail-wr1-x42c.google.com with SMTP id j15so23120680wrq.3 for ; Tue, 15 Nov 2022 01:08:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9/S6c5LA/uGjh5irR7dj+to8pxwOG/wP47q4LrRMbC8=; b=qBnZbjlYrrd6D4FBjccvVqbbFzyFbQ+93kw6V4qQTeD07KsXSFqDTot5RxsTDesoL0 sfDd8MD/I64C9xSiQkUDzX0Ht6VI7qZ6XNuUEucSUIGs8BBXca1cgfizhLQKebYioFtC YKIzaryIZQvcrKQw1CUMU+c5eYWUKgyVNo03UA7HcXThc5WMdlmq/K2IH3vtoJ/D2RLT Ty6KmnjxH1xF+u5rBHjkdsCOVAncLlimsLhcqDENdyRSAsG1zA38xeyHdGRZ6F6C9g+i Fl4QhYJ3KfatL0bWO3lb77ITCCgoEJ3YHKmqOLCFAVRO4Xd+Tdm8/usqT943rJu41rDY JLMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9/S6c5LA/uGjh5irR7dj+to8pxwOG/wP47q4LrRMbC8=; b=1Vt+wAJe2B3ds9aj6fy8vhQgMzLC0PCpKsogkAGK+/fS9+dvxN21ZRl18og5Y/X3ex EmdI35rSrQxHnkSpZWp8QpfgARLU8oBRqHR5INkyBJ7CG/E5Z+rwKizgAn8VJxMt9C5A RThyFqcw0qqDED1f6UhY5Gs+cMgRmA/Prbs5LC/YeX6wkCNWp1uTQAfZ8JL3qneHlfvL 6TS04ZZCDxVWWkZbZovLfz18umscn5vUfWL2wnjVBEOJr0uwsr6jjKX5DEsq3U85ixLp KTlL9IBWqNOvQtnVnG0/KriR63lZHPsBdngGMEomXpOxBlxgf/33cXL4z9L8huCxIEIC ZE2w== X-Gm-Message-State: ANoB5pm/CJ2TyDzEcv3P3DL0Zjx1s2A8URufHu4onA02Fqeg5U8tmcgq P2vEu9toKTie8NfJ81HypelNuYgoJ/7d2Q== X-Google-Smtp-Source: AA0mqf7HTr44cTeoudvCbfO6JwyrGnuLwjDy4nZcKPQkCZab6SrpM5IQlvIXMZB8LmM7GYbstLlKgg== X-Received: by 2002:a05:6000:12d0:b0:22e:2c8b:993d with SMTP id l16-20020a05600012d000b0022e2c8b993dmr9590821wrx.523.1668503281582; Tue, 15 Nov 2022 01:08:01 -0800 (PST) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id p25-20020a05600c1d9900b003c64c186206sm15143696wms.16.2022.11.15.01.08.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Nov 2022 01:08:01 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Tue, 15 Nov 2022 09:07:49 +0000 Message-Id: <20221115090755.2921-3-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221115090755.2921-1-naush@raspberrypi.com> References: <20221115090755.2921-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v6 2/8] pipeline: raspberrypi: Switch to RPi::DelayedControls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Switch the Raspberry Pi pipeline handler to use the DelayedControls implementation in the RPi:: namespace. This will allow us to use Raspberry Pi specific API changes in future commits. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman --- src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 343f8cb2c7ed..f3be4ee3b730 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -33,7 +33,6 @@ #include "libcamera/internal/bayer_format.h" #include "libcamera/internal/camera.h" #include "libcamera/internal/camera_sensor.h" -#include "libcamera/internal/delayed_controls.h" #include "libcamera/internal/device_enumerator.h" #include "libcamera/internal/framebuffer.h" #include "libcamera/internal/ipa_manager.h" @@ -41,6 +40,7 @@ #include "libcamera/internal/pipeline_handler.h" #include "libcamera/internal/v4l2_videodevice.h" +#include "delayed_controls.h" #include "dma_heaps.h" #include "rpi_stream.h" @@ -243,7 +243,7 @@ public: RPi::DmaHeap dmaHeap_; SharedFD lsTable_; - std::unique_ptr delayedCtrls_; + std::unique_ptr delayedCtrls_; bool sensorMetadata_; /* @@ -1294,13 +1294,13 @@ int PipelineHandlerRPi::registerCamera(MediaDevice *unicam, MediaDevice *isp, Me * Setup our delayed control writer with the sensor default * gain and exposure delays. Mark VBLANK for priority write. */ - std::unordered_map params = { + std::unordered_map params = { { V4L2_CID_ANALOGUE_GAIN, { result.sensorConfig.gainDelay, false } }, { V4L2_CID_EXPOSURE, { result.sensorConfig.exposureDelay, false } }, { V4L2_CID_HBLANK, { result.sensorConfig.hblankDelay, false } }, { V4L2_CID_VBLANK, { result.sensorConfig.vblankDelay, true } } }; - data->delayedCtrls_ = std::make_unique(data->sensor_->device(), params); + data->delayedCtrls_ = std::make_unique(data->sensor_->device(), params); data->sensorMetadata_ = result.sensorConfig.sensorMetadata; /* Register initial controls that the Raspberry Pi IPA can handle. */ From patchwork Tue Nov 15 09:07:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 17795 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3214CBE08B for ; Tue, 15 Nov 2022 09:08:07 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 670D06308E; Tue, 15 Nov 2022 10:08:05 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1668503285; bh=MfJGzVF0xEaHzNXFhd8igfaQOfe1CDgC3OPO2CFqVWs=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=Lo31s/RG4flgOT5WHtas6VgycKeI6fWLhVcUy/5Kuygj7hBwvYIwOJnJU3y7ott/Z pj1+azQmVIjC8FAiVDKLVo5DqudyjdLjxPRO8OmDwIU+bfHm8HgDv/SA4jdTNyZODw ytubN1efmaLcRPomsaqqzR5zDo2fLsRQ73QozWEGTxJYOi04nUmh0dJo0TU70rEGHk Uj5RYX/iQ2I34Fa40iiYTOz2Eq9Ok8wMGfbLt+Eb/Cfr1zCrdu5hGA2WAyCZeCL6ev G/14CxKTFa6HFGhFF4mrIWURJlH7UIJtoCm8x1PYL0bdl0Vu3HkeQ5ipeV3tNklUgy SuzaAeJi37ASQ== Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 11AC06307F for ; Tue, 15 Nov 2022 10:08:03 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="VftRBPRV"; dkim-atps=neutral Received: by mail-wr1-x431.google.com with SMTP id a14so23107936wru.5 for ; Tue, 15 Nov 2022 01:08:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9FLYHppxCNcOu0dK4WMQh2Nl/lRPe5xDxdYMrwqkS1s=; b=VftRBPRVTZxOp1ZTHk9r+v6tFnA3mJHco6eCeDunopuvTP/OVCR7N42QH/2hiqjawa h+kLdtMwwxMbreob8aHBF9fguSngO5MiV7JcyieOMLbwECw9NuqOBI7mSiYXhQByzTrF n4g2nbhLbPTyMqxotTgNfDqpIMzZ+xxnDky/2zkPQTpk4D3jex9Wn4O1/KLEz/a9X1Kq BfM77F6wSXqp+hnzNhn/N6vMc2bzDnCn47rXF3wMdjjjtILgC1ymienymjfdCZs+3gcw ObA/AffTbinQsyruX1o3keymzNU3mMG8WqUY0utkn9Aws/hllrkT46BCPbqZJJb3fur2 sxQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9FLYHppxCNcOu0dK4WMQh2Nl/lRPe5xDxdYMrwqkS1s=; b=WmAqAv50S5DBmE438Swivz0+60mZcRoUp6HK2kcUwsAv9zoaeMtnSZFAM7yiP5XCKk zSQZ/7vbJik2Rjfny+6mBoglOdL+scG7KiworzHKVPXWFET5KyYvcvZsG2ueTcmC0Xib GrV5xVv0PA2lBki+lLsN/q3+Df9M2t7y0mQ9AsYFE1DKAI6CCn2m95uiXAwHNjiqDA16 QhDiFeG5nZtiK9Ya01vksS4EeEaS6a4JCFW4zb6dnd4tNrBic9BATfWjrzOtK/QcBsd4 Yej+DhnMkdgL2mryHA4vkZW3PnmhqXI+YD9LZ3EyUSLe9hpBFsXSkE4Co2sWomwYhXge 4n5g== X-Gm-Message-State: ANoB5pm6zqZ8qmlWFQEtbEz7YmNwzkOTn5hEmLIeUmJGkES57SBqOcR6 x5tEp5PbK+RgrsXf+RtnWanoFyH/pR7U2g== X-Google-Smtp-Source: AA0mqf7X+W3C1SlspDynAKcMpXTJbSNGA3lGtw0bvq2fMGRq/nM4PEpmEFjMI25RuyCHHr1LKBxGCQ== X-Received: by 2002:adf:f502:0:b0:238:c1d5:3293 with SMTP id q2-20020adff502000000b00238c1d53293mr9733918wro.446.1668503282525; Tue, 15 Nov 2022 01:08:02 -0800 (PST) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id p25-20020a05600c1d9900b003c64c186206sm15143696wms.16.2022.11.15.01.08.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Nov 2022 01:08:01 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Tue, 15 Nov 2022 09:07:50 +0000 Message-Id: <20221115090755.2921-4-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221115090755.2921-1-naush@raspberrypi.com> References: <20221115090755.2921-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v6 3/8] pipeline: raspberrypi: delayed_controls: Template the ControlRingBuffer class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Convert ControlRingBuffer to a templated class to allow arbitrary ring buffer array types to be defined. Rename ControlRingBuffer to RingBuffer to indicate this. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman --- .../pipeline/raspberrypi/delayed_controls.h | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/delayed_controls.h b/src/libcamera/pipeline/raspberrypi/delayed_controls.h index f7f246482968..238b86ab6cb4 100644 --- a/src/libcamera/pipeline/raspberrypi/delayed_controls.h +++ b/src/libcamera/pipeline/raspberrypi/delayed_controls.h @@ -56,17 +56,18 @@ private: }; static constexpr int listSize = 16; - class ControlRingBuffer : public std::array + template + class RingBuffer : public std::array { public: - Info &operator[](unsigned int index) + T &operator[](unsigned int index) { - return std::array::operator[](index % listSize); + return std::array::operator[](index % listSize); } - const Info &operator[](unsigned int index) const + const T &operator[](unsigned int index) const { - return std::array::operator[](index % listSize); + return std::array::operator[](index % listSize); } }; @@ -76,7 +77,7 @@ private: uint32_t queueCount_; uint32_t writeCount_; - std::unordered_map values_; + std::unordered_map> values_; }; } /* namespace RPi */ From patchwork Tue Nov 15 09:07:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 17796 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id DD54EC3285 for ; Tue, 15 Nov 2022 09:08:07 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BB0AE63079; Tue, 15 Nov 2022 10:08:06 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1668503286; bh=J6cFEdUNUolwDllVMsFx8kpijHwSyHWPGi/UH1SmfYc=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=VdyriAh3jmQ+xXDmyTW3AgLE9BKC3E/d8baelRuGPaZzWH8dzJVJW+IZJ55cp0nhA IK9Uaumh2OPOEgSU2RB5lZKmnHXd2vZ2z0U+9x/BaVtY2bHuNk37pbEc1VwW9U2lmX jvfjngL+17Qq9TdwrRIk4/6+88hm8nT73vO5t88NKRRBPGIvFfNBlujGn6HVImGAYn mxMdm93pGpmBpEW/mN5kytdOWCdGL61+LwtVVJQlrFqLgrf6OH56Ji7wyBGDA155ew QkD51c/0UoUzPhyxMwr751K/3icDgJYZmuX+ZI37/J/toEPaz6ozuBk3hPxueeCVMJ uwCh7CEItlO2Q== Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C69726307F for ; Tue, 15 Nov 2022 10:08:03 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="p1fT+sUh"; dkim-atps=neutral Received: by mail-wr1-x42c.google.com with SMTP id j15so23120810wrq.3 for ; Tue, 15 Nov 2022 01:08:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=g4e6aCLgztXw7aLK8j8CpH1wz8V389BXNzUdmUAx/Uc=; b=p1fT+sUheMGfIcnAhb/GLseNdTIb/BAU3YFN4b3+O8vhHXl1+P0O6+7YbzFyi6/lW3 e8rrTnwfhROiWCZdDjfE9eyV6qfQ3rbCVouli5mkVaVSAZ6dAFfztW+B/wBnKZioVtV3 BbEoPJCNkRAF/cBQjMC2OlilhiU3u7pem4VcBwBHr7tKh5GZuvSwmSQQGSBCVMq2Ggi2 qr78HJMQgSmSLTWSdqUw04qYm7HwIQaLVpZ8qrv/DqSEMRhdDLc8IvlEyLf1Uc3GDI+O TXqxdjTWQ2jpVThAIWRWMpLw6dOifAvrA6/QW5aFOGQI68W/HxQRy7S2IIvtVY/Aynsi Gn1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=g4e6aCLgztXw7aLK8j8CpH1wz8V389BXNzUdmUAx/Uc=; b=crSqkokpVF5kv983a8uPyAFeIrvM1Sx9Kl2Aydfw2deDkTJOwhc+VZKEucm0LlvklC o8BpH9gz+WM1N66Jxty/KFqavdCii3i7GCKuBLw+GtmZLTqu4hg3hPAN58y7CPE57JyF gGVcfQk8lUtfiANoxxlezdz5o20jPBUhS1ZTdBO0gq+XhQEuFZK01Hdz54SGBjVq92RD v0fxs6NBNQIpjwEx5QBgto5X0KmRBRkiFjlU2xdFIim0X6Y73yaSbtPYPKNucLEyTLDW jsSqWJbA1VVGPrAn79LQ4yKsXzj8B4avUVgu4otdhxiLk3MQtlZJw6IzKES4np+TKwrd R6Eg== X-Gm-Message-State: ANoB5pnVJa/4KhFsewhBUVjfp5zOnrOIcD1lrnTZJdsuUT8QhbA76qlk egHStDxlQs/2G+16jN0eOByRGbgNlc4Rhg== X-Google-Smtp-Source: AA0mqf6ZYqbkIJk5ZZrmR2pjvprAbUPtQ8LKypPIy98Ri6o5erjlK0iSv9jX6F+Ej5FilvqyoGo3QQ== X-Received: by 2002:adf:ff85:0:b0:235:c5de:75c3 with SMTP id j5-20020adfff85000000b00235c5de75c3mr9782712wrr.285.1668503283260; Tue, 15 Nov 2022 01:08:03 -0800 (PST) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id p25-20020a05600c1d9900b003c64c186206sm15143696wms.16.2022.11.15.01.08.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Nov 2022 01:08:02 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Tue, 15 Nov 2022 09:07:51 +0000 Message-Id: <20221115090755.2921-5-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221115090755.2921-1-naush@raspberrypi.com> References: <20221115090755.2921-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v6 4/8] pipeline: raspberrypi: delayed_controls: Add user cookie to DelayedControls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Allow the caller to provide a cookie value to DelayedControls::reset and DelayedControls::push. This cookie value is returned from DelayedControls::get for the frame that has the control values applied to it. The cookie value is useful in tracking when a set of controls has been applied to a frame. In a subsequent commit, it will be used by the Raspberry Pi IPA to track the IPA context used when setting the control values. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman --- .../pipeline/raspberrypi/delayed_controls.cpp | 14 ++++++++------ .../pipeline/raspberrypi/delayed_controls.h | 8 +++++--- src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 6 +++--- 3 files changed, 16 insertions(+), 12 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/delayed_controls.cpp b/src/libcamera/pipeline/raspberrypi/delayed_controls.cpp index 867e3866cc46..3db92e7d24fb 100644 --- a/src/libcamera/pipeline/raspberrypi/delayed_controls.cpp +++ b/src/libcamera/pipeline/raspberrypi/delayed_controls.cpp @@ -108,7 +108,7 @@ DelayedControls::DelayedControls(V4L2Device *device, maxDelay_ = std::max(maxDelay_, controlParams_[id].delay); } - reset(); + reset(0); } /** @@ -117,10 +117,11 @@ DelayedControls::DelayedControls(V4L2Device *device, * Resets the state machine to a starting position based on control values * retrieved from the device. */ -void DelayedControls::reset() +void DelayedControls::reset(unsigned int cookie) { queueCount_ = 1; writeCount_ = 0; + cookies_[0] = cookie; /* Retrieve control as reported by the device. */ std::vector ids; @@ -150,7 +151,7 @@ void DelayedControls::reset() * * \returns true if \a controls are accepted, or false otherwise */ -bool DelayedControls::push(const ControlList &controls) +bool DelayedControls::push(const ControlList &controls, const unsigned int cookie) { /* Copy state from previous frame. */ for (auto &ctrl : values_) { @@ -184,6 +185,7 @@ bool DelayedControls::push(const ControlList &controls) << " at index " << queueCount_; } + cookies_[queueCount_] = cookie; queueCount_++; return true; @@ -204,7 +206,7 @@ bool DelayedControls::push(const ControlList &controls) * * \return The controls at \a sequence number */ -ControlList DelayedControls::get(uint32_t sequence) +std::pair DelayedControls::get(uint32_t sequence) { unsigned int index = std::max(0, sequence - maxDelay_); @@ -221,7 +223,7 @@ ControlList DelayedControls::get(uint32_t sequence) << " at index " << index; } - return out; + return { out, cookies_[index] }; } /** @@ -280,7 +282,7 @@ void DelayedControls::applyControls(uint32_t sequence) while (writeCount_ > queueCount_) { LOG(RPiDelayedControls, Debug) << "Queue is empty, auto queue no-op."; - push({}); + push({}, cookies_[queueCount_ - 1]); } device_->setControls(&out); diff --git a/src/libcamera/pipeline/raspberrypi/delayed_controls.h b/src/libcamera/pipeline/raspberrypi/delayed_controls.h index 238b86ab6cb4..61f755f0fddd 100644 --- a/src/libcamera/pipeline/raspberrypi/delayed_controls.h +++ b/src/libcamera/pipeline/raspberrypi/delayed_controls.h @@ -11,6 +11,7 @@ #include #include +#include #include @@ -31,10 +32,10 @@ public: DelayedControls(V4L2Device *device, const std::unordered_map &controlParams); - void reset(); + void reset(unsigned int cookie); - bool push(const ControlList &controls); - ControlList get(uint32_t sequence); + bool push(const ControlList &controls, unsigned int cookie); + std::pair get(uint32_t sequence); void applyControls(uint32_t sequence); @@ -78,6 +79,7 @@ private: uint32_t queueCount_; uint32_t writeCount_; std::unordered_map> values_; + RingBuffer cookies_; }; } /* namespace RPi */ diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index f3be4ee3b730..2c12ed1fba95 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -1064,7 +1064,7 @@ int PipelineHandlerRPi::start(Camera *camera, const ControlList *controls) * Reset the delayed controls with the gain and exposure values set by * the IPA. */ - data->delayedCtrls_->reset(); + data->delayedCtrls_->reset(0); data->state_ = RPiCameraData::State::Idle; @@ -1794,7 +1794,7 @@ void RPiCameraData::setIspControls(const ControlList &controls) void RPiCameraData::setDelayedControls(const ControlList &controls) { - if (!delayedCtrls_->push(controls)) + if (!delayedCtrls_->push(controls, 0)) LOG(RPI, Error) << "V4L2 DelayedControl set failed"; handleState(); } @@ -1867,7 +1867,7 @@ void RPiCameraData::unicamBufferDequeue(FrameBuffer *buffer) * Lookup the sensor controls used for this frame sequence from * DelayedControl and queue them along with the frame buffer. */ - ControlList ctrl = delayedCtrls_->get(buffer->metadata().sequence); + auto [ctrl, cookie] = delayedCtrls_->get(buffer->metadata().sequence); /* * Add the frame timestamp to the ControlList for the IPA to use * as it does not receive the FrameBuffer object. From patchwork Tue Nov 15 09:07:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 17797 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 90D04BE08B for ; Tue, 15 Nov 2022 09:08:10 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4470363088; Tue, 15 Nov 2022 10:08:10 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1668503290; bh=s2YTQygTvSeLyPnuHS2nRt//taGrUtun4/B03wSKMcQ=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=N2UqrqBazNrcxqR8AK8Hj+acuYhBYIqO8d2RwCc89yj0WV3Sfs2b3o+DCZkIHONYl NfV2o2pt2MZEfVEL9ch2vIkFLWfMKBSdDFRCDVqXq4CnK9BzjmxEE2hWPINkO4Rv3R wXbkfLvTl84ufgEdULsTUPr2BgNlJ08h6ksh2ArdRzNtq/ptAHd9gIeH4jq/cWJITB I0avyAVzDvSY3JQWT8AWfQ/iLayORFFDU8j4aPG/9xDth6r3ncKKOs9A4cF+3SRBla rrYZtJaPtjbrLaBge81OGvwjy008V2ykceVW9ugyQ2B7NDjWnu8c8G8zq/TODQ6lAU oeR9tzQEJTUmQ== Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B29BD63094 for ; Tue, 15 Nov 2022 10:08:04 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="OicspHLj"; dkim-atps=neutral Received: by mail-wr1-x431.google.com with SMTP id z14so22945306wrn.7 for ; Tue, 15 Nov 2022 01:08:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=U1yeWuA5xm3xCJOhL9gRZMk/o1jcW8BC8T8GrBzPs80=; b=OicspHLjy7TfH0tH5RathjH5fnXbaIoDIpYvTk559N6oc5vNK4NZM07pfLjyL0coRg dAqHEluOjSd/meUvj/zQ0/06jTqSTp2iwdy1AccDh2L5iDzUawZHj71ee+nPrVRQ0+ss ZI12eupJ/hnIQb6DwVmGu5o6wv6l7iBYvOqHL5li4JQUlakc8c/g+IirFjkGNYl4/zt6 Batde4Gf5oPuOWDuu8aVThRl3dqAt3icxYNOjmzwUM6XcIDah3aCtQTeI+F9dHGVuWgH gVpF0psZ+BNt5ThzOZsJRr1SbWUvTwpj8CB4ab+0dqnpdyALZaAA0fY3lHOlHi+Pme9T ceBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=U1yeWuA5xm3xCJOhL9gRZMk/o1jcW8BC8T8GrBzPs80=; b=wuxX59nZsBko7hI+qivM11K7to9z32marehTCg5lkc2XrYoLMIqN/PM3n1SHlvTZWJ tO1W9HUxBSAT2vXQyfr0RmHtuUU1u4mtzfkTvmPCXAMlqOUcIEzkrL4jCmj+nX2AQsV1 AJ/3qjOoVlQ0Q0Ik6crSAKATLutknoI+PF32XAgBPzroEWXYlLh1wzPqyv1kVhz5ZJNK voTiRLtFGuTwkQT+ZzSEjfHmSuYsMv5RPIGOSepaabejP3pOfcl3k/uc/JgY6YLq/VLN 1S5iaHg19HOKtKC3DXmY3UA8WxEIrKsQcf16uRk+W6L9TrPka0pn2FfsBYEydaA8oIE0 AWNw== X-Gm-Message-State: ANoB5pmOZzTHGT0PEJKqdDl5SS0Qe26svb2XUo5JEV9Gw2P4rdhydCzP n/JCxp9xAkXanuyCZ1DIpQ+0TwmI48mJUg== X-Google-Smtp-Source: AA0mqf5Yjpiy1UQK1tGn7OBxFiqjRIuZ5+neWAji8wUYJQoYA6jJPd09pisku+Pb30UnR97vhm6qxw== X-Received: by 2002:a5d:4051:0:b0:236:715c:aaa6 with SMTP id w17-20020a5d4051000000b00236715caaa6mr9815435wrp.25.1668503284068; Tue, 15 Nov 2022 01:08:04 -0800 (PST) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id p25-20020a05600c1d9900b003c64c186206sm15143696wms.16.2022.11.15.01.08.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Nov 2022 01:08:03 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Tue, 15 Nov 2022 09:07:52 +0000 Message-Id: <20221115090755.2921-6-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221115090755.2921-1-naush@raspberrypi.com> References: <20221115090755.2921-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v6 5/8] ipa: raspberrypi: Add RPiController::Metadata::mergeCopy X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a new member function to RPiController::Metadata that copies unique key/value pairs from one object to the other. This is different from std::map::merge that would remove the key/value pairs from the source object. Signed-off-by: Naushir Patuck Reviewed-by: Kieran Bingham Reviewed-by: David Plowman --- src/ipa/raspberrypi/controller/metadata.h | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/src/ipa/raspberrypi/controller/metadata.h b/src/ipa/raspberrypi/controller/metadata.h index 0f7ebfaf4c25..ddce491784e9 100644 --- a/src/ipa/raspberrypi/controller/metadata.h +++ b/src/ipa/raspberrypi/controller/metadata.h @@ -78,6 +78,16 @@ public: data_.merge(other.data_); } + void mergeCopy(const Metadata &other) + { + std::scoped_lock lock(mutex_, other.mutex_); + /* + * If the metadata key exists, ignore this item and copy only + * unique key/value pairs. + */ + data_.insert(other.data_.begin(), other.data_.end()); + } + template T *getLocked(std::string const &tag) { From patchwork Tue Nov 15 09:07:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 17798 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 36034BE08B for ; Tue, 15 Nov 2022 09:08:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E316563089; Tue, 15 Nov 2022 10:08:10 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1668503290; bh=jlRfxeMNJmHpGoiVrybdPgGhig7tY5AWa9GoNAXL7UI=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=UlrBDyOiYjup7vPPYfs24omnJcAffF/47OyjHlpAK2U8M0isVP8iMTBmN4CfPwRJC E8eUYAMlhyB0+VOPPhhuKqmsYej7ZQtNQcawsUBVxJJZGMYMIdYxZYOUUB/geySz4o zkxppRI8+tz5U/2hxWX0aK+ZveRjc4znXhtJngajHbcH6NX2VJOgEtEYWzsNi1XEIm IJWNqwt9CQbrJPsASkyLNqverGwfCX0vzVxsnLgO1B1ZNdLyPGh0mbk4u1bdvkXI9r qeCi6ylcyL4a97WzYUYA/vmZVLn875nIdvLQpZ7t/Pw2fEqxqFceOTzuOHPTtEkxeR MNchDahARPsGg== Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D0A8863097 for ; Tue, 15 Nov 2022 10:08:05 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="XwLnmbtt"; dkim-atps=neutral Received: by mail-wr1-x431.google.com with SMTP id z14so22945398wrn.7 for ; Tue, 15 Nov 2022 01:08:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7pCZVHEGkH4rkfudxa5QoNd2EeOZUCc279G5kmfq7Bk=; b=XwLnmbttdCZAw1Ab1iumLNvE3cbdXp3aQmRmOUuIYf9GA2bHwf2lacfFB37jvZ+Gu4 Qp1I8ePXu7c+D/UEQX7b8hUOwYuqg4INQlyAu7fvEr4G63R/SBaEidQ+hsb0wpQN84vL ioBk6yrC4NZ310lztXCh77auiLKs4xsQHXh6ixb5BdTos7zwgds0kHmIZ/NKKh+xAKMb FzlyNtKd8lpgHIpggDJoUNueCrmt4PUJYgvYBF+SgYuZbqOnr/fznzwFOzdT8HvpNNB/ WCin3uA/EybsHARZo47j6h4GpX6hlaa/lh+48QCCf5kWkKJmPfR/sBHydDvWqiIfwWmH 4z0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7pCZVHEGkH4rkfudxa5QoNd2EeOZUCc279G5kmfq7Bk=; b=awtT142bEtYoM87abqosl2Hz8tQ6CvUFEGqxIu6RKDjlfu+/h+jgKnfuvKlyNgiJHG ijSpKqc3zMQedomtPix+q4WzWMxabsi68EjeJkvwUHoHTW+7qu4cfJyWNhimH9Y8BKJ0 xW6cwSNV8HCIyiWho7kVHxUFjnszLW2pkBK1UayXPMWzf15MxYDYxNV6cSS0Xc0PF9g+ +S7StY3CbsqspRqSB0MmdGn6f5/q/tqHKkWQBwvBiC3c0hs6xLqEL1XkJQaM4Q0kC/b/ wJri578Rw5QxPozcCTr+6kjqUZidBuU5Gb29aGwFtRRxwVOZUWVyjXzNQSJfxsU89daG 1Cfg== X-Gm-Message-State: ANoB5pkmtX1udu8fCPN3K4aYal6sIFrXnXfg2bqMYr2ES3hFiaLKulE4 im7kdm0PnSQumdWs3JCDdA9xmhOhDo/uCQ== X-Google-Smtp-Source: AA0mqf49O+mFJiDRfrcBwN7wQIVAicNQ2nFaqctkuluGLqnHyt2V+oWGWSiOl9v5xWdVGDtto3KPzA== X-Received: by 2002:a5d:46c6:0:b0:236:8710:41f2 with SMTP id g6-20020a5d46c6000000b00236871041f2mr9968225wrs.239.1668503285110; Tue, 15 Nov 2022 01:08:05 -0800 (PST) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id p25-20020a05600c1d9900b003c64c186206sm15143696wms.16.2022.11.15.01.08.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Nov 2022 01:08:04 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Tue, 15 Nov 2022 09:07:53 +0000 Message-Id: <20221115090755.2921-7-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221115090755.2921-1-naush@raspberrypi.com> References: <20221115090755.2921-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v6 6/8] ipa: raspberrypi: Use an array of RPiController::Metadata objects X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Allow the IPA to cycle through an array of RPiController::Metadata objects, one per prepare()/process() invocation, when running the controller algorithms. This allows historical metadata objects to be retained, and subsequently passed into the controller algorithms on future frames. At present, only a single index into this array is used. This change provides a route to fixing a problem with the AGC algorithm, where if a manual shutter/gain is requested, the algorithm does not currently retain any context of the total exposure that it has calculated. As a result, the wrong digital gain would be applied when the frame with the manual shutter/gain is processed by the ISP. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Tested-by: David Plowman Reviewed-by: Kieran Bingham --- src/ipa/raspberrypi/raspberrypi.cpp | 63 ++++++++++++++++------------- 1 file changed, 35 insertions(+), 28 deletions(-) diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index b74f1ecf738f..799a4fe70000 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -57,6 +57,9 @@ namespace libcamera { using namespace std::literals::chrono_literals; using utils::Duration; +/* Number of metadata objects available in the context list. */ +constexpr unsigned int numMetadataContexts = 16; + /* Configure the sensor with these values initially. */ constexpr double defaultAnalogueGain = 1.0; constexpr Duration defaultExposureTime = 20.0ms; @@ -163,7 +166,7 @@ private: /* Raspberry Pi controller specific defines. */ std::unique_ptr helper_; RPiController::Controller controller_; - RPiController::Metadata rpiMetadata_; + std::array rpiMetadata_; /* * We count frames to decide if the frame must be hidden (e.g. from @@ -539,14 +542,15 @@ void IPARPi::signalIspPrepare(const ISPConfig &data) void IPARPi::reportMetadata() { - std::unique_lock lock(rpiMetadata_); + RPiController::Metadata &rpiMetadata = rpiMetadata_[0]; + std::unique_lock lock(rpiMetadata); /* * Certain information about the current frame and how it will be * processed can be extracted and placed into the libcamera metadata * buffer, where an application could query it. */ - DeviceStatus *deviceStatus = rpiMetadata_.getLocked("device.status"); + DeviceStatus *deviceStatus = rpiMetadata.getLocked("device.status"); if (deviceStatus) { libcameraMetadata_.set(controls::ExposureTime, deviceStatus->shutterSpeed.get()); @@ -557,24 +561,24 @@ void IPARPi::reportMetadata() libcameraMetadata_.set(controls::SensorTemperature, *deviceStatus->sensorTemperature); } - AgcStatus *agcStatus = rpiMetadata_.getLocked("agc.status"); + AgcStatus *agcStatus = rpiMetadata.getLocked("agc.status"); if (agcStatus) { libcameraMetadata_.set(controls::AeLocked, agcStatus->locked); libcameraMetadata_.set(controls::DigitalGain, agcStatus->digitalGain); } - LuxStatus *luxStatus = rpiMetadata_.getLocked("lux.status"); + LuxStatus *luxStatus = rpiMetadata.getLocked("lux.status"); if (luxStatus) libcameraMetadata_.set(controls::Lux, luxStatus->lux); - AwbStatus *awbStatus = rpiMetadata_.getLocked("awb.status"); + AwbStatus *awbStatus = rpiMetadata.getLocked("awb.status"); if (awbStatus) { libcameraMetadata_.set(controls::ColourGains, { static_cast(awbStatus->gainR), static_cast(awbStatus->gainB) }); libcameraMetadata_.set(controls::ColourTemperature, awbStatus->temperatureK); } - BlackLevelStatus *blackLevelStatus = rpiMetadata_.getLocked("black_level.status"); + BlackLevelStatus *blackLevelStatus = rpiMetadata.getLocked("black_level.status"); if (blackLevelStatus) libcameraMetadata_.set(controls::SensorBlackLevels, { static_cast(blackLevelStatus->blackLevelR), @@ -582,7 +586,7 @@ void IPARPi::reportMetadata() static_cast(blackLevelStatus->blackLevelG), static_cast(blackLevelStatus->blackLevelB) }); - FocusStatus *focusStatus = rpiMetadata_.getLocked("focus.status"); + FocusStatus *focusStatus = rpiMetadata.getLocked("focus.status"); if (focusStatus && focusStatus->num == 12) { /* * We get a 4x3 grid of regions by default. Calculate the average @@ -593,7 +597,7 @@ void IPARPi::reportMetadata() libcameraMetadata_.set(controls::FocusFoM, focusFoM); } - CcmStatus *ccmStatus = rpiMetadata_.getLocked("ccm.status"); + CcmStatus *ccmStatus = rpiMetadata.getLocked("ccm.status"); if (ccmStatus) { float m[9]; for (unsigned int i = 0; i < 9; i++) @@ -1006,9 +1010,10 @@ void IPARPi::prepareISP(const ISPConfig &data) { int64_t frameTimestamp = data.controls.get(controls::SensorTimestamp).value_or(0); RPiController::Metadata lastMetadata; + RPiController::Metadata &rpiMetadata = rpiMetadata_[0]; Span embeddedBuffer; - lastMetadata = std::move(rpiMetadata_); + lastMetadata = std::move(rpiMetadata); fillDeviceStatus(data.controls); if (data.embeddedBufferPresent) { @@ -1025,7 +1030,7 @@ void IPARPi::prepareISP(const ISPConfig &data) * This may overwrite the DeviceStatus using values from the sensor * metadata, and may also do additional custom processing. */ - helper_->prepare(embeddedBuffer, rpiMetadata_); + helper_->prepare(embeddedBuffer, rpiMetadata); /* Done with embedded data now, return to pipeline handler asap. */ if (data.embeddedBufferPresent) @@ -1041,7 +1046,7 @@ void IPARPi::prepareISP(const ISPConfig &data) * current frame, or any other bits of metadata that were added * in helper_->Prepare(). */ - rpiMetadata_.merge(lastMetadata); + rpiMetadata.merge(lastMetadata); processPending_ = false; return; } @@ -1051,48 +1056,48 @@ void IPARPi::prepareISP(const ISPConfig &data) ControlList ctrls(ispCtrls_); - controller_.prepare(&rpiMetadata_); + controller_.prepare(&rpiMetadata); /* Lock the metadata buffer to avoid constant locks/unlocks. */ - std::unique_lock lock(rpiMetadata_); + std::unique_lock lock(rpiMetadata); - AwbStatus *awbStatus = rpiMetadata_.getLocked("awb.status"); + AwbStatus *awbStatus = rpiMetadata.getLocked("awb.status"); if (awbStatus) applyAWB(awbStatus, ctrls); - CcmStatus *ccmStatus = rpiMetadata_.getLocked("ccm.status"); + CcmStatus *ccmStatus = rpiMetadata.getLocked("ccm.status"); if (ccmStatus) applyCCM(ccmStatus, ctrls); - AgcStatus *dgStatus = rpiMetadata_.getLocked("agc.status"); + AgcStatus *dgStatus = rpiMetadata.getLocked("agc.status"); if (dgStatus) applyDG(dgStatus, ctrls); - AlscStatus *lsStatus = rpiMetadata_.getLocked("alsc.status"); + AlscStatus *lsStatus = rpiMetadata.getLocked("alsc.status"); if (lsStatus) applyLS(lsStatus, ctrls); - ContrastStatus *contrastStatus = rpiMetadata_.getLocked("contrast.status"); + ContrastStatus *contrastStatus = rpiMetadata.getLocked("contrast.status"); if (contrastStatus) applyGamma(contrastStatus, ctrls); - BlackLevelStatus *blackLevelStatus = rpiMetadata_.getLocked("black_level.status"); + BlackLevelStatus *blackLevelStatus = rpiMetadata.getLocked("black_level.status"); if (blackLevelStatus) applyBlackLevel(blackLevelStatus, ctrls); - GeqStatus *geqStatus = rpiMetadata_.getLocked("geq.status"); + GeqStatus *geqStatus = rpiMetadata.getLocked("geq.status"); if (geqStatus) applyGEQ(geqStatus, ctrls); - DenoiseStatus *denoiseStatus = rpiMetadata_.getLocked("denoise.status"); + DenoiseStatus *denoiseStatus = rpiMetadata.getLocked("denoise.status"); if (denoiseStatus) applyDenoise(denoiseStatus, ctrls); - SharpenStatus *sharpenStatus = rpiMetadata_.getLocked("sharpen.status"); + SharpenStatus *sharpenStatus = rpiMetadata.getLocked("sharpen.status"); if (sharpenStatus) applySharpen(sharpenStatus, ctrls); - DpcStatus *dpcStatus = rpiMetadata_.getLocked("dpc.status"); + DpcStatus *dpcStatus = rpiMetadata.getLocked("dpc.status"); if (dpcStatus) applyDPC(dpcStatus, ctrls); @@ -1116,11 +1121,13 @@ void IPARPi::fillDeviceStatus(const ControlList &sensorControls) LOG(IPARPI, Debug) << "Metadata - " << deviceStatus; - rpiMetadata_.set("device.status", deviceStatus); + rpiMetadata_[0].set("device.status", deviceStatus); } void IPARPi::processStats(unsigned int bufferId) { + RPiController::Metadata &rpiMetadata = rpiMetadata_[0]; + auto it = buffers_.find(bufferId); if (it == buffers_.end()) { LOG(IPARPI, Error) << "Could not find stats buffer!"; @@ -1130,11 +1137,11 @@ void IPARPi::processStats(unsigned int bufferId) Span mem = it->second.planes()[0]; bcm2835_isp_stats *stats = reinterpret_cast(mem.data()); RPiController::StatisticsPtr statistics = std::make_shared(*stats); - helper_->process(statistics, rpiMetadata_); - controller_.process(statistics, &rpiMetadata_); + helper_->process(statistics, rpiMetadata); + controller_.process(statistics, &rpiMetadata); struct AgcStatus agcStatus; - if (rpiMetadata_.get("agc.status", agcStatus) == 0) { + if (rpiMetadata.get("agc.status", agcStatus) == 0) { ControlList ctrls(sensorCtrls_); applyAGC(&agcStatus, ctrls); From patchwork Tue Nov 15 09:07:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 17799 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9F849C3285 for ; Tue, 15 Nov 2022 09:08:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 57C7563098; Tue, 15 Nov 2022 10:08:11 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1668503291; bh=BcdzumNDkt3pVKjdZHPGzaZultlq4a0Ap82bjymxZEU=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=hC5E9CVCacPRNMwZIUKJVM/gaOHj3aHhqFyHr0i1hhk119qhCy9xltWPmx8BZDTWX daFYgFnX3WoMCya1aVUUVTdDsxo8zq8g4lO2fgXVvBPYg8DDCWh5BGXkY14oq+zZ3P yLrbfgyFtRYcjZ6S5v1umoh03qEmQSZu2qeFSl/f1FZG9QiHjh0s2Tl8Amq98km6eF EsrTH1rRIspRZVF2XuftoXN2ScMwxcvty5HXhqFg4sm8GvF1SqihngjIJ5ZCP92itf ZB5jkNNFbX7ZD0H+6swMNrRLOAFDEAt9RpDKba2P5w0uJL7N45wFMGMArMcuVW1xQU 8cQlUcjs2f4zg== Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7A4CF63088 for ; Tue, 15 Nov 2022 10:08:06 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="iITqZCyQ"; dkim-atps=neutral Received: by mail-wr1-x42e.google.com with SMTP id d9so18381855wrm.13 for ; Tue, 15 Nov 2022 01:08:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xMING90SQKUvR7yzkp4yjNn91MarCln/xYFvrXKiQIk=; b=iITqZCyQ6DreIMGwwf4ddHBkzRFIb2NpOdCI1X3vZiKhL9bpT8r8asnXU7xcLdI5ef rly+ZQ6mSxifR4sFxYsLqEggpGYWpJvNUhHcfCt20/RhMvjhZJfko615dKusZRuCCrFK 2TcDysUnA8WSu5cxECTV5OicalVVnNIC/Gp1CK0xVNJks6v4fUIAGkGL1AFBOZnRQ4DS w1MDX9JniatMuZztEAvkKGdPghULvKDTMmCgHoelutcADKOAemotP8JuN3izrD/y3VuI El23BCUDYBNdmM6irUKhsyms/uEEw8qJjA2F+5cLqZSGEauGYDXPfD29Kcr1NflY3+mW mFRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xMING90SQKUvR7yzkp4yjNn91MarCln/xYFvrXKiQIk=; b=i11eKgyb1z8JFhOcacsFbT6jyAvBS3LDzARxmGoJfp7EZG6tJZANX0RjVEh9uxYKNM Hkq7QVJBDIHWxRS6vaIYEKlT0hOiNuasd/+PE74xtiHtq8qUwL/ugmLCRJ1eo1qMRkmY GgNuMRTl092BmgBG/4wqE9HNNvfqw1r2zKS1X1WYRWAIiIf7ydwoWvNEYELRrQh3gTmk 3eTAcxeA0jf/YMC1/Bh9LDJ0pYYA92WOAbFjx0O9fTE0nIadTu4WMY1Vp4znwpHqbLyv F077KxlGbMlZj1NOZkuQi7nxhoqiepx2m3QNIAQmleuwsgbvOo+FTMqcsf2ZNL3PuT1D RQNg== X-Gm-Message-State: ANoB5pkKPCzsOqN0tbmc4zWNgtdmOFZkY41+P/bay/eWNSrmQxqRgtmS ETQo/RAxMFTCoMd5lOnpCps+Y2emM5SIKQ== X-Google-Smtp-Source: AA0mqf70agn1WuqZ9veoYnw28j3aswQyI8s63UraIJoqCPKw9NHTg76ZoYjdnb+te4RGPHwCbignUw== X-Received: by 2002:adf:f18c:0:b0:236:862b:c457 with SMTP id h12-20020adff18c000000b00236862bc457mr9703951wro.154.1668503285698; Tue, 15 Nov 2022 01:08:05 -0800 (PST) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id p25-20020a05600c1d9900b003c64c186206sm15143696wms.16.2022.11.15.01.08.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Nov 2022 01:08:05 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Tue, 15 Nov 2022 09:07:54 +0000 Message-Id: <20221115090755.2921-8-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221115090755.2921-1-naush@raspberrypi.com> References: <20221115090755.2921-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v6 7/8] pipeline: ipa: raspberrypi: Use IPA cookies X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Pass an IPA cookie from the pipeline handler to the IPA and eventually back to the pipeline handler through the setDelayedControls signal. This cookie is used to index the RPiController::Metadata object to be used for the frame. The IPA cookie is then returned from DelayedControls when the frame with the applied controls has been returned from the sensor, and eventually passed back to the IPA from the signalIspPrepare signal. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Tested-by: David Plowman --- include/libcamera/ipa/raspberrypi.mojom | 6 ++- src/ipa/raspberrypi/raspberrypi.cpp | 42 ++++++++++--------- .../pipeline/raspberrypi/raspberrypi.cpp | 16 ++++--- 3 files changed, 37 insertions(+), 27 deletions(-) diff --git a/include/libcamera/ipa/raspberrypi.mojom b/include/libcamera/ipa/raspberrypi.mojom index 40f78d9e3b3f..325d2d855bc0 100644 --- a/include/libcamera/ipa/raspberrypi.mojom +++ b/include/libcamera/ipa/raspberrypi.mojom @@ -37,6 +37,8 @@ struct ISPConfig { uint32 bayerBufferId; bool embeddedBufferPresent; libcamera.ControlList controls; + uint32 ipaContext; + uint32 delayContext; }; struct IPAConfig { @@ -127,7 +129,7 @@ interface IPARPiInterface { */ unmapBuffers(array ids); - [async] signalStatReady(uint32 bufferId); + [async] signalStatReady(uint32 bufferId, uint32 ipaContext); [async] signalQueueRequest(libcamera.ControlList controls); [async] signalIspPrepare(ISPConfig data); }; @@ -137,5 +139,5 @@ interface IPARPiEventInterface { runIsp(uint32 bufferId); embeddedComplete(uint32 bufferId); setIspControls(libcamera.ControlList controls); - setDelayedControls(libcamera.ControlList controls); + setDelayedControls(libcamera.ControlList controls, uint32 delayContext); }; diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 799a4fe70000..194171a8bc96 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -126,7 +126,7 @@ public: ControlList *controls, IPAConfigResult *result) override; void mapBuffers(const std::vector &buffers) override; void unmapBuffers(const std::vector &ids) override; - void signalStatReady(const uint32_t bufferId) override; + void signalStatReady(const uint32_t bufferId, uint32_t ipaContext) override; void signalQueueRequest(const ControlList &controls) override; void signalIspPrepare(const ISPConfig &data) override; @@ -137,9 +137,9 @@ private: void queueRequest(const ControlList &controls); void returnEmbeddedBuffer(unsigned int bufferId); void prepareISP(const ISPConfig &data); - void reportMetadata(); - void fillDeviceStatus(const ControlList &sensorControls); - void processStats(unsigned int bufferId); + void reportMetadata(unsigned int ipaContext); + void fillDeviceStatus(const ControlList &sensorControls, unsigned int ipaContext); + void processStats(unsigned int bufferId, unsigned int ipaContext); void applyFrameDurations(Duration minFrameDuration, Duration maxFrameDuration); void applyAGC(const struct AgcStatus *agcStatus, ControlList &ctrls); void applyAWB(const struct AwbStatus *awbStatus, ControlList &ctrls); @@ -509,14 +509,16 @@ void IPARPi::unmapBuffers(const std::vector &ids) } } -void IPARPi::signalStatReady(uint32_t bufferId) +void IPARPi::signalStatReady(uint32_t bufferId, uint32_t ipaContext) { + unsigned int context = ipaContext % rpiMetadata_.size(); + if (++checkCount_ != frameCount_) /* assert here? */ LOG(IPARPI, Error) << "WARNING: Prepare/Process mismatch!!!"; if (processPending_ && frameCount_ > mistrustCount_) - processStats(bufferId); + processStats(bufferId, context); - reportMetadata(); + reportMetadata(context); statsMetadataComplete.emit(bufferId & MaskID, libcameraMetadata_); } @@ -540,9 +542,9 @@ void IPARPi::signalIspPrepare(const ISPConfig &data) runIsp.emit(data.bayerBufferId & MaskID); } -void IPARPi::reportMetadata() +void IPARPi::reportMetadata(unsigned int ipaContext) { - RPiController::Metadata &rpiMetadata = rpiMetadata_[0]; + RPiController::Metadata &rpiMetadata = rpiMetadata_[ipaContext]; std::unique_lock lock(rpiMetadata); /* @@ -1009,12 +1011,12 @@ void IPARPi::returnEmbeddedBuffer(unsigned int bufferId) void IPARPi::prepareISP(const ISPConfig &data) { int64_t frameTimestamp = data.controls.get(controls::SensorTimestamp).value_or(0); - RPiController::Metadata lastMetadata; - RPiController::Metadata &rpiMetadata = rpiMetadata_[0]; + unsigned int ipaContext = data.ipaContext % rpiMetadata_.size(); + RPiController::Metadata &rpiMetadata = rpiMetadata_[ipaContext]; Span embeddedBuffer; - lastMetadata = std::move(rpiMetadata); - fillDeviceStatus(data.controls); + rpiMetadata.clear(); + fillDeviceStatus(data.controls, ipaContext); if (data.embeddedBufferPresent) { /* @@ -1046,7 +1048,9 @@ void IPARPi::prepareISP(const ISPConfig &data) * current frame, or any other bits of metadata that were added * in helper_->Prepare(). */ - rpiMetadata.merge(lastMetadata); + RPiController::Metadata &lastMetadata = + rpiMetadata_[ipaContext ? ipaContext : rpiMetadata_.size()]; + rpiMetadata.mergeCopy(lastMetadata); processPending_ = false; return; } @@ -1105,7 +1109,7 @@ void IPARPi::prepareISP(const ISPConfig &data) setIspControls.emit(ctrls); } -void IPARPi::fillDeviceStatus(const ControlList &sensorControls) +void IPARPi::fillDeviceStatus(const ControlList &sensorControls, unsigned int ipaContext) { DeviceStatus deviceStatus = {}; @@ -1121,12 +1125,12 @@ void IPARPi::fillDeviceStatus(const ControlList &sensorControls) LOG(IPARPI, Debug) << "Metadata - " << deviceStatus; - rpiMetadata_[0].set("device.status", deviceStatus); + rpiMetadata_[ipaContext].set("device.status", deviceStatus); } -void IPARPi::processStats(unsigned int bufferId) +void IPARPi::processStats(unsigned int bufferId, unsigned int ipaContext) { - RPiController::Metadata &rpiMetadata = rpiMetadata_[0]; + RPiController::Metadata &rpiMetadata = rpiMetadata_[ipaContext]; auto it = buffers_.find(bufferId); if (it == buffers_.end()) { @@ -1145,7 +1149,7 @@ void IPARPi::processStats(unsigned int bufferId) ControlList ctrls(sensorCtrls_); applyAGC(&agcStatus, ctrls); - setDelayedControls.emit(ctrls); + setDelayedControls.emit(ctrls, ipaContext); } } diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 2c12ed1fba95..6f560fc2782e 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -206,7 +206,7 @@ public: void runIsp(uint32_t bufferId); void embeddedComplete(uint32_t bufferId); void setIspControls(const ControlList &controls); - void setDelayedControls(const ControlList &controls); + void setDelayedControls(const ControlList &controls, uint32_t delayContext); void setSensorControls(ControlList &controls); void unicamTimeout(); @@ -262,6 +262,7 @@ public: struct BayerFrame { FrameBuffer *buffer; ControlList controls; + unsigned int delayContext; }; std::queue bayerQueue_; @@ -1792,9 +1793,9 @@ void RPiCameraData::setIspControls(const ControlList &controls) handleState(); } -void RPiCameraData::setDelayedControls(const ControlList &controls) +void RPiCameraData::setDelayedControls(const ControlList &controls, uint32_t delayContext) { - if (!delayedCtrls_->push(controls, 0)) + if (!delayedCtrls_->push(controls, delayContext)) LOG(RPI, Error) << "V4L2 DelayedControl set failed"; handleState(); } @@ -1867,13 +1868,13 @@ void RPiCameraData::unicamBufferDequeue(FrameBuffer *buffer) * Lookup the sensor controls used for this frame sequence from * DelayedControl and queue them along with the frame buffer. */ - auto [ctrl, cookie] = delayedCtrls_->get(buffer->metadata().sequence); + auto [ctrl, delayContext] = delayedCtrls_->get(buffer->metadata().sequence); /* * Add the frame timestamp to the ControlList for the IPA to use * as it does not receive the FrameBuffer object. */ ctrl.set(controls::SensorTimestamp, buffer->metadata().timestamp); - bayerQueue_.push({ buffer, std::move(ctrl) }); + bayerQueue_.push({ buffer, std::move(ctrl), delayContext }); } else { embeddedQueue_.push(buffer); } @@ -1923,7 +1924,8 @@ void RPiCameraData::ispOutputDequeue(FrameBuffer *buffer) * application until after the IPA signals so. */ if (stream == &isp_[Isp::Stats]) { - ipa_->signalStatReady(ipa::RPi::MaskStats | static_cast(index)); + ipa_->signalStatReady(ipa::RPi::MaskStats | static_cast(index), + requestQueue_.front()->sequence()); } else { /* Any other ISP output can be handed back to the application now. */ handleStreamBuffer(buffer, stream); @@ -2168,6 +2170,8 @@ void RPiCameraData::tryRunPipeline() ipa::RPi::ISPConfig ispPrepare; ispPrepare.bayerBufferId = ipa::RPi::MaskBayerData | bayerId; ispPrepare.controls = std::move(bayerFrame.controls); + ispPrepare.ipaContext = request->sequence(); + ispPrepare.delayContext = bayerFrame.delayContext; if (embeddedBuffer) { unsigned int embeddedId = unicam_[Unicam::Embedded].getBufferId(embeddedBuffer); From patchwork Tue Nov 15 09:07:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 17800 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4982DBE08B for ; Tue, 15 Nov 2022 09:08:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D50ED63095; Tue, 15 Nov 2022 10:08:11 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1668503291; bh=YhIxN2di9TyNXMbx5fMifQcQb3B04ncA8Y/T2+CW3v4=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=CCWv0EP9062B61IIsx2KyAm+uQbFUkgLuS8wdUOz2OL5Ud85/0lN7ARgABnIS8Zd3 i9rn9Vq5NBDX4pp/MUOPn+Ck/Wa/l0dWhJhRHANAy1/SphyfhUZRQRwgQAzD5tiNBD wDR3JO1jIDA688xr2qhrlxEPSfSAVR05BeXZbzM74d170qjcAVwI3lZs7yEtaKjj8W wYt0Pb8fUmqEBj+pppKt1j/vajcc86XChHSFXgJXv/Zf4ijOwHOsfh/4A+n4R7nrUW +lJH6BWmTvm7B1jtagP2vKi8fQyX6tDukzLcFqL5Qi33dYoJ57DjdCaqtOxLPnvNcM ovZ5luZEJr2sg== Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5742B63089 for ; Tue, 15 Nov 2022 10:08:07 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="kc+I44dj"; dkim-atps=neutral Received: by mail-wr1-x434.google.com with SMTP id v1so23055638wrt.11 for ; Tue, 15 Nov 2022 01:08:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zdhz+1j/NkLDJJXH7usJMOs+Zyvf3jcELZrgcTTFIAE=; b=kc+I44djMYwTNTtT3Hb/KU8C6b4ZzYBMlVa6tZmuGyFTWtReRI6ZVZ5EVZG0bP88Nw EA6Kf1U3HWxAT2RRdtyk3xOGQ/D6+LYq0Mgi8YzjdoiInZTG9R3OZ53Ovme/hHTxmP/M szTzJbGnzMzLchnUZnV0jyWpzM8nHbkV5KcARJnTTqImUo/CZa+3O2wlKNn1r0xE1Orw ufPWRDysc6MwtMUqbIJueYlciwW+ZdvKBhTR5kgS1v04+SBB+MVW+DvCijO5D1AiQwYD 7iwDQbP1cHanc/e5yrFFTWhM4smz2Vp1whnveeKQ3GXT7r7komVOvJ2JGd9DFJaUngpF llBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zdhz+1j/NkLDJJXH7usJMOs+Zyvf3jcELZrgcTTFIAE=; b=QMVMC1gY3dXuqzu16z1liZr3qx2kfjKtbYm99KBlYKbRuRhaI44GD7ERbe3yKwQL4U D0SkqScccbeFV4cppsRPRVFkCwVy1DAOHdQ77eYx1ypNPOJsDw2/uuNv5tcZ+bjcQJ1D axX2La95dN1dkPqEYZmcUWLYQBeA8i6O4MO+ZqyLeefQ51OyrI/oqaZFKr3wZThAqSMu BZ6YWwVZRTyUnrXML1sq2kgbOotyW/hX4cYjiyh5dsce8li3d7cXKJD/bZqEZ0MVLygf evsJYSju0i9syftHJ88ltOzBFD31IrCkzGRSWMgKfdkntFlJzp/NUfpCGl4Xbc6HfpR7 9U5A== X-Gm-Message-State: ANoB5pkEBdXNLp3Qrz0cjOUU9Yrbo7povWeTQ78DkqrveA1AmHg3guxm EisaXO+7d9v4cFNoesrXAxvFNkUE1Ry9nA== X-Google-Smtp-Source: AA0mqf5eugk5u7ukkFVyUeihFIzhdvHrGdOdnr4AG/lLyy4jWWkcSCg9PDfnGUzH1zs9/IRfenwS3Q== X-Received: by 2002:adf:edc6:0:b0:236:6a28:eeda with SMTP id v6-20020adfedc6000000b002366a28eedamr9778463wro.315.1668503286518; Tue, 15 Nov 2022 01:08:06 -0800 (PST) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id p25-20020a05600c1d9900b003c64c186206sm15143696wms.16.2022.11.15.01.08.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Nov 2022 01:08:05 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Tue, 15 Nov 2022 09:07:55 +0000 Message-Id: <20221115090755.2921-9-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221115090755.2921-1-naush@raspberrypi.com> References: <20221115090755.2921-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v6 8/8] ipa: raspberrypi: agc: Fix digital gain calculation for manual mode X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The digital gain calculation uses a total exposure value computed with the current AGC state. However, this is wrong in the case of manual shutter/gain controls, as the total exposure value used must be the value computed when the AGC sent the manual shutter/gain controls to the pipeline handler to action. To fix this, the IPA now adds the historical AgcStatus structure to the metadata (tagged with "agc.delayed_status"). This historical AgcStatus structure contains the total exposure value calculated when the AGC sent the manual shutter/gain controls to the pipeline handler. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Tested-by: David Plowman Reviewed-by: Kieran Bingham --- src/ipa/raspberrypi/controller/rpi/agc.cpp | 10 ++++++++-- src/ipa/raspberrypi/raspberrypi.cpp | 11 +++++++++++ 2 files changed, 19 insertions(+), 2 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp index bd54a639d637..b18bd7b5b19e 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp @@ -408,6 +408,12 @@ void Agc::switchMode(CameraMode const &cameraMode, void Agc::prepare(Metadata *imageMetadata) { + Duration totalExposureValue = status_.totalExposureValue; + AgcStatus delayedStatus; + + if (!imageMetadata->get("agc.delayed_status", delayedStatus)) + totalExposureValue = delayedStatus.totalExposureValue; + status_.digitalGain = 1.0; fetchAwbStatus(imageMetadata); /* always fetch it so that Process knows it's been done */ @@ -418,8 +424,8 @@ void Agc::prepare(Metadata *imageMetadata) Duration actualExposure = deviceStatus.shutterSpeed * deviceStatus.analogueGain; if (actualExposure) { - status_.digitalGain = status_.totalExposureValue / actualExposure; - LOG(RPiAgc, Debug) << "Want total exposure " << status_.totalExposureValue; + status_.digitalGain = totalExposureValue / actualExposure; + LOG(RPiAgc, Debug) << "Want total exposure " << totalExposureValue; /* * Never ask for a gain < 1.0, and also impose * some upper limit. Make it customisable? diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 194171a8bc96..7904f2b696dd 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -1028,6 +1028,17 @@ void IPARPi::prepareISP(const ISPConfig &data) embeddedBuffer = it->second.planes()[0]; } + /* + * AGC wants to know the algorithm status from the time it actioned the + * sensor exposure/gain changes. So fetch it from the metadata list + * indexed by the IPA cookie returned, and put it in the current frame + * metadata. + */ + AgcStatus agcStatus; + RPiController::Metadata &delayedMetadata = rpiMetadata_[data.delayContext]; + if (!delayedMetadata.get("agc.status", agcStatus)) + rpiMetadata.set("agc.delayed_status", agcStatus); + /* * This may overwrite the DeviceStatus using values from the sensor * metadata, and may also do additional custom processing.