From patchwork Tue Nov 15 09:07:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 17793 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A93B0BE08B for ; Tue, 15 Nov 2022 09:08:04 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6939863088; Tue, 15 Nov 2022 10:08:04 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1668503284; bh=DByEoimg1lvh8ErW1sv4VviuI8HSuOZoLi0E4p+TQ7w=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=WmOolv35PZwvDTjRnS+lDSj/mW7UsJ6sD8PJF3A+dl85UbZ8BA738R0fT/q0xIQ7J iHbUmtEdIQaJxPAB0Eu0x6yQk0MrTwYh1ELYnMy4h/cQyLdIuk/WE9RIny+66HYhI2 pbPxrdVG9j8IqUhxYRugTfGuz5eUE9i13dr1N9I+nSW914BDobP5vj3miXibStapSy U1swO06bg7mkV2Uxewt3TatFUoSAaA2fo8GyoOkHe5i+hJRkQryujERQG/1gzAk1if SpydOD8GTtq+E8RLmkO9XOhRobT8GNqMOy8F+AApePKEStF7iQWY3j8c388hXxBd/z EjKHjUCdarzVA== Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E5C0263089 for ; Tue, 15 Nov 2022 10:08:01 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="tQsaDZRQ"; dkim-atps=neutral Received: by mail-wm1-x32f.google.com with SMTP id ay14-20020a05600c1e0e00b003cf6ab34b61so12613467wmb.2 for ; Tue, 15 Nov 2022 01:08:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=XUEQUArpIfN6duNUMuOAcBmxj99qc641z9vVt3gWhPQ=; b=tQsaDZRQkPLzjwh9PERIJnFmq3D6177SeqrAXeOna2R9L8Of5n7wvlPTJgKLzVFq4/ tfLJQD/mhkeLM+BoRGxtOEryOrhCqrQQ8HadZYuVk1gEeKa2gY2/n555HBgR3O7UHn9I +PjARn1qY4+Fl7cIuyI1GniV6Gk2EYpn+fS3Wucnk0tfQ+091sbu3dojykdvMgfX4qHM O0xq5zmZC0bRdB9ocLKY1IA230dmNN/eC0kM/atNJZN1h48Lrpt8vNDrhtv8THVuV/gr ynY8KEAZQPrMNxiI3FFWsarl67X/Uf2mr08Iv1Jo9cI0ZptCIblKbGb1HWZ9dkSrlZAd aK0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XUEQUArpIfN6duNUMuOAcBmxj99qc641z9vVt3gWhPQ=; b=hCILExgdQ6cQ+wtrHlbcFf9FtcHsYa9jd8zfAe+brDRLwpRWhzpUIpiepN2DtcnXY9 gufKgp0aZwbBLeAIxLdTu4uw3enRGt3lRC03vyB4gr8MDmIDJ8B3+xKxfY6/b7LV5DQB fcZ+dFbJkZ1Fvaj10PQFNuCHCqxXbPe2zWQc+vtaD/1uRF5yrs/6zYRUeTXsKow+Z5ls 2Pw8K1Ng2uYLdWbrtuh2cAihtfr+1tUYYlNmrE1fk7FnsjrhlrWj4GLCpP9DJup4RANX jOa7tuOgKRAdDOH5fQwehivH/mNX6Qu2ECGTj0OfYgOPMrr1FSH+7c5intk4e7wP2C84 43ZQ== X-Gm-Message-State: ANoB5pn2PCx77VL9BjUULicqkRvSrQd91ea6APqO/HtqsvS26pKvaK50 fUnPtBnGK+aK9SqK7AVFRY4YC/idSbcLdg== X-Google-Smtp-Source: AA0mqf7B0OB0FfOkzvZiW6WnLqhgpFqDCxcTSbc0/MVrj3L2mqS3xZLgyE0tuafzb1PslRmG9HaOJA== X-Received: by 2002:a7b:c010:0:b0:3cf:eaf5:77c6 with SMTP id c16-20020a7bc010000000b003cfeaf577c6mr677213wmb.56.1668503280934; Tue, 15 Nov 2022 01:08:00 -0800 (PST) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id p25-20020a05600c1d9900b003c64c186206sm15143696wms.16.2022.11.15.01.08.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Nov 2022 01:08:00 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Tue, 15 Nov 2022 09:07:48 +0000 Message-Id: <20221115090755.2921-2-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221115090755.2921-1-naush@raspberrypi.com> References: <20221115090755.2921-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v6 1/8] pipeline: raspberrypi: Fork DelayedControls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Fork the libcamera::DelayedControls implementation in the RPi:: namespace, with the intention of updating the API for Raspberry Pi specific features. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Reviewed-by: Kieran Bingham --- .../pipeline/raspberrypi/delayed_controls.cpp | 291 ++++++++++++++++++ .../pipeline/raspberrypi/delayed_controls.h | 84 +++++ .../pipeline/raspberrypi/meson.build | 1 + 3 files changed, 376 insertions(+) create mode 100644 src/libcamera/pipeline/raspberrypi/delayed_controls.cpp create mode 100644 src/libcamera/pipeline/raspberrypi/delayed_controls.h diff --git a/src/libcamera/pipeline/raspberrypi/delayed_controls.cpp b/src/libcamera/pipeline/raspberrypi/delayed_controls.cpp new file mode 100644 index 000000000000..867e3866cc46 --- /dev/null +++ b/src/libcamera/pipeline/raspberrypi/delayed_controls.cpp @@ -0,0 +1,291 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Raspberry Pi Ltd + * + * delayed_controls.cpp - Helper to deal with controls that take effect with a delay + * + * Note: This has been forked from the libcamera core implementation. + */ + +#include "delayed_controls.h" + +#include + +#include + +#include "libcamera/internal/v4l2_device.h" + +/** + * \file delayed_controls.h + * \brief Helper to deal with controls that take effect with a delay + */ + +namespace libcamera { + +LOG_DEFINE_CATEGORY(RPiDelayedControls) + +namespace RPi { + +/** + * \class DelayedControls + * \brief Helper to deal with controls that take effect with a delay + * + * Some sensor controls take effect with a delay as the sensor needs time to + * adjust, for example exposure and analog gain. This is a helper class to deal + * with such controls and the intended users are pipeline handlers. + * + * The idea is to extend the concept of the buffer depth of a pipeline the + * application needs to maintain to also cover controls. Just as with buffer + * depth if the application keeps the number of requests queued above the + * control depth the controls are guaranteed to take effect for the correct + * request. The control depth is determined by the control with the greatest + * delay. + */ + +/** + * \struct DelayedControls::ControlParams + * \brief Parameters associated with controls handled by the \a DelayedControls + * helper class + * + * \var ControlParams::delay + * \brief Frame delay from setting the control on a sensor device to when it is + * consumed during framing. + * + * \var ControlParams::priorityWrite + * \brief Flag to indicate that this control must be applied ahead of, and + * separately from the other controls. + * + * Typically set for the \a V4L2_CID_VBLANK control so that the device driver + * does not reject \a V4L2_CID_EXPOSURE control values that may be outside of + * the existing vertical blanking specified bounds, but are within the new + * blanking bounds. + */ + +/** + * \brief Construct a DelayedControls instance + * \param[in] device The V4L2 device the controls have to be applied to + * \param[in] controlParams Map of the numerical V4L2 control ids to their + * associated control parameters. + * + * The control parameters comprise of delays (in frames) and a priority write + * flag. If this flag is set, the relevant control is written separately from, + * and ahead of the rest of the batched controls. + * + * Only controls specified in \a controlParams are handled. If it's desired to + * mix delayed controls and controls that take effect immediately the immediate + * controls must be listed in the \a controlParams map with a delay value of 0. + */ +DelayedControls::DelayedControls(V4L2Device *device, + const std::unordered_map &controlParams) + : device_(device), maxDelay_(0) +{ + const ControlInfoMap &controls = device_->controls(); + + /* + * Create a map of control ids to delays for controls exposed by the + * device. + */ + for (auto const ¶m : controlParams) { + auto it = controls.find(param.first); + if (it == controls.end()) { + LOG(RPiDelayedControls, Error) + << "Delay request for control id " + << utils::hex(param.first) + << " but control is not exposed by device " + << device_->deviceNode(); + continue; + } + + const ControlId *id = it->first; + + controlParams_[id] = param.second; + + LOG(RPiDelayedControls, Debug) + << "Set a delay of " << controlParams_[id].delay + << " and priority write flag " << controlParams_[id].priorityWrite + << " for " << id->name(); + + maxDelay_ = std::max(maxDelay_, controlParams_[id].delay); + } + + reset(); +} + +/** + * \brief Reset state machine + * + * Resets the state machine to a starting position based on control values + * retrieved from the device. + */ +void DelayedControls::reset() +{ + queueCount_ = 1; + writeCount_ = 0; + + /* Retrieve control as reported by the device. */ + std::vector ids; + for (auto const ¶m : controlParams_) + ids.push_back(param.first->id()); + + ControlList controls = device_->getControls(ids); + + /* Seed the control queue with the controls reported by the device. */ + values_.clear(); + for (const auto &ctrl : controls) { + const ControlId *id = device_->controls().idmap().at(ctrl.first); + /* + * Do not mark this control value as updated, it does not need + * to be written to to device on startup. + */ + values_[id][0] = Info(ctrl.second, false); + } +} + +/** + * \brief Push a set of controls on the queue + * \param[in] controls List of controls to add to the device queue + * + * Push a set of controls to the control queue. This increases the control queue + * depth by one. + * + * \returns true if \a controls are accepted, or false otherwise + */ +bool DelayedControls::push(const ControlList &controls) +{ + /* Copy state from previous frame. */ + for (auto &ctrl : values_) { + Info &info = ctrl.second[queueCount_]; + info = values_[ctrl.first][queueCount_ - 1]; + info.updated = false; + } + + /* Update with new controls. */ + const ControlIdMap &idmap = device_->controls().idmap(); + for (const auto &control : controls) { + const auto &it = idmap.find(control.first); + if (it == idmap.end()) { + LOG(RPiDelayedControls, Warning) + << "Unknown control " << control.first; + return false; + } + + const ControlId *id = it->second; + + if (controlParams_.find(id) == controlParams_.end()) + return false; + + Info &info = values_[id][queueCount_]; + + info = Info(control.second); + + LOG(RPiDelayedControls, Debug) + << "Queuing " << id->name() + << " to " << info.toString() + << " at index " << queueCount_; + } + + queueCount_++; + + return true; +} + +/** + * \brief Read back controls in effect at a sequence number + * \param[in] sequence The sequence number to get controls for + * + * Read back what controls where in effect at a specific sequence number. The + * history is a ring buffer of 16 entries where new and old values coexist. It's + * the callers responsibility to not read too old sequence numbers that have been + * pushed out of the history. + * + * Historic values are evicted by pushing new values onto the queue using + * push(). The max history from the current sequence number that yields valid + * values are thus 16 minus number of controls pushed. + * + * \return The controls at \a sequence number + */ +ControlList DelayedControls::get(uint32_t sequence) +{ + unsigned int index = std::max(0, sequence - maxDelay_); + + ControlList out(device_->controls()); + for (const auto &ctrl : values_) { + const ControlId *id = ctrl.first; + const Info &info = ctrl.second[index]; + + out.set(id->id(), info); + + LOG(RPiDelayedControls, Debug) + << "Reading " << id->name() + << " to " << info.toString() + << " at index " << index; + } + + return out; +} + +/** + * \brief Inform DelayedControls of the start of a new frame + * \param[in] sequence Sequence number of the frame that started + * + * Inform the state machine that a new frame has started and of its sequence + * number. Any user of these helpers is responsible to inform the helper about + * the start of any frame. This can be connected with ease to the start of a + * exposure (SOE) V4L2 event. + */ +void DelayedControls::applyControls(uint32_t sequence) +{ + LOG(RPiDelayedControls, Debug) << "frame " << sequence << " started"; + + /* + * Create control list peeking ahead in the value queue to ensure + * values are set in time to satisfy the sensor delay. + */ + ControlList out(device_->controls()); + for (auto &ctrl : values_) { + const ControlId *id = ctrl.first; + unsigned int delayDiff = maxDelay_ - controlParams_[id].delay; + unsigned int index = std::max(0, writeCount_ - delayDiff); + Info &info = ctrl.second[index]; + + if (info.updated) { + if (controlParams_[id].priorityWrite) { + /* + * This control must be written now, it could + * affect validity of the other controls. + */ + ControlList priority(device_->controls()); + priority.set(id->id(), info); + device_->setControls(&priority); + } else { + /* + * Batch up the list of controls and write them + * at the end of the function. + */ + out.set(id->id(), info); + } + + LOG(RPiDelayedControls, Debug) + << "Setting " << id->name() + << " to " << info.toString() + << " at index " << index; + + /* Done with this update, so mark as completed. */ + info.updated = false; + } + } + + writeCount_ = sequence + 1; + + while (writeCount_ > queueCount_) { + LOG(RPiDelayedControls, Debug) + << "Queue is empty, auto queue no-op."; + push({}); + } + + device_->setControls(&out); +} + +} /* namespace RPi */ + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/raspberrypi/delayed_controls.h b/src/libcamera/pipeline/raspberrypi/delayed_controls.h new file mode 100644 index 000000000000..f7f246482968 --- /dev/null +++ b/src/libcamera/pipeline/raspberrypi/delayed_controls.h @@ -0,0 +1,84 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Raspberry Pi Ltd + * + * delayed_controls.h - Helper to deal with controls that take effect with a delay + * + * Note: This has been forked from the libcamera core implementation. + */ + +#pragma once + +#include +#include + +#include + +namespace libcamera { + +class V4L2Device; + +namespace RPi { + +class DelayedControls +{ +public: + struct ControlParams { + unsigned int delay; + bool priorityWrite; + }; + + DelayedControls(V4L2Device *device, + const std::unordered_map &controlParams); + + void reset(); + + bool push(const ControlList &controls); + ControlList get(uint32_t sequence); + + void applyControls(uint32_t sequence); + +private: + class Info : public ControlValue + { + public: + Info() + : updated(false) + { + } + + Info(const ControlValue &v, bool updated_ = true) + : ControlValue(v), updated(updated_) + { + } + + bool updated; + }; + + static constexpr int listSize = 16; + class ControlRingBuffer : public std::array + { + public: + Info &operator[](unsigned int index) + { + return std::array::operator[](index % listSize); + } + + const Info &operator[](unsigned int index) const + { + return std::array::operator[](index % listSize); + } + }; + + V4L2Device *device_; + std::unordered_map controlParams_; + unsigned int maxDelay_; + + uint32_t queueCount_; + uint32_t writeCount_; + std::unordered_map values_; +}; + +} /* namespace RPi */ + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/raspberrypi/meson.build b/src/libcamera/pipeline/raspberrypi/meson.build index f1a2f5ee72c2..6064a3f00122 100644 --- a/src/libcamera/pipeline/raspberrypi/meson.build +++ b/src/libcamera/pipeline/raspberrypi/meson.build @@ -1,6 +1,7 @@ # SPDX-License-Identifier: CC0-1.0 libcamera_sources += files([ + 'delayed_controls.cpp', 'dma_heaps.cpp', 'raspberrypi.cpp', 'rpi_stream.cpp',