From patchwork Wed Nov 27 09:25:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 22112 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 712C5C3213 for ; Wed, 27 Nov 2024 09:26:46 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D6C0B660C3; Wed, 27 Nov 2024 10:26:45 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="jBFo3plp"; dkim-atps=neutral Received: from mail-ot1-x335.google.com (mail-ot1-x335.google.com [IPv6:2607:f8b0:4864:20::335]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 14DAE660AD for ; Wed, 27 Nov 2024 10:26:41 +0100 (CET) Received: by mail-ot1-x335.google.com with SMTP id 46e09a7af769-71d42a79a14so1652478a34.3 for ; Wed, 27 Nov 2024 01:26:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1732699599; x=1733304399; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=W9hw/h8DkwpQ2JaAwAse5GJuP66gmF3X+z6FR3Gk7gc=; b=jBFo3plpHI1hVtllOfs4k5J8lyHGdWlfJe8vG3PYZQ7jOg1d4ooyVX47FQD4UZ0Les bf6yOP0sxEhnlAvQL9j/bppbvl+jNTHVEcY9qxg8bhqG33EasY6p0d8IRJdXY1nJXKeK EzlsnW4Ow1/meumLAn958CK4Zg66GDnG2iA4Y= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732699599; x=1733304399; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=W9hw/h8DkwpQ2JaAwAse5GJuP66gmF3X+z6FR3Gk7gc=; b=TH2DpyrLu06StJiVOtJYJu28iQRlsR6OHsZYbfnpXfWbwOdky+yxl1kMKl3vILc1X+ Wusim/j1mQlCJRahOFDlgkH6oLx+268EFlD4Zgw5NeVYlA1jPsnkw2syVkZdmfVbfVkb GIQIMMtltGz7fB7ZY5yCzm15aogw4eXlpicJvyajLuWRRPnfCA0kAKNG6QkTwFCyjigQ TSmXDWXT/FOwBF6SzMrzkqTGPyRuKs5jW4katJEWs5P9b6yM3lF2GJtWjECFoE0Ydyon 2p47R6htjmuecZSmayN7xmqLZbyFYXkKottH84RqfzMIhZ2rOW6xV4bKJpHZX68kT+Ny n75w== X-Gm-Message-State: AOJu0YydG6Zk9YbUEnErEq5Mi78UMydWzJYNNfOhxay+zIt0JYxUUQFh cV7D/jjbbRXFZXXDvibjLc54Wytd8OG5G0dTsNEUZuKBX5ekJiZmeWI/1afQavonf+JzHtBTPR8 = X-Gm-Gg: ASbGncvFY50wSq1HfIhX+3UUyZn1mirC/BmvKVYPWsNDOUtcj1WTtjPT73FvquH6NOf B0J+rAGD03Uz6ONzGmXSx3AGLhBHmK65OTKfs1uhcdtuosRYejV6euRl5TnMZ/AoLNmX3U402H3 f6+OW6v/FqsjjsQaj8X6MKXvYsTn0I67YIxCj51XVjGirTfh1B9hkeNTW2LVm2VnxNdi9OpLCVz xiC+gl8ivMrQdXO5C5TEDSQJhXuMWpgE1UZFpJSp7GBTQCGwVdP+7VoNTq3NX6tDhyvXPAO9qlu /MNiJAUbL4RWVS72EBlBjkUBz0IIUSpXF3GYsJy0jil2REmehg/FnA== X-Google-Smtp-Source: AGHT+IGX39X0MMD9q3e5WIa5y/wqcLhDtSLtr/WxnTf/+Ultwg7Caumei5Wdr09Df7d+iUNRbB17qw== X-Received: by 2002:a05:6830:6dc6:b0:718:17b6:56f5 with SMTP id 46e09a7af769-71d65d01867mr2225754a34.29.1732699599128; Wed, 27 Nov 2024 01:26:39 -0800 (PST) Received: from chenghaoyang-low.c.googlers.com.com (27.247.221.35.bc.googleusercontent.com. [35.221.247.27]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7fbcbfc41f9sm8693027a12.8.2024.11.27.01.26.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Nov 2024 01:26:38 -0800 (PST) From: Harvey Yang To: libcamera-devel@lists.libcamera.org Cc: Han-Lin Chen , Harvey Yang Subject: [PATCH v2 1/9] libcamera: Camera: Add signals for completion of metadata as a partial result Date: Wed, 27 Nov 2024 09:25:51 +0000 Message-ID: <20241127092632.3145984-2-chenghaoyang@chromium.org> X-Mailer: git-send-email 2.47.0.338.g60cca15819-goog In-Reply-To: <20241127092632.3145984-1-chenghaoyang@chromium.org> References: <20241127092632.3145984-1-chenghaoyang@chromium.org> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Han-Lin Chen Allows pipeline handler to signal metadata completion by adding the following signals to Camera: Signal metadataAvailable; Together with the bufferCompleted signal, the pipeline handler is allowed to return buffers and partial metadata at any stage of processing. Signed-off-by: Han-Lin Chen Co-developed-by: Harvey Yang --- include/libcamera/camera.h | 1 + include/libcamera/internal/pipeline_handler.h | 1 + include/libcamera/internal/request.h | 4 ++ include/libcamera/request.h | 1 + src/libcamera/camera.cpp | 6 +++ src/libcamera/pipeline_handler.cpp | 41 +++++++++++++++++++ src/libcamera/request.cpp | 21 ++++++++++ 7 files changed, 75 insertions(+) diff --git a/include/libcamera/camera.h b/include/libcamera/camera.h index 94cee7bd8..eb7cdf81b 100644 --- a/include/libcamera/camera.h +++ b/include/libcamera/camera.h @@ -122,6 +122,7 @@ public: const std::string &id() const; + Signal metadataAvailable; Signal bufferCompleted; Signal requestCompleted; Signal<> disconnected; diff --git a/include/libcamera/internal/pipeline_handler.h b/include/libcamera/internal/pipeline_handler.h index fb28a18d0..6c6cad66f 100644 --- a/include/libcamera/internal/pipeline_handler.h +++ b/include/libcamera/internal/pipeline_handler.h @@ -58,6 +58,7 @@ public: void registerRequest(Request *request); void queueRequest(Request *request); + void completeMetadata(Request *request, const ControlList &metadata); bool completeBuffer(Request *request, FrameBuffer *buffer); void completeRequest(Request *request); void cancelRequest(Request *request); diff --git a/include/libcamera/internal/request.h b/include/libcamera/internal/request.h index 4e7d05b1e..286cd9d76 100644 --- a/include/libcamera/internal/request.h +++ b/include/libcamera/internal/request.h @@ -43,6 +43,8 @@ public: void prepare(std::chrono::milliseconds timeout = 0ms); Signal<> prepared; + ControlList addCompletedMetadata(const ControlList &metadata); + private: friend class PipelineHandler; friend std::ostream &operator<<(std::ostream &out, const Request &r); @@ -60,6 +62,8 @@ private: std::unordered_set pending_; std::map> notifiers_; std::unique_ptr timer_; + + std::unordered_set completedMetadata_; }; } /* namespace libcamera */ diff --git a/include/libcamera/request.h b/include/libcamera/request.h index e214a9d13..2c78d9bb4 100644 --- a/include/libcamera/request.h +++ b/include/libcamera/request.h @@ -12,6 +12,7 @@ #include #include #include +#include #include #include diff --git a/src/libcamera/camera.cpp b/src/libcamera/camera.cpp index 7507e9dda..22484721a 100644 --- a/src/libcamera/camera.cpp +++ b/src/libcamera/camera.cpp @@ -892,6 +892,12 @@ const std::string &Camera::id() const * completed */ +/** + * \var Camera::metadataAvailable + * \brief Signal emitted when some metadata for a request is available as a + * partial result + */ + /** * \var Camera::requestCompleted * \brief Signal emitted when a request queued to the camera has completed diff --git a/src/libcamera/pipeline_handler.cpp b/src/libcamera/pipeline_handler.cpp index 991b06f26..4ba96cfa2 100644 --- a/src/libcamera/pipeline_handler.cpp +++ b/src/libcamera/pipeline_handler.cpp @@ -531,6 +531,32 @@ bool PipelineHandler::completeBuffer(Request *request, FrameBuffer *buffer) return request->_d()->completeBuffer(buffer); } +/** + * \brief Complete part of metadata for a request + * \param[in] request The request the metadata belongs to + * \param[in] metadata The partial metadata that has completed + * + * This function could be called by pipeline handlers to signal availability of + * \a metadata before \a request completes. Early metadata completion allows to + * notify applications about the availability of a partial metadata buffer + * before the associated Request has completed. + * + * A metadata key is expected to be completed at most once. If it's completed + * more than once, the key will be dropped since the second time. + * + * \context This function shall be called from the CameraManager thread. + */ +void PipelineHandler::completeMetadata(Request *request, const ControlList &metadata) +{ + const ControlList validMetadata = request->_d()->addCompletedMetadata(metadata); + if (!validMetadata.empty()) { + request->metadata().merge(validMetadata); + + Camera *camera = request->_d()->camera(); + camera->metadataAvailable.emit(request, validMetadata); + } +} + /** * \brief Signal request completion * \param[in] request The request that has completed @@ -553,6 +579,21 @@ void PipelineHandler::completeRequest(Request *request) Camera::Private *data = camera->_d(); + /* + * Collect metadata which is not yet completed by the Camera, and + * create one partial result to cover the missing metadata before + * completing the whole request. This guarantees the aggregation of + * metadata in completed partial results equals to the global metadata + * in the request. + * + * \todo: Forbid merging metadata into request.metadata() directly and + * force calling completeMetadata() to report metadata. + */ + const ControlList validMetadata = request->_d()->addCompletedMetadata( + request->metadata()); + if (!validMetadata.empty()) + camera->metadataAvailable.emit(request, validMetadata); + while (!data->queuedRequests_.empty()) { Request *req = data->queuedRequests_.front(); if (req->status() == Request::RequestPending) diff --git a/src/libcamera/request.cpp b/src/libcamera/request.cpp index 8c56ed30d..ae5cdeb19 100644 --- a/src/libcamera/request.cpp +++ b/src/libcamera/request.cpp @@ -178,6 +178,7 @@ void Request::Private::reset() pending_.clear(); notifiers_.clear(); timer_.reset(); + completedMetadata_.clear(); } /* @@ -270,6 +271,26 @@ void Request::Private::prepare(std::chrono::milliseconds timeout) * if they have failed preparing. */ +/** + * \brief Add completed metadata, as a partial result + * \param[in] metadata The metadata completed + * + * Request will record the entries that has been sent to the application, to + * prevent duplicated controls. + * + * \return ControlList that hasn't been completed before + */ +ControlList Request::Private::addCompletedMetadata(const ControlList &metadata) +{ + ControlList resultMetadata; + for (auto &[id, value] : metadata) { + if (!completedMetadata_.count(id)) + resultMetadata.set(id, value); + } + + return resultMetadata; +} + void Request::Private::notifierActivated(FrameBuffer *buffer) { /* Close the fence if successfully signalled. */