From patchwork Tue Nov 15 09:07:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 17799 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9F849C3285 for ; Tue, 15 Nov 2022 09:08:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 57C7563098; Tue, 15 Nov 2022 10:08:11 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1668503291; bh=BcdzumNDkt3pVKjdZHPGzaZultlq4a0Ap82bjymxZEU=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=hC5E9CVCacPRNMwZIUKJVM/gaOHj3aHhqFyHr0i1hhk119qhCy9xltWPmx8BZDTWX daFYgFnX3WoMCya1aVUUVTdDsxo8zq8g4lO2fgXVvBPYg8DDCWh5BGXkY14oq+zZ3P yLrbfgyFtRYcjZ6S5v1umoh03qEmQSZu2qeFSl/f1FZG9QiHjh0s2Tl8Amq98km6eF EsrTH1rRIspRZVF2XuftoXN2ScMwxcvty5HXhqFg4sm8GvF1SqihngjIJ5ZCP92itf ZB5jkNNFbX7ZD0H+6swMNrRLOAFDEAt9RpDKba2P5w0uJL7N45wFMGMArMcuVW1xQU 8cQlUcjs2f4zg== Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7A4CF63088 for ; Tue, 15 Nov 2022 10:08:06 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="iITqZCyQ"; dkim-atps=neutral Received: by mail-wr1-x42e.google.com with SMTP id d9so18381855wrm.13 for ; Tue, 15 Nov 2022 01:08:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xMING90SQKUvR7yzkp4yjNn91MarCln/xYFvrXKiQIk=; b=iITqZCyQ6DreIMGwwf4ddHBkzRFIb2NpOdCI1X3vZiKhL9bpT8r8asnXU7xcLdI5ef rly+ZQ6mSxifR4sFxYsLqEggpGYWpJvNUhHcfCt20/RhMvjhZJfko615dKusZRuCCrFK 2TcDysUnA8WSu5cxECTV5OicalVVnNIC/Gp1CK0xVNJks6v4fUIAGkGL1AFBOZnRQ4DS w1MDX9JniatMuZztEAvkKGdPghULvKDTMmCgHoelutcADKOAemotP8JuN3izrD/y3VuI El23BCUDYBNdmM6irUKhsyms/uEEw8qJjA2F+5cLqZSGEauGYDXPfD29Kcr1NflY3+mW mFRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xMING90SQKUvR7yzkp4yjNn91MarCln/xYFvrXKiQIk=; b=i11eKgyb1z8JFhOcacsFbT6jyAvBS3LDzARxmGoJfp7EZG6tJZANX0RjVEh9uxYKNM Hkq7QVJBDIHWxRS6vaIYEKlT0hOiNuasd/+PE74xtiHtq8qUwL/ugmLCRJ1eo1qMRkmY GgNuMRTl092BmgBG/4wqE9HNNvfqw1r2zKS1X1WYRWAIiIf7ydwoWvNEYELRrQh3gTmk 3eTAcxeA0jf/YMC1/Bh9LDJ0pYYA92WOAbFjx0O9fTE0nIadTu4WMY1Vp4znwpHqbLyv F077KxlGbMlZj1NOZkuQi7nxhoqiepx2m3QNIAQmleuwsgbvOo+FTMqcsf2ZNL3PuT1D RQNg== X-Gm-Message-State: ANoB5pkKPCzsOqN0tbmc4zWNgtdmOFZkY41+P/bay/eWNSrmQxqRgtmS ETQo/RAxMFTCoMd5lOnpCps+Y2emM5SIKQ== X-Google-Smtp-Source: AA0mqf70agn1WuqZ9veoYnw28j3aswQyI8s63UraIJoqCPKw9NHTg76ZoYjdnb+te4RGPHwCbignUw== X-Received: by 2002:adf:f18c:0:b0:236:862b:c457 with SMTP id h12-20020adff18c000000b00236862bc457mr9703951wro.154.1668503285698; Tue, 15 Nov 2022 01:08:05 -0800 (PST) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id p25-20020a05600c1d9900b003c64c186206sm15143696wms.16.2022.11.15.01.08.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Nov 2022 01:08:05 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Tue, 15 Nov 2022 09:07:54 +0000 Message-Id: <20221115090755.2921-8-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221115090755.2921-1-naush@raspberrypi.com> References: <20221115090755.2921-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v6 7/8] pipeline: ipa: raspberrypi: Use IPA cookies X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Pass an IPA cookie from the pipeline handler to the IPA and eventually back to the pipeline handler through the setDelayedControls signal. This cookie is used to index the RPiController::Metadata object to be used for the frame. The IPA cookie is then returned from DelayedControls when the frame with the applied controls has been returned from the sensor, and eventually passed back to the IPA from the signalIspPrepare signal. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Tested-by: David Plowman --- include/libcamera/ipa/raspberrypi.mojom | 6 ++- src/ipa/raspberrypi/raspberrypi.cpp | 42 ++++++++++--------- .../pipeline/raspberrypi/raspberrypi.cpp | 16 ++++--- 3 files changed, 37 insertions(+), 27 deletions(-) diff --git a/include/libcamera/ipa/raspberrypi.mojom b/include/libcamera/ipa/raspberrypi.mojom index 40f78d9e3b3f..325d2d855bc0 100644 --- a/include/libcamera/ipa/raspberrypi.mojom +++ b/include/libcamera/ipa/raspberrypi.mojom @@ -37,6 +37,8 @@ struct ISPConfig { uint32 bayerBufferId; bool embeddedBufferPresent; libcamera.ControlList controls; + uint32 ipaContext; + uint32 delayContext; }; struct IPAConfig { @@ -127,7 +129,7 @@ interface IPARPiInterface { */ unmapBuffers(array ids); - [async] signalStatReady(uint32 bufferId); + [async] signalStatReady(uint32 bufferId, uint32 ipaContext); [async] signalQueueRequest(libcamera.ControlList controls); [async] signalIspPrepare(ISPConfig data); }; @@ -137,5 +139,5 @@ interface IPARPiEventInterface { runIsp(uint32 bufferId); embeddedComplete(uint32 bufferId); setIspControls(libcamera.ControlList controls); - setDelayedControls(libcamera.ControlList controls); + setDelayedControls(libcamera.ControlList controls, uint32 delayContext); }; diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 799a4fe70000..194171a8bc96 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -126,7 +126,7 @@ public: ControlList *controls, IPAConfigResult *result) override; void mapBuffers(const std::vector &buffers) override; void unmapBuffers(const std::vector &ids) override; - void signalStatReady(const uint32_t bufferId) override; + void signalStatReady(const uint32_t bufferId, uint32_t ipaContext) override; void signalQueueRequest(const ControlList &controls) override; void signalIspPrepare(const ISPConfig &data) override; @@ -137,9 +137,9 @@ private: void queueRequest(const ControlList &controls); void returnEmbeddedBuffer(unsigned int bufferId); void prepareISP(const ISPConfig &data); - void reportMetadata(); - void fillDeviceStatus(const ControlList &sensorControls); - void processStats(unsigned int bufferId); + void reportMetadata(unsigned int ipaContext); + void fillDeviceStatus(const ControlList &sensorControls, unsigned int ipaContext); + void processStats(unsigned int bufferId, unsigned int ipaContext); void applyFrameDurations(Duration minFrameDuration, Duration maxFrameDuration); void applyAGC(const struct AgcStatus *agcStatus, ControlList &ctrls); void applyAWB(const struct AwbStatus *awbStatus, ControlList &ctrls); @@ -509,14 +509,16 @@ void IPARPi::unmapBuffers(const std::vector &ids) } } -void IPARPi::signalStatReady(uint32_t bufferId) +void IPARPi::signalStatReady(uint32_t bufferId, uint32_t ipaContext) { + unsigned int context = ipaContext % rpiMetadata_.size(); + if (++checkCount_ != frameCount_) /* assert here? */ LOG(IPARPI, Error) << "WARNING: Prepare/Process mismatch!!!"; if (processPending_ && frameCount_ > mistrustCount_) - processStats(bufferId); + processStats(bufferId, context); - reportMetadata(); + reportMetadata(context); statsMetadataComplete.emit(bufferId & MaskID, libcameraMetadata_); } @@ -540,9 +542,9 @@ void IPARPi::signalIspPrepare(const ISPConfig &data) runIsp.emit(data.bayerBufferId & MaskID); } -void IPARPi::reportMetadata() +void IPARPi::reportMetadata(unsigned int ipaContext) { - RPiController::Metadata &rpiMetadata = rpiMetadata_[0]; + RPiController::Metadata &rpiMetadata = rpiMetadata_[ipaContext]; std::unique_lock lock(rpiMetadata); /* @@ -1009,12 +1011,12 @@ void IPARPi::returnEmbeddedBuffer(unsigned int bufferId) void IPARPi::prepareISP(const ISPConfig &data) { int64_t frameTimestamp = data.controls.get(controls::SensorTimestamp).value_or(0); - RPiController::Metadata lastMetadata; - RPiController::Metadata &rpiMetadata = rpiMetadata_[0]; + unsigned int ipaContext = data.ipaContext % rpiMetadata_.size(); + RPiController::Metadata &rpiMetadata = rpiMetadata_[ipaContext]; Span embeddedBuffer; - lastMetadata = std::move(rpiMetadata); - fillDeviceStatus(data.controls); + rpiMetadata.clear(); + fillDeviceStatus(data.controls, ipaContext); if (data.embeddedBufferPresent) { /* @@ -1046,7 +1048,9 @@ void IPARPi::prepareISP(const ISPConfig &data) * current frame, or any other bits of metadata that were added * in helper_->Prepare(). */ - rpiMetadata.merge(lastMetadata); + RPiController::Metadata &lastMetadata = + rpiMetadata_[ipaContext ? ipaContext : rpiMetadata_.size()]; + rpiMetadata.mergeCopy(lastMetadata); processPending_ = false; return; } @@ -1105,7 +1109,7 @@ void IPARPi::prepareISP(const ISPConfig &data) setIspControls.emit(ctrls); } -void IPARPi::fillDeviceStatus(const ControlList &sensorControls) +void IPARPi::fillDeviceStatus(const ControlList &sensorControls, unsigned int ipaContext) { DeviceStatus deviceStatus = {}; @@ -1121,12 +1125,12 @@ void IPARPi::fillDeviceStatus(const ControlList &sensorControls) LOG(IPARPI, Debug) << "Metadata - " << deviceStatus; - rpiMetadata_[0].set("device.status", deviceStatus); + rpiMetadata_[ipaContext].set("device.status", deviceStatus); } -void IPARPi::processStats(unsigned int bufferId) +void IPARPi::processStats(unsigned int bufferId, unsigned int ipaContext) { - RPiController::Metadata &rpiMetadata = rpiMetadata_[0]; + RPiController::Metadata &rpiMetadata = rpiMetadata_[ipaContext]; auto it = buffers_.find(bufferId); if (it == buffers_.end()) { @@ -1145,7 +1149,7 @@ void IPARPi::processStats(unsigned int bufferId) ControlList ctrls(sensorCtrls_); applyAGC(&agcStatus, ctrls); - setDelayedControls.emit(ctrls); + setDelayedControls.emit(ctrls, ipaContext); } } diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 2c12ed1fba95..6f560fc2782e 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -206,7 +206,7 @@ public: void runIsp(uint32_t bufferId); void embeddedComplete(uint32_t bufferId); void setIspControls(const ControlList &controls); - void setDelayedControls(const ControlList &controls); + void setDelayedControls(const ControlList &controls, uint32_t delayContext); void setSensorControls(ControlList &controls); void unicamTimeout(); @@ -262,6 +262,7 @@ public: struct BayerFrame { FrameBuffer *buffer; ControlList controls; + unsigned int delayContext; }; std::queue bayerQueue_; @@ -1792,9 +1793,9 @@ void RPiCameraData::setIspControls(const ControlList &controls) handleState(); } -void RPiCameraData::setDelayedControls(const ControlList &controls) +void RPiCameraData::setDelayedControls(const ControlList &controls, uint32_t delayContext) { - if (!delayedCtrls_->push(controls, 0)) + if (!delayedCtrls_->push(controls, delayContext)) LOG(RPI, Error) << "V4L2 DelayedControl set failed"; handleState(); } @@ -1867,13 +1868,13 @@ void RPiCameraData::unicamBufferDequeue(FrameBuffer *buffer) * Lookup the sensor controls used for this frame sequence from * DelayedControl and queue them along with the frame buffer. */ - auto [ctrl, cookie] = delayedCtrls_->get(buffer->metadata().sequence); + auto [ctrl, delayContext] = delayedCtrls_->get(buffer->metadata().sequence); /* * Add the frame timestamp to the ControlList for the IPA to use * as it does not receive the FrameBuffer object. */ ctrl.set(controls::SensorTimestamp, buffer->metadata().timestamp); - bayerQueue_.push({ buffer, std::move(ctrl) }); + bayerQueue_.push({ buffer, std::move(ctrl), delayContext }); } else { embeddedQueue_.push(buffer); } @@ -1923,7 +1924,8 @@ void RPiCameraData::ispOutputDequeue(FrameBuffer *buffer) * application until after the IPA signals so. */ if (stream == &isp_[Isp::Stats]) { - ipa_->signalStatReady(ipa::RPi::MaskStats | static_cast(index)); + ipa_->signalStatReady(ipa::RPi::MaskStats | static_cast(index), + requestQueue_.front()->sequence()); } else { /* Any other ISP output can be handed back to the application now. */ handleStreamBuffer(buffer, stream); @@ -2168,6 +2170,8 @@ void RPiCameraData::tryRunPipeline() ipa::RPi::ISPConfig ispPrepare; ispPrepare.bayerBufferId = ipa::RPi::MaskBayerData | bayerId; ispPrepare.controls = std::move(bayerFrame.controls); + ispPrepare.ipaContext = request->sequence(); + ispPrepare.delayContext = bayerFrame.delayContext; if (embeddedBuffer) { unsigned int embeddedId = unicam_[Unicam::Embedded].getBufferId(embeddedBuffer);