{"id":17089,"url":"https://patchwork.libcamera.org/api/patches/17089/?format=json","web_url":"https://patchwork.libcamera.org/patch/17089/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20220812090838.1784703-4-hanlinchen@chromium.org>","date":"2022-08-12T09:08:34","name":"[libcamera-devel,v2,3/7] libcamera: ipu3: returning partial results","commit_ref":null,"pull_url":null,"state":"new","archived":false,"hash":"55031edef7af459af9dc960744a49d3d667a13cc","submitter":{"id":98,"url":"https://patchwork.libcamera.org/api/people/98/?format=json","name":"Hanlin Chen","email":"hanlinchen@chromium.org"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/17089/mbox/","series":[{"id":3408,"url":"https://patchwork.libcamera.org/api/series/3408/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=3408","date":"2022-08-12T09:08:31","name":"Implement Android Partial Result Featrue","version":2,"mbox":"https://patchwork.libcamera.org/series/3408/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/17089/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/17089/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 68A34C3272\n\tfor ;\n\tFri, 12 Aug 2022 09:08:55 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 352EC63334;\n\tFri, 12 Aug 2022 11:08:55 +0200 (CEST)","from mail-pf1-x435.google.com (mail-pf1-x435.google.com\n\t[IPv6:2607:f8b0:4864:20::435])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id E615563326\n\tfor ;\n\tFri, 12 Aug 2022 11:08:53 +0200 (CEST)","by mail-pf1-x435.google.com with SMTP id h28so412939pfq.11\n\tfor ;\n\tFri, 12 Aug 2022 02:08:53 -0700 (PDT)","from localhost ([2401:fa00:1:17:1705:d284:d114:2e24])\n\tby smtp.gmail.com with UTF8SMTPSA id\n\tr38-20020a632066000000b00419acadde52sm962612pgm.46.2022.08.12.02.08.51\n\t(version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);\n\tFri, 12 Aug 2022 02:08:51 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1660295335;\n\tbh=ngwFPQ51YQid1jL1yJ0V8rBtQg4g+bDYOVeFeyd7ah8=;\n\th=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:\n\tFrom;\n\tb=cxi1G8eJiiqvEFldr8O92Xik0mYJkr7U+ca8x3vXwwSRSU97vy5gAGeuKYsTRh/Su\n\tSmyxzgrkzHjmUlprxfItOnOsEnfLASRM8SfqDNjw8UFQWakwVtNgWTeqXQx3mLZMFL\n\tkKz6gpA8IhLIkDiUZUmzUXn9K6NYc5oUSOIW276TBjANK3vbLeM7UDkC2G6ehgWBLR\n\t5z2zJXibLilTdl/ZW4rY1bQIEzrRc1yoF70fZTTAijq8rH7Tjox6foXnkTFC8RNFK0\n\tCvMUw0cKA8bmRifDhK+s8BFuGw2Gw0cYl5gqJvzqv0JN+kxCrR3ErNfHu9pMzXnhhp\n\t1I2pePg1xJUxA==","v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org;\n\ts=google; \n\th=content-transfer-encoding:mime-version:references:in-reply-to\n\t:message-id:date:subject:cc:to:from:from:to:cc;\n\tbh=EVORGy2cSfxQ44qSix18weyzDNUIZ9Zn8Yw/5wddF80=;\n\tb=CLimVt+7oQwN24pSh8jx19uI1BV18GhZBRBI+JKXR2wwyPutHIcEeRQtDaP6J66wQJ\n\tE6TVRHkdqGAujdgunpy0gStJM5la55gPexM8m3rLHgDf6LzhaxX7PSJLqITnIQii/SuR\n\tqtSiLu+RkvLgoIsP1WOylvA19nOP6evzOy4K8="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=chromium.org\n\theader.i=@chromium.org header.b=\"CLimVt+7\"; \n\tdkim-atps=neutral","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20210112;\n\th=content-transfer-encoding:mime-version:references:in-reply-to\n\t:message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc;\n\tbh=EVORGy2cSfxQ44qSix18weyzDNUIZ9Zn8Yw/5wddF80=;\n\tb=w6j5Ph24T4aP3u9nHjKfFCQPPOLkPIMeJiG4r5vd81SdslWIlFPkCuZgatbhlWgxdX\n\tTy5wudjGK/aEilMhcS9j3ES/koldbdHFBJQVOW4KVY/3GhxH1pUBQYj7roB65PtM5892\n\t1LdwuCNKCfG83OWJZdy6Yjyc0Te92Y2Tue4dhExwXrvWsNfFGqhjroByMCLOq4ncO82k\n\tBSy9y8vQgzIetqy2eQReZMn/5vH/zUdqUaCvrx76G9q5cGMpqdkV0suvQm7XxiIX5+Om\n\tF5pzL9nqFr2tNENzohNZiXBPqEq7wcabV1wcD5Fzmqb1WllxWVitTY1v5oNfs0BHRfu4\n\tSL5g==","X-Gm-Message-State":"ACgBeo3tYbdOFmR3AL1APF1Kj9eQxQQqUUoUnpt+JCHNu11Q5kQscGuj\n\t1omd5m/vZIUSpHb4J0DkBpTh3lOd+5U6PQ==","X-Google-Smtp-Source":"AA6agR4PPhnEvuLriaEutRZj27Ok7j4TDsYyDACLHSZG3M4x+fHwE5/d26VqGj/OR6vDBYh8MaxVEA==","X-Received":"by 2002:aa7:9637:0:b0:52d:b082:cd28 with SMTP id\n\tr23-20020aa79637000000b0052db082cd28mr3011647pfg.43.1660295332179; \n\tFri, 12 Aug 2022 02:08:52 -0700 (PDT)","To":"libcamera-devel@lists.libcamera.org","Date":"Fri, 12 Aug 2022 17:08:34 +0800","Message-Id":"<20220812090838.1784703-4-hanlinchen@chromium.org>","X-Mailer":"git-send-email 2.37.1.595.g718a3a8f04-goog","In-Reply-To":"<20220812090838.1784703-1-hanlinchen@chromium.org>","References":"<20220812090838.1784703-1-hanlinchen@chromium.org>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Subject":"[libcamera-devel] [PATCH v2 3/7] libcamera: ipu3: returning partial\n\tresults","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Han-Lin Chen via libcamera-devel ","Reply-To":"Han-Lin Chen ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Return metadata at earlier stage.\n\nSigned-off-by: Han-Lin Chen \n---\n src/ipa/ipu3/ipu3.cpp | 2 -\n src/libcamera/pipeline/ipu3/ipu3.cpp | 65 ++++++++++++++++++----------\n 2 files changed, 42 insertions(+), 25 deletions(-)","diff":"diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp\nindex 2f6bb672..5e921875 100644\n--- a/src/ipa/ipu3/ipu3.cpp\n+++ b/src/ipa/ipu3/ipu3.cpp\n@@ -594,8 +594,6 @@ void IPAIPU3::processStatsBuffer(const uint32_t frame,\n \n \tctrls.set(controls::ColourTemperature, context_.activeState.awb.temperatureK);\n \n-\tctrls.set(controls::ExposureTime, frameContext.sensor.exposure * lineDuration);\n-\n \t/*\n \t * \\todo The Metadata provides a path to getting extended data\n \t * out to the application. Further data such as a simplifed Histogram\ndiff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp\nindex 4fe52f74..f65db3c8 100644\n--- a/src/libcamera/pipeline/ipu3/ipu3.cpp\n+++ b/src/libcamera/pipeline/ipu3/ipu3.cpp\n@@ -41,6 +41,8 @@ namespace libcamera {\n \n LOG_DEFINE_CATEGORY(IPU3)\n \n+using namespace std::literals::chrono_literals;\n+\n static const ControlInfoMap::Map IPU3Controls = {\n \t{ &controls::draft::PipelineDepth, ControlInfo(2, 3) },\n };\n@@ -86,6 +88,8 @@ public:\n \n \tControlInfoMap ipaControls_;\n \n+\tIPACameraSensorInfo sensorInfo_;\n+\n private:\n \tvoid metadataReady(unsigned int id, const ControlList &metadata);\n \tvoid paramsBufferReady(unsigned int id);\n@@ -993,7 +997,7 @@ int PipelineHandlerIPU3::initControls(IPU3CameraData *data)\n int PipelineHandlerIPU3::updateControls(IPU3CameraData *data)\n {\n \tCameraSensor *sensor = data->cio2_.sensor();\n-\tIPACameraSensorInfo sensorInfo{};\n+\tIPACameraSensorInfo &sensorInfo = data->sensorInfo_;\n \n \tint ret = sensor->sensorInfo(&sensorInfo);\n \tif (ret)\n@@ -1300,7 +1304,7 @@ void IPU3CameraData::metadataReady(unsigned int id, const ControlList &metadata)\n \t\treturn;\n \n \tRequest *request = info->request;\n-\trequest->metadata().merge(metadata);\n+\tpipe()->completeMetadata(request, metadata);\n \n \tinfo->metadataProcessed = true;\n \tif (frameInfos_.tryComplete(info))\n@@ -1324,16 +1328,8 @@ void IPU3CameraData::imguOutputBufferReady(FrameBuffer *buffer)\n \t\treturn;\n \n \tRequest *request = info->request;\n-\n \tpipe()->completeBuffer(request, buffer);\n \n-\trequest->metadata().set(controls::draft::PipelineDepth, 3);\n-\t/* \\todo Actually apply the scaler crop region to the ImgU. */\n-\tconst auto &scalerCrop = request->controls().get(controls::ScalerCrop);\n-\tif (scalerCrop)\n-\t\tcropRegion_ = *scalerCrop;\n-\trequest->metadata().set(controls::ScalerCrop, cropRegion_);\n-\n \tif (frameInfos_.tryComplete(info))\n \t\tpipe()->completeRequest(request);\n }\n@@ -1372,13 +1368,35 @@ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer)\n \t * \\todo The sensor timestamp should be better estimated by connecting\n \t * to the V4L2Device::frameStart signal.\n \t */\n-\trequest->metadata().set(controls::SensorTimestamp,\n-\t\t\t\tbuffer->metadata().timestamp);\n \n-\tinfo->effectiveSensorControls = delayedCtrls_->get(buffer->metadata().sequence);\n+\tControlList &sensorCtrl = info->effectiveSensorControls;\n+\tsensorCtrl = delayedCtrls_->get(buffer->metadata().sequence);\n \n-\tif (request->findBuffer(&rawStream_))\n+\tControlList metadata;\n+\tmetadata.set(controls::draft::PipelineDepth, 3);\n+\tmetadata.set(controls::SensorTimestamp, buffer->metadata().timestamp);\n+\n+\t/* \\todo Actually apply the scaler crop region to the ImgU. */\n+\tconst auto &scalerCrop = request->controls().get(controls::ScalerCrop);\n+\tif (scalerCrop)\n+\t\tcropRegion_ = *scalerCrop;\n+\tmetadata.set(controls::ScalerCrop, cropRegion_);\n+\n+\t/*\n+\t * \\todo Move reporting the raw frame related metadata from the IPA\n+\t * to here: AnalogueGain, FrameDuration, etc.\n+\t */\n+\tutils::Duration lineDuration = sensorInfo_.lineLength * 1.0s / sensorInfo_.pixelRate;\n+\tif (sensorCtrl.contains(V4L2_CID_EXPOSURE)) {\n+\t\tint32_t exposureTime = sensorCtrl.get(V4L2_CID_EXPOSURE).get();\n+\t\tmetadata.set(controls::ExposureTime, exposureTime * lineDuration.get());\n+\t}\n+\n+\tpipe()->completeMetadata(request, metadata);\n+\n+\tif (request->findBuffer(&rawStream_)) {\n \t\tpipe()->completeBuffer(request, buffer);\n+\t}\n \n \tipa_->fillParamsBuffer(info->id, info->paramBuffer->cookie());\n }\n@@ -1455,20 +1473,21 @@ void IPU3CameraData::frameStart(uint32_t sequence)\n \tRequest *request = processingRequests_.front();\n \tprocessingRequests_.pop();\n \n-\tconst auto &testPatternMode = request->controls().get(controls::draft::TestPatternMode);\n-\tif (!testPatternMode)\n-\t\treturn;\n+\tint32_t testPatternMode = controls::draft::TestPatternModeOff;\n+\tconst auto &testPatternControl = request->controls().get(controls::draft::TestPatternMode);\n+\tif (testPatternControl)\n+\t\ttestPatternMode = *testPatternControl;\n \n \tint ret = cio2_.sensor()->setTestPatternMode(\n-\t\tstatic_cast(*testPatternMode));\n+\t\tstatic_cast(testPatternMode));\n \tif (ret) {\n-\t\tLOG(IPU3, Error) << \"Failed to set test pattern mode: \"\n-\t\t\t\t << ret;\n-\t\treturn;\n+\t\tLOG(IPU3, Error) << \"Failed to set test pattern mode: \" << ret;\n+\t\ttestPatternMode = controls::draft::TestPatternModeOff;\n \t}\n \n-\trequest->metadata().set(controls::draft::TestPatternMode,\n-\t\t\t\t*testPatternMode);\n+\tControlList metadata;\n+\tmetadata.set(controls::draft::TestPatternMode, testPatternMode);\n+\tpipe()->completeMetadata(request, metadata);\n }\n \n REGISTER_PIPELINE_HANDLER(PipelineHandlerIPU3)\n","prefixes":["libcamera-devel","v2","3/7"]}