From patchwork Fri Aug 12 09:08:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanlin Chen X-Patchwork-Id: 17089 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 68A34C3272 for ; Fri, 12 Aug 2022 09:08:55 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 352EC63334; Fri, 12 Aug 2022 11:08:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1660295335; bh=ngwFPQ51YQid1jL1yJ0V8rBtQg4g+bDYOVeFeyd7ah8=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=cxi1G8eJiiqvEFldr8O92Xik0mYJkr7U+ca8x3vXwwSRSU97vy5gAGeuKYsTRh/Su SmyxzgrkzHjmUlprxfItOnOsEnfLASRM8SfqDNjw8UFQWakwVtNgWTeqXQx3mLZMFL kKz6gpA8IhLIkDiUZUmzUXn9K6NYc5oUSOIW276TBjANK3vbLeM7UDkC2G6ehgWBLR 5z2zJXibLilTdl/ZW4rY1bQIEzrRc1yoF70fZTTAijq8rH7Tjox6foXnkTFC8RNFK0 CvMUw0cKA8bmRifDhK+s8BFuGw2Gw0cYl5gqJvzqv0JN+kxCrR3ErNfHu9pMzXnhhp 1I2pePg1xJUxA== Received: from mail-pf1-x435.google.com (mail-pf1-x435.google.com [IPv6:2607:f8b0:4864:20::435]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E615563326 for ; Fri, 12 Aug 2022 11:08:53 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="CLimVt+7"; dkim-atps=neutral Received: by mail-pf1-x435.google.com with SMTP id h28so412939pfq.11 for ; Fri, 12 Aug 2022 02:08:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=EVORGy2cSfxQ44qSix18weyzDNUIZ9Zn8Yw/5wddF80=; b=CLimVt+7oQwN24pSh8jx19uI1BV18GhZBRBI+JKXR2wwyPutHIcEeRQtDaP6J66wQJ E6TVRHkdqGAujdgunpy0gStJM5la55gPexM8m3rLHgDf6LzhaxX7PSJLqITnIQii/SuR qtSiLu+RkvLgoIsP1WOylvA19nOP6evzOy4K8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=EVORGy2cSfxQ44qSix18weyzDNUIZ9Zn8Yw/5wddF80=; b=w6j5Ph24T4aP3u9nHjKfFCQPPOLkPIMeJiG4r5vd81SdslWIlFPkCuZgatbhlWgxdX Ty5wudjGK/aEilMhcS9j3ES/koldbdHFBJQVOW4KVY/3GhxH1pUBQYj7roB65PtM5892 1LdwuCNKCfG83OWJZdy6Yjyc0Te92Y2Tue4dhExwXrvWsNfFGqhjroByMCLOq4ncO82k BSy9y8vQgzIetqy2eQReZMn/5vH/zUdqUaCvrx76G9q5cGMpqdkV0suvQm7XxiIX5+Om F5pzL9nqFr2tNENzohNZiXBPqEq7wcabV1wcD5Fzmqb1WllxWVitTY1v5oNfs0BHRfu4 SL5g== X-Gm-Message-State: ACgBeo3tYbdOFmR3AL1APF1Kj9eQxQQqUUoUnpt+JCHNu11Q5kQscGuj 1omd5m/vZIUSpHb4J0DkBpTh3lOd+5U6PQ== X-Google-Smtp-Source: AA6agR4PPhnEvuLriaEutRZj27Ok7j4TDsYyDACLHSZG3M4x+fHwE5/d26VqGj/OR6vDBYh8MaxVEA== X-Received: by 2002:aa7:9637:0:b0:52d:b082:cd28 with SMTP id r23-20020aa79637000000b0052db082cd28mr3011647pfg.43.1660295332179; Fri, 12 Aug 2022 02:08:52 -0700 (PDT) Received: from localhost ([2401:fa00:1:17:1705:d284:d114:2e24]) by smtp.gmail.com with UTF8SMTPSA id r38-20020a632066000000b00419acadde52sm962612pgm.46.2022.08.12.02.08.51 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 12 Aug 2022 02:08:51 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Fri, 12 Aug 2022 17:08:34 +0800 Message-Id: <20220812090838.1784703-4-hanlinchen@chromium.org> X-Mailer: git-send-email 2.37.1.595.g718a3a8f04-goog In-Reply-To: <20220812090838.1784703-1-hanlinchen@chromium.org> References: <20220812090838.1784703-1-hanlinchen@chromium.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 3/7] libcamera: ipu3: returning partial results X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Han-Lin Chen via libcamera-devel From: Hanlin Chen Reply-To: Han-Lin Chen Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Return metadata at earlier stage. Signed-off-by: Han-Lin Chen --- src/ipa/ipu3/ipu3.cpp | 2 - src/libcamera/pipeline/ipu3/ipu3.cpp | 65 ++++++++++++++++++---------- 2 files changed, 42 insertions(+), 25 deletions(-) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 2f6bb672..5e921875 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -594,8 +594,6 @@ void IPAIPU3::processStatsBuffer(const uint32_t frame, ctrls.set(controls::ColourTemperature, context_.activeState.awb.temperatureK); - ctrls.set(controls::ExposureTime, frameContext.sensor.exposure * lineDuration); - /* * \todo The Metadata provides a path to getting extended data * out to the application. Further data such as a simplifed Histogram diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 4fe52f74..f65db3c8 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -41,6 +41,8 @@ namespace libcamera { LOG_DEFINE_CATEGORY(IPU3) +using namespace std::literals::chrono_literals; + static const ControlInfoMap::Map IPU3Controls = { { &controls::draft::PipelineDepth, ControlInfo(2, 3) }, }; @@ -86,6 +88,8 @@ public: ControlInfoMap ipaControls_; + IPACameraSensorInfo sensorInfo_; + private: void metadataReady(unsigned int id, const ControlList &metadata); void paramsBufferReady(unsigned int id); @@ -993,7 +997,7 @@ int PipelineHandlerIPU3::initControls(IPU3CameraData *data) int PipelineHandlerIPU3::updateControls(IPU3CameraData *data) { CameraSensor *sensor = data->cio2_.sensor(); - IPACameraSensorInfo sensorInfo{}; + IPACameraSensorInfo &sensorInfo = data->sensorInfo_; int ret = sensor->sensorInfo(&sensorInfo); if (ret) @@ -1300,7 +1304,7 @@ void IPU3CameraData::metadataReady(unsigned int id, const ControlList &metadata) return; Request *request = info->request; - request->metadata().merge(metadata); + pipe()->completeMetadata(request, metadata); info->metadataProcessed = true; if (frameInfos_.tryComplete(info)) @@ -1324,16 +1328,8 @@ void IPU3CameraData::imguOutputBufferReady(FrameBuffer *buffer) return; Request *request = info->request; - pipe()->completeBuffer(request, buffer); - request->metadata().set(controls::draft::PipelineDepth, 3); - /* \todo Actually apply the scaler crop region to the ImgU. */ - const auto &scalerCrop = request->controls().get(controls::ScalerCrop); - if (scalerCrop) - cropRegion_ = *scalerCrop; - request->metadata().set(controls::ScalerCrop, cropRegion_); - if (frameInfos_.tryComplete(info)) pipe()->completeRequest(request); } @@ -1372,13 +1368,35 @@ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer) * \todo The sensor timestamp should be better estimated by connecting * to the V4L2Device::frameStart signal. */ - request->metadata().set(controls::SensorTimestamp, - buffer->metadata().timestamp); - info->effectiveSensorControls = delayedCtrls_->get(buffer->metadata().sequence); + ControlList &sensorCtrl = info->effectiveSensorControls; + sensorCtrl = delayedCtrls_->get(buffer->metadata().sequence); - if (request->findBuffer(&rawStream_)) + ControlList metadata; + metadata.set(controls::draft::PipelineDepth, 3); + metadata.set(controls::SensorTimestamp, buffer->metadata().timestamp); + + /* \todo Actually apply the scaler crop region to the ImgU. */ + const auto &scalerCrop = request->controls().get(controls::ScalerCrop); + if (scalerCrop) + cropRegion_ = *scalerCrop; + metadata.set(controls::ScalerCrop, cropRegion_); + + /* + * \todo Move reporting the raw frame related metadata from the IPA + * to here: AnalogueGain, FrameDuration, etc. + */ + utils::Duration lineDuration = sensorInfo_.lineLength * 1.0s / sensorInfo_.pixelRate; + if (sensorCtrl.contains(V4L2_CID_EXPOSURE)) { + int32_t exposureTime = sensorCtrl.get(V4L2_CID_EXPOSURE).get(); + metadata.set(controls::ExposureTime, exposureTime * lineDuration.get()); + } + + pipe()->completeMetadata(request, metadata); + + if (request->findBuffer(&rawStream_)) { pipe()->completeBuffer(request, buffer); + } ipa_->fillParamsBuffer(info->id, info->paramBuffer->cookie()); } @@ -1455,20 +1473,21 @@ void IPU3CameraData::frameStart(uint32_t sequence) Request *request = processingRequests_.front(); processingRequests_.pop(); - const auto &testPatternMode = request->controls().get(controls::draft::TestPatternMode); - if (!testPatternMode) - return; + int32_t testPatternMode = controls::draft::TestPatternModeOff; + const auto &testPatternControl = request->controls().get(controls::draft::TestPatternMode); + if (testPatternControl) + testPatternMode = *testPatternControl; int ret = cio2_.sensor()->setTestPatternMode( - static_cast(*testPatternMode)); + static_cast(testPatternMode)); if (ret) { - LOG(IPU3, Error) << "Failed to set test pattern mode: " - << ret; - return; + LOG(IPU3, Error) << "Failed to set test pattern mode: " << ret; + testPatternMode = controls::draft::TestPatternModeOff; } - request->metadata().set(controls::draft::TestPatternMode, - *testPatternMode); + ControlList metadata; + metadata.set(controls::draft::TestPatternMode, testPatternMode); + pipe()->completeMetadata(request, metadata); } REGISTER_PIPELINE_HANDLER(PipelineHandlerIPU3)