{"id":15489,"url":"https://patchwork.libcamera.org/api/1.1/patches/15489/?format=json","web_url":"https://patchwork.libcamera.org/patch/15489/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20220321124828.1845058-1-naush@raspberrypi.com>","date":"2022-03-21T12:48:28","name":"[libcamera-devel] pipeline: raspberrypi: Add a sensor dequeue timeout","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"b11f244a9ec07ce403706637bd093d9c1a53fe8f","submitter":{"id":34,"url":"https://patchwork.libcamera.org/api/1.1/people/34/?format=json","name":"Naushir Patuck","email":"naush@raspberrypi.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/15489/mbox/","series":[{"id":2978,"url":"https://patchwork.libcamera.org/api/1.1/series/2978/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=2978","date":"2022-03-21T12:48:28","name":"[libcamera-devel] pipeline: raspberrypi: Add a sensor dequeue timeout","version":1,"mbox":"https://patchwork.libcamera.org/series/2978/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/15489/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/15489/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 265C2BDE17\n\tfor ;\n\tMon, 21 Mar 2022 12:48:36 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 8F046604D4;\n\tMon, 21 Mar 2022 13:48:35 +0100 (CET)","from mail-wr1-x433.google.com (mail-wr1-x433.google.com\n\t[IPv6:2a00:1450:4864:20::433])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id A03CD604C6\n\tfor ;\n\tMon, 21 Mar 2022 13:48:33 +0100 (CET)","by mail-wr1-x433.google.com with SMTP id b19so20512421wrh.11\n\tfor ;\n\tMon, 21 Mar 2022 05:48:33 -0700 (PDT)","from naush-laptop.pitowers.org\n\t([2a00:1098:3142:14:77bd:7c6c:a8b9:8855])\n\tby smtp.gmail.com with ESMTPSA id\n\tl12-20020a05600c4f0c00b0038be825b774sm15120339wmq.45.2022.03.21.05.48.32\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tMon, 21 Mar 2022 05:48:32 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1647866915;\n\tbh=l6FGZX125eO/R0eapnZwu2hSRp/wg9PDyas3WDx4bbE=;\n\th=To:Date:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post:\n\tList-Help:List-Subscribe:From:Reply-To:From;\n\tb=1VGJzdOkTUOVl+M9Letjf3HWoEDJIDVLuwkUdWTOHZfQ0EKfwqLRa9ghR7E9SHsiY\n\twCnueh4KpPb4cgGdGu/Ivcz8SG/tdsZ+x2iTqQ3pFqo3W9zO2Dy1C1BRyalz4BnRDE\n\t2syv3utroyAIGcyuWGhkaxBvuUjqVT6dabByj+gym7LuxjsiW+QI/34Zmk0/FiFnVS\n\tySBA0eIpV9OJ6CBAN5QLaJYldf9w4DEenCZIbt2RNPQUZuhJ99LWWySAp8dPjU4V3o\n\tVseLulw7MPSu0BFFLE6zrobvk4OyBuDTwRJEvkJRWMuqBMe+/jPhS7uuZYNA3pOdNh\n\tLAc63eimOYWjQ==","v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=raspberrypi.com; s=google;\n\th=from:to:cc:subject:date:message-id:mime-version\n\t:content-transfer-encoding;\n\tbh=eom49egC79+V85At1spf0wwWgp6r3T8TKStanhXsZ+Y=;\n\tb=MHD7rvvolcaDBwffV5mlVqRngck5+uw2/zvjhejGWJFtQ6Y/8Lu84nbqTSWmJY96qr\n\tr3/4mLO82lFdmeWzm8n10PKob7kXefhuOrULk7sh3N4iOy02+0OZZXOISuLkGD04vYrX\n\tnEu1BpaUhys3Y8JYbQR5DEIrJg0WOWC7aMLbeuoXe3iUV9hAjdCSQw7X2slOGP/onvxh\n\tY/B/EzvMagqqGH18JzFaV/BOo+Om1RO/nqKMZuhRVFkWg63GJP5kcYSHN2A1w4CoIk7w\n\tBCcdOlLAfYsdCeaA/iqnn5DRqSPJR4Nf8XdLEdL5BY4YlbTf3gYbgwi4dGzFuvLq6SPu\n\tt1ow=="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key; \n\tunprotected) header.d=raspberrypi.com\n\theader.i=@raspberrypi.com\n\theader.b=\"MHD7rvvo\"; dkim-atps=neutral","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20210112;\n\th=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version\n\t:content-transfer-encoding;\n\tbh=eom49egC79+V85At1spf0wwWgp6r3T8TKStanhXsZ+Y=;\n\tb=QpMe7XfCmZN4hXO4AF29J6Qe5QghROCQBRk6bbG021XExhZgdsh4oMp8EytNhTuqPD\n\thup5Q2S0RCO3As13Xpomk9fbaPBL/WSAUuUEZ1BLND0DaHuo3mE6SPggS1BKsZ5p2C9f\n\t+0Mq5x6gWWvSdl+xG6oCJf/SD5Qs6ijuREXlj/kZkiR4m6ct0vE8yan3pvCoyjfJNO3I\n\t4//dh1cteNix/JYxJVHKqh2fsUSYCE+puti201xkEuxqoVMJAgeagbX+D6A7xTu300ha\n\tQE9ROvUsNQgVq1o7LegyNGmzMJWvcuq4Vp/bJXIuMEFGpjcbQkhMtougUvc593Zza59f\n\tt2nQ==","X-Gm-Message-State":"AOAM533Lm0Rf4By69xqRRE/njq8l3oL1VhoTxchCQkwgGWLFHFBWMequ\n\tCfCayrMsaLytj9SKxAs9+R5asNTmoVsSng==","X-Google-Smtp-Source":"ABdhPJw7arhMSovWUEEO8P/nxsF4cSka59qqclVEPRWavhAu324Djm8nl5Pw0rKrO7qQszMQYHLEIg==","X-Received":"by 2002:a5d:6707:0:b0:203:e60e:49fa with SMTP id\n\to7-20020a5d6707000000b00203e60e49famr18522224wru.603.1647866912920; \n\tMon, 21 Mar 2022 05:48:32 -0700 (PDT)","To":"libcamera-devel@lists.libcamera.org","Date":"Mon, 21 Mar 2022 12:48:28 +0000","Message-Id":"<20220321124828.1845058-1-naush@raspberrypi.com>","X-Mailer":"git-send-email 2.25.1","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Subject":"[libcamera-devel] [PATCH] pipeline: raspberrypi: Add a sensor\n\tdequeue timeout","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Naushir Patuck via libcamera-devel\n\t","Reply-To":"Naushir Patuck ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Add a timer to the pipeline handler that get set on start(), and subsequently\nreset on each frame dequeue from the sensor. If the timeout expires, log an\nerror message indicating so. This is useful for debugging sensor failures were\nthe device just stops streaming frames to Unicam, or the pipeline handler has\nfailed to queue buffers to the Unicam device driver.\n\nThe timeout is calculated as 2x the maximum frame length possible for a given\nmode.\n\nSigned-off-by: Naushir Patuck \n---\n include/libcamera/ipa/raspberrypi.mojom | 1 +\n src/ipa/raspberrypi/raspberrypi.cpp | 7 ++++++\n .../pipeline/raspberrypi/raspberrypi.cpp | 22 +++++++++++++++++++\n 3 files changed, 30 insertions(+)","diff":"diff --git a/include/libcamera/ipa/raspberrypi.mojom b/include/libcamera/ipa/raspberrypi.mojom\nindex acd3cafe6c91..33d2a97ca916 100644\n--- a/include/libcamera/ipa/raspberrypi.mojom\n+++ b/include/libcamera/ipa/raspberrypi.mojom\n@@ -41,6 +41,7 @@ struct IPAConfig {\n struct StartConfig {\n \tlibcamera.ControlList controls;\n \tint32 dropFrameCount;\n+\tuint32 sensorTimeoutMs;\n };\n \n interface IPARPiInterface {\ndiff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp\nindex fd8fecb07f81..983d6e998b4c 100644\n--- a/src/ipa/raspberrypi/raspberrypi.cpp\n+++ b/src/ipa/raspberrypi/raspberrypi.cpp\n@@ -281,6 +281,13 @@ void IPARPi::start(const ControlList &controls, ipa::RPi::StartConfig *startConf\n \n \tstartConfig->dropFrameCount = dropFrameCount_;\n \n+\t/*\n+\t * Set the pipeline handler's sensor timeout to 2x the maximum possible\n+\t * frame duration for this mode.\n+\t */\n+\tconst Duration maxSensorFrameDuration = mode_.max_frame_length * mode_.line_length;\n+\tstartConfig->sensorTimeoutMs = 2 * maxSensorFrameDuration.get();\n+\n \tfirstStart_ = false;\n \tlastRunTimestamp_ = 0;\n }\ndiff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp\nindex c2230199fed7..86d952b52aed 100644\n--- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp\n+++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp\n@@ -15,6 +15,7 @@\n #include \n \n #include \n+#include \n #include \n \n #include \n@@ -202,6 +203,7 @@ public:\n \tvoid setIspControls(const ControlList &controls);\n \tvoid setDelayedControls(const ControlList &controls);\n \tvoid setSensorControls(ControlList &controls);\n+\tvoid sensorTimeout();\n \n \t/* bufferComplete signal handlers. */\n \tvoid unicamBufferDequeue(FrameBuffer *buffer);\n@@ -279,6 +281,10 @@ public:\n \t */\n \tstd::optional notifyGainsUnity_;\n \n+\t/* Timer to ensure the sensor is producing frames for the pipeline handler. */\n+\tTimer sensorTimeout_;\n+\tstd::chrono::milliseconds sensorTimeoutDuration_;\n+\n private:\n \tvoid checkRequestCompleted();\n \tvoid fillRequestMetadata(const ControlList &bufferControls,\n@@ -1032,6 +1038,11 @@ int PipelineHandlerRPi::start(Camera *camera, const ControlList *controls)\n \t\t}\n \t}\n \n+\tLOG(RPI, Debug) << \"Setting sensor timeout to \" << startConfig.sensorTimeoutMs << \" ms\";\n+\tdata->sensorTimeoutDuration_ = std::chrono::milliseconds(startConfig.sensorTimeoutMs);\n+\tdata->sensorTimeout_.start(data->sensorTimeoutDuration_);\n+\tdata->sensorTimeout_.timeout.connect(data, &RPiCameraData::sensorTimeout);\n+\n \treturn 0;\n }\n \n@@ -1040,6 +1051,8 @@ void PipelineHandlerRPi::stopDevice(Camera *camera)\n \tRPiCameraData *data = cameraData(camera);\n \n \tdata->state_ = RPiCameraData::State::Stopped;\n+\tdata->sensorTimeout_.timeout.disconnect();\n+\tdata->sensorTimeout_.stop();\n \n \t/* Disable SOF event generation. */\n \tdata->unicam_[Unicam::Image].dev()->setFrameStartEnabled(false);\n@@ -1757,6 +1770,12 @@ void RPiCameraData::setSensorControls(ControlList &controls)\n \tsensor_->setControls(&controls);\n }\n \n+void RPiCameraData::sensorTimeout()\n+{\n+\tLOG(RPI, Error) << \"Sensor has timed out after \"\n+\t\t\t<< sensorTimeoutDuration_.count() << \" ms!\";\n+}\n+\n void RPiCameraData::unicamBufferDequeue(FrameBuffer *buffer)\n {\n \tRPi::Stream *stream = nullptr;\n@@ -1792,6 +1811,9 @@ void RPiCameraData::unicamBufferDequeue(FrameBuffer *buffer)\n \t\t */\n \t\tctrl.set(controls::SensorTimestamp, buffer->metadata().timestamp);\n \t\tbayerQueue_.push({ buffer, std::move(ctrl) });\n+\n+\t\t/* Restart the sensor timer. */\n+\t\tsensorTimeout_.start(sensorTimeoutDuration_);\n \t} else {\n \t\tembeddedQueue_.push(buffer);\n \t}\n","prefixes":["libcamera-devel"]}