From patchwork Thu Mar 24 09:34:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 15530 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A464ABD80A for ; Thu, 24 Mar 2022 09:34:14 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BFB77604DA; Thu, 24 Mar 2022 10:34:13 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1648114453; bh=zILa/s1ntW+kghEdxq25c2IT3/MqXufb22HdwyIDlaQ=; h=To:Date:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=P7zzfEmtnu+Hd8BQ7zeqqVw4k5UH91to50N4DYwCNCeLpHpq/tLzOhK5xM+iWMmVp WqrbmYTe2fVetyi0lv7+GjxONINxg5unKqsJ40jXTe0bAoyen/APlW7eZy/kgKUc93 YhC3aT2LSra2FtXHmxpNXRGsf05tppHgFoS+PrfFuxz4xptweTyxJcJLpC+VEuIIfd lv9RSzYTRQOhGLz0CsR72nPGdK+7FW1ncwtbwSXCWX6bQtBzmjPLKWhDIJ5PSv5Ptk Shou/3yGgPtOaYyS9+uFwwmkq273XymhEjgSEC6uAk6DjYgCLeK1IMM9a7wJqPVngm o+rI/s4I9r+Fw== Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8D3DE601F5 for ; Thu, 24 Mar 2022 10:34:12 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="qwYDpSJM"; dkim-atps=neutral Received: by mail-wr1-x42f.google.com with SMTP id t11so5732917wrm.5 for ; Thu, 24 Mar 2022 02:34:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=ipNazOf+GDYpo5NVgHVuDTgMHP3J4Xmz8ZP+fnMDeXE=; b=qwYDpSJMyAXbuY1joW4PyF3BnGNsdTElGIsm6q18AEttLO/Ug8fy7wDBKpQCZsqBVy CS++ALMYed0kVfj5ZOyUWwZ3ys0Dylve/1eqGBnzrOThF+2Rga9+ha11bHu5OZAOu/wh J5OFgfKk0VNsRLIKd4h5db4fKPRu50vXOX1nlHcnfOyudGy36ow+qxddunV3y/1RNTff uahhOuG//O6oTxTyYumt+Iv3amfQi+CMYHISrLMv80wNtMuprYdhQcPAGGiSTfZbZc25 /807wbuan4FOw+2cZ5XqaYSl1WfEbcq9tHlawaqYWBgRNkLGkAhIdvoxX3SXhtwZIbMo WU4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=ipNazOf+GDYpo5NVgHVuDTgMHP3J4Xmz8ZP+fnMDeXE=; b=kag4zuaWMD9xsDUIbYq7KeMGBzryNcjnSkf4i1jTPtiP7Fv+vFEDZa8Eo80ycpsD/K ZtONkdbN0FvqpWJ2+ooHcguVgHM1gE60gMORe7ZmzFVI5GM8kBBoIT0tjUjOwbjNPmxf Pvb7PTZCr0EqURFrFcWLy6hCCpqdTsTo0V1Cw28axCt37HmHVhxlFEaVXgfEM9fYVNMa WERdjhCNAwLEGVA34bPAaeKbDStS9GDArxdniFOMtWdxznymQzfhmwA4kG6XGcS/WCbv QxMuWsp4DWqUnVtFiq8Ure1nJI5Eqix4uOMi/WH3HDjIStsztApAhZ0yS20dXdbMi9if XEhQ== X-Gm-Message-State: AOAM531DmNtpOzEY5IeB+fwWta5etUDwhr25+s/518x9iimswnrxyyn6 m29Zlcee35SMTuxXltkyJWDSwOblt0SyjA== X-Google-Smtp-Source: ABdhPJy1VftVIwWyw3CjFXxCudIlqlkZH5ydjFQTvSViF+IM6XE3hO5ggI4iiyiXMb2ngjgSTtd9PA== X-Received: by 2002:adf:dfc2:0:b0:1f0:262a:d831 with SMTP id q2-20020adfdfc2000000b001f0262ad831mr3715271wrn.589.1648114451724; Thu, 24 Mar 2022 02:34:11 -0700 (PDT) Received: from naush-laptop.pitowers.org ([2a00:1098:3142:14:18f1:1b78:a3ab:7eda]) by smtp.gmail.com with ESMTPSA id v15-20020a056000144f00b002057eac999fsm2243889wrx.76.2022.03.24.02.34.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Mar 2022 02:34:11 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Thu, 24 Mar 2022 09:34:08 +0000 Message-Id: <20220324093409.29247-1-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 1/2] libcamera: v4l2_videodevice: Add a dequeue timer X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a timer that gets reset on every buffer dequeue event. If the timeout expires, optionally call a slot in the pipeline handler to handle this condition. This may be useful in detecting and handling stalls in either the hardware or device driver. Signed-off-by: Naushir Patuck --- include/libcamera/internal/v4l2_videodevice.h | 10 ++++ src/libcamera/v4l2_videodevice.cpp | 49 +++++++++++++++++++ 2 files changed, 59 insertions(+) diff --git a/include/libcamera/internal/v4l2_videodevice.h b/include/libcamera/internal/v4l2_videodevice.h index 2d2ccc477c91..c34cc6585612 100644 --- a/include/libcamera/internal/v4l2_videodevice.h +++ b/include/libcamera/internal/v4l2_videodevice.h @@ -9,6 +9,7 @@ #include #include +#include #include #include #include @@ -20,6 +21,7 @@ #include #include #include +#include #include #include @@ -216,6 +218,9 @@ public: int streamOn(); int streamOff(); + void setDequeueTimeout(std::chrono::milliseconds msec); + Signal<> dequeueTimeout; + static std::unique_ptr fromEntityName(const MediaDevice *media, const std::string &entity); @@ -246,6 +251,8 @@ private: void bufferAvailable(); FrameBuffer *dequeueBuffer(); + void watchdogExpired(); + V4L2Capability caps_; V4L2DeviceFormat format_; const PixelFormatInfo *formatInfo_; @@ -259,6 +266,9 @@ private: EventNotifier *fdBufferNotifier_; bool streaming_; + + Timer watchdog_; + std::chrono::milliseconds watchdogDuration_; }; class V4L2M2MDevice diff --git a/src/libcamera/v4l2_videodevice.cpp b/src/libcamera/v4l2_videodevice.cpp index 5f36ee20710d..29225f3058b0 100644 --- a/src/libcamera/v4l2_videodevice.cpp +++ b/src/libcamera/v4l2_videodevice.cpp @@ -526,6 +526,7 @@ V4L2VideoDevice::V4L2VideoDevice(const std::string &deviceNode) V4L2VideoDevice::V4L2VideoDevice(const MediaEntity *entity) : V4L2VideoDevice(entity->deviceNode()) { + watchdog_.timeout.connect(this, &V4L2VideoDevice::watchdogExpired); } V4L2VideoDevice::~V4L2VideoDevice() @@ -1695,6 +1696,9 @@ FrameBuffer *V4L2VideoDevice::dequeueBuffer() return nullptr; } + if (watchdogDuration_.count()) + watchdog_.start(watchdogDuration_); + cache_->put(buf.index); FrameBuffer *buffer = it->second; @@ -1797,6 +1801,8 @@ int V4L2VideoDevice::streamOn() } streaming_ = true; + if (watchdogDuration_.count()) + watchdog_.start(watchdogDuration_); return 0; } @@ -1821,6 +1827,9 @@ int V4L2VideoDevice::streamOff() if (!streaming_ && queuedBuffers_.empty()) return 0; + if (watchdogDuration_.count()) + watchdog_.stop(); + ret = ioctl(VIDIOC_STREAMOFF, &bufferType_); if (ret < 0) { LOG(V4L2, Error) @@ -1843,6 +1852,46 @@ int V4L2VideoDevice::streamOff() return 0; } +/** + * \brief Set the dequeue timeout value + * \param[in] msec The timeout value to be used + * + * Sets a timeout value, given by \a msec, that will be used by a watchdog timer + * to ensure buffer dequeue events are periodically occurring when the device is + * streaming. The watchdog timer is only active when the device is streaming, so + * it is not necessary to disable it when the device stops streaming. The timeout + * value can be safely updated at any time. + * + * If the timer expires, the \ref V4L2VideoDevice::dequeueTimeout signal is + * emitted. This can typically be used by pipeline handlers to be notified of + * stalled devices. + * + * Set \a msec to 0 to disable the watchdog timer. + */ +void V4L2VideoDevice::setDequeueTimeout(std::chrono::milliseconds msec) +{ + watchdogDuration_ = msec; + + watchdog_.stop(); + if (watchdogDuration_.count() && streaming_) + watchdog_.start(msec); +} + +/** + * \brief Slot to handle an expired dequeue timer. + * + * When this slot is called, the time between successive dequeue events is over + * the required timeout. Emit the \ref V4L2VideoDevice::dequeueTimeout signal. + */ +void V4L2VideoDevice::watchdogExpired() +{ + LOG(V4L2, Warning) + << "Dequeue timer of " << watchdogDuration_.count() + << " ms has expired!"; + + dequeueTimeout.emit(); +} + /** * \brief Create a new video device instance from \a entity in media device * \a media From patchwork Thu Mar 24 09:34:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 15531 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 14311BD80A for ; Thu, 24 Mar 2022 09:34:16 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7D3D4604C5; Thu, 24 Mar 2022 10:34:15 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1648114455; bh=cvsUhuGJdi2+O8B/WNlehg1JM8ArO7iMbY7zdhfE6Aw=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=XGVQpMA4SUWCF0Z/z6lNF6V5eLqJ/y7syFAv0eeXtatRIuXcxoisp6lV36dv2cfML DyNrjHXeXa5BhwWw44zFyoAPyjcTWgrjpIjUc7PlUQxJWAMV6LLLARN31U81YPI2Dc 9GQn1kvbqug+BCfqJbdB0d8+foHf/KzvjqGHhLRXbAofTvEs1/L9Z/IgGOD60/lvLv X5f7AbS5f8/DwZ8YMhR16ZHy7u2v4MeWJjtHMMIzwjXtMod9OrPh0dOGJssA6tRqfd rHr5jaDN3JitSIZxtHxPWw99pyVkC/sON80LBTaifmuxnAYq0ccLZhMl1oCcJXpDGE 46dQWwoA8Z55A== Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2E707601F5 for ; Thu, 24 Mar 2022 10:34:13 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="mhnNwlId"; dkim-atps=neutral Received: by mail-wr1-x42d.google.com with SMTP id u3so5752972wrg.3 for ; Thu, 24 Mar 2022 02:34:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=J177DCPIrCbnU3M3SWgf1jMhEAT05MNMBbvF1zi1x+I=; b=mhnNwlIdzUzDEWS0nZSddNYhMm33dq0QmJgi7sZrs9U4ubo1VjHrT2lhnXaeq+289h YpjTrceag+BjUPvnjzdN1dSPpcF+cga2xikmSUqFBclDrrYXRLxO9mKvihqRXqs7VpdV tTgfcXXgjpKFxY4EWwrZuycKlc9bnfY2CwoQThUqp0omeLtNLcedAgPNqWbNVPZwFgif gDdx+VErpITX2oo0nyBjoP02CQw+V6BpEJWuMe+43MxYvg+krIm7a/KLZMvnjEhmdXe6 +OPgnn7SLJrxrNek2hdLwE7WF1KuU1VD+TMmmM4vW1jR3rjVh8DxLp2TeTa+Eb2ENVrC TF/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=J177DCPIrCbnU3M3SWgf1jMhEAT05MNMBbvF1zi1x+I=; b=6VZZMteu7qG8vdjwMxw04V1l49iYMK1uyY2cH/jaEicw+KFuz/AhLL/i23mkRfMq4n cCppmLF2tMKHrla1X4tbUrF54PF7FCb1dDACK3GQsM7x5aFpivVBlGBZCTTFv792kJKs lrM5Td1DnW1pHMFleZbJ8z/1LpnzOzXWHSR1wKyw+K0AvIO3ZDgoGtTl8Exlf18odf0Z Bv/F2IpVe1Y2gChFna/QlMNz2thLaupWB3AIsQN+qGBDVxK+++9jK285Bc1DQtJNAzL+ VAi+V6I2n/OqkQV+hY/sE4gnW5Id7dfE7+MU+KbHrb7PPZwZVISCXpQCioi3z5wMFJqK 9loA== X-Gm-Message-State: AOAM531ID6P3Jq5DBcI/srDkqg9n47B4VkGkCZimbUqRfGIlxa2vX3Wv uCIk/dA5JGMO6hqyYkoKx/dojZiVmSAACQ== X-Google-Smtp-Source: ABdhPJxLXGjFXdUAXtuOVL84sBS6K8T1G4N36p+Oo3NtIKZqCPDVw8CCa93j/D97LWP0zolBYLvMaw== X-Received: by 2002:a05:6000:1547:b0:204:213:ae28 with SMTP id 7-20020a056000154700b002040213ae28mr3786796wry.114.1648114452617; Thu, 24 Mar 2022 02:34:12 -0700 (PDT) Received: from naush-laptop.pitowers.org ([2a00:1098:3142:14:18f1:1b78:a3ab:7eda]) by smtp.gmail.com with ESMTPSA id v15-20020a056000144f00b002057eac999fsm2243889wrx.76.2022.03.24.02.34.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Mar 2022 02:34:12 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Thu, 24 Mar 2022 09:34:09 +0000 Message-Id: <20220324093409.29247-2-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220324093409.29247-1-naush@raspberrypi.com> References: <20220324093409.29247-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 2/2] pipeline: raspberrypi: Add a Unicam dequeue timeout X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Enable the V4L2VideoDevice dequeue timeout for the Unicam Image node, and connect the timeout signal to a slot in the pipeline handler. This slot will log a fatal error message informing the user of a possible hardware stall. The timeout is calculated as 2x the maximum frame length possible for a given mode, returned by the IPA. Signed-off-by: Naushir Patuck --- include/libcamera/ipa/raspberrypi.mojom | 1 + src/ipa/raspberrypi/raspberrypi.cpp | 2 ++ .../pipeline/raspberrypi/raspberrypi.cpp | 15 +++++++++++++++ 3 files changed, 18 insertions(+) diff --git a/include/libcamera/ipa/raspberrypi.mojom b/include/libcamera/ipa/raspberrypi.mojom index acd3cafe6c91..5a228b75cd2f 100644 --- a/include/libcamera/ipa/raspberrypi.mojom +++ b/include/libcamera/ipa/raspberrypi.mojom @@ -41,6 +41,7 @@ struct IPAConfig { struct StartConfig { libcamera.ControlList controls; int32 dropFrameCount; + uint32 maxSensorFrameLengthMs; }; interface IPARPiInterface { diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 1bf4e270b062..89767a9db471 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -280,6 +280,8 @@ void IPARPi::start(const ControlList &controls, ipa::RPi::StartConfig *startConf } startConfig->dropFrameCount = dropFrameCount_; + const Duration maxSensorFrameDuration = mode_.max_frame_length * mode_.line_length; + startConfig->maxSensorFrameLengthMs = maxSensorFrameDuration.get(); firstStart_ = false; lastRunTimestamp_ = 0; diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index c2230199fed7..58b1179c4294 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -202,6 +202,7 @@ public: void setIspControls(const ControlList &controls); void setDelayedControls(const ControlList &controls); void setSensorControls(ControlList &controls); + void unicamTimeout(); /* bufferComplete signal handlers. */ void unicamBufferDequeue(FrameBuffer *buffer); @@ -1032,6 +1033,11 @@ int PipelineHandlerRPi::start(Camera *camera, const ControlList *controls) } } + /* Set the dequeue timeout to 2x the maximum possible frame duration. */ + std::chrono::milliseconds msec(startConfig.maxSensorFrameLengthMs * 2); + data->unicam_[Unicam::Image].dev()->setDequeueTimeout(msec); + LOG(RPI, Debug) << "Setting Unicam timeout to " << msec << " ms."; + return 0; } @@ -1176,6 +1182,7 @@ int PipelineHandlerRPi::registerCamera(MediaDevice *unicam, MediaDevice *isp, Me data->isp_[Isp::Stats] = RPi::Stream("ISP Stats", ispCapture3); /* Wire up all the buffer connections. */ + data->unicam_[Unicam::Image].dev()->dequeueTimeout.connect(data.get(), &RPiCameraData::unicamTimeout); data->unicam_[Unicam::Image].dev()->frameStart.connect(data.get(), &RPiCameraData::frameStarted); data->unicam_[Unicam::Image].dev()->bufferReady.connect(data.get(), &RPiCameraData::unicamBufferDequeue); data->isp_[Isp::Input].dev()->bufferReady.connect(data.get(), &RPiCameraData::ispInputDequeue); @@ -1757,6 +1764,14 @@ void RPiCameraData::setSensorControls(ControlList &controls) sensor_->setControls(&controls); } +void RPiCameraData::unicamTimeout() +{ + LOG(RPI, Error) + << "Unicam has timed out!\n" + "Please check that your camera sensor connector is attached securely.\n" + "Alternatively, try another cable and/or sensor."; +} + void RPiCameraData::unicamBufferDequeue(FrameBuffer *buffer) { RPi::Stream *stream = nullptr;