[{"id":16352,"web_url":"https://patchwork.libcamera.org/comment/16352/","msgid":"<fdd0fa50-e9a3-561d-f6cd-bf20c77dc5a9@ideasonboard.com>","date":"2021-04-19T13:35:27","subject":"Re: [libcamera-devel] [PATCH 02/13] libcamera: ipu3: Report sensor\n\ttimestamp","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"Hi Jacopo,\n\nOn 19/04/2021 14:14, Jacopo Mondi wrote:\n> Report the sensor's timestamp in the Request metadata by using the\n> CIO2 buffer timestamp as an initial approximation.\n> \n> The buffer's timestamp is recorded at DMA-transfer time, and it does not\n> theoretically matches the 'start of exposure' definition, but when used\n> to compare two consecutive frames it gives an acceptable estimation of\n> the sensor frame period duration.\n> \n> Reviewed-by: Hirokazu Honda <hiroh@chromium.org>\n> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>\n> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>\n> ---\n>  src/libcamera/pipeline/ipu3/ipu3.cpp | 9 +++++++++\n>  1 file changed, 9 insertions(+)\n> \n> diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp\n> index 51446fcf5bc1..28e849a43a3e 100644\n> --- a/src/libcamera/pipeline/ipu3/ipu3.cpp\n> +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp\n> @@ -1255,6 +1255,15 @@ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer)\n>  \n>  \tRequest *request = info->request;\n>  \n> +\t/*\n> +\t * Record the sensor's timestamp in the request metadata.\n> +\t *\n> +\t * \\todo The sensor timestamp should be better estimated by connecting\n> +\t * to the V4L2Device::frameStart signal.\n\nShouldn't that be handled by the sensor driver itself then?\n\nWhat defines the temporal location/point of the timestamp that is\nreturned currently?\n\nI would have thought getting the time at the frameStart() signal would\nbe susceptible to more latency due to the signal handling and calling up\nto userspace?\n\nThis must surely be something better handled by the kernel drivers right?\n\n\n> +\t */\n> +\trequest->metadata().set(controls::SensorTimestamp,\n> +\t\t\t\tbuffer->metadata().timestamp);\n> +\n>  \t/* If the buffer is cancelled force a complete of the whole request. */\n>  \tif (buffer->metadata().status == FrameMetadata::FrameCancelled) {\n>  \t\tfor (auto it : request->buffers())\n>","headers":{"Return-Path":"<libcamera-devel-bounces@lists.libcamera.org>","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 9893DBD814\n\tfor <parsemail@patchwork.libcamera.org>;\n\tMon, 19 Apr 2021 13:35:31 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 5AC1068830;\n\tMon, 19 Apr 2021 15:35:31 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id E511E602CD\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tMon, 19 Apr 2021 15:35:29 +0200 (CEST)","from [192.168.0.20]\n\t(cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 8E089D4A;\n\tMon, 19 Apr 2021 15:35:29 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"cKkgMSN3\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1618839329;\n\tbh=mW22fso6UeaPgK25tT64jCU0AKx3Eq/BhSUKZuI9qCc=;\n\th=Reply-To:Subject:To:References:From:Date:In-Reply-To:From;\n\tb=cKkgMSN3k6lu2El3LRJaKPd5QRinE92Bi1WQpZPefaMIUWilp4xD1pMIUX/4yTFtW\n\t8vtYqm4dPOhZ15E2vn3k6dJSVaFywlK1H6SdKs3ToxO30RvHMIlGlJkT3n6M3fOvpg\n\tcizs6coKh4+PGhNq1OoxGU/WYCR2sR/k2fg4UyEI=","To":"Jacopo Mondi <jacopo@jmondi.org>, libcamera-devel@lists.libcamera.org","References":"<20210419131433.22920-1-jacopo@jmondi.org>\n\t<20210419131433.22920-3-jacopo@jmondi.org>","From":"Kieran Bingham <kieran.bingham@ideasonboard.com>","Organization":"Ideas on Board","Message-ID":"<fdd0fa50-e9a3-561d-f6cd-bf20c77dc5a9@ideasonboard.com>","Date":"Mon, 19 Apr 2021 14:35:27 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101\n\tThunderbird/78.7.1","MIME-Version":"1.0","In-Reply-To":"<20210419131433.22920-3-jacopo@jmondi.org>","Content-Language":"en-GB","Subject":"Re: [libcamera-devel] [PATCH 02/13] libcamera: ipu3: Report sensor\n\ttimestamp","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"<libcamera-devel.lists.libcamera.org>","List-Unsubscribe":"<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>","List-Archive":"<https://lists.libcamera.org/pipermail/libcamera-devel/>","List-Post":"<mailto:libcamera-devel@lists.libcamera.org>","List-Help":"<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>","List-Subscribe":"<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>","Reply-To":"kieran.bingham@ideasonboard.com","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>"}},{"id":16353,"web_url":"https://patchwork.libcamera.org/comment/16353/","msgid":"<27b26dd2-586a-f6cd-78f4-2024a7967813@ideasonboard.com>","date":"2021-04-19T13:36:28","subject":"Re: [libcamera-devel] [PATCH 02/13] libcamera: ipu3: Report sensor\n\ttimestamp","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"On 19/04/2021 14:35, Kieran Bingham wrote:\n> Hi Jacopo,\n> \n> On 19/04/2021 14:14, Jacopo Mondi wrote:\n>> Report the sensor's timestamp in the Request metadata by using the\n>> CIO2 buffer timestamp as an initial approximation.\n>>\n>> The buffer's timestamp is recorded at DMA-transfer time, and it does not\n>> theoretically matches the 'start of exposure' definition, but when used\n>> to compare two consecutive frames it gives an acceptable estimation of\n>> the sensor frame period duration.\n>>\n>> Reviewed-by: Hirokazu Honda <hiroh@chromium.org>\n>> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>\n>> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>\n>> ---\n>>  src/libcamera/pipeline/ipu3/ipu3.cpp | 9 +++++++++\n>>  1 file changed, 9 insertions(+)\n>>\n>> diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp\n>> index 51446fcf5bc1..28e849a43a3e 100644\n>> --- a/src/libcamera/pipeline/ipu3/ipu3.cpp\n>> +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp\n>> @@ -1255,6 +1255,15 @@ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer)\n>>  \n>>  \tRequest *request = info->request;\n>>  \n>> +\t/*\n>> +\t * Record the sensor's timestamp in the request metadata.\n>> +\t *\n>> +\t * \\todo The sensor timestamp should be better estimated by connecting\n>> +\t * to the V4L2Device::frameStart signal.\n> \n> Shouldn't that be handled by the sensor driver itself then?\n> \n> What defines the temporal location/point of the timestamp that is\n> returned currently?\n> \n> I would have thought getting the time at the frameStart() signal would\n> be susceptible to more latency due to the signal handling and calling up\n> to userspace?\n> \n> This must surely be something better handled by the kernel drivers right?\n> \n> \n>> +\t */\n>> +\trequest->metadata().set(controls::SensorTimestamp,\n>> +\t\t\t\tbuffer->metadata().timestamp);\n\nBut for this....\n\nReviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>\n\nif the todo is correctly applicable then of course it can stay ;)\n\n\n\n>> +\n>>  \t/* If the buffer is cancelled force a complete of the whole request. */\n>>  \tif (buffer->metadata().status == FrameMetadata::FrameCancelled) {\n>>  \t\tfor (auto it : request->buffers())\n>>\n>","headers":{"Return-Path":"<libcamera-devel-bounces@lists.libcamera.org>","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 0AFEEBD812\n\tfor <parsemail@patchwork.libcamera.org>;\n\tMon, 19 Apr 2021 13:36:34 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id C2AD368830;\n\tMon, 19 Apr 2021 15:36:33 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 35F11602CD\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tMon, 19 Apr 2021 15:36:32 +0200 (CEST)","from [192.168.0.20]\n\t(cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id C7D3DD4A;\n\tMon, 19 Apr 2021 15:36:31 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"LspFhGHR\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1618839391;\n\tbh=PvorQPzJhDTiZjdH9hD4bVvBNX09P0UzcjPDlzBLzyY=;\n\th=Reply-To:Subject:From:To:References:Date:In-Reply-To:From;\n\tb=LspFhGHRd9gX266Zj7+/PAs3q3KATiM9lC9nNPyVxa5PkBJU8YROXMgAuIN5lLpwo\n\tWtV0XnIZxNae9lN1498J2Vmgwed0e05WFGQ2kIThDe3yPAE7QtICgww+Mv4YGKh/e1\n\tJB2TUTOUN9o7EVXJKqPUS7qWX+ZZpCcA8XEcB/4E=","From":"Kieran Bingham <kieran.bingham@ideasonboard.com>","To":"Jacopo Mondi <jacopo@jmondi.org>, libcamera-devel@lists.libcamera.org","References":"<20210419131433.22920-1-jacopo@jmondi.org>\n\t<20210419131433.22920-3-jacopo@jmondi.org>\n\t<fdd0fa50-e9a3-561d-f6cd-bf20c77dc5a9@ideasonboard.com>","Organization":"Ideas on Board","Message-ID":"<27b26dd2-586a-f6cd-78f4-2024a7967813@ideasonboard.com>","Date":"Mon, 19 Apr 2021 14:36:28 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101\n\tThunderbird/78.7.1","MIME-Version":"1.0","In-Reply-To":"<fdd0fa50-e9a3-561d-f6cd-bf20c77dc5a9@ideasonboard.com>","Content-Language":"en-GB","Subject":"Re: [libcamera-devel] [PATCH 02/13] libcamera: ipu3: Report sensor\n\ttimestamp","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"<libcamera-devel.lists.libcamera.org>","List-Unsubscribe":"<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>","List-Archive":"<https://lists.libcamera.org/pipermail/libcamera-devel/>","List-Post":"<mailto:libcamera-devel@lists.libcamera.org>","List-Help":"<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>","List-Subscribe":"<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>","Reply-To":"kieran.bingham@ideasonboard.com","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>"}},{"id":16359,"web_url":"https://patchwork.libcamera.org/comment/16359/","msgid":"<20210419135342.h7snlzheujfm6diw@uno.localdomain>","date":"2021-04-19T13:53:42","subject":"Re: [libcamera-devel] [PATCH 02/13] libcamera: ipu3: Report sensor\n\ttimestamp","submitter":{"id":3,"url":"https://patchwork.libcamera.org/api/people/3/","name":"Jacopo Mondi","email":"jacopo@jmondi.org"},"content":"Hi Kieran,\n\nOn Mon, Apr 19, 2021 at 02:35:27PM +0100, Kieran Bingham wrote:\n> Hi Jacopo,\n>\n> On 19/04/2021 14:14, Jacopo Mondi wrote:\n> > Report the sensor's timestamp in the Request metadata by using the\n> > CIO2 buffer timestamp as an initial approximation.\n> >\n> > The buffer's timestamp is recorded at DMA-transfer time, and it does not\n> > theoretically matches the 'start of exposure' definition, but when used\n> > to compare two consecutive frames it gives an acceptable estimation of\n> > the sensor frame period duration.\n> >\n> > Reviewed-by: Hirokazu Honda <hiroh@chromium.org>\n> > Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>\n> > Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>\n> > ---\n> >  src/libcamera/pipeline/ipu3/ipu3.cpp | 9 +++++++++\n> >  1 file changed, 9 insertions(+)\n> >\n> > diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp\n> > index 51446fcf5bc1..28e849a43a3e 100644\n> > --- a/src/libcamera/pipeline/ipu3/ipu3.cpp\n> > +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp\n> > @@ -1255,6 +1255,15 @@ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer)\n> >\n> >  \tRequest *request = info->request;\n> >\n> > +\t/*\n> > +\t * Record the sensor's timestamp in the request metadata.\n> > +\t *\n> > +\t * \\todo The sensor timestamp should be better estimated by connecting\n> > +\t * to the V4L2Device::frameStart signal.\n>\n> Shouldn't that be handled by the sensor driver itself then?\n>\n\nI'm not aware of a mechanism to transport that information from the\nsubdevice up, nor from what I've see the typical sensor driver has a\nmean to connect to that event and timestamp it.\n\nWe could rely on the receiver's SOF interrupt, from which usually the\nV4L2_EVENT_FRAME_SYNC is signaled to userspace\n\n> What defines the temporal location/point of the timestamp that is\n> returned currently?\n>\n\nIt depends on the platform. When the buffer whose timestamp is used to\npopulate the SensorTimestamp property is the application facing buffer\n(uvc, simple, rkisp1) it is usually the time whene the EOF interrupt\nis triggered. Platforms with an off-line design where the raw frame is\naccessible to libcamera use the raw frametimestamp, which is for the\nCIO2 in example, the EOF interrupt time of the raw frame.\n\n> I would have thought getting the time at the frameStart() signal would\n> be susceptible to more latency due to the signal handling and calling up\n> to userspace?\n\nProbably yes, I cannot really quantify that at the moment..\n\n>\n> This must surely be something better handled by the kernel drivers right?\n>\n\nBuffers are timestamped by the kernel. Thing is, even when we get the\nraw frame timestamp it usually refers to the end of the DMA transfer,\nnot the start of the exposure.\n\n>\n> > +\t */\n> > +\trequest->metadata().set(controls::SensorTimestamp,\n> > +\t\t\t\tbuffer->metadata().timestamp);\n> > +\n> >  \t/* If the buffer is cancelled force a complete of the whole request. */\n> >  \tif (buffer->metadata().status == FrameMetadata::FrameCancelled) {\n> >  \t\tfor (auto it : request->buffers())\n> >\n>\n> --\n> Regards\n> --\n> Kieran","headers":{"Return-Path":"<libcamera-devel-bounces@lists.libcamera.org>","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 41AE6BD812\n\tfor <parsemail@patchwork.libcamera.org>;\n\tMon, 19 Apr 2021 13:53:05 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 0CED668831;\n\tMon, 19 Apr 2021 15:53:05 +0200 (CEST)","from relay5-d.mail.gandi.net (relay5-d.mail.gandi.net\n\t[217.70.183.197])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 76C4B68824\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tMon, 19 Apr 2021 15:53:03 +0200 (CEST)","from uno.localdomain (93-34-118-233.ip49.fastwebnet.it\n\t[93.34.118.233]) (Authenticated sender: jacopo@jmondi.org)\n\tby relay5-d.mail.gandi.net (Postfix) with ESMTPSA id 021BC1C000D;\n\tMon, 19 Apr 2021 13:53:02 +0000 (UTC)"],"X-Originating-IP":"93.34.118.233","Date":"Mon, 19 Apr 2021 15:53:42 +0200","From":"Jacopo Mondi <jacopo@jmondi.org>","To":"Kieran Bingham <kieran.bingham@ideasonboard.com>","Message-ID":"<20210419135342.h7snlzheujfm6diw@uno.localdomain>","References":"<20210419131433.22920-1-jacopo@jmondi.org>\n\t<20210419131433.22920-3-jacopo@jmondi.org>\n\t<fdd0fa50-e9a3-561d-f6cd-bf20c77dc5a9@ideasonboard.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<fdd0fa50-e9a3-561d-f6cd-bf20c77dc5a9@ideasonboard.com>","Subject":"Re: [libcamera-devel] [PATCH 02/13] libcamera: ipu3: Report sensor\n\ttimestamp","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"<libcamera-devel.lists.libcamera.org>","List-Unsubscribe":"<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>","List-Archive":"<https://lists.libcamera.org/pipermail/libcamera-devel/>","List-Post":"<mailto:libcamera-devel@lists.libcamera.org>","List-Help":"<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>","List-Subscribe":"<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>"}},{"id":16402,"web_url":"https://patchwork.libcamera.org/comment/16402/","msgid":"<YH9CFk8VIFyJnBwI@pendragon.ideasonboard.com>","date":"2021-04-20T21:05:26","subject":"Re: [libcamera-devel] [PATCH 02/13] libcamera: ipu3: Report sensor\n\ttimestamp","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hello,\n\nOn Mon, Apr 19, 2021 at 03:53:42PM +0200, Jacopo Mondi wrote:\n> On Mon, Apr 19, 2021 at 02:35:27PM +0100, Kieran Bingham wrote:\n> > On 19/04/2021 14:14, Jacopo Mondi wrote:\n> > > Report the sensor's timestamp in the Request metadata by using the\n> > > CIO2 buffer timestamp as an initial approximation.\n> > >\n> > > The buffer's timestamp is recorded at DMA-transfer time, and it does not\n> > > theoretically matches the 'start of exposure' definition, but when used\n> > > to compare two consecutive frames it gives an acceptable estimation of\n> > > the sensor frame period duration.\n> > >\n> > > Reviewed-by: Hirokazu Honda <hiroh@chromium.org>\n> > > Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>\n> > > Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>\n> > > ---\n> > >  src/libcamera/pipeline/ipu3/ipu3.cpp | 9 +++++++++\n> > >  1 file changed, 9 insertions(+)\n> > >\n> > > diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp\n> > > index 51446fcf5bc1..28e849a43a3e 100644\n> > > --- a/src/libcamera/pipeline/ipu3/ipu3.cpp\n> > > +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp\n> > > @@ -1255,6 +1255,15 @@ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer)\n> > >\n> > >  \tRequest *request = info->request;\n> > >\n> > > +\t/*\n> > > +\t * Record the sensor's timestamp in the request metadata.\n> > > +\t *\n> > > +\t * \\todo The sensor timestamp should be better estimated by connecting\n> > > +\t * to the V4L2Device::frameStart signal.\n> >\n> > Shouldn't that be handled by the sensor driver itself then?\n> \n> I'm not aware of a mechanism to transport that information from the\n> subdevice up, nor from what I've see the typical sensor driver has a\n> mean to connect to that event and timestamp it.\n> \n> We could rely on the receiver's SOF interrupt, from which usually the\n> V4L2_EVENT_FRAME_SYNC is signaled to userspace\n\nIt's currently messy in V4L2. Drivers report timestamps in different\nways, and most of them don't match the specification. The specification\nitself has some useful features (a driver can for instance opt to report\nthe SOF timestamp if it has access to that information, but whether the\ndriver does so or not can't be queried). I think we'll have to solve\nthis in V4L2 first.\n\n> > What defines the temporal location/point of the timestamp that is\n> > returned currently?\n> \n> It depends on the platform. When the buffer whose timestamp is used to\n> populate the SensorTimestamp property is the application facing buffer\n> (uvc, simple, rkisp1) it is usually the time whene the EOF interrupt\n> is triggered. Platforms with an off-line design where the raw frame is\n> accessible to libcamera use the raw frametimestamp, which is for the\n> CIO2 in example, the EOF interrupt time of the raw frame.\n> \n> > I would have thought getting the time at the frameStart() signal would\n> > be susceptible to more latency due to the signal handling and calling up\n> > to userspace?\n> \n> Probably yes, I cannot really quantify that at the moment..\n> \n> > This must surely be something better handled by the kernel drivers right?\n> \n> Buffers are timestamped by the kernel. Thing is, even when we get the\n> raw frame timestamp it usually refers to the end of the DMA transfer,\n> not the start of the exposure.\n\nWhen it comes to what drivers should do, I'd say they should provide\nenough information for userspace to obtain the most accurate timestamp,\nbut not more. We will need to implement clock recovery logic based on\nstatistics for some platforms, and that doesn't belong to kernelspace.\n\n> > > +\t */\n> > > +\trequest->metadata().set(controls::SensorTimestamp,\n> > > +\t\t\t\tbuffer->metadata().timestamp);\n> > > +\n> > >  \t/* If the buffer is cancelled force a complete of the whole request. */\n> > >  \tif (buffer->metadata().status == FrameMetadata::FrameCancelled) {\n> > >  \t\tfor (auto it : request->buffers())","headers":{"Return-Path":"<libcamera-devel-bounces@lists.libcamera.org>","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 55FE4BDB15\n\tfor <parsemail@patchwork.libcamera.org>;\n\tTue, 20 Apr 2021 21:05:33 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id C32EB68840;\n\tTue, 20 Apr 2021 23:05:32 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 3CCC260516\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tTue, 20 Apr 2021 23:05:31 +0200 (CEST)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id A06F9411;\n\tTue, 20 Apr 2021 23:05:30 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"RqtIa0xb\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1618952730;\n\tbh=mlP57wpeBwyekesbxWeNd4tcTmpAt/rMpSFkXIjIJtU=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=RqtIa0xb06w1Czo4krOrM5Gbaz3PC4rUIafAZGhqEGa/U23OMmYkkPIZQ99nFYQ0/\n\ttCCXcJe5YuJpXLTwc3n4HiCuapqv/kXRI1ZalGGftsfWaG5SvfUy+ykbm7K4IlSM/G\n\thZMTU6pCI1+y/gMtKZ+Oo+8fhS3a5fabImq81FVU=","Date":"Wed, 21 Apr 2021 00:05:26 +0300","From":"Laurent Pinchart <laurent.pinchart@ideasonboard.com>","To":"Jacopo Mondi <jacopo@jmondi.org>","Message-ID":"<YH9CFk8VIFyJnBwI@pendragon.ideasonboard.com>","References":"<20210419131433.22920-1-jacopo@jmondi.org>\n\t<20210419131433.22920-3-jacopo@jmondi.org>\n\t<fdd0fa50-e9a3-561d-f6cd-bf20c77dc5a9@ideasonboard.com>\n\t<20210419135342.h7snlzheujfm6diw@uno.localdomain>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<20210419135342.h7snlzheujfm6diw@uno.localdomain>","Subject":"Re: [libcamera-devel] [PATCH 02/13] libcamera: ipu3: Report sensor\n\ttimestamp","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"<libcamera-devel.lists.libcamera.org>","List-Unsubscribe":"<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>","List-Archive":"<https://lists.libcamera.org/pipermail/libcamera-devel/>","List-Post":"<mailto:libcamera-devel@lists.libcamera.org>","List-Help":"<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>","List-Subscribe":"<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>"}}]