[{"id":3942,"web_url":"https://patchwork.libcamera.org/comment/3942/","msgid":"","date":"2020-03-05T16:26:03","subject":"Re: [libcamera-devel] [PATCH 24/31] libcamera: byte_stream_buffer:\n\tAdd zero-copy read() variant","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"On 29/02/2020 16:42, Laurent Pinchart wrote:\n> Add a read() function to ByteStreamBuffer that returns a pointer to the\n> data instead of copying it. Overflow check is still handled by the\n> class, but the caller must check the returned pointer explicitly.\n> \n\nI'm wondering why this doesn't return a span, but perhaps I'm about to\nfind out in a later patch if this is to implement some feature for\nhandling span ...\n\nBut still - this byte_stream_buffer.h/cpp knows how to deal with spans\nalready...\n\n> Signed-off-by: Laurent Pinchart \n\nMinors aside,\n\nReviewed-by: Kieran Bingham \n\n> ---\n> src/libcamera/byte_stream_buffer.cpp | 39 ++++++++++++++++++++++\n> src/libcamera/include/byte_stream_buffer.h | 9 +++++\n> 2 files changed, 48 insertions(+)\n> \n> diff --git a/src/libcamera/byte_stream_buffer.cpp b/src/libcamera/byte_stream_buffer.cpp\n> index 40380bf0434a..576f556f54a7 100644\n> --- a/src/libcamera/byte_stream_buffer.cpp\n> +++ b/src/libcamera/byte_stream_buffer.cpp\n> @@ -241,6 +241,19 @@ int ByteStreamBuffer::skip(size_t size)\n> * \\retval -ENOSPC no more space is available in the managed memory buffer\n> */\n> \n> +/**\n> + * \\fn template const T *ByteStreamBuffer::read(size_t count)\n> + * \\brief Read data from the managed memory buffer without copy\n\nwithout a copy? or \"without performing a copy\"\n\n> + * \\param[in] count Number of data items to read\n> + *\n> + * This function read \\a count elements of type \\a T from the buffer. Unlike\n\n\ns/read/reads/\n\n> + * the other read variants, it doesn't copy the data but returns a pointer to\n> + * the first element. If data can't be read for any reason (usually due to\n> + * reading more data than available), the function returns nullptr.\n\nreturns 'a' nullptr?\n\n> + *\n> + * \\return A pointer to the data on success, or nullptr otherwise\n> + */\n> +\n> /**\n> * \\fn template int ByteStreamBuffer::write(const T *t)\n> * \\brief Write \\a t to the managed memory buffer\n> @@ -259,6 +272,32 @@ int ByteStreamBuffer::skip(size_t size)\n> * \\retval -ENOSPC no more space is available in the managed memory buffer\n> */\n> \n> +const uint8_t *ByteStreamBuffer::read(size_t size, size_t count)\n> +{\n> +\tif (!read_)\n> +\t\treturn nullptr;\n> +\n> +\tif (overflow_)\n> +\t\treturn nullptr;\n> +\n> +\tsize_t bytes;\n> +\tif (__builtin_mul_overflow(size, count, &bytes)) {\n\nIs this a gcc builtin? or supported in clang too? (I assume it is ...)\n\n> +\t\tsetOverflow();\n> +\t\treturn nullptr;\n> +\t}\n> +\n> +\tif (read_ + bytes > base_ + size_) {\n> +\t\tLOG(Serialization, Error)\n> +\t\t\t<< \"Unable to read \" << bytes << \" bytes: out of bounds\";\n> +\t\tsetOverflow();\n> +\t\treturn nullptr;\n> +\t}\n> +\n> +\tconst uint8_t *data = read_;\n> +\tread_ += bytes;\n> +\treturn data;\n> +}\n> +\n> int ByteStreamBuffer::read(uint8_t *data, size_t size)\n> {\n> \tif (!read_)\n> diff --git a/src/libcamera/include/byte_stream_buffer.h b/src/libcamera/include/byte_stream_buffer.h\n> index 17cb0146061e..b3aaa8b9fb28 100644\n> --- a/src/libcamera/include/byte_stream_buffer.h\n> +++ b/src/libcamera/include/byte_stream_buffer.h\n> @@ -9,6 +9,7 @@\n> \n> #include \n> #include \n> +#include \n> \n> #include \n> \n> @@ -43,6 +44,13 @@ public:\n> \t\t\t data.size_bytes());\n> \t}\n> \n> +\ttemplate\n> +\tconst std::remove_reference_t *read(size_t count = 1)\n> +\t{\n> +\t\tusing return_type = const std::remove_reference_t *;\n> +\t\treturn reinterpret_cast(read(sizeof(T), count));\n> +\t}\n> +\n> \ttemplate\n> \tint write(const T *t)\n> \t{\n> @@ -63,6 +71,7 @@ private:\n> \tvoid setOverflow();\n> \n> \tint read(uint8_t *data, size_t size);\n> +\tconst uint8_t *read(size_t size, size_t count);\n> \tint write(const uint8_t *data, size_t size);\n> \n> \tByteStreamBuffer *parent_;\n>","headers":{"Return-Path":"","Received":["from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id A11B460427\n\tfor ;\n\tThu, 5 Mar 2020 17:26:18 +0100 (CET)","from [192.168.0.20]\n\t(cpc89242-aztw30-2-0-cust488.18-1.cable.virginm.net [86.31.129.233])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 05011312;\n\tThu, 5 Mar 2020 17:26:17 +0100 (CET)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1583425578;\n\tbh=cgknVsxEoKrgl2SBOP1LIKBqDPkVFKcJMDXdsgXnRZk=;\n\th=Reply-To:Subject:To:References:From:Date:In-Reply-To:From;\n\tb=rs1qcngWiwMoGVFE97DbETJOtUriwHJO56lssVAjXkE9Mz1HYEHQTyWuGickXnHkS\n\tNIErvFbC/g2EsxmBTSB64Y+1h5jhPE+3fswjKvg9sFTDhYDMxt/c/l01H2sqcF9I0A\n\tGSp2uGmSXU3gj9VpOjmCNfIMOPW+5LEIlavE7R6U=","Reply-To":"kieran.bingham@ideasonboard.com","To":"Laurent Pinchart ,\n\tlibcamera-devel@lists.libcamera.org","References":"<20200229164254.23604-1-laurent.pinchart@ideasonboard.com>\n\t<20200229164254.23604-25-laurent.pinchart@ideasonboard.com>","From":"Kieran Bingham ","Openpgp":"preference=signencrypt","Autocrypt":"addr=kieran.bingham@ideasonboard.com; keydata=\n\tmQINBFYE/WYBEACs1PwjMD9rgCu1hlIiUA1AXR4rv2v+BCLUq//vrX5S5bjzxKAryRf0uHat\n\tV/zwz6hiDrZuHUACDB7X8OaQcwhLaVlq6byfoBr25+hbZG7G3+5EUl9cQ7dQEdvNj6V6y/SC\n\trRanWfelwQThCHckbobWiQJfK9n7rYNcPMq9B8e9F020LFH7Kj6YmO95ewJGgLm+idg1Kb3C\n\tpotzWkXc1xmPzcQ1fvQMOfMwdS+4SNw4rY9f07Xb2K99rjMwZVDgESKIzhsDB5GY465sCsiQ\n\tcSAZRxqE49RTBq2+EQsbrQpIc8XiffAB8qexh5/QPzCmR4kJgCGeHIXBtgRj+nIkCJPZvZtf\n\tKr2EAbc6tgg6DkAEHJb+1okosV09+0+TXywYvtEop/WUOWQ+zo+Y/OBd+8Ptgt1pDRyOBzL8\n\tRXa8ZqRf0Mwg75D+dKntZeJHzPRJyrlfQokngAAs4PaFt6UfS+ypMAF37T6CeDArQC41V3ko\n\tlPn1yMsVD0p+6i3DPvA/GPIksDC4owjnzVX9kM8Zc5Cx+XoAN0w5Eqo4t6qEVbuettxx55gq\n\t8K8FieAjgjMSxngo/HST8TpFeqI5nVeq0/lqtBRQKumuIqDg+Bkr4L1V/PSB6XgQcOdhtd36\n\tOe9X9dXB8YSNt7VjOcO7BTmFn/Z8r92mSAfHXpb07YJWJosQOQARAQABtDBLaWVyYW4gQmlu\n\tZ2hhbSA8a2llcmFuLmJpbmdoYW1AaWRlYXNvbmJvYXJkLmNvbT6JAlcEEwEKAEECGwMFCwkI\n\tBwIGFQgJCgsCBBYCAwECHgECF4ACGQEWIQSQLdeYP70o/eNy1HqhHkZyEKRh/QUCXWTtygUJ\n\tCyJXZAAKCRChHkZyEKRh/f8dEACTDsbLN2nioNZMwyLuQRUAFcXNolDX48xcUXsWS2QjxaPm\n\tVsJx8Uy8aYkS85mdPBh0C83OovQR/OVbr8AxhGvYqBs3nQvbWuTl/+4od7DfK2VZOoKBAu5S\n\tQK2FYuUcikDqYcFWJ8DQnubxfE8dvzojHEkXw0sA4igINHDDFX3HJGZtLio+WpEFQtCbfTAG\n\tYZslasz1YZRbwEdSsmO3/kqy5eMnczlm8a21A3fKUo3g8oAZEFM+f4DUNzqIltg31OAB/kZS\n\tenKZQ/SWC8PmLg/ZXBrReYakxXtkP6w3FwMlzOlhGxqhIRNiAJfXJBaRhuUWzPOpEDE9q5YJ\n\tBmqQL2WJm1VSNNVxbXJHpaWMH1sA2R00vmvRrPXGwyIO0IPYeUYQa3gsy6k+En/aMQJd27dp\n\taScf9am9PFICPY5T4ppneeJLif2lyLojo0mcHOV+uyrds9XkLpp14GfTkeKPdPMrLLTsHRfH\n\tfA4I4OBpRrEPiGIZB/0im98MkGY/Mu6qxeZmYLCcgD6qz4idOvfgVOrNh+aA8HzIVR+RMW8H\n\tQGBN9f0E3kfwxuhl3omo6V7lDw8XOdmuWZNC9zPq1UfryVHANYbLGz9KJ4Aw6M+OgBC2JpkD\n\thXMdHUkC+d20dwXrwHTlrJi1YNp6rBc+xald3wsUPOZ5z8moTHUX/uPA/qhGsbkCDQRWBP1m\n\tARAAzijkb+Sau4hAncr1JjOY+KyFEdUNxRy+hqTJdJfaYihxyaj0Ee0P0zEi35CbE6lgU0Uz\n\ttih9fiUbSV3wfsWqg1Ut3/5rTKu7kLFp15kF7eqvV4uezXRD3Qu4yjv/rMmEJbbD4cTvGCYI\n\td6MDC417f7vK3hCbCVIZSp3GXxyC1LU+UQr3fFcOyCwmP9vDUR9JV0BSqHHxRDdpUXE26Dk6\n\tmhf0V1YkspE5St814ETXpEus2urZE5yJIUROlWPIL+hm3NEWfAP06vsQUyLvr/GtbOT79vXl\n\tEn1aulcYyu20dRRxhkQ6iILaURcxIAVJJKPi8dsoMnS8pB0QW12AHWuirPF0g6DiuUfPmrA5\n\tPKe56IGlpkjc8cO51lIxHkWTpCMWigRdPDexKX+Sb+W9QWK/0JjIc4t3KBaiG8O4yRX8ml2R\n\t+rxfAVKM6V769P/hWoRGdgUMgYHFpHGSgEt80OKK5HeUPy2cngDUXzwrqiM5Sz6Od0qw5pCk\n\tNlXqI0W/who0iSVM+8+RmyY0OEkxEcci7rRLsGnM15B5PjLJjh1f2ULYkv8s4SnDwMZ/kE04\n\t/UqCMK/KnX8pwXEMCjz0h6qWNpGwJ0/tYIgQJZh6bqkvBrDogAvuhf60Sogw+mH8b+PBlx1L\n\toeTK396wc+4c3BfiC6pNtUS5GpsPMMjYMk7kVvEAEQEAAYkCPAQYAQoAJgIbDBYhBJAt15g/\n\tvSj943LUeqEeRnIQpGH9BQJdizzIBQkLSKZiAAoJEKEeRnIQpGH9eYgQAJpjaWNgqNOnMTmD\n\tMJggbwjIotypzIXfhHNCeTkG7+qCDlSaBPclcPGYrTwCt0YWPU2TgGgJrVhYT20ierN8LUvj\n\t6qOPTd+Uk7NFzL65qkh80ZKNBFddx1AabQpSVQKbdcLb8OFs85kuSvFdgqZwgxA1vl4TFhNz\n\tPZ79NAmXLackAx3sOVFhk4WQaKRshCB7cSl+RIng5S/ThOBlwNlcKG7j7W2MC06BlTbdEkUp\n\tECzuuRBv8wX4OQl+hbWbB/VKIx5HKlLu1eypen/5lNVzSqMMIYkkZcjV2SWQyUGxSwq0O/sx\n\tS0A8/atCHUXOboUsn54qdxrVDaK+6jIAuo8JiRWctP16KjzUM7MO0/+4zllM8EY57rXrj48j\n\tsbEYX0YQnzaj+jO6kJtoZsIaYR7rMMq9aUAjyiaEZpmP1qF/2sYenDx0Fg2BSlLvLvXM0vU8\n\tpQk3kgDu7kb/7PRYrZvBsr21EIQoIjXbZxDz/o7z95frkP71EaICttZ6k9q5oxxA5WC6sTXc\n\tMW8zs8avFNuA9VpXt0YupJd2ijtZy2mpZNG02fFVXhIn4G807G7+9mhuC4XG5rKlBBUXTvPU\n\tAfYnB4JBDLmLzBFavQfvonSfbitgXwCG3vS+9HEwAjU30Bar1PEOmIbiAoMzuKeRm2LVpmq4\n\tWZw01QYHU/GUV/zHJSFk","Organization":"Ideas on Board","Message-ID":"","Date":"Thu, 5 Mar 2020 16:26:03 +0000","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101\n\tThunderbird/60.9.1","MIME-Version":"1.0","In-Reply-To":"<20200229164254.23604-25-laurent.pinchart@ideasonboard.com>","Content-Type":"text/plain; charset=utf-8","Content-Language":"en-GB","Content-Transfer-Encoding":"8bit","Subject":"Re: [libcamera-devel] [PATCH 24/31] libcamera: byte_stream_buffer:\n\tAdd zero-copy read() variant","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","X-List-Received-Date":"Thu, 05 Mar 2020 16:26:20 -0000"}},{"id":3954,"web_url":"https://patchwork.libcamera.org/comment/3954/","msgid":"<20200306145435.GK4878@pendragon.ideasonboard.com>","date":"2020-03-06T14:54:35","subject":"Re: [libcamera-devel] [PATCH 24/31] libcamera: byte_stream_buffer:\n\tAdd zero-copy read() variant","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Kieran,\n\nOn Thu, Mar 05, 2020 at 04:26:03PM +0000, Kieran Bingham wrote:\n> On 29/02/2020 16:42, Laurent Pinchart wrote:\n> > Add a read() function to ByteStreamBuffer that returns a pointer to the\n> > data instead of copying it. Overflow check is still handled by the\n> > class, but the caller must check the returned pointer explicitly.\n> \n> I'm wondering why this doesn't return a span, but perhaps I'm about to\n> find out in a later patch if this is to implement some feature for\n> handling span ...\n\nThe idea is to offer a safe way to access memory in the byte stream\nbuffer through a typed pointer, without casting explicitly in the caller\nas that would be unsafe.\n\n> But still - this byte_stream_buffer.h/cpp knows how to deal with spans\n> already...\n> \n> > Signed-off-by: Laurent Pinchart \n> \n> Minors aside,\n> \n> Reviewed-by: Kieran Bingham \n> \n> > ---\n> > src/libcamera/byte_stream_buffer.cpp | 39 ++++++++++++++++++++++\n> > src/libcamera/include/byte_stream_buffer.h | 9 +++++\n> > 2 files changed, 48 insertions(+)\n> > \n> > diff --git a/src/libcamera/byte_stream_buffer.cpp b/src/libcamera/byte_stream_buffer.cpp\n> > index 40380bf0434a..576f556f54a7 100644\n> > --- a/src/libcamera/byte_stream_buffer.cpp\n> > +++ b/src/libcamera/byte_stream_buffer.cpp\n> > @@ -241,6 +241,19 @@ int ByteStreamBuffer::skip(size_t size)\n> > * \\retval -ENOSPC no more space is available in the managed memory buffer\n> > */\n> > \n> > +/**\n> > + * \\fn template const T *ByteStreamBuffer::read(size_t count)\n> > + * \\brief Read data from the managed memory buffer without copy\n> \n> without a copy? or \"without performing a copy\"\n\nI've used the second option.\n\n> > + * \\param[in] count Number of data items to read\n> > + *\n> > + * This function read \\a count elements of type \\a T from the buffer. Unlike\n> \n> s/read/reads/\n> \n> > + * the other read variants, it doesn't copy the data but returns a pointer to\n> > + * the first element. If data can't be read for any reason (usually due to\n> > + * reading more data than available), the function returns nullptr.\n> \n> returns 'a' nullptr?\n\n'nullptr' is a value, like NULL\n(https://en.cppreference.com/w/cpp/language/nullptr), so I think this is\ncorrect.\n\n> \n> > + *\n> > + * \\return A pointer to the data on success, or nullptr otherwise\n> > + */\n> > +\n> > /**\n> > * \\fn template int ByteStreamBuffer::write(const T *t)\n> > * \\brief Write \\a t to the managed memory buffer\n> > @@ -259,6 +272,32 @@ int ByteStreamBuffer::skip(size_t size)\n> > * \\retval -ENOSPC no more space is available in the managed memory buffer\n> > */\n> > \n> > +const uint8_t *ByteStreamBuffer::read(size_t size, size_t count)\n> > +{\n> > +\tif (!read_)\n> > +\t\treturn nullptr;\n> > +\n> > +\tif (overflow_)\n> > +\t\treturn nullptr;\n> > +\n> > +\tsize_t bytes;\n> > +\tif (__builtin_mul_overflow(size, count, &bytes)) {\n> \n> Is this a gcc builtin? or supported in clang too? (I assume it is ...)\n\nIt's supported by both gcc and clang.\n\n> > +\t\tsetOverflow();\n> > +\t\treturn nullptr;\n> > +\t}\n> > +\n> > +\tif (read_ + bytes > base_ + size_) {\n> > +\t\tLOG(Serialization, Error)\n> > +\t\t\t<< \"Unable to read \" << bytes << \" bytes: out of bounds\";\n> > +\t\tsetOverflow();\n> > +\t\treturn nullptr;\n> > +\t}\n> > +\n> > +\tconst uint8_t *data = read_;\n> > +\tread_ += bytes;\n> > +\treturn data;\n> > +}\n> > +\n> > int ByteStreamBuffer::read(uint8_t *data, size_t size)\n> > {\n> > \tif (!read_)\n> > diff --git a/src/libcamera/include/byte_stream_buffer.h b/src/libcamera/include/byte_stream_buffer.h\n> > index 17cb0146061e..b3aaa8b9fb28 100644\n> > --- a/src/libcamera/include/byte_stream_buffer.h\n> > +++ b/src/libcamera/include/byte_stream_buffer.h\n> > @@ -9,6 +9,7 @@\n> > \n> > #include \n> > #include \n> > +#include \n> > \n> > #include \n> > \n> > @@ -43,6 +44,13 @@ public:\n> > \t\t\t data.size_bytes());\n> > \t}\n> > \n> > +\ttemplate\n> > +\tconst std::remove_reference_t *read(size_t count = 1)\n> > +\t{\n> > +\t\tusing return_type = const std::remove_reference_t *;\n> > +\t\treturn reinterpret_cast(read(sizeof(T), count));\n> > +\t}\n> > +\n> > \ttemplate\n> > \tint write(const T *t)\n> > \t{\n> > @@ -63,6 +71,7 @@ private:\n> > \tvoid setOverflow();\n> > \n> > \tint read(uint8_t *data, size_t size);\n> > +\tconst uint8_t *read(size_t size, size_t count);\n> > \tint write(const uint8_t *data, size_t size);\n> > \n> > \tByteStreamBuffer *parent_;","headers":{"Return-Path":"","Received":["from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 4BB5860424\n\tfor ;\n\tFri, 6 Mar 2020 15:54:38 +0100 (CET)","from pendragon.ideasonboard.com (81-175-216-236.bb.dnainternet.fi\n\t[81.175.216.236])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id AE8A024B;\n\tFri, 6 Mar 2020 15:54:37 +0100 (CET)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1583506477;\n\tbh=b90AhKpDc929Q91JTKhV2tEXgUHMnTRM05VUFu+/98o=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=PZugNm1DcaD4fSCcLQMS3UYTZPvgPLmjKYV4P5rpzUjOlB+8Y8ZGOa//C9PVAPbxe\n\tLy34QjXSvsCOXmKp62w+ak/VJAm5suoZPSrWDvOHYd+4Dp7G22Vog3//mOTpnvU/Fl\n\thC/G2jKj4jSX6aEx7Y9H8EwwkV8COl8uZ6TjYks8=","Date":"Fri, 6 Mar 2020 16:54:35 +0200","From":"Laurent Pinchart ","To":"Kieran Bingham ","Cc":"libcamera-devel@lists.libcamera.org","Message-ID":"<20200306145435.GK4878@pendragon.ideasonboard.com>","References":"<20200229164254.23604-1-laurent.pinchart@ideasonboard.com>\n\t<20200229164254.23604-25-laurent.pinchart@ideasonboard.com>\n\t","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"","User-Agent":"Mutt/1.10.1 (2018-07-13)","Subject":"Re: [libcamera-devel] [PATCH 24/31] libcamera: byte_stream_buffer:\n\tAdd zero-copy read() variant","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","X-List-Received-Date":"Fri, 06 Mar 2020 14:54:38 -0000"}}]