[{"id":15360,"web_url":"https://patchwork.libcamera.org/comment/15360/","msgid":"<150c1b51-81d8-19ab-27fe-9a65f8526e71@ideasonboard.com>","date":"2021-03-01T20:23:54","subject":"Re: [libcamera-devel] [PATCH 12/20] libcamera: pipeline: simple:\n\tDocument the pipeline handler design","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"On 31/01/2021 22:46, Laurent Pinchart wrote:\n> The simple pipeline handler has grown over time, and isn't that simple\n> anymore that it can easily be understood by an unfamiliar reader.\n> Document the design to explicitly state the expectations of the pipeline\n> handler, and to explain how it operates.\n\nShould it be renamed?\n\nGenericPipelineHandler... something else ?\n\n\n> Signed-off-by: Laurent Pinchart \n> ---\n> src/libcamera/pipeline/simple/simple.cpp | 83 ++++++++++++++++++++++++\n> 1 file changed, 83 insertions(+)\n> \n> diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp\n> index f072e0f1fa81..7c5a56a2f395 100644\n> --- a/src/libcamera/pipeline/simple/simple.cpp\n> +++ b/src/libcamera/pipeline/simple/simple.cpp\n> @@ -38,6 +38,89 @@ namespace libcamera {\n> \n> LOG_DEFINE_CATEGORY(SimplePipeline)\n> \n> +/* -----------------------------------------------------------------------------\n> + *\n> + * Overview\n> + * --------\n> + *\n> + * The SimplePipelineHandler relies on generic kernel APIs to control a camera\n> + * device, without any device-specific code and with limited device-specific\n> + * static data.\n> + *\n> + * To quality for support by the simple pipeline handler, a device shall\n\ns/quality/qualify/\n\n> + *\n> + * - be supported by V4L2 drivers, exposing the Media Controller API, the V4L2\n> + * subdev APIs and the media bus format-based enumeration extension for the\n> + * VIDIOC_ENUM_FMT ioctl ;\n> + * - not expose any device-specific API from drivers to userspace ;\n> + * - include one or more camera sensor media entities and one or more video\n> + * capture devices ;\n> + * - have a capture pipeline with linear paths from the camera sensors to the\n> + * video capture devices ; and\n> + * - have an optional memory-to-memory device to perform format conversion\n> + * and/or scaling, exposed as a V4L2 M2M device.\n> + *\n> + * As devices that require a specific pipeline handler may still match the\n> + * above characteristics, the simple pipeline handler doesn't attempt to\n> + * automatically determine which devices it can support. It instead relies on\n> + * an explicit list of supported devices, provided in the supportedDevices\n> + * array.\n> + *\n> + * When matching a device, the pipeline handler enumerates all camera sensors\n> + * and attempts, for each of them, to find a path to a video captude video node.\n\ns/captude/capture/\n\n> + * It does so by traversing the media graph, following the first non permanently\n> + * disabled downstream link. If such a path is found, the pipeline handler\n> + * creates a corresponding SimpleCameraData instance, and stores the media graph\n> + * path in its entities_ list.\n> + *\n> + * A more complex graph search algorithm could be implemented if a device that\n> + * would otherwise be compatible with the pipeline handler isn't correctly\n> + * handled by this heuristic.\n> + *\n> + * Once the camera data instances have been created, the match() function\n> + * creates a V4L2Subdevice instance for each entity used by any of the cameras\n> + * and stores the instances in SimplePipelineHandler::subdevs_, accessible by\n> + * the SimpleCameraData class through the SimplePipelineHandler::subdev()\n> + * function. This avoids duplication of subdev instances between different\n> + * cameras when the same entity is used in multiple paths. A similar mechanism\n> + * is used for V4L2VideoDevice instances, but instances are in this case created\n> + * on demand when access through SimplePipelineHandler::video() instead of all\n\ns/access/accessed/\n\n> + * in one go at initialization time.\n> + *\n> + * Finally, all camera data instances are initialized to gather information\n> + * about the possible pipeline configurations for the corresponding camera. If\n> + * valid pipeline configurations are found, a Camera is registered for the\n> + * SimpleCameraData instance.\n> + *\n> + * Pipeline Configuration\n> + * ----------------------\n> + *\n> + * The simple pipeline handler configures the pipeline by propagating V4L2\n> + * subdev formats from the camera sensor to the video node. The format is first\n> + * set on the camera sensor's output, using the native camera sensor\n> + * resolution. Then, on every link in the pipeline, the format is retrieved on\n> + * the link source and set unmodified on the link sink.\n> + *\n> + * When initializating the camera data, this above procedure is repeated for\n> + * every media bus format supported by the camera sensor. Upon reaching the\n> + * video node, the pixel formats compatible with the media bus format are\n> + * enumerated. Each of those pixel formats correspond to one possible pipeline\n\ns/correspond/corresponds/\t\n\n> + * configuration, stored as an intsance of SimpleCameraData::Configuration in\n\ns/intsance/instance/\n\n> + * the SimpleCameraData::formats_ map.\n> + *\n> + * Format Conversion and Scaling\n> + * -----------------------------\n> + *\n> + * The capture pipeline isn't expected to include a scaler, and if a scaler is\n> + * available, it is ignored when configuring the pipeline. However, the simple\n> + * pipeline handler supports optional memory-to-memory converters to scale the\n> + * image and convert it to a different pixel format. If such a converter is\n> + * present, the pipeline handler enumerates, for each pipeline configuration,\n> + * the pixel formats and sizes that the converter can produce for the output of\n> + * the capture video node, and stores the information in the outputFormats and\n> + * outputSizes of the SimpleCameraData::Configuration structure.\n> + */\n> +\n> class SimplePipelineHandler;\n> \n\nWith all that fixed,\n\nReviewed-by: Kieran Bingham \n\n> struct SimplePipelineInfo {\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 9D730BD1F1\n\tfor ;\n\tMon, 1 Mar 2021 20:23:59 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 02DE668A7D;\n\tMon, 1 Mar 2021 21:23:59 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id B78CD60521\n\tfor ;\n\tMon, 1 Mar 2021 21:23:57 +0100 (CET)","from [192.168.0.20]\n\t(cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 178FE41;\n\tMon, 1 Mar 2021 21:23:57 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"Pvr0ZtRc\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1614630237;\n\tbh=xLS0lv3b2V9oHh/JvCA8PbvMqEapYgqMPX5NDbPUp+s=;\n\th=Reply-To:Subject:To:Cc:References:From:Date:In-Reply-To:From;\n\tb=Pvr0ZtRcfqWLzhH1jNEDrCc/Emdo58ZGqeacJ5h51/zNXvG/oj//VZPECmZZUSOH/\n\tHD8RYW882givyNFRKRLHtePgOOEybcFOdtdEEydRmOMKuv6QfakYaoO/PNmSjsAe12\n\tJ6mEEiqCQptjPaP5XRQKxAXiU8rPkwelZOmPu7AU=","To":"Laurent Pinchart ,\n\tlibcamera-devel@lists.libcamera.org","References":"<20210131224702.8838-1-laurent.pinchart@ideasonboard.com>\n\t<20210131224702.8838-13-laurent.pinchart@ideasonboard.com>","From":"Kieran Bingham ","Autocrypt":"addr=kieran.bingham@ideasonboard.com; keydata=\n\tmQINBFYE/WYBEACs1PwjMD9rgCu1hlIiUA1AXR4rv2v+BCLUq//vrX5S5bjzxKAryRf0uHat\n\tV/zwz6hiDrZuHUACDB7X8OaQcwhLaVlq6byfoBr25+hbZG7G3+5EUl9cQ7dQEdvNj6V6y/SC\n\trRanWfelwQThCHckbobWiQJfK9n7rYNcPMq9B8e9F020LFH7Kj6YmO95ewJGgLm+idg1Kb3C\n\tpotzWkXc1xmPzcQ1fvQMOfMwdS+4SNw4rY9f07Xb2K99rjMwZVDgESKIzhsDB5GY465sCsiQ\n\tcSAZRxqE49RTBq2+EQsbrQpIc8XiffAB8qexh5/QPzCmR4kJgCGeHIXBtgRj+nIkCJPZvZtf\n\tKr2EAbc6tgg6DkAEHJb+1okosV09+0+TXywYvtEop/WUOWQ+zo+Y/OBd+8Ptgt1pDRyOBzL8\n\tRXa8ZqRf0Mwg75D+dKntZeJHzPRJyrlfQokngAAs4PaFt6UfS+ypMAF37T6CeDArQC41V3ko\n\tlPn1yMsVD0p+6i3DPvA/GPIksDC4owjnzVX9kM8Zc5Cx+XoAN0w5Eqo4t6qEVbuettxx55gq\n\t8K8FieAjgjMSxngo/HST8TpFeqI5nVeq0/lqtBRQKumuIqDg+Bkr4L1V/PSB6XgQcOdhtd36\n\tOe9X9dXB8YSNt7VjOcO7BTmFn/Z8r92mSAfHXpb07YJWJosQOQARAQABtDBLaWVyYW4gQmlu\n\tZ2hhbSA8a2llcmFuLmJpbmdoYW1AaWRlYXNvbmJvYXJkLmNvbT6JAlcEEwEKAEECGwMFCwkI\n\tBwIGFQgJCgsCBBYCAwECHgECF4ACGQEWIQSQLdeYP70o/eNy1HqhHkZyEKRh/QUCXWTtygUJ\n\tCyJXZAAKCRChHkZyEKRh/f8dEACTDsbLN2nioNZMwyLuQRUAFcXNolDX48xcUXsWS2QjxaPm\n\tVsJx8Uy8aYkS85mdPBh0C83OovQR/OVbr8AxhGvYqBs3nQvbWuTl/+4od7DfK2VZOoKBAu5S\n\tQK2FYuUcikDqYcFWJ8DQnubxfE8dvzojHEkXw0sA4igINHDDFX3HJGZtLio+WpEFQtCbfTAG\n\tYZslasz1YZRbwEdSsmO3/kqy5eMnczlm8a21A3fKUo3g8oAZEFM+f4DUNzqIltg31OAB/kZS\n\tenKZQ/SWC8PmLg/ZXBrReYakxXtkP6w3FwMlzOlhGxqhIRNiAJfXJBaRhuUWzPOpEDE9q5YJ\n\tBmqQL2WJm1VSNNVxbXJHpaWMH1sA2R00vmvRrPXGwyIO0IPYeUYQa3gsy6k+En/aMQJd27dp\n\taScf9am9PFICPY5T4ppneeJLif2lyLojo0mcHOV+uyrds9XkLpp14GfTkeKPdPMrLLTsHRfH\n\tfA4I4OBpRrEPiGIZB/0im98MkGY/Mu6qxeZmYLCcgD6qz4idOvfgVOrNh+aA8HzIVR+RMW8H\n\tQGBN9f0E3kfwxuhl3omo6V7lDw8XOdmuWZNC9zPq1UfryVHANYbLGz9KJ4Aw6M+OgBC2JpkD\n\thXMdHUkC+d20dwXrwHTlrJi1YNp6rBc+xald3wsUPOZ5z8moTHUX/uPA/qhGsbkCDQRWBP1m\n\tARAAzijkb+Sau4hAncr1JjOY+KyFEdUNxRy+hqTJdJfaYihxyaj0Ee0P0zEi35CbE6lgU0Uz\n\ttih9fiUbSV3wfsWqg1Ut3/5rTKu7kLFp15kF7eqvV4uezXRD3Qu4yjv/rMmEJbbD4cTvGCYI\n\td6MDC417f7vK3hCbCVIZSp3GXxyC1LU+UQr3fFcOyCwmP9vDUR9JV0BSqHHxRDdpUXE26Dk6\n\tmhf0V1YkspE5St814ETXpEus2urZE5yJIUROlWPIL+hm3NEWfAP06vsQUyLvr/GtbOT79vXl\n\tEn1aulcYyu20dRRxhkQ6iILaURcxIAVJJKPi8dsoMnS8pB0QW12AHWuirPF0g6DiuUfPmrA5\n\tPKe56IGlpkjc8cO51lIxHkWTpCMWigRdPDexKX+Sb+W9QWK/0JjIc4t3KBaiG8O4yRX8ml2R\n\t+rxfAVKM6V769P/hWoRGdgUMgYHFpHGSgEt80OKK5HeUPy2cngDUXzwrqiM5Sz6Od0qw5pCk\n\tNlXqI0W/who0iSVM+8+RmyY0OEkxEcci7rRLsGnM15B5PjLJjh1f2ULYkv8s4SnDwMZ/kE04\n\t/UqCMK/KnX8pwXEMCjz0h6qWNpGwJ0/tYIgQJZh6bqkvBrDogAvuhf60Sogw+mH8b+PBlx1L\n\toeTK396wc+4c3BfiC6pNtUS5GpsPMMjYMk7kVvEAEQEAAYkCPAQYAQoAJgIbDBYhBJAt15g/\n\tvSj943LUeqEeRnIQpGH9BQJdizzIBQkLSKZiAAoJEKEeRnIQpGH9eYgQAJpjaWNgqNOnMTmD\n\tMJggbwjIotypzIXfhHNCeTkG7+qCDlSaBPclcPGYrTwCt0YWPU2TgGgJrVhYT20ierN8LUvj\n\t6qOPTd+Uk7NFzL65qkh80ZKNBFddx1AabQpSVQKbdcLb8OFs85kuSvFdgqZwgxA1vl4TFhNz\n\tPZ79NAmXLackAx3sOVFhk4WQaKRshCB7cSl+RIng5S/ThOBlwNlcKG7j7W2MC06BlTbdEkUp\n\tECzuuRBv8wX4OQl+hbWbB/VKIx5HKlLu1eypen/5lNVzSqMMIYkkZcjV2SWQyUGxSwq0O/sx\n\tS0A8/atCHUXOboUsn54qdxrVDaK+6jIAuo8JiRWctP16KjzUM7MO0/+4zllM8EY57rXrj48j\n\tsbEYX0YQnzaj+jO6kJtoZsIaYR7rMMq9aUAjyiaEZpmP1qF/2sYenDx0Fg2BSlLvLvXM0vU8\n\tpQk3kgDu7kb/7PRYrZvBsr21EIQoIjXbZxDz/o7z95frkP71EaICttZ6k9q5oxxA5WC6sTXc\n\tMW8zs8avFNuA9VpXt0YupJd2ijtZy2mpZNG02fFVXhIn4G807G7+9mhuC4XG5rKlBBUXTvPU\n\tAfYnB4JBDLmLzBFavQfvonSfbitgXwCG3vS+9HEwAjU30Bar1PEOmIbiAoMzuKeRm2LVpmq4\n\tWZw01QYHU/GUV/zHJSFk","Organization":"Ideas on Board","Message-ID":"<150c1b51-81d8-19ab-27fe-9a65f8526e71@ideasonboard.com>","Date":"Mon, 1 Mar 2021 20:23:54 +0000","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101\n\tThunderbird/68.10.0","MIME-Version":"1.0","In-Reply-To":"<20210131224702.8838-13-laurent.pinchart@ideasonboard.com>","Content-Language":"en-GB","Subject":"Re: [libcamera-devel] [PATCH 12/20] libcamera: pipeline: simple:\n\tDocument the pipeline handler design","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Reply-To":"kieran.bingham@ideasonboard.com","Cc":"Phi-Bang Nguyen ","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":15365,"web_url":"https://patchwork.libcamera.org/comment/15365/","msgid":"","date":"2021-03-01T22:46:56","subject":"Re: [libcamera-devel] [PATCH 12/20] libcamera: pipeline: simple:\n\tDocument the pipeline handler design","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Kieran,\n\nOn Mon, Mar 01, 2021 at 08:23:54PM +0000, Kieran Bingham wrote:\n> On 31/01/2021 22:46, Laurent Pinchart wrote:\n> > The simple pipeline handler has grown over time, and isn't that simple\n> > anymore that it can easily be understood by an unfamiliar reader.\n> > Document the design to explicitly state the expectations of the pipeline\n> > handler, and to explain how it operates.\n> \n> Should it be renamed?\n> \n> GenericPipelineHandler... something else ?\n\nPossibly, especially that it will get less and less simple over time :-)\nI'd rather do this on top of this series though.\n\n> > Signed-off-by: Laurent Pinchart \n> > ---\n> > src/libcamera/pipeline/simple/simple.cpp | 83 ++++++++++++++++++++++++\n> > 1 file changed, 83 insertions(+)\n> > \n> > diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp\n> > index f072e0f1fa81..7c5a56a2f395 100644\n> > --- a/src/libcamera/pipeline/simple/simple.cpp\n> > +++ b/src/libcamera/pipeline/simple/simple.cpp\n> > @@ -38,6 +38,89 @@ namespace libcamera {\n> > \n> > LOG_DEFINE_CATEGORY(SimplePipeline)\n> > \n> > +/* -----------------------------------------------------------------------------\n> > + *\n> > + * Overview\n> > + * --------\n> > + *\n> > + * The SimplePipelineHandler relies on generic kernel APIs to control a camera\n> > + * device, without any device-specific code and with limited device-specific\n> > + * static data.\n> > + *\n> > + * To quality for support by the simple pipeline handler, a device shall\n> \n> s/quality/qualify/\n> \n> > + *\n> > + * - be supported by V4L2 drivers, exposing the Media Controller API, the V4L2\n> > + * subdev APIs and the media bus format-based enumeration extension for the\n> > + * VIDIOC_ENUM_FMT ioctl ;\n> > + * - not expose any device-specific API from drivers to userspace ;\n> > + * - include one or more camera sensor media entities and one or more video\n> > + * capture devices ;\n> > + * - have a capture pipeline with linear paths from the camera sensors to the\n> > + * video capture devices ; and\n> > + * - have an optional memory-to-memory device to perform format conversion\n> > + * and/or scaling, exposed as a V4L2 M2M device.\n> > + *\n> > + * As devices that require a specific pipeline handler may still match the\n> > + * above characteristics, the simple pipeline handler doesn't attempt to\n> > + * automatically determine which devices it can support. It instead relies on\n> > + * an explicit list of supported devices, provided in the supportedDevices\n> > + * array.\n> > + *\n> > + * When matching a device, the pipeline handler enumerates all camera sensors\n> > + * and attempts, for each of them, to find a path to a video captude video node.\n> \n> s/captude/capture/\n> \n> > + * It does so by traversing the media graph, following the first non permanently\n> > + * disabled downstream link. If such a path is found, the pipeline handler\n> > + * creates a corresponding SimpleCameraData instance, and stores the media graph\n> > + * path in its entities_ list.\n> > + *\n> > + * A more complex graph search algorithm could be implemented if a device that\n> > + * would otherwise be compatible with the pipeline handler isn't correctly\n> > + * handled by this heuristic.\n> > + *\n> > + * Once the camera data instances have been created, the match() function\n> > + * creates a V4L2Subdevice instance for each entity used by any of the cameras\n> > + * and stores the instances in SimplePipelineHandler::subdevs_, accessible by\n> > + * the SimpleCameraData class through the SimplePipelineHandler::subdev()\n> > + * function. This avoids duplication of subdev instances between different\n> > + * cameras when the same entity is used in multiple paths. A similar mechanism\n> > + * is used for V4L2VideoDevice instances, but instances are in this case created\n> > + * on demand when access through SimplePipelineHandler::video() instead of all\n> \n> s/access/accessed/\n> \n> > + * in one go at initialization time.\n> > + *\n> > + * Finally, all camera data instances are initialized to gather information\n> > + * about the possible pipeline configurations for the corresponding camera. If\n> > + * valid pipeline configurations are found, a Camera is registered for the\n> > + * SimpleCameraData instance.\n> > + *\n> > + * Pipeline Configuration\n> > + * ----------------------\n> > + *\n> > + * The simple pipeline handler configures the pipeline by propagating V4L2\n> > + * subdev formats from the camera sensor to the video node. The format is first\n> > + * set on the camera sensor's output, using the native camera sensor\n> > + * resolution. Then, on every link in the pipeline, the format is retrieved on\n> > + * the link source and set unmodified on the link sink.\n> > + *\n> > + * When initializating the camera data, this above procedure is repeated for\n> > + * every media bus format supported by the camera sensor. Upon reaching the\n> > + * video node, the pixel formats compatible with the media bus format are\n> > + * enumerated. Each of those pixel formats correspond to one possible pipeline\n> \n> s/correspond/corresponds/\t\n> \n> > + * configuration, stored as an intsance of SimpleCameraData::Configuration in\n> \n> s/intsance/instance/\n> \n> > + * the SimpleCameraData::formats_ map.\n> > + *\n> > + * Format Conversion and Scaling\n> > + * -----------------------------\n> > + *\n> > + * The capture pipeline isn't expected to include a scaler, and if a scaler is\n> > + * available, it is ignored when configuring the pipeline. However, the simple\n> > + * pipeline handler supports optional memory-to-memory converters to scale the\n> > + * image and convert it to a different pixel format. If such a converter is\n> > + * present, the pipeline handler enumerates, for each pipeline configuration,\n> > + * the pixel formats and sizes that the converter can produce for the output of\n> > + * the capture video node, and stores the information in the outputFormats and\n> > + * outputSizes of the SimpleCameraData::Configuration structure.\n> > + */\n> > +\n> > class SimplePipelineHandler;\n> > \n> \n> With all that fixed,\n> \n> Reviewed-by: Kieran Bingham \n> \n> > struct SimplePipelineInfo {\n> >","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 3ACDCBD1F1\n\tfor ;\n\tMon, 1 Mar 2021 22:47:27 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id B049C68A92;\n\tMon, 1 Mar 2021 23:47:26 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 7030F68A69\n\tfor ;\n\tMon, 1 Mar 2021 23:47:25 +0100 (CET)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id C831D332;\n\tMon, 1 Mar 2021 23:47:24 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"mbYWSqNG\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1614638845;\n\tbh=/IoOsHTH/DL54gQ9cDrAofxQP7hP4nmyzG11RwrXbSs=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=mbYWSqNG7AWYFp8cPK5dShGqR+XiIP4MMBFLpBWLhh4ppKN+dRvwdWCcW/YWEaLfQ\n\tSeuTzK3tiAPyyFmkuKpi6MDhrJZY+Z/QOFUtpgRavV0CwfJjoKhMhC1J8xVCxDIMVn\n\tutdEZcxo7getycckRmlQBXhQibIRQtRSXH7/qLFw=","Date":"Tue, 2 Mar 2021 00:46:56 +0200","From":"Laurent Pinchart ","To":"Kieran Bingham ","Message-ID":"","References":"<20210131224702.8838-1-laurent.pinchart@ideasonboard.com>\n\t<20210131224702.8838-13-laurent.pinchart@ideasonboard.com>\n\t<150c1b51-81d8-19ab-27fe-9a65f8526e71@ideasonboard.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<150c1b51-81d8-19ab-27fe-9a65f8526e71@ideasonboard.com>","Subject":"Re: [libcamera-devel] [PATCH 12/20] libcamera: pipeline: simple:\n\tDocument the pipeline handler design","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Phi-Bang Nguyen ,\n\tlibcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":15378,"web_url":"https://patchwork.libcamera.org/comment/15378/","msgid":"<20210302005544.GK3084@pyrite.rasen.tech>","date":"2021-03-02T00:55:44","subject":"Re: [libcamera-devel] [PATCH 12/20] libcamera: pipeline: simple:\n\tDocument the pipeline handler design","submitter":{"id":17,"url":"https://patchwork.libcamera.org/api/people/17/","name":"Paul Elder","email":"paul.elder@ideasonboard.com"},"content":"Hi Laurent,\n\nOn Mon, Feb 01, 2021 at 12:46:54AM +0200, Laurent Pinchart wrote:\n> The simple pipeline handler has grown over time, and isn't that simple\n> anymore that it can easily be understood by an unfamiliar reader.\n> Document the design to explicitly state the expectations of the pipeline\n> handler, and to explain how it operates.\n> \n> Signed-off-by: Laurent Pinchart \n\nWith all the fixes suggested by Kieran,\n\nReviewed-by: Paul Elder \n\n> ---\n> src/libcamera/pipeline/simple/simple.cpp | 83 ++++++++++++++++++++++++\n> 1 file changed, 83 insertions(+)\n> \n> diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp\n> index f072e0f1fa81..7c5a56a2f395 100644\n> --- a/src/libcamera/pipeline/simple/simple.cpp\n> +++ b/src/libcamera/pipeline/simple/simple.cpp\n> @@ -38,6 +38,89 @@ namespace libcamera {\n> \n> LOG_DEFINE_CATEGORY(SimplePipeline)\n> \n> +/* -----------------------------------------------------------------------------\n> + *\n> + * Overview\n> + * --------\n> + *\n> + * The SimplePipelineHandler relies on generic kernel APIs to control a camera\n> + * device, without any device-specific code and with limited device-specific\n> + * static data.\n> + *\n> + * To quality for support by the simple pipeline handler, a device shall\n> + *\n> + * - be supported by V4L2 drivers, exposing the Media Controller API, the V4L2\n> + * subdev APIs and the media bus format-based enumeration extension for the\n> + * VIDIOC_ENUM_FMT ioctl ;\n> + * - not expose any device-specific API from drivers to userspace ;\n> + * - include one or more camera sensor media entities and one or more video\n> + * capture devices ;\n> + * - have a capture pipeline with linear paths from the camera sensors to the\n> + * video capture devices ; and\n> + * - have an optional memory-to-memory device to perform format conversion\n> + * and/or scaling, exposed as a V4L2 M2M device.\n> + *\n> + * As devices that require a specific pipeline handler may still match the\n> + * above characteristics, the simple pipeline handler doesn't attempt to\n> + * automatically determine which devices it can support. It instead relies on\n> + * an explicit list of supported devices, provided in the supportedDevices\n> + * array.\n> + *\n> + * When matching a device, the pipeline handler enumerates all camera sensors\n> + * and attempts, for each of them, to find a path to a video captude video node.\n> + * It does so by traversing the media graph, following the first non permanently\n> + * disabled downstream link. If such a path is found, the pipeline handler\n> + * creates a corresponding SimpleCameraData instance, and stores the media graph\n> + * path in its entities_ list.\n> + *\n> + * A more complex graph search algorithm could be implemented if a device that\n> + * would otherwise be compatible with the pipeline handler isn't correctly\n> + * handled by this heuristic.\n> + *\n> + * Once the camera data instances have been created, the match() function\n> + * creates a V4L2Subdevice instance for each entity used by any of the cameras\n> + * and stores the instances in SimplePipelineHandler::subdevs_, accessible by\n> + * the SimpleCameraData class through the SimplePipelineHandler::subdev()\n> + * function. This avoids duplication of subdev instances between different\n> + * cameras when the same entity is used in multiple paths. A similar mechanism\n> + * is used for V4L2VideoDevice instances, but instances are in this case created\n> + * on demand when access through SimplePipelineHandler::video() instead of all\n> + * in one go at initialization time.\n> + *\n> + * Finally, all camera data instances are initialized to gather information\n> + * about the possible pipeline configurations for the corresponding camera. If\n> + * valid pipeline configurations are found, a Camera is registered for the\n> + * SimpleCameraData instance.\n> + *\n> + * Pipeline Configuration\n> + * ----------------------\n> + *\n> + * The simple pipeline handler configures the pipeline by propagating V4L2\n> + * subdev formats from the camera sensor to the video node. The format is first\n> + * set on the camera sensor's output, using the native camera sensor\n> + * resolution. Then, on every link in the pipeline, the format is retrieved on\n> + * the link source and set unmodified on the link sink.\n> + *\n> + * When initializating the camera data, this above procedure is repeated for\n> + * every media bus format supported by the camera sensor. Upon reaching the\n> + * video node, the pixel formats compatible with the media bus format are\n> + * enumerated. Each of those pixel formats correspond to one possible pipeline\n> + * configuration, stored as an intsance of SimpleCameraData::Configuration in\n> + * the SimpleCameraData::formats_ map.\n> + *\n> + * Format Conversion and Scaling\n> + * -----------------------------\n> + *\n> + * The capture pipeline isn't expected to include a scaler, and if a scaler is\n> + * available, it is ignored when configuring the pipeline. However, the simple\n> + * pipeline handler supports optional memory-to-memory converters to scale the\n> + * image and convert it to a different pixel format. If such a converter is\n> + * present, the pipeline handler enumerates, for each pipeline configuration,\n> + * the pixel formats and sizes that the converter can produce for the output of\n> + * the capture video node, and stores the information in the outputFormats and\n> + * outputSizes of the SimpleCameraData::Configuration structure.\n> + */\n> +\n> class SimplePipelineHandler;\n> \n> struct SimplePipelineInfo {\n> -- \n> Regards,\n> \n> Laurent Pinchart\n> \n> _______________________________________________\n> libcamera-devel mailing list\n> libcamera-devel@lists.libcamera.org\n> https://lists.libcamera.org/listinfo/libcamera-devel","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id F1A04BD1F1\n\tfor ;\n\tTue, 2 Mar 2021 00:55:53 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 6FA6268A8F;\n\tTue, 2 Mar 2021 01:55:53 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 48A0B602E9\n\tfor ;\n\tTue, 2 Mar 2021 01:55:52 +0100 (CET)","from pyrite.rasen.tech (unknown\n\t[IPv6:2400:4051:61:600:2c71:1b79:d06d:5032])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 5437645D;\n\tTue, 2 Mar 2021 01:55:49 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"jgtKghDp\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1614646551;\n\tbh=umnZyVrFLmIubu/CKA+RzdGGl8uEhSxLL9IY/Ee0W+s=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=jgtKghDpQctiKUyUwHcJiiB5WXeSm8phCvCOSzaxibaVAXmQhu6uI3rWnqEjmsgnT\n\te5oOkG6tDHvs7qEhLwhBfRkxEz6TleMDwRdto3hBOIpxqINJu6KMDx3uqxnKXny00e\n\txr2mOw974JoeCB+vnKFsnZd7kH20GZmVCuCKWFrA=","Date":"Tue, 2 Mar 2021 09:55:44 +0900","From":"paul.elder@ideasonboard.com","To":"Laurent Pinchart ","Message-ID":"<20210302005544.GK3084@pyrite.rasen.tech>","References":"<20210131224702.8838-1-laurent.pinchart@ideasonboard.com>\n\t<20210131224702.8838-13-laurent.pinchart@ideasonboard.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<20210131224702.8838-13-laurent.pinchart@ideasonboard.com>","Subject":"Re: [libcamera-devel] [PATCH 12/20] libcamera: pipeline: simple:\n\tDocument the pipeline handler design","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Phi-Bang Nguyen ,\n\tlibcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]