[{"id":15444,"web_url":"https://patchwork.libcamera.org/comment/15444/","msgid":"<811ea08f-15e8-fc7d-57e0-04c27c6d2045@collabora.com>","date":"2021-03-03T14:21:08","subject":"Re: [libcamera-devel] [PATCH v2] libcamera: pipeline: simple:\n\tSupport camera sensors that contain an ISP","submitter":{"id":46,"url":"https://patchwork.libcamera.org/api/people/46/","name":"Dafna Hirschfeld","email":"dafna.hirschfeld@collabora.com"},"content":"Hi,\n\nOn 03.03.21 00:53, Laurent Pinchart wrote:\n> Camera sensors can include an ISP. For instance, the AP1302 external ISP\n> can be connected to up to two raw camera sensors, and the combination of\n> the sensors and ISP is considered as a (smart) camera sensor from\n> libcamera's point of view.\n> \n> The CameraSensor class has limited support for this already. Extend the\n> simple pipeline handler to support such sensors, by using the media\n> entity corresponding to the ISP instead of the raw camera sensor's\n> entity.\n> \n> We don't need to handle the case where an entity in the SoC would expose\n> the MEDIA_ENT_F_PROC_VIDEO_ISP function, as pipeline containing an ISP\n> would have a dedicated pipeline handler.\n> \n> The implementation is limited as it won't support other multi-entity\n> camera sensors (such as CCS). While this would be worth supporting, we\n> don't have a test platform with a CCS-compatible sensor at this point,\n> so let's not over-engineer the solution. Extending support to CCS (and\n> possibly other sensor topologies) will likely involve helpers that can\n> be used by other pipeline handlers (such as generic graph walk helpers\n> for instance) and extensions to the CameraSensor class.\n> \n> Signed-off-by: Laurent Pinchart \n> ---\n> Changes since v1:\n> \n> - Fix link handling\n> ---\n> src/libcamera/pipeline/simple/simple.cpp | 88 ++++++++++++++++++++----\n> 1 file changed, 75 insertions(+), 13 deletions(-)\n> \n> diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp\n> index 36f55e8709f1..aa6155f0596f 100644\n> --- a/src/libcamera/pipeline/simple/simple.cpp\n> +++ b/src/libcamera/pipeline/simple/simple.cpp\n> @@ -245,6 +245,8 @@ private:\n> \t\t\tPipelineHandler::cameraData(camera));\n> \t}\n> \n> +\tstd::vector locateSensors();\n> +\n> \tvoid bufferReady(FrameBuffer *buffer);\n> \tvoid converterInputDone(FrameBuffer *buffer);\n> \tvoid converterOutputDone(FrameBuffer *buffer);\n> @@ -869,6 +871,78 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request)\n> * Match and Setup\n> */\n> \n> +std::vector SimplePipelineHandler::locateSensors()\n> +{\n> +\tstd::vector entities;\n> +\n> +\t/*\n> +\t * Gather all the camera sensor entities based on the function they\n> +\t * expose.\n> +\t */\n> +\tfor (MediaEntity *entity : media_->entities()) {\n> +\t\tif (entity->function() == MEDIA_ENT_F_CAM_SENSOR)\n> +\t\t\tentities.push_back(entity);\n> +\t}\n> +\n> +\tif (entities.empty())\n> +\t\treturn {};\n> +\n> +\t/*\n> +\t * Sensors can be made of multiple entities. For instance, a raw sensor\n> +\t * can be connected to an ISP, and the combination of both should be\n> +\t * treated as one sensor. To support this, as a crude heuristic, check\n> +\t * the downstream entity from the camera sensor, and if it is an ISP,\n> +\t * use it instead of the sensor.\n> +\t */\n> +\tstd::vector sensors;\n> +\n> +\tfor (MediaEntity *entity : entities) {\n> +\t\t/*\n> +\t\t * Locate the downstream entity by following the first enabled\n> +\t\t * link from a source pad.\n> +\t\t */\n> +\t\tconst MediaPad *pad = nullptr;\n> +\t\tconst MediaLink *link = nullptr;\n> +\n> +\t\tfor (const MediaPad *p : entity->pads()) {\n> +\t\t\tif (p->flags() & MEDIA_PAD_FL_SOURCE) {\n> +\t\t\t\tpad = p;\n> +\t\t\t\tbreak;\n> +\t\t\t}\n> +\t\t}\n> +\n> +\t\tif (!pad)\n> +\t\t\tcontinue;\n> +\n> +\t\tfor (const MediaLink *l : pad->links()) {\n> +\t\t\tif (l->flags() & MEDIA_LNK_FL_ENABLED ||\n> +\t\t\t !(l->flags() & MEDIA_LNK_FL_IMMUTABLE)) {\n\nSame issue as discussed in the irc, this condition is always true.\n\nThanks,\nDafna\n\n> +\t\t\t\tlink = l;\n> +\t\t\t\tbreak;\n> +\t\t\t}\n> +\t\t}\n> +\n> +\t\tif (!link)\n> +\t\t\tcontinue;\n> +\n> +\t\tMediaEntity *remote = link->sink()->entity();\n> +\t\tif (remote->function() == MEDIA_ENT_F_PROC_VIDEO_ISP)\n> +\t\t\tsensors.push_back(remote);\n> +\t\telse\n> +\t\t\tsensors.push_back(entity);\n> +\t}\n> +\n> +\t/*\n> +\t * Remove duplicates, in case multiple sensors are connected to the\n> +\t * same ISP.\n> +\t */\n> +\tstd::sort(sensors.begin(), sensors.end());\n> +\tauto last = std::unique(sensors.begin(), sensors.end());\n> +\tsensors.erase(last, sensors.end());\n> +\n> +\treturn sensors;\n> +}\n> +\n> bool SimplePipelineHandler::match(DeviceEnumerator *enumerator)\n> {\n> \tconst SimplePipelineInfo *info = nullptr;\n> @@ -892,19 +966,7 @@ bool SimplePipelineHandler::match(DeviceEnumerator *enumerator)\n> \t}\n> \n> \t/* Locate the sensors. */\n> -\tstd::vector sensors;\n> -\n> -\tfor (MediaEntity *entity : media_->entities()) {\n> -\t\tswitch (entity->function()) {\n> -\t\tcase MEDIA_ENT_F_CAM_SENSOR:\n> -\t\t\tsensors.push_back(entity);\n> -\t\t\tbreak;\n> -\n> -\t\tdefault:\n> -\t\t\tbreak;\n> -\t\t}\n> -\t}\n> -\n> +\tstd::vector sensors = locateSensors();\n> \tif (sensors.empty()) {\n> \t\tLOG(SimplePipeline, Error) << \"No sensor found\";\n> \t\treturn false;\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 5840DBD80C\n\tfor ;\n\tWed, 3 Mar 2021 14:21:12 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id E064668A98;\n\tWed, 3 Mar 2021 15:21:11 +0100 (CET)","from bhuna.collabora.co.uk (bhuna.collabora.co.uk\n\t[IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id A82C268A7E\n\tfor ;\n\tWed, 3 Mar 2021 15:21:10 +0100 (CET)","from [IPv6:2003:c7:cf38:3800:7d25:342b:1528:aa90]\n\t(p200300c7cf3838007d25342b1528aa90.dip0.t-ipconnect.de\n\t[IPv6:2003:c7:cf38:3800:7d25:342b:1528:aa90])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128\n\tbits))\n\t(No client certificate requested) (Authenticated sender: dafna)\n\tby bhuna.collabora.co.uk (Postfix) with ESMTPSA id 5E7BA1F45D83;\n\tWed, 3 Mar 2021 14:21:10 +0000 (GMT)"],"To":"Laurent Pinchart ,\n\tlibcamera-devel@lists.libcamera.org","References":"<20210302235348.2878-1-laurent.pinchart@ideasonboard.com>","From":"Dafna Hirschfeld ","Message-ID":"<811ea08f-15e8-fc7d-57e0-04c27c6d2045@collabora.com>","Date":"Wed, 3 Mar 2021 15:21:08 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101\n\tThunderbird/68.10.0","MIME-Version":"1.0","In-Reply-To":"<20210302235348.2878-1-laurent.pinchart@ideasonboard.com>","Content-Language":"en-US","Subject":"Re: [libcamera-devel] [PATCH v2] libcamera: pipeline: simple:\n\tSupport camera sensors that contain an ISP","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Content-Transfer-Encoding":"7bit","Content-Type":"text/plain; charset=\"us-ascii\"; Format=\"flowed\"","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]