[{"id":4352,"web_url":"https://patchwork.libcamera.org/comment/4352/","msgid":"<3a2c39a7-5e77-d8ac-0457-e9c2839cd6bb@linaro.org>","date":"2020-03-30T19:25:07","subject":"Re: [libcamera-devel] [PATCH v3 09/11] libcamera: pipeline: Add a\n\tsimple pipeline handler","submitter":{"id":25,"url":"https://patchwork.libcamera.org/api/people/25/","name":"Andrey Konovalov","email":"andrey.konovalov@linaro.org"},"content":"Hi Laurent,\n\nThank you for this patch set!\n\nI've been using it with a sensor which supports just one format, but\nrecently Loic volunteered to give it a try on his db410c with ov5640\nsensor. (The kernel and the libcamera branches used have few quick hacks\nnot appropriate to be sent upstream but required for the things to work\nwith this hardware configuration; I am working on rewriting them in a\nproper way).\n\nOne issue has revealed - see below.\n\nOn 20.03.2020 04:48, Laurent Pinchart wrote:\n> From: Martijn Braam \n> \n> This new pipeline handler aims at supporting any simple device without\n> requiring any device-specific code. Simple devices are currently defined\n> as a graph made of one or multiple camera sensors and a single video\n> node, with each sensor connected to the video node through a linear\n> pipeline.\n> \n> The simple pipeline handler will automatically parse the media graph,\n> enumerate sensors, build supported stream configurations, and configure\n> the pipeline, without any device-specific knowledge. It doesn't support\n> configuration of any processing in the pipeline at the moment, but may\n> be extended to support simple processing such as format conversion or\n> scaling in the future.\n> \n> The only device-specific information in the pipeline handler is the list\n> of supported drivers, required for device matching. We may be able to\n> remove this in the future by matching with the simple pipeline handler\n> as a last resort option, after all other pipeline handlers have been\n> tried.\n> \n> Signed-off-by: Martijn Braam \n> Signed-off-by: Laurent Pinchart \n> ---\n> Changes since v2:\n> \n> - Log an error when setupFormats() fail\n> - Propagate getFormat() and setFormat() errors to the caller of\n> setupFormats()\n> - Reorder variable declarations in validate()\n> - Add \\todo comment related to the selection of the default format\n> - Use log Error instead of Info if pipeline isn't valid\n> - Rebase on top of V4L2PixelFormat\n> \n> Changes since v1:\n> \n> - Rebase on top of buffer API rework\n> - Expose stream formats\n> - Rework camera data config\n> ---\n> src/libcamera/pipeline/meson.build | 1 +\n> src/libcamera/pipeline/simple/meson.build | 3 +\n> src/libcamera/pipeline/simple/simple.cpp | 710 ++++++++++++++++++++++\n> 3 files changed, 714 insertions(+)\n> create mode 100644 src/libcamera/pipeline/simple/meson.build\n> create mode 100644 src/libcamera/pipeline/simple/simple.cpp\n> \n> diff --git a/src/libcamera/pipeline/meson.build b/src/libcamera/pipeline/meson.build\n> index 0d466225a72e..606ba31a0319 100644\n> --- a/src/libcamera/pipeline/meson.build\n> +++ b/src/libcamera/pipeline/meson.build\n> @@ -5,3 +5,4 @@ libcamera_sources += files([\n> \n> subdir('ipu3')\n> subdir('rkisp1')\n> +subdir('simple')\n> diff --git a/src/libcamera/pipeline/simple/meson.build b/src/libcamera/pipeline/simple/meson.build\n> new file mode 100644\n> index 000000000000..4945a3e173cf\n> --- /dev/null\n> +++ b/src/libcamera/pipeline/simple/meson.build\n> @@ -0,0 +1,3 @@\n> +libcamera_sources += files([\n> + 'simple.cpp',\n> +])\n> diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp\n> new file mode 100644\n> index 000000000000..545a99fe31ca\n> --- /dev/null\n> +++ b/src/libcamera/pipeline/simple/simple.cpp\n\n\n\n> +int SimpleCameraData::init()\n> +{\n> +\tSimplePipelineHandler *pipe = static_cast(pipe_);\n> +\tV4L2VideoDevice *video = pipe->video();\n> +\tint ret;\n> +\n> +\t/*\n> +\t * Enumerate the possible pipeline configurations. For each media bus\n> +\t * format supported by the sensor, propagate the formats through the\n> +\t * pipeline, and enumerate the corresponding possible V4L2 pixel\n> +\t * formats on the video node.\n> +\t */\n> +\tfor (unsigned int code : sensor_->mbusCodes()) {\n> +\t\tV4L2SubdeviceFormat format{ code, sensor_->resolution() };\n> +\n> +\t\t/*\n> +\t\t * Setup links first as some subdev drivers take active links\n> +\t\t * into account to propaget TRY formats. So is life :-(\n> +\t\t */\n> +\t\tret = setupLinks();\n> +\t\tif (ret < 0)\n> +\t\t\treturn ret;\n> +\n> +\t\tret = setupFormats(&format, V4L2Subdevice::TryFormat);\n> +\t\tif (ret < 0) {\n> +\t\t\tLOG(SimplePipeline, Error)\n> +\t\t\t\t<< \"Failed to setup pipeline for media bus code \"\n> +\t\t\t\t<< utils::hex(code, 4);\n> +\t\t\treturn ret;\n\nIf some of the formats supported by the camera sensor are not supported by an\nelement down the pipeline, this return statement aborts the enumeration thus\nskipping the rest of the formats which might work.\n\nThis has been observed with ov5640 sensor connected to db410c board\n(Thanks Loic for your help with testing and debugging this on your hardware).\nThe first value assigned to the code variable is RGB565_2X8_BE, so the\ncodes enumeration is aborted immediately, and doesn't get to trying the YUV\nformats.\n\nReplacing the above 'return' statement with 'continue' works much better -\nconfigurations for a few YUV formats are added OK.\n\nThanks,\nAndrey\n\n> +\t\t}\n> +\n> +\t\tstd::map> videoFormats =\n> +\t\t\tvideo->formats(format.mbus_code);\n> +\n> +\t\tLOG(SimplePipeline, Debug)\n> +\t\t\t<< \"Adding configuration for \" << format.size.toString()\n> +\t\t\t<< \" in pixel formats [ \"\n> +\t\t\t<< utils::join(videoFormats, \", \",\n> +\t\t\t\t [](const auto &f) {\n> +\t\t\t\t\t return f.first.toString();\n> +\t\t\t\t })\n> +\t\t\t<< \" ]\";\n> +\n> +\t\t/*\n> +\t\t * Store the configuration in the formats_ map, mapping\n> +\t\t * PixelFormat to configuration. Any previously stored value is\n> +\t\t * overwritten, as the pipeline handler currently doesn't care\n> +\t\t * about how a particular PixelFormat is achieved.\n> +\t\t */\n> +\t\tfor (const auto &videoFormat : videoFormats) {\n> +\t\t\tPixelFormat pixelFormat = video->toPixelFormat(videoFormat.first);\n> +\t\t\tif (!pixelFormat)\n> +\t\t\t\tcontinue;\n> +\n> +\t\t\tConfiguration config;\n> +\t\t\tconfig.code = code;\n> +\t\t\tconfig.pixelFormat = pixelFormat;\n> +\t\t\tconfig.size = format.size;\n> +\n> +\t\t\tformats_[pixelFormat] = config;\n> +\t\t}\n> +\t}\n> +\n> +\tif (formats_.empty()) {\n> +\t\tLOG(SimplePipeline, Error) << \"No valid configuration found\";\n> +\t\treturn -EINVAL;\n> +\t}\n> +\n> +\treturn 0;\n> +}\n\n","headers":{"Return-Path":"","Received":["from mail-lf1-x143.google.com (mail-lf1-x143.google.com\n\t[IPv6:2a00:1450:4864:20::143])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 1F23C6040D\n\tfor ;\n\tMon, 30 Mar 2020 21:25:12 +0200 (CEST)","by mail-lf1-x143.google.com with SMTP id j188so15224448lfj.11\n\tfor ;\n\tMon, 30 Mar 2020 12:25:12 -0700 (PDT)","from [192.168.118.216] (37-144-159-139.broadband.corbina.ru.\n\t[37.144.159.139]) by smtp.gmail.com with ESMTPSA id\n\tq30sm5307319lfn.18.2020.03.30.12.25.09\n\t(version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);\n\tMon, 30 Mar 2020 12:25:10 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key; \n\tunprotected) header.d=linaro.org\n\theader.i=@linaro.org header.b=\"wCcz1If2\"; \n\tdkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=subject:to:cc:references:from:message-id:date:user-agent\n\t:mime-version:in-reply-to:content-language:content-transfer-encoding; \n\tbh=Rh8mzm89vbzMfeufF+l9tDEF2FStB7FUqviVVFLz3gw=;\n\tb=wCcz1If2ZSmVggSmXQ2TgT6S8vdqi3aX7U1CRvzpmFli7sLC8Nl2SXYyYdlL5SY/bG\n\tfEkSN6dFX6GoffYF1dY+Vp7f26qsCcgW7++WK+qomEnBeiyp7x4Dc+r1htrGnaiSb3JH\n\te6cdg6ATbXxAIefZydHOS+tMxNmnbUbBopPhS4/nxiH3TeSu5xxfAGF8VIT4MuLVMhOy\n\tI5FzcL0lbdPeb56hVEsN50BK19s0xCT8wvVf+M9pTox1Y5LvQOMrhwumnUEDyJIVSo9b\n\t8evRluM0+FdwkcEAlO5Kts4dtmCmuRDIMeJSbLV64jrO3GyD8WpBglnXGx2xpv/vzfWP\n\thYtw==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:subject:to:cc:references:from:message-id:date\n\t:user-agent:mime-version:in-reply-to:content-language\n\t:content-transfer-encoding;\n\tbh=Rh8mzm89vbzMfeufF+l9tDEF2FStB7FUqviVVFLz3gw=;\n\tb=cjZdIa6EKNCRuKk9GRcft3A1lS5bh8gn02gRccAyVQqrRbSJOPHHqLYdIGdzeAE6Xq\n\t5NjBPHFHZDj2NsX07Cwz6L2BAGRSwuVHGEU1Yo+DpvuO4IRb/TtUjvtgm4/Mdwojhby/\n\tizjagnyzYmzUCn0cUhAtVGuQzmSz9hKT0nUObvnxJD+8z+uXO9to4XRS3Gipzh9IirkJ\n\tMjTZwrvTMJJ2a+F1Mwt287ZbzJbWF4DKp42Qb3ZChkTSqz48GbxoF/UpkhDqv/iGMZ/I\n\tr+teI2Q2in2OeJmfh3wkb/p/BmPcaSWeAgSEu7ipJ+p3J0wvcBP6+basdktxP4F7rdA6\n\tvWnA==","X-Gm-Message-State":"AGi0Pubp9hpqUj1rudiGTEacKDCmydWi8ZuYHxmSz/aMCW7SIY/0HBnp\n\tnLGpj9m3T4WnBVtBp73ycTdrsg==","X-Google-Smtp-Source":"APiQypKGXnH+9r0hdhdCpnMUQklBK0dTWeaXm6aPj70i4DOWy035TB+2QitJl5D2lrisecboGkVQYg==","X-Received":"by 2002:a05:6512:1095:: with SMTP id\n\tj21mr9005322lfg.118.1585596310867; \n\tMon, 30 Mar 2020 12:25:10 -0700 (PDT)","To":"Laurent Pinchart ,\n\tlibcamera-devel@lists.libcamera.org","Cc":"Martijn Braam ,\n\tBenjamin GAIGNARD ,\n\tLoic Poulain ","References":"<20200320014839.14683-1-laurent.pinchart@ideasonboard.com>\n\t<20200320014839.14683-10-laurent.pinchart@ideasonboard.com>","From":"Andrey Konovalov ","Message-ID":"<3a2c39a7-5e77-d8ac-0457-e9c2839cd6bb@linaro.org>","Date":"Mon, 30 Mar 2020 22:25:07 +0300","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101\n\tThunderbird/68.4.1","MIME-Version":"1.0","In-Reply-To":"<20200320014839.14683-10-laurent.pinchart@ideasonboard.com>","Content-Type":"text/plain; charset=utf-8; format=flowed","Content-Language":"en-US","Content-Transfer-Encoding":"7bit","Subject":"Re: [libcamera-devel] [PATCH v3 09/11] libcamera: pipeline: Add a\n\tsimple pipeline handler","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","X-List-Received-Date":"Mon, 30 Mar 2020 19:25:13 -0000"}},{"id":4354,"web_url":"https://patchwork.libcamera.org/comment/4354/","msgid":"<43484c6b-44ac-4bd7-88e4-b46070840315@ideasonboard.com>","date":"2020-03-31T11:45:14","subject":"Re: [libcamera-devel] [PATCH v3 09/11] libcamera: pipeline: Add a\n\tsimple pipeline handler","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"Hi Laurent (/Martijn...)\n\n\nOn 20/03/2020 01:48, Laurent Pinchart wrote:\n> From: Martijn Braam \n> \n> This new pipeline handler aims at supporting any simple device without\n> requiring any device-specific code. Simple devices are currently defined\n> as a graph made of one or multiple camera sensors and a single video\n> node, with each sensor connected to the video node through a linear\n> pipeline.\n> \n> The simple pipeline handler will automatically parse the media graph,\n> enumerate sensors, build supported stream configurations, and configure\n> the pipeline, without any device-specific knowledge. It doesn't support\n> configuration of any processing in the pipeline at the moment, but may\n> be extended to support simple processing such as format conversion or\n> scaling in the future.\n> \n> The only device-specific information in the pipeline handler is the list\n> of supported drivers, required for device matching. We may be able to\n> remove this in the future by matching with the simple pipeline handler\n> as a last resort option, after all other pipeline handlers have been\n> tried.\n> \n> Signed-off-by: Martijn Braam \n> Signed-off-by: Laurent Pinchart \n\nReviewed-by: Kieran Bingham \n\nSome comments and discussion points below, but I think we should work\ntowards trying to get this integrated so it can actually be used (which\nis where we'll find the nuances and corner cases as people test it)\n\n--\nKieran\n\n\n\n> ---\n> Changes since v2:\n> \n> - Log an error when setupFormats() fail\n> - Propagate getFormat() and setFormat() errors to the caller of\n> setupFormats()\n> - Reorder variable declarations in validate()\n> - Add \\todo comment related to the selection of the default format\n> - Use log Error instead of Info if pipeline isn't valid\n> - Rebase on top of V4L2PixelFormat\n> \n> Changes since v1:\n> \n> - Rebase on top of buffer API rework\n> - Expose stream formats\n> - Rework camera data config\n> ---\n> src/libcamera/pipeline/meson.build | 1 +\n> src/libcamera/pipeline/simple/meson.build | 3 +\n> src/libcamera/pipeline/simple/simple.cpp | 710 ++++++++++++++++++++++\n> 3 files changed, 714 insertions(+)\n> create mode 100644 src/libcamera/pipeline/simple/meson.build\n> create mode 100644 src/libcamera/pipeline/simple/simple.cpp\n> \n> diff --git a/src/libcamera/pipeline/meson.build b/src/libcamera/pipeline/meson.build\n> index 0d466225a72e..606ba31a0319 100644\n> --- a/src/libcamera/pipeline/meson.build\n> +++ b/src/libcamera/pipeline/meson.build\n> @@ -5,3 +5,4 @@ libcamera_sources += files([\n> \n> subdir('ipu3')\n> subdir('rkisp1')\n> +subdir('simple')\n> diff --git a/src/libcamera/pipeline/simple/meson.build b/src/libcamera/pipeline/simple/meson.build\n> new file mode 100644\n> index 000000000000..4945a3e173cf\n> --- /dev/null\n> +++ b/src/libcamera/pipeline/simple/meson.build\n> @@ -0,0 +1,3 @@\n> +libcamera_sources += files([\n> + 'simple.cpp',\n> +])\n> diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp\n> new file mode 100644\n> index 000000000000..545a99fe31ca\n> --- /dev/null\n> +++ b/src/libcamera/pipeline/simple/simple.cpp\n> @@ -0,0 +1,710 @@\n> +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n> +/*\n> + * Copyright (C) 2020, Laurent Pinchart\n> + * Copyright (C) 2019, Martijn Braam\n> + *\n> + * simple.cpp - Pipeline handler for simple pipelines\n> + */\n> +\n> +#include \n> +#include \n> +#include \n> +#include \n> +#include \n> +#include \n> +#include \n> +#include \n> +#include \n> +#include \n> +\n> +#include \n> +\n> +#include \n> +#include \n> +#include \n> +\n> +#include \"camera_sensor.h\"\n> +#include \"device_enumerator.h\"\n> +#include \"log.h\"\n> +#include \"media_device.h\"\n> +#include \"pipeline_handler.h\"\n> +#include \"v4l2_subdevice.h\"\n> +#include \"v4l2_videodevice.h\"\n> +\n> +namespace libcamera {\n> +\n> +LOG_DEFINE_CATEGORY(SimplePipeline)\n> +\n> +class SimplePipelineHandler;\n> +\n> +class SimpleCameraData : public CameraData\n> +{\n> +public:\n> +\tSimpleCameraData(PipelineHandler *pipe, MediaEntity *sensor,\n> +\t\t\t MediaEntity *video);\n> +\n> +\tbool isValid() const { return sensor_ != nullptr; }\n> +\tstd::set streams() { return { &stream_ }; }\n> +\n> +\tint init();\n> +\tint setupLinks();\n> +\tint setupFormats(V4L2SubdeviceFormat *format,\n> +\t\t\t V4L2Subdevice::Whence whence);\n> +\n> +\tstruct Entity {\n> +\t\tMediaEntity *entity;\n> +\t\tMediaLink *link;\n> +\t};\n> +\n> +\tstruct Configuration {\n> +\t\tuint32_t code;\n> +\t\tPixelFormat pixelFormat;\n> +\t\tSize size;\n> +\t};\n> +\n> +\tStream stream_;\n> +\tstd::unique_ptr sensor_;\n> +\tstd::list entities_;\n> +\n> +\tstd::vector configs_;\n> +\tstd::map formats_;\n> +};\n> +\n> +class SimpleCameraConfiguration : public CameraConfiguration\n> +{\n> +public:\n> +\tSimpleCameraConfiguration(Camera *camera, SimpleCameraData *data);\n> +\n> +\tStatus validate() override;\n> +\n> +\tconst V4L2SubdeviceFormat &sensorFormat() { return sensorFormat_; }\n> +\n> +private:\n> +\t/*\n> +\t * The SimpleCameraData instance is guaranteed to be valid as long as\n> +\t * the corresponding Camera instance is valid. In order to borrow a\n> +\t * reference to the camera data, store a new reference to the camera.\n> +\t */\n> +\tstd::shared_ptr camera_;\n> +\tconst SimpleCameraData *data_;\n> +\n> +\tV4L2SubdeviceFormat sensorFormat_;\n> +};\n> +\n> +class SimplePipelineHandler : public PipelineHandler\n> +{\n> +public:\n> +\tSimplePipelineHandler(CameraManager *manager);\n> +\t~SimplePipelineHandler();\n> +\n> +\tCameraConfiguration *generateConfiguration(Camera *camera,\n> +\t\t\t\t\t\t const StreamRoles &roles) override;\n> +\tint configure(Camera *camera, CameraConfiguration *config) override;\n> +\n> +\tint exportFrameBuffers(Camera *camera, Stream *stream,\n> +\t\t\t std::vector> *buffers) override;\n> +\n> +\tint start(Camera *camera) override;\n> +\tvoid stop(Camera *camera) override;\n> +\n> +\tbool match(DeviceEnumerator *enumerator) override;\n> +\n> +\tV4L2VideoDevice *video() { return video_; }\n> +\tV4L2Subdevice *subdev(const MediaEntity *entity);\n> +\n> +protected:\n> +\tint queueRequestDevice(Camera *camera, Request *request) override;\n> +\n> +private:\n> +\tSimpleCameraData *cameraData(const Camera *camera)\n> +\t{\n> +\t\treturn static_cast(\n> +\t\t\tPipelineHandler::cameraData(camera));\n> +\t}\n> +\n> +\tint initLinks();\n> +\n> +\tint createCamera(MediaEntity *sensor);\n> +\n> +\tvoid bufferReady(FrameBuffer *buffer);\n> +\n> +\tMediaDevice *media_;\n> +\tV4L2VideoDevice *video_;\n> +\tstd::map subdevs_;\n> +\n> +\tCamera *activeCamera_;\n> +};\n> +\n> +/* -----------------------------------------------------------------------------\n> + * Camera Data\n> + */\n> +SimpleCameraData::SimpleCameraData(PipelineHandler *pipe, MediaEntity *sensor,\n> +\t\t\t\t MediaEntity *video)\n> +\t: CameraData(pipe)\n> +{\n> +\tint ret;\n> +\n> +\t/*\n> +\t * Walk the pipeline towards the video node and store all entities\n> +\t * along the way.\n> +\t */\n> +\tMediaEntity *source = sensor;\n> +\n> +\twhile (source) {\n> +\t\t/* If we have reached the video node, we're done. */\n> +\t\tif (source == video)\n> +\t\t\tbreak;\n> +\n> +\t\t/* Use the first output pad that has links. */\n> +\t\tMediaPad *sourcePad = nullptr;\n> +\t\tfor (MediaPad *pad : source->pads()) {\n> +\t\t\tif ((pad->flags() & MEDIA_PAD_FL_SOURCE) &&\n> +\t\t\t !pad->links().empty()) {\n> +\t\t\t\tsourcePad = pad;\n> +\t\t\t\tbreak;\n> +\t\t\t}\n> +\t\t}\n> +\n> +\t\tif (!sourcePad)\n> +\t\t\treturn;\n> +\n> +\t\t/* Use the first link that isn't immutable and disabled. */\n> +\t\tMediaLink *sourceLink = nullptr;\n> +\t\tfor (MediaLink *link : sourcePad->links()) {\n> +\t\t\tif ((link->flags() & MEDIA_LNK_FL_ENABLED) ||\n> +\t\t\t !(link->flags() & MEDIA_LNK_FL_IMMUTABLE)) {\n> +\t\t\t\tsourceLink = link;\n> +\t\t\t\tbreak;\n> +\t\t\t}\n> +\t\t}\n> +\n> +\t\tif (!sourceLink)\n> +\t\t\treturn;\n> +\n> +\t\tentities_.push_back({ source, sourceLink });\n> +\n> +\t\tsource = sourceLink->sink()->entity();\n> +\n> +\t\t/* Avoid infinite loops. */\n> +\t\tauto iter = std::find_if(entities_.begin(), entities_.end(),\n> +\t\t\t\t\t [&](const Entity &entity) {\n> +\t\t\t\t\t\t return entity.entity == source;\n> +\t\t\t\t\t });\n> +\t\tif (iter != entities_.end()) {\n> +\t\t\tLOG(SimplePipeline, Info) << \"Loop detected in pipeline\";\n> +\t\t\treturn;\n> +\t\t}\n> +\t}\n> +\n> +\t/* We have a valid pipeline, create the camera sensor. */\n> +\tsensor_ = std::make_unique(sensor);\n> +\tret = sensor_->init();\n> +\tif (ret) {\n> +\t\tsensor_.reset();\n> +\t\treturn;\n> +\t}\n> +}\n> +\n> +int SimpleCameraData::init()\n> +{\n> +\tSimplePipelineHandler *pipe = static_cast(pipe_);\n> +\tV4L2VideoDevice *video = pipe->video();\n> +\tint ret;\n> +\n> +\t/*\n> +\t * Enumerate the possible pipeline configurations. For each media bus\n> +\t * format supported by the sensor, propagate the formats through the\n> +\t * pipeline, and enumerate the corresponding possible V4L2 pixel\n> +\t * formats on the video node.\n> +\t */\n> +\tfor (unsigned int code : sensor_->mbusCodes()) {\n> +\t\tV4L2SubdeviceFormat format{ code, sensor_->resolution() };\n> +\n> +\t\t/*\n> +\t\t * Setup links first as some subdev drivers take active links\n> +\t\t * into account to propaget TRY formats. So is life :-(\n\n/propaget/propagate/\n\n/So is life/Such is life/\n\n\n> +\t\t */\n> +\t\tret = setupLinks();\n> +\t\tif (ret < 0)\n> +\t\t\treturn ret;\n> +\n> +\t\tret = setupFormats(&format, V4L2Subdevice::TryFormat);\n> +\t\tif (ret < 0) {\n> +\t\t\tLOG(SimplePipeline, Error)\n> +\t\t\t\t<< \"Failed to setup pipeline for media bus code \"\n> +\t\t\t\t<< utils::hex(code, 4);\n\nOh dear, seems we need mbusCode.toString() too! hehe\n\n> +\t\t\treturn ret;\n\nAndrey states that this return prevents iterating potentially successful\nmbus-codes, and should instead 'continue'...\n\n\n> +\t\t}\n> +\n> +\t\tstd::map> videoFormats =\n> +\t\t\tvideo->formats(format.mbus_code);\n> +\n> +\t\tLOG(SimplePipeline, Debug)\n> +\t\t\t<< \"Adding configuration for \" << format.size.toString()\n> +\t\t\t<< \" in pixel formats [ \"\n> +\t\t\t<< utils::join(videoFormats, \", \",\n> +\t\t\t\t [](const auto &f) {\n> +\t\t\t\t\t return f.first.toString();\n> +\t\t\t\t })\n\nAha, I kinda like the utils::join!\n\n> +\t\t\t<< \" ]\";\n> +\n> +\t\t/*\n> +\t\t * Store the configuration in the formats_ map, mapping\n> +\t\t * PixelFormat to configuration. Any previously stored value is\n\n/to configuration/to the current configuration/ ?\n\n> +\t\t * overwritten, as the pipeline handler currently doesn't care\n> +\t\t * about how a particular PixelFormat is achieved.\n\nAs long as there's an output, I agree. If devices have specific\nconstraints such as better performance or such with a particular\nmbus-code, then they need to create a platform specific pipeline handler\nto deal with the nuances.\n\nMaybe there might be tuning hints or something later, but certainly not now.\n\n\n> +\t\t */\n> +\t\tfor (const auto &videoFormat : videoFormats) {\n> +\t\t\tPixelFormat pixelFormat = video->toPixelFormat(videoFormat.first);\n> +\t\t\tif (!pixelFormat)\n> +\t\t\t\tcontinue;\n> +\n> +\t\t\tConfiguration config;\n> +\t\t\tconfig.code = code;\n> +\t\t\tconfig.pixelFormat = pixelFormat;\n> +\t\t\tconfig.size = format.size;\n> +\n> +\t\t\tformats_[pixelFormat] = config;\n> +\t\t}\n> +\t}\n> +\n> +\tif (formats_.empty()) {\n> +\t\tLOG(SimplePipeline, Error) << \"No valid configuration found\";\n> +\t\treturn -EINVAL;\n> +\t}\n> +\n> +\treturn 0;\n> +}\n> +\n> +int SimpleCameraData::setupLinks()\n> +{\n> +\tint ret;\n> +\n> +\t/*\n> +\t * Configure all links along the pipeline. Some entities may not allow\n> +\t * multiple sink links to be enabled together, even on different sink\n> +\t * pads. We must thus start by disabling all sink links (but the one we\n> +\t * want to enable) before enabling the pipeline link.\n\nEeek - this sounds like it could have potential system effects,\n(disabling existing configurations) but I think this is the\n'Such-is-life' comment I read earlier.\n\n\n> +\t */\n> +\tfor (SimpleCameraData::Entity &e : entities_) {\n> +\t\tfor (MediaPad *pad : e.link->sink()->entity()->pads()) {\n\nFor readability, I would be tempted to pull e.link->sink()->entity() to\na local var called Entity &remote, or Entity &endpoint, or such,\nessentially to make this line read as:\n\n for (MediaPad *pad : remote->pads()) {\n\n> +\t\t\tfor (MediaLink *link : pad->links()) {\n> +\t\t\t\tif (link == e.link)\n> +\t\t\t\t\tcontinue;\n> +\n> +\t\t\t\tif ((link->flags() & MEDIA_LNK_FL_ENABLED) &&\n> +\t\t\t\t !(link->flags() & MEDIA_LNK_FL_IMMUTABLE)) {\n> +\t\t\t\t\tret = link->setEnabled(false);\n> +\t\t\t\t\tif (ret < 0)\n> +\t\t\t\t\t\treturn ret;\n> +\t\t\t\t}\n> +\t\t\t}\n> +\t\t}\n> +\n\nI'm not going to try it out, but would this be easier / clearer if we\nsimply disabled *all* links, then explicitly enable links on our pipeline?\n\n(only a devils-advocate discussion point, not something that's needed).\n\n> +\t\tif (!(e.link->flags() & MEDIA_LNK_FL_ENABLED)) {\n> +\t\t\tret = e.link->setEnabled(true);\n> +\t\t\tif (ret < 0)\n> +\t\t\t\treturn ret;\n> +\t\t}\n> +\t}\n> +\n> +\treturn 0;\n> +}\n> +\n> +int SimpleCameraData::setupFormats(V4L2SubdeviceFormat *format,\n> +\t\t\t\t V4L2Subdevice::Whence whence)\n> +{\n> +\tSimplePipelineHandler *pipe = static_cast(pipe_);\n> +\tint ret;\n> +\n> +\t/*\n> +\t * Configure the format on the sensor output and propagate it through\n> +\t * the pipeline.\n> +\t */\n> +\tret = sensor_->setFormat(format);\n> +\tif (ret < 0)\n> +\t\treturn ret;\n> +\n> +\tfor (const Entity &e : entities_) {\n> +\t\tMediaLink *link = e.link;\n> +\t\tMediaPad *source = link->source();\n> +\t\tMediaPad *sink = link->sink();\n> +\n> +\t\tif (source->entity() != sensor_->entity()) {\n> +\t\t\tV4L2Subdevice *subdev = pipe->subdev(source->entity());\n> +\t\t\tret = subdev->getFormat(source->index(), format, whence);\n> +\t\t\tif (ret < 0)\n> +\t\t\t\treturn ret;\n> +\t\t}\n> +\n> +\t\tif (sink->entity()->function() != MEDIA_ENT_F_IO_V4L) {\n> +\t\t\tV4L2Subdevice *subdev = pipe->subdev(sink->entity());\n> +\t\t\tret = subdev->setFormat(sink->index(), format, whence);\n> +\t\t\tif (ret < 0)\n> +\t\t\t\treturn ret;\n> +\t\t}\n> +\n> +\t\tLOG(SimplePipeline, Debug)\n> +\t\t\t<< \"Link '\" << source->entity()->name()\n> +\t\t\t<< \"':\" << source->index()\n> +\t\t\t<< \" -> '\" << sink->entity()->name()\n> +\t\t\t<< \"':\" << sink->index()\n> +\t\t\t<< \" configured with format \" << format->toString();\n\nI wonder if we can get the whole pipeline configuration string to a\nsingle line... but not required now.\n\n> +\t}\n> +\n> +\treturn 0;\n> +}\n> +\n> +/* -----------------------------------------------------------------------------\n> + * Camera Configuration\n> + */\n> +\n> +SimpleCameraConfiguration::SimpleCameraConfiguration(Camera *camera,\n> +\t\t\t\t\t\t SimpleCameraData *data)\n> +\t: CameraConfiguration(), camera_(camera->shared_from_this()),\n> +\t data_(data)\n> +{\n> +}\n> +\n> +CameraConfiguration::Status SimpleCameraConfiguration::validate()\n> +{\n> +\tStatus status = Valid;\n> +\n> +\tif (config_.empty())\n> +\t\treturn Invalid;\n> +\n> +\t/* Cap the number of entries to the available streams. */\n> +\tif (config_.size() > 1) {\n> +\t\tconfig_.resize(1);\n> +\t\tstatus = Adjusted;\n> +\t}\n> +\n> +\tStreamConfiguration &cfg = config_[0];\n> +\n> +\t/* Adjust the pixel format. */\n> +\tauto it = data_->formats_.find(cfg.pixelFormat);\n> +\tif (it == data_->formats_.end())\n> +\t\tit = data_->formats_.begin();\n> +\n> +\tPixelFormat pixelFormat = it->first;\n> +\tif (cfg.pixelFormat != pixelFormat) {\n> +\t\tLOG(SimplePipeline, Debug) << \"Adjusting pixel format\";\n> +\t\tcfg.pixelFormat = pixelFormat;\n> +\t\tstatus = Adjusted;\n> +\t}\n> +\n> +\tconst SimpleCameraData::Configuration &pipeConfig = it->second;\n> +\tif (cfg.size != pipeConfig.size) {\n> +\t\tLOG(SimplePipeline, Debug)\n> +\t\t\t<< \"Adjusting size from \" << cfg.size.toString()\n> +\t\t\t<< \" to \" << pipeConfig.size.toString();\n> +\t\tcfg.size = pipeConfig.size;\n> +\t\tstatus = Adjusted;\n> +\t}\n> +\n> +\tcfg.bufferCount = 3;\n> +\n> +\treturn status;\n> +}\n> +\n> +/* -----------------------------------------------------------------------------\n> + * Pipeline Handler\n> + */\n> +\n> +SimplePipelineHandler::SimplePipelineHandler(CameraManager *manager)\n> +\t: PipelineHandler(manager), video_(nullptr)\n> +{\n> +}\n> +\n> +SimplePipelineHandler::~SimplePipelineHandler()\n> +{\n> +\tdelete video_;\n> +}\n> +\n> +CameraConfiguration *SimplePipelineHandler::generateConfiguration(Camera *camera,\n> +\t\t\t\t\t\t\t\t const StreamRoles &roles)\n> +{\n> +\tSimpleCameraData *data = cameraData(camera);\n> +\tCameraConfiguration *config =\n> +\t\tnew SimpleCameraConfiguration(camera, data);\n> +\n> +\tif (roles.empty())\n> +\t\treturn config;\n> +\n> +\t/* Create the formats map. */\n> +\tstd::map> formats;\n> +\tstd::transform(data->formats_.begin(), data->formats_.end(),\n> +\t\t std::inserter(formats, formats.end()),\n> +\t\t [](const auto &format) -> decltype(formats)::value_type {\n> +\t\t\t const PixelFormat &pixelFormat = format.first;\n> +\t\t\t const Size &size = format.second.size;\n> +\t\t\t return { pixelFormat, { size } };\n> +\t\t });\n\nThis is quite unfriendly to the reader :-( ... not that I have a better\nalternative ... but eugh\n\n> +\n> +\t/*\n> +\t * Create the stream configuration. Take the first entry in the formats\n> +\t * map as the default, for lack of a better option.\n> +\t *\n> +\t * \\todo Implement a better way to pick the default format\n\nIs there one? It's a generic configuration for a pipeline handler.\n\nWhat might be better as a general case, and presumably any logic added\nhere would potentially be applicable to most pipeline handlers too.\n\nThe biggest Size? The smallest Size? or the nearest size to a list of\npredefined sizes for a given role?\n\n> +\t */\n> +\tStreamConfiguration cfg{ StreamFormats{ formats } };\n> +\tcfg.pixelFormat = formats.begin()->first;\n> +\tcfg.size = formats.begin()->second[0].max;\n> +\n> +\tconfig->addConfiguration(cfg);\n> +\n> +\tconfig->validate();\n> +\n> +\treturn config;\n> +}\n> +\n> +int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c)\n> +{\n> +\tSimpleCameraConfiguration *config =\n> +\t\tstatic_cast(c);\n> +\tSimpleCameraData *data = cameraData(camera);\n> +\tStreamConfiguration &cfg = config->at(0);\n> +\tint ret;\n> +\n> +\t/*\n> +\t * Configure links on the pipeline and propagate formats from the\n> +\t * sensor to the video node.\n> +\t */\n> +\tret = data->setupLinks();\n> +\tif (ret < 0)\n> +\t\treturn ret;\n> +\n> +\tconst SimpleCameraData::Configuration &pipeConfig =\n> +\t\tdata->formats_[cfg.pixelFormat];\n> +\n> +\tV4L2SubdeviceFormat format{ pipeConfig.code, data->sensor_->resolution() };\n> +\n> +\tret = data->setupFormats(&format, V4L2Subdevice::ActiveFormat);\n> +\tif (ret < 0)\n> +\t\treturn ret;\n> +\n> +\t/* Configure the video node. */\n> +\tV4L2PixelFormat videoFormat = video_->toV4L2PixelFormat(cfg.pixelFormat);\n> +\n> +\tV4L2DeviceFormat outputFormat = {};\n> +\toutputFormat.fourcc = videoFormat;\n> +\toutputFormat.size = cfg.size;\n> +\n> +\tret = video_->setFormat(&outputFormat);\n> +\tif (ret)\n> +\t\treturn ret;\n> +\n> +\tif (outputFormat.size != cfg.size || outputFormat.fourcc != videoFormat) {\n> +\t\tLOG(SimplePipeline, Error)\n> +\t\t\t<< \"Unable to configure capture in \" << cfg.toString();\n> +\t\treturn -EINVAL;\n> +\t}\n> +\n> +\tcfg.setStream(&data->stream_);\n> +\n> +\treturn 0;\n> +}\n> +\n> +int SimplePipelineHandler::exportFrameBuffers(Camera *camera, Stream *stream,\n> +\t\t\t\t\t std::vector> *buffers)\n> +{\n> +\tunsigned int count = stream->configuration().bufferCount;\n> +\n> +\treturn video_->exportBuffers(count, buffers);\n> +}\n> +\n> +int SimplePipelineHandler::start(Camera *camera)\n> +{\n> +\tSimpleCameraData *data = cameraData(camera);\n> +\tunsigned int count = data->stream_.configuration().bufferCount;\n> +\n> +\tint ret = video_->importBuffers(count);\n> +\tif (ret < 0)\n> +\t\treturn ret;\n> +\n> +\tret = video_->streamOn();\n> +\tif (ret < 0) {\n> +\t\tvideo_->releaseBuffers();\n\nI wonder if video_->importBuffers() should be something that now happens\nat video_->streamOn()?\n\nI presume that it is now more of an operation to set up V4L2 with the\ninternal buffer structures for passing the information on 'which' buffer\nis in use to V4L2...\n\nEvery video stream will have to do this?\n\nOtherwise, if conceptually we don't want to mis-manage buffer handling\nin V4L2VideoDevice::streamOn() we might want to have a V4L2Stream object\nto do so ...\n\n(clearly those are just V4L2VideoDevice API discussion points, and not\nactions on this patch)\n\n\n> +\t\treturn ret;\n> +\t}\n> +\n> +\tactiveCamera_ = camera;\n\nWill simple pipeline handler support more than one camera?\n\nThe activeCamera_ feels like a way of supporting more than one camera on\na pipeline, which could otherwise have been a bool cameraActive to\nrepresent it's state. In fact I'm not sure it's even needed to do\nthat... The only actual check is an assert to ensure that it's set\nbefore using it in the bufferReady() callbacks, which could likely have\naccessed the camera directly?\n\nBut if it's worth keeping this as it is to keep it close to other\npipeline handlers then so be it.\n\n\n Looks like it is used actaully, as the Camera is only created and\nregistered with the CameraManager, so we don't really have direct access\nto the Camera within the PipelineHandler, and we have to be given it..\n(even though /we/ created it :-D)\n\n\n> +\n> +\treturn 0;\n> +}\n> +\n> +void SimplePipelineHandler::stop(Camera *camera)\n> +{\n> +\tvideo_->streamOff();\n> +\tvideo_->releaseBuffers();\n> +\tactiveCamera_ = nullptr;\n> +}\n> +\n> +int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request)\n> +{\n> +\tSimpleCameraData *data = cameraData(camera);\n> +\tStream *stream = &data->stream_;\n> +\n> +\tFrameBuffer *buffer = request->findBuffer(stream);\n> +\tif (!buffer) {\n> +\t\tLOG(SimplePipeline, Error)\n> +\t\t\t<< \"Attempt to queue request with invalid stream\";\n> +\t\treturn -ENOENT;\n> +\t}\n> +\n> +\treturn video_->queueBuffer(buffer);\n> +}\n> +\n> +/* -----------------------------------------------------------------------------\n> + * Match and Setup\n> + */\n> +\n> +bool SimplePipelineHandler::match(DeviceEnumerator *enumerator)\n> +{\n> +\tstatic const char * const drivers[] = {\n> +\t\t\"imx7-csi\",\n> +\t\t\"sun6i-csi\",\n> +\t};\n> +\n> +\tfor (const char *driver : drivers) {\n> +\t\tDeviceMatch dm(driver);\n> +\t\tmedia_ = acquireMediaDevice(enumerator, dm);\n> +\t\tif (media_)\n> +\t\t\tbreak;\n> +\t}\n> +\n> +\tif (!media_)\n> +\t\treturn false;\n> +\n> +\t/*\n> +\t * Locate sensors and video nodes. We only support pipelines with at\n> +\t * least one sensor and exactly one video captude node.\n\n/captude/capture/\n\n> +\t */\n> +\tstd::vector sensors;\n> +\tstd::vector videos;\n> +\n> +\tfor (MediaEntity *entity : media_->entities()) {\n> +\t\tswitch (entity->function()) {\n> +\t\tcase MEDIA_ENT_F_CAM_SENSOR:\n> +\t\t\tsensors.push_back(entity);\n> +\t\t\tbreak;\n> +\n> +\t\tcase MEDIA_ENT_F_IO_V4L:\n> +\t\t\tif (entity->pads().size() == 1 &&\n> +\t\t\t (entity->pads()[0]->flags() & MEDIA_PAD_FL_SINK))\n> +\t\t\t\tvideos.push_back(entity);\n> +\t\t\tbreak;\n> +\n> +\t\tdefault:\n> +\t\t\tbreak;\n> +\t\t}\n> +\t}\n> +\n> +\tif (sensors.empty()) {\n> +\t\tLOG(SimplePipeline, Error) << \"No sensor found\";\n> +\t\treturn false;\n> +\t}\n> +\n> +\tif (videos.size() != 1) {\n> +\t\tLOG(SimplePipeline, Error)\n> +\t\t\t<< \"Pipeline with \" << videos.size()\n> +\t\t\t<< \" video capture nodes is not supported\";\n> +\t\treturn false;\n> +\t}\n> +\n> +\t/* Locate and open the capture video node. */\n> +\tvideo_ = new V4L2VideoDevice(videos[0]);\n> +\tif (video_->open() < 0)\n> +\t\treturn false;\n> +\n> +\tif (video_->caps().isMultiplanar()) {\n> +\t\tLOG(SimplePipeline, Error)\n> +\t\t\t<< \"V4L2 multiplanar devices are not supported\";\n> +\t\treturn false;\n> +\t}\n> +\n> +\tvideo_->bufferReady.connect(this, &SimplePipelineHandler::bufferReady);\n> +\n> +\t/*\n> +\t * Create one camera data instance for each sensor and gather all\n> +\t * entities in all pipelines.\n> +\t */\n> +\tstd::vector> pipelines;\n> +\tstd::set entities;\n> +\n> +\tpipelines.reserve(sensors.size());\n> +\n> +\tfor (MediaEntity *sensor : sensors) {\n> +\t\tstd::unique_ptr data =\n> +\t\t\tstd::make_unique(this, sensor,\n> +\t\t\t\t\t\t\t videos[0]);\n> +\t\tif (!data->isValid()) {\n> +\t\t\tLOG(SimplePipeline, Error)\n> +\t\t\t\t<< \"No valid pipeline for sensor '\"\n> +\t\t\t\t<< sensor->name() << \"', skipping\";\n> +\t\t\tcontinue;\n> +\t\t}\n> +\n> +\t\tfor (SimpleCameraData::Entity &entity : data->entities_)\n> +\t\t\tentities.insert(entity.entity);\n> +\n> +\t\tpipelines.push_back(std::move(data));\n> +\t}\n> +\n> +\tif (entities.empty())\n> +\t\treturn false;\n> +\n> +\t/* Create and open V4L2Subdev instances for all the entities. */\n> +\tfor (MediaEntity *entity : entities) {\n\n\t\t/* Create a new map entry with the entity, and a\n\t\t * constructed V4L2Subdevice() using the entity for the\n\t\t * default constructor. */\n\nor some such? (Otherwise, emplacing the entity twice looks really odd\nand obfuscated)\n\n> +\t\tauto elem = subdevs_.emplace(std::piecewise_construct,\n> +\t\t\t\t\t std::forward_as_tuple(entity),\n> +\t\t\t\t\t std::forward_as_tuple(entity));\n\n\nIs a piecewise_construct required here?\n\n\tauto elem = subdevs_.emplace(entity, entity);\n\nshould do the trick, (and at least compiles).\n\n\n> +\t\tV4L2Subdevice *subdev = &elem.first->second;\n\nThe first second. Well I really do love pairs :-)\n\n> +\t\tint ret = subdev->open();\n> +\t\tif (ret < 0) {\n> +\t\t\tLOG(SimplePipeline, Error)\n> +\t\t\t\t<< \"Failed to open \" << subdev->deviceNode()\n> +\t\t\t\t<< \": \" << strerror(-ret);\n> +\t\t\treturn false;\n> +\t\t}\n> +\t}\n> +\n> +\t/* Initialize each pipeline and register a corresponding camera. */\n> +\tfor (std::unique_ptr &data : pipelines) {\n> +\t\tint ret = data->init();\n> +\t\tif (ret < 0)\n> +\t\t\tcontinue;\n> +\n> +\t\tstd::shared_ptr camera =\n> +\t\t\tCamera::create(this, data->sensor_->entity()->name(),\n> +\t\t\t\t data->streams());\n> +\t\tregisterCamera(std::move(camera), std::move(data));\n> +\t}\n> +\n> +\treturn true;\n> +}\n> +\n> +V4L2Subdevice *SimplePipelineHandler::subdev(const MediaEntity *entity)\n> +{\n> +\tauto iter = subdevs_.find(entity);\n> +\tif (iter == subdevs_.end())\n> +\t\treturn nullptr;\n> +\n> +\treturn &iter->second;\n> +}\n> +\n> +/* -----------------------------------------------------------------------------\n> + * Buffer Handling\n> + */\n> +\n> +void SimplePipelineHandler::bufferReady(FrameBuffer *buffer)\n> +{\n> +\tASSERT(activeCamera_);\n> +\tRequest *request = buffer->request();\n> +\tcompleteBuffer(activeCamera_, request, buffer);\n> +\tcompleteRequest(activeCamera_, request);\n> +}\n> +\n> +REGISTER_PIPELINE_HANDLER(SimplePipelineHandler);\n> +\n> +} /* namespace libcamera */\n>","headers":{"Return-Path":"","Received":["from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 85E356040B\n\tfor ;\n\tTue, 31 Mar 2020 13:45:17 +0200 (CEST)","from [192.168.0.20]\n\t(cpc89242-aztw30-2-0-cust488.18-1.cable.virginm.net [86.31.129.233])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id BD8E2999;\n\tTue, 31 Mar 2020 13:45:16 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ideasonboard.com\n\theader.i=@ideasonboard.com\n\theader.b=\"kWlWiR9g\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1585655117;\n\tbh=YodMwCfV3dhmbaYDYwuimRh6JdCZ4XuSPOUuSkXz5JY=;\n\th=Reply-To:Subject:To:Cc:References:From:Date:In-Reply-To:From;\n\tb=kWlWiR9gHSDn0P4XJz+MKza9t3Ri2VnPZ0TMeai15y6wlCi7rILf1ZQj/wFV6PGa8\n\tD8X7ZTR6p2Vk3zNXQrxtjDQno57p9W3pk+bciA3y6QHjI+IAhD6oXS3BQD08diELTo\n\tVyc0ihAnw/jTRmVvER92y6+cktOAoUAkqWTd8hBc=","Reply-To":"kieran.bingham@ideasonboard.com","To":"Laurent Pinchart ,\n\tlibcamera-devel@lists.libcamera.org","Cc":"Martijn Braam ,\n\tBenjamin GAIGNARD ","References":"<20200320014839.14683-1-laurent.pinchart@ideasonboard.com>\n\t<20200320014839.14683-10-laurent.pinchart@ideasonboard.com>","From":"Kieran Bingham ","Openpgp":"preference=signencrypt","Autocrypt":"addr=kieran.bingham@ideasonboard.com; keydata=\n\tmQINBFYE/WYBEACs1PwjMD9rgCu1hlIiUA1AXR4rv2v+BCLUq//vrX5S5bjzxKAryRf0uHat\n\tV/zwz6hiDrZuHUACDB7X8OaQcwhLaVlq6byfoBr25+hbZG7G3+5EUl9cQ7dQEdvNj6V6y/SC\n\trRanWfelwQThCHckbobWiQJfK9n7rYNcPMq9B8e9F020LFH7Kj6YmO95ewJGgLm+idg1Kb3C\n\tpotzWkXc1xmPzcQ1fvQMOfMwdS+4SNw4rY9f07Xb2K99rjMwZVDgESKIzhsDB5GY465sCsiQ\n\tcSAZRxqE49RTBq2+EQsbrQpIc8XiffAB8qexh5/QPzCmR4kJgCGeHIXBtgRj+nIkCJPZvZtf\n\tKr2EAbc6tgg6DkAEHJb+1okosV09+0+TXywYvtEop/WUOWQ+zo+Y/OBd+8Ptgt1pDRyOBzL8\n\tRXa8ZqRf0Mwg75D+dKntZeJHzPRJyrlfQokngAAs4PaFt6UfS+ypMAF37T6CeDArQC41V3ko\n\tlPn1yMsVD0p+6i3DPvA/GPIksDC4owjnzVX9kM8Zc5Cx+XoAN0w5Eqo4t6qEVbuettxx55gq\n\t8K8FieAjgjMSxngo/HST8TpFeqI5nVeq0/lqtBRQKumuIqDg+Bkr4L1V/PSB6XgQcOdhtd36\n\tOe9X9dXB8YSNt7VjOcO7BTmFn/Z8r92mSAfHXpb07YJWJosQOQARAQABtDBLaWVyYW4gQmlu\n\tZ2hhbSA8a2llcmFuLmJpbmdoYW1AaWRlYXNvbmJvYXJkLmNvbT6JAlcEEwEKAEECGwMFCwkI\n\tBwIGFQgJCgsCBBYCAwECHgECF4ACGQEWIQSQLdeYP70o/eNy1HqhHkZyEKRh/QUCXWTtygUJ\n\tCyJXZAAKCRChHkZyEKRh/f8dEACTDsbLN2nioNZMwyLuQRUAFcXNolDX48xcUXsWS2QjxaPm\n\tVsJx8Uy8aYkS85mdPBh0C83OovQR/OVbr8AxhGvYqBs3nQvbWuTl/+4od7DfK2VZOoKBAu5S\n\tQK2FYuUcikDqYcFWJ8DQnubxfE8dvzojHEkXw0sA4igINHDDFX3HJGZtLio+WpEFQtCbfTAG\n\tYZslasz1YZRbwEdSsmO3/kqy5eMnczlm8a21A3fKUo3g8oAZEFM+f4DUNzqIltg31OAB/kZS\n\tenKZQ/SWC8PmLg/ZXBrReYakxXtkP6w3FwMlzOlhGxqhIRNiAJfXJBaRhuUWzPOpEDE9q5YJ\n\tBmqQL2WJm1VSNNVxbXJHpaWMH1sA2R00vmvRrPXGwyIO0IPYeUYQa3gsy6k+En/aMQJd27dp\n\taScf9am9PFICPY5T4ppneeJLif2lyLojo0mcHOV+uyrds9XkLpp14GfTkeKPdPMrLLTsHRfH\n\tfA4I4OBpRrEPiGIZB/0im98MkGY/Mu6qxeZmYLCcgD6qz4idOvfgVOrNh+aA8HzIVR+RMW8H\n\tQGBN9f0E3kfwxuhl3omo6V7lDw8XOdmuWZNC9zPq1UfryVHANYbLGz9KJ4Aw6M+OgBC2JpkD\n\thXMdHUkC+d20dwXrwHTlrJi1YNp6rBc+xald3wsUPOZ5z8moTHUX/uPA/qhGsbkCDQRWBP1m\n\tARAAzijkb+Sau4hAncr1JjOY+KyFEdUNxRy+hqTJdJfaYihxyaj0Ee0P0zEi35CbE6lgU0Uz\n\ttih9fiUbSV3wfsWqg1Ut3/5rTKu7kLFp15kF7eqvV4uezXRD3Qu4yjv/rMmEJbbD4cTvGCYI\n\td6MDC417f7vK3hCbCVIZSp3GXxyC1LU+UQr3fFcOyCwmP9vDUR9JV0BSqHHxRDdpUXE26Dk6\n\tmhf0V1YkspE5St814ETXpEus2urZE5yJIUROlWPIL+hm3NEWfAP06vsQUyLvr/GtbOT79vXl\n\tEn1aulcYyu20dRRxhkQ6iILaURcxIAVJJKPi8dsoMnS8pB0QW12AHWuirPF0g6DiuUfPmrA5\n\tPKe56IGlpkjc8cO51lIxHkWTpCMWigRdPDexKX+Sb+W9QWK/0JjIc4t3KBaiG8O4yRX8ml2R\n\t+rxfAVKM6V769P/hWoRGdgUMgYHFpHGSgEt80OKK5HeUPy2cngDUXzwrqiM5Sz6Od0qw5pCk\n\tNlXqI0W/who0iSVM+8+RmyY0OEkxEcci7rRLsGnM15B5PjLJjh1f2ULYkv8s4SnDwMZ/kE04\n\t/UqCMK/KnX8pwXEMCjz0h6qWNpGwJ0/tYIgQJZh6bqkvBrDogAvuhf60Sogw+mH8b+PBlx1L\n\toeTK396wc+4c3BfiC6pNtUS5GpsPMMjYMk7kVvEAEQEAAYkCPAQYAQoAJgIbDBYhBJAt15g/\n\tvSj943LUeqEeRnIQpGH9BQJdizzIBQkLSKZiAAoJEKEeRnIQpGH9eYgQAJpjaWNgqNOnMTmD\n\tMJggbwjIotypzIXfhHNCeTkG7+qCDlSaBPclcPGYrTwCt0YWPU2TgGgJrVhYT20ierN8LUvj\n\t6qOPTd+Uk7NFzL65qkh80ZKNBFddx1AabQpSVQKbdcLb8OFs85kuSvFdgqZwgxA1vl4TFhNz\n\tPZ79NAmXLackAx3sOVFhk4WQaKRshCB7cSl+RIng5S/ThOBlwNlcKG7j7W2MC06BlTbdEkUp\n\tECzuuRBv8wX4OQl+hbWbB/VKIx5HKlLu1eypen/5lNVzSqMMIYkkZcjV2SWQyUGxSwq0O/sx\n\tS0A8/atCHUXOboUsn54qdxrVDaK+6jIAuo8JiRWctP16KjzUM7MO0/+4zllM8EY57rXrj48j\n\tsbEYX0YQnzaj+jO6kJtoZsIaYR7rMMq9aUAjyiaEZpmP1qF/2sYenDx0Fg2BSlLvLvXM0vU8\n\tpQk3kgDu7kb/7PRYrZvBsr21EIQoIjXbZxDz/o7z95frkP71EaICttZ6k9q5oxxA5WC6sTXc\n\tMW8zs8avFNuA9VpXt0YupJd2ijtZy2mpZNG02fFVXhIn4G807G7+9mhuC4XG5rKlBBUXTvPU\n\tAfYnB4JBDLmLzBFavQfvonSfbitgXwCG3vS+9HEwAjU30Bar1PEOmIbiAoMzuKeRm2LVpmq4\n\tWZw01QYHU/GUV/zHJSFk","Organization":"Ideas on Board","Message-ID":"<43484c6b-44ac-4bd7-88e4-b46070840315@ideasonboard.com>","Date":"Tue, 31 Mar 2020 12:45:14 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101\n\tThunderbird/60.9.1","MIME-Version":"1.0","In-Reply-To":"<20200320014839.14683-10-laurent.pinchart@ideasonboard.com>","Content-Type":"text/plain; charset=utf-8","Content-Language":"en-GB","Content-Transfer-Encoding":"8bit","Subject":"Re: [libcamera-devel] [PATCH v3 09/11] libcamera: pipeline: Add a\n\tsimple pipeline handler","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","X-List-Received-Date":"Tue, 31 Mar 2020 11:45:17 -0000"}},{"id":4370,"web_url":"https://patchwork.libcamera.org/comment/4370/","msgid":"<0519333c-3d17-325a-704b-664e475f556c@linaro.org>","date":"2020-04-03T13:13:35","subject":"Re: [libcamera-devel] [PATCH v3 09/11] libcamera: pipeline: Add a\n\tsimple pipeline handler","submitter":{"id":25,"url":"https://patchwork.libcamera.org/api/people/25/","name":"Andrey Konovalov","email":"andrey.konovalov@linaro.org"},"content":"Hi,\n\nOn 31.03.2020 14:45, Kieran Bingham wrote:\n> Hi Laurent (/Martijn...)\n> \n> \n> On 20/03/2020 01:48, Laurent Pinchart wrote:\n>> From: Martijn Braam \n>>\n>> This new pipeline handler aims at supporting any simple device without\n>> requiring any device-specific code. Simple devices are currently defined\n>> as a graph made of one or multiple camera sensors and a single video\n>> node, with each sensor connected to the video node through a linear\n>> pipeline.\n>>\n>> The simple pipeline handler will automatically parse the media graph,\n>> enumerate sensors, build supported stream configurations, and configure\n>> the pipeline, without any device-specific knowledge. It doesn't support\n>> configuration of any processing in the pipeline at the moment, but may\n>> be extended to support simple processing such as format conversion or\n>> scaling in the future.\n>>\n>> The only device-specific information in the pipeline handler is the list\n>> of supported drivers, required for device matching. We may be able to\n>> remove this in the future by matching with the simple pipeline handler\n>> as a last resort option, after all other pipeline handlers have been\n>> tried.\n>>\n>> Signed-off-by: Martijn Braam \n>> Signed-off-by: Laurent Pinchart \n> \n> Reviewed-by: Kieran Bingham \n> \n> Some comments and discussion points below, but I think we should work\n> towards trying to get this integrated so it can actually be used (which\n> is where we'll find the nuances and corner cases as people test it)\n> \n> --\n> Kieran\n> \n> \n> \n>> ---\n>> Changes since v2:\n>>\n>> - Log an error when setupFormats() fail\n>> - Propagate getFormat() and setFormat() errors to the caller of\n>> setupFormats()\n>> - Reorder variable declarations in validate()\n>> - Add \\todo comment related to the selection of the default format\n>> - Use log Error instead of Info if pipeline isn't valid\n>> - Rebase on top of V4L2PixelFormat\n>>\n>> Changes since v1:\n>>\n>> - Rebase on top of buffer API rework\n>> - Expose stream formats\n>> - Rework camera data config\n>> ---\n>> src/libcamera/pipeline/meson.build | 1 +\n>> src/libcamera/pipeline/simple/meson.build | 3 +\n>> src/libcamera/pipeline/simple/simple.cpp | 710 ++++++++++++++++++++++\n>> 3 files changed, 714 insertions(+)\n>> create mode 100644 src/libcamera/pipeline/simple/meson.build\n>> create mode 100644 src/libcamera/pipeline/simple/simple.cpp\n>>\n>> diff --git a/src/libcamera/pipeline/meson.build b/src/libcamera/pipeline/meson.build\n>> index 0d466225a72e..606ba31a0319 100644\n>> --- a/src/libcamera/pipeline/meson.build\n>> +++ b/src/libcamera/pipeline/meson.build\n>> @@ -5,3 +5,4 @@ libcamera_sources += files([\n>> \n>> subdir('ipu3')\n>> subdir('rkisp1')\n>> +subdir('simple')\n>> diff --git a/src/libcamera/pipeline/simple/meson.build b/src/libcamera/pipeline/simple/meson.build\n>> new file mode 100644\n>> index 000000000000..4945a3e173cf\n>> --- /dev/null\n>> +++ b/src/libcamera/pipeline/simple/meson.build\n>> @@ -0,0 +1,3 @@\n>> +libcamera_sources += files([\n>> + 'simple.cpp',\n>> +])\n>> diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp\n>> new file mode 100644\n\n\n\n>> +int SimpleCameraData::init()\n>> +{\n>> +\tSimplePipelineHandler *pipe = static_cast(pipe_);\n>> +\tV4L2VideoDevice *video = pipe->video();\n>> +\tint ret;\n>> +\n>> +\t/*\n>> +\t * Enumerate the possible pipeline configurations. For each media bus\n>> +\t * format supported by the sensor, propagate the formats through the\n>> +\t * pipeline, and enumerate the corresponding possible V4L2 pixel\n>> +\t * formats on the video node.\n>> +\t */\n>> +\tfor (unsigned int code : sensor_->mbusCodes()) {\n>> +\t\tV4L2SubdeviceFormat format{ code, sensor_->resolution() };\n>> +\n>> +\t\t/*\n>> +\t\t * Setup links first as some subdev drivers take active links\n>> +\t\t * into account to propaget TRY formats. So is life :-(\n> \n> /propaget/propagate/\n> \n> /So is life/Such is life/\n> \n> \n>> +\t\t */\n>> +\t\tret = setupLinks();\n>> +\t\tif (ret < 0)\n>> +\t\t\treturn ret;\n>> +\n>> +\t\tret = setupFormats(&format, V4L2Subdevice::TryFormat);\n>> +\t\tif (ret < 0) {\n>> +\t\t\tLOG(SimplePipeline, Error)\n>> +\t\t\t\t<< \"Failed to setup pipeline for media bus code \"\n>> +\t\t\t\t<< utils::hex(code, 4);\n> \n> Oh dear, seems we need mbusCode.toString() too! hehe\n> \n>> +\t\t\treturn ret;\n> \n> Andrey states that this return prevents iterating potentially successful\n> mbus-codes, and should instead 'continue'...\n\nI've taken a deeper look at the ov5640 + db410c logs I've received, and the above\nreturn *could* be a potential problem, but definitely not in this (ov5640 + db410c)\ncase.\n\nWe only saw SimpleCameraData::setupFormats() returning error due to failing\nsensor_->setFormat(format) call.\nThis happened if before the cam is started (and the stream cfg isn't set to a\nparticular width*height) the sensor is configured for the frame rate greater than 15 fps.\nIn particular, in the ov5640's driver probe function the initial mode is set to\n\"YUV422 UYVY VGA@30fps\".\nov5640_enum_frame_size() doesn't consider the current fps at all, so the maximum\nframe size reported is 2592x1944. But it only works at 15fps max, and an attempt\nto set the sensor format to a one with the maximum possible frame size of 2592x1944\nresults in -EINVAL returned by SUBDEV_S_FMT.\n(So in our ov5640 + db410c case replacing the \"return\" with \"continue\" does let the pipeline\nhandler to try all the other media bus codes if the first one fails, but they all fail the\nsame way as the first one does.)\n\nIf this issue with the frame rate is worked around in some way, the enumeration\nof the possible pipeline configurations completes kind of OK: a good configuration\nis selected in the end. But during the enumeration all the configurations, the\nbroken ones included are considered as the working ones:\n\nThe links are set up this way (\"entity name\":pad_number]):\n\n\"ov5640 4-0076\":0 -> \"msm_csiphy0\":0,\"msm_csiphy0\":1 -> \"msm_csid0\":0,\"msm_csid0\":1 ->\n\t-> \"msm_ispif0\":0,\"msm_ispif0\":1 -> \"msm_vfe0_rdi0\":0\n\n\nSimpleCameraData::setupFormats() creates a broken pipeline when it tries the sensor format\nnot supported by the rest of the entities. This results in e.g. the following:\n\n\"ov5640 4-0076\":0[MEDIA_BUS_FMT_RGB565_2X8_BE] -> [MEDIA_BUS_FMT_UYVY8_2X8]\"msm_csiphy0\":0,\n \"msm_csiphy0\":1[MEDIA_BUS_FMT_UYVY8_2X8] -> [MEDIA_BUS_FMT_UYVY8_2X8]\"msm_csid0\":0, etc\n\nI.e. an attempt to set fmt on \"msm_csiphy0\":0 to MEDIA_BUS_FMT_RGB565_2X8_BE results in the\nformat set to MEDIA_BUS_FMT_UYVY8_2X8 and no error is returned here (and down the pipeline\nas MEDIA_BUS_FMT_UYVY8_2X8 is set OK for all the other pads). So we get the first link misconfigured,\nbut the result is considered as a good pipeline as no error was returned while the format\nfrom the sensor output was propagated from the sensor output down to \"msm_vfe0_rdi0\":0.\nAnd the broken configuration is stored in the formats_ map (in the end, it works in our\nov5640+db410c case as the broken configurations are rewritten with the valid ones later).\n\nWho/what is to ensure that for each of the links, the source and the sink pads of the same link\nare configured to the same format?\n\nThanks,\nAndrey\n\n>> +\t\t}\n>> +\n>> +\t\tstd::map> videoFormats =\n>> +\t\t\tvideo->formats(format.mbus_code);\n>> +\n>> +\t\tLOG(SimplePipeline, Debug)\n>> +\t\t\t<< \"Adding configuration for \" << format.size.toString()\n>> +\t\t\t<< \" in pixel formats [ \"\n>> +\t\t\t<< utils::join(videoFormats, \", \",\n>> +\t\t\t\t [](const auto &f) {\n>> +\t\t\t\t\t return f.first.toString();\n>> +\t\t\t\t })\n> \n> Aha, I kinda like the utils::join!\n> \n>> +\t\t\t<< \" ]\";\n>> +\n>> +\t\t/*\n>> +\t\t * Store the configuration in the formats_ map, mapping\n>> +\t\t * PixelFormat to configuration. Any previously stored value is\n> \n> /to configuration/to the current configuration/ ?\n> \n>> +\t\t * overwritten, as the pipeline handler currently doesn't care\n>> +\t\t * about how a particular PixelFormat is achieved.\n> \n> As long as there's an output, I agree. If devices have specific\n> constraints such as better performance or such with a particular\n> mbus-code, then they need to create a platform specific pipeline handler\n> to deal with the nuances.\n> \n> Maybe there might be tuning hints or something later, but certainly not now.\n> \n> \n>> +\t\t */\n>> +\t\tfor (const auto &videoFormat : videoFormats) {\n>> +\t\t\tPixelFormat pixelFormat = video->toPixelFormat(videoFormat.first);\n>> +\t\t\tif (!pixelFormat)\n>> +\t\t\t\tcontinue;\n>> +\n>> +\t\t\tConfiguration config;\n>> +\t\t\tconfig.code = code;\n>> +\t\t\tconfig.pixelFormat = pixelFormat;\n>> +\t\t\tconfig.size = format.size;\n>> +\n>> +\t\t\tformats_[pixelFormat] = config;\n>> +\t\t}\n>> +\t}\n>> +\n>> +\tif (formats_.empty()) {\n>> +\t\tLOG(SimplePipeline, Error) << \"No valid configuration found\";\n>> +\t\treturn -EINVAL;\n>> +\t}\n>> +\n>> +\treturn 0;\n>> +}\n>> +\n>> +int SimpleCameraData::setupLinks()\n>> +{\n>> +\tint ret;\n>> +\n>> +\t/*\n>> +\t * Configure all links along the pipeline. Some entities may not allow\n>> +\t * multiple sink links to be enabled together, even on different sink\n>> +\t * pads. We must thus start by disabling all sink links (but the one we\n>> +\t * want to enable) before enabling the pipeline link.\n> \n> Eeek - this sounds like it could have potential system effects,\n> (disabling existing configurations) but I think this is the\n> 'Such-is-life' comment I read earlier.\n> \n> \n>> +\t */\n>> +\tfor (SimpleCameraData::Entity &e : entities_) {\n>> +\t\tfor (MediaPad *pad : e.link->sink()->entity()->pads()) {\n> \n> For readability, I would be tempted to pull e.link->sink()->entity() to\n> a local var called Entity &remote, or Entity &endpoint, or such,\n> essentially to make this line read as:\n> \n> for (MediaPad *pad : remote->pads()) {\n> \n>> +\t\t\tfor (MediaLink *link : pad->links()) {\n>> +\t\t\t\tif (link == e.link)\n>> +\t\t\t\t\tcontinue;\n>> +\n>> +\t\t\t\tif ((link->flags() & MEDIA_LNK_FL_ENABLED) &&\n>> +\t\t\t\t !(link->flags() & MEDIA_LNK_FL_IMMUTABLE)) {\n>> +\t\t\t\t\tret = link->setEnabled(false);\n>> +\t\t\t\t\tif (ret < 0)\n>> +\t\t\t\t\t\treturn ret;\n>> +\t\t\t\t}\n>> +\t\t\t}\n>> +\t\t}\n>> +\n> \n> I'm not going to try it out, but would this be easier / clearer if we\n> simply disabled *all* links, then explicitly enable links on our pipeline?\n> \n> (only a devils-advocate discussion point, not something that's needed).\n> \n>> +\t\tif (!(e.link->flags() & MEDIA_LNK_FL_ENABLED)) {\n>> +\t\t\tret = e.link->setEnabled(true);\n>> +\t\t\tif (ret < 0)\n>> +\t\t\t\treturn ret;\n>> +\t\t}\n>> +\t}\n>> +\n>> +\treturn 0;\n>> +}\n>> +\n>> +int SimpleCameraData::setupFormats(V4L2SubdeviceFormat *format,\n>> +\t\t\t\t V4L2Subdevice::Whence whence)\n>> +{\n>> +\tSimplePipelineHandler *pipe = static_cast(pipe_);\n>> +\tint ret;\n>> +\n>> +\t/*\n>> +\t * Configure the format on the sensor output and propagate it through\n>> +\t * the pipeline.\n>> +\t */\n>> +\tret = sensor_->setFormat(format);\n>> +\tif (ret < 0)\n>> +\t\treturn ret;\n>> +\n>> +\tfor (const Entity &e : entities_) {\n>> +\t\tMediaLink *link = e.link;\n>> +\t\tMediaPad *source = link->source();\n>> +\t\tMediaPad *sink = link->sink();\n>> +\n>> +\t\tif (source->entity() != sensor_->entity()) {\n>> +\t\t\tV4L2Subdevice *subdev = pipe->subdev(source->entity());\n>> +\t\t\tret = subdev->getFormat(source->index(), format, whence);\n>> +\t\t\tif (ret < 0)\n>> +\t\t\t\treturn ret;\n>> +\t\t}\n>> +\n>> +\t\tif (sink->entity()->function() != MEDIA_ENT_F_IO_V4L) {\n>> +\t\t\tV4L2Subdevice *subdev = pipe->subdev(sink->entity());\n>> +\t\t\tret = subdev->setFormat(sink->index(), format, whence);\n>> +\t\t\tif (ret < 0)\n>> +\t\t\t\treturn ret;\n>> +\t\t}\n>> +\n>> +\t\tLOG(SimplePipeline, Debug)\n>> +\t\t\t<< \"Link '\" << source->entity()->name()\n>> +\t\t\t<< \"':\" << source->index()\n>> +\t\t\t<< \" -> '\" << sink->entity()->name()\n>> +\t\t\t<< \"':\" << sink->index()\n>> +\t\t\t<< \" configured with format \" << format->toString();\n> \n> I wonder if we can get the whole pipeline configuration string to a\n> single line... but not required now.\n> \n>> +\t}\n>> +\n>> +\treturn 0;\n>> +}\n>> +\n>> +/* -----------------------------------------------------------------------------\n>> + * Camera Configuration\n>> + */\n>> +\n>> +SimpleCameraConfiguration::SimpleCameraConfiguration(Camera *camera,\n>> +\t\t\t\t\t\t SimpleCameraData *data)\n>> +\t: CameraConfiguration(), camera_(camera->shared_from_this()),\n>> +\t data_(data)\n>> +{\n>> +}\n>> +\n>> +CameraConfiguration::Status SimpleCameraConfiguration::validate()\n>> +{\n>> +\tStatus status = Valid;\n>> +\n>> +\tif (config_.empty())\n>> +\t\treturn Invalid;\n>> +\n>> +\t/* Cap the number of entries to the available streams. */\n>> +\tif (config_.size() > 1) {\n>> +\t\tconfig_.resize(1);\n>> +\t\tstatus = Adjusted;\n>> +\t}\n>> +\n>> +\tStreamConfiguration &cfg = config_[0];\n>> +\n>> +\t/* Adjust the pixel format. */\n>> +\tauto it = data_->formats_.find(cfg.pixelFormat);\n>> +\tif (it == data_->formats_.end())\n>> +\t\tit = data_->formats_.begin();\n>> +\n>> +\tPixelFormat pixelFormat = it->first;\n>> +\tif (cfg.pixelFormat != pixelFormat) {\n>> +\t\tLOG(SimplePipeline, Debug) << \"Adjusting pixel format\";\n>> +\t\tcfg.pixelFormat = pixelFormat;\n>> +\t\tstatus = Adjusted;\n>> +\t}\n>> +\n>> +\tconst SimpleCameraData::Configuration &pipeConfig = it->second;\n>> +\tif (cfg.size != pipeConfig.size) {\n>> +\t\tLOG(SimplePipeline, Debug)\n>> +\t\t\t<< \"Adjusting size from \" << cfg.size.toString()\n>> +\t\t\t<< \" to \" << pipeConfig.size.toString();\n>> +\t\tcfg.size = pipeConfig.size;\n>> +\t\tstatus = Adjusted;\n>> +\t}\n>> +\n>> +\tcfg.bufferCount = 3;\n>> +\n>> +\treturn status;\n>> +}\n>> +\n>> +/* -----------------------------------------------------------------------------\n>> + * Pipeline Handler\n>> + */\n>> +\n>> +SimplePipelineHandler::SimplePipelineHandler(CameraManager *manager)\n>> +\t: PipelineHandler(manager), video_(nullptr)\n>> +{\n>> +}\n>> +\n>> +SimplePipelineHandler::~SimplePipelineHandler()\n>> +{\n>> +\tdelete video_;\n>> +}\n>> +\n>> +CameraConfiguration *SimplePipelineHandler::generateConfiguration(Camera *camera,\n>> +\t\t\t\t\t\t\t\t const StreamRoles &roles)\n>> +{\n>> +\tSimpleCameraData *data = cameraData(camera);\n>> +\tCameraConfiguration *config =\n>> +\t\tnew SimpleCameraConfiguration(camera, data);\n>> +\n>> +\tif (roles.empty())\n>> +\t\treturn config;\n>> +\n>> +\t/* Create the formats map. */\n>> +\tstd::map> formats;\n>> +\tstd::transform(data->formats_.begin(), data->formats_.end(),\n>> +\t\t std::inserter(formats, formats.end()),\n>> +\t\t [](const auto &format) -> decltype(formats)::value_type {\n>> +\t\t\t const PixelFormat &pixelFormat = format.first;\n>> +\t\t\t const Size &size = format.second.size;\n>> +\t\t\t return { pixelFormat, { size } };\n>> +\t\t });\n> \n> This is quite unfriendly to the reader :-( ... not that I have a better\n> alternative ... but eugh\n> \n>> +\n>> +\t/*\n>> +\t * Create the stream configuration. Take the first entry in the formats\n>> +\t * map as the default, for lack of a better option.\n>> +\t *\n>> +\t * \\todo Implement a better way to pick the default format\n> \n> Is there one? It's a generic configuration for a pipeline handler.\n> \n> What might be better as a general case, and presumably any logic added\n> here would potentially be applicable to most pipeline handlers too.\n> \n> The biggest Size? The smallest Size? or the nearest size to a list of\n> predefined sizes for a given role?\n> \n>> +\t */\n>> +\tStreamConfiguration cfg{ StreamFormats{ formats } };\n>> +\tcfg.pixelFormat = formats.begin()->first;\n>> +\tcfg.size = formats.begin()->second[0].max;\n>> +\n>> +\tconfig->addConfiguration(cfg);\n>> +\n>> +\tconfig->validate();\n>> +\n>> +\treturn config;\n>> +}\n>> +\n>> +int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c)\n>> +{\n>> +\tSimpleCameraConfiguration *config =\n>> +\t\tstatic_cast(c);\n>> +\tSimpleCameraData *data = cameraData(camera);\n>> +\tStreamConfiguration &cfg = config->at(0);\n>> +\tint ret;\n>> +\n>> +\t/*\n>> +\t * Configure links on the pipeline and propagate formats from the\n>> +\t * sensor to the video node.\n>> +\t */\n>> +\tret = data->setupLinks();\n>> +\tif (ret < 0)\n>> +\t\treturn ret;\n>> +\n>> +\tconst SimpleCameraData::Configuration &pipeConfig =\n>> +\t\tdata->formats_[cfg.pixelFormat];\n>> +\n>> +\tV4L2SubdeviceFormat format{ pipeConfig.code, data->sensor_->resolution() };\n>> +\n>> +\tret = data->setupFormats(&format, V4L2Subdevice::ActiveFormat);\n>> +\tif (ret < 0)\n>> +\t\treturn ret;\n>> +\n>> +\t/* Configure the video node. */\n>> +\tV4L2PixelFormat videoFormat = video_->toV4L2PixelFormat(cfg.pixelFormat);\n>> +\n>> +\tV4L2DeviceFormat outputFormat = {};\n>> +\toutputFormat.fourcc = videoFormat;\n>> +\toutputFormat.size = cfg.size;\n>> +\n>> +\tret = video_->setFormat(&outputFormat);\n>> +\tif (ret)\n>> +\t\treturn ret;\n>> +\n>> +\tif (outputFormat.size != cfg.size || outputFormat.fourcc != videoFormat) {\n>> +\t\tLOG(SimplePipeline, Error)\n>> +\t\t\t<< \"Unable to configure capture in \" << cfg.toString();\n>> +\t\treturn -EINVAL;\n>> +\t}\n>> +\n>> +\tcfg.setStream(&data->stream_);\n>> +\n>> +\treturn 0;\n>> +}\n>> +\n>> +int SimplePipelineHandler::exportFrameBuffers(Camera *camera, Stream *stream,\n>> +\t\t\t\t\t std::vector> *buffers)\n>> +{\n>> +\tunsigned int count = stream->configuration().bufferCount;\n>> +\n>> +\treturn video_->exportBuffers(count, buffers);\n>> +}\n>> +\n>> +int SimplePipelineHandler::start(Camera *camera)\n>> +{\n>> +\tSimpleCameraData *data = cameraData(camera);\n>> +\tunsigned int count = data->stream_.configuration().bufferCount;\n>> +\n>> +\tint ret = video_->importBuffers(count);\n>> +\tif (ret < 0)\n>> +\t\treturn ret;\n>> +\n>> +\tret = video_->streamOn();\n>> +\tif (ret < 0) {\n>> +\t\tvideo_->releaseBuffers();\n> \n> I wonder if video_->importBuffers() should be something that now happens\n> at video_->streamOn()?\n> \n> I presume that it is now more of an operation to set up V4L2 with the\n> internal buffer structures for passing the information on 'which' buffer\n> is in use to V4L2...\n> \n> Every video stream will have to do this?\n> \n> Otherwise, if conceptually we don't want to mis-manage buffer handling\n> in V4L2VideoDevice::streamOn() we might want to have a V4L2Stream object\n> to do so ...\n> \n> (clearly those are just V4L2VideoDevice API discussion points, and not\n> actions on this patch)\n> \n> \n>> +\t\treturn ret;\n>> +\t}\n>> +\n>> +\tactiveCamera_ = camera;\n> \n> Will simple pipeline handler support more than one camera?\n> \n> The activeCamera_ feels like a way of supporting more than one camera on\n> a pipeline, which could otherwise have been a bool cameraActive to\n> represent it's state. In fact I'm not sure it's even needed to do\n> that... The only actual check is an assert to ensure that it's set\n> before using it in the bufferReady() callbacks, which could likely have\n> accessed the camera directly?\n> \n> But if it's worth keeping this as it is to keep it close to other\n> pipeline handlers then so be it.\n> \n> \n> Looks like it is used actaully, as the Camera is only created and\n> registered with the CameraManager, so we don't really have direct access\n> to the Camera within the PipelineHandler, and we have to be given it..\n> (even though /we/ created it :-D)\n> \n> \n>> +\n>> +\treturn 0;\n>> +}\n>> +\n>> +void SimplePipelineHandler::stop(Camera *camera)\n>> +{\n>> +\tvideo_->streamOff();\n>> +\tvideo_->releaseBuffers();\n>> +\tactiveCamera_ = nullptr;\n>> +}\n>> +\n>> +int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request)\n>> +{\n>> +\tSimpleCameraData *data = cameraData(camera);\n>> +\tStream *stream = &data->stream_;\n>> +\n>> +\tFrameBuffer *buffer = request->findBuffer(stream);\n>> +\tif (!buffer) {\n>> +\t\tLOG(SimplePipeline, Error)\n>> +\t\t\t<< \"Attempt to queue request with invalid stream\";\n>> +\t\treturn -ENOENT;\n>> +\t}\n>> +\n>> +\treturn video_->queueBuffer(buffer);\n>> +}\n>> +\n>> +/* -----------------------------------------------------------------------------\n>> + * Match and Setup\n>> + */\n>> +\n>> +bool SimplePipelineHandler::match(DeviceEnumerator *enumerator)\n>> +{\n>> +\tstatic const char * const drivers[] = {\n>> +\t\t\"imx7-csi\",\n>> +\t\t\"sun6i-csi\",\n>> +\t};\n>> +\n>> +\tfor (const char *driver : drivers) {\n>> +\t\tDeviceMatch dm(driver);\n>> +\t\tmedia_ = acquireMediaDevice(enumerator, dm);\n>> +\t\tif (media_)\n>> +\t\t\tbreak;\n>> +\t}\n>> +\n>> +\tif (!media_)\n>> +\t\treturn false;\n>> +\n>> +\t/*\n>> +\t * Locate sensors and video nodes. We only support pipelines with at\n>> +\t * least one sensor and exactly one video captude node.\n> \n> /captude/capture/\n> \n>> +\t */\n>> +\tstd::vector sensors;\n>> +\tstd::vector videos;\n>> +\n>> +\tfor (MediaEntity *entity : media_->entities()) {\n>> +\t\tswitch (entity->function()) {\n>> +\t\tcase MEDIA_ENT_F_CAM_SENSOR:\n>> +\t\t\tsensors.push_back(entity);\n>> +\t\t\tbreak;\n>> +\n>> +\t\tcase MEDIA_ENT_F_IO_V4L:\n>> +\t\t\tif (entity->pads().size() == 1 &&\n>> +\t\t\t (entity->pads()[0]->flags() & MEDIA_PAD_FL_SINK))\n>> +\t\t\t\tvideos.push_back(entity);\n>> +\t\t\tbreak;\n>> +\n>> +\t\tdefault:\n>> +\t\t\tbreak;\n>> +\t\t}\n>> +\t}\n>> +\n>> +\tif (sensors.empty()) {\n>> +\t\tLOG(SimplePipeline, Error) << \"No sensor found\";\n>> +\t\treturn false;\n>> +\t}\n>> +\n>> +\tif (videos.size() != 1) {\n>> +\t\tLOG(SimplePipeline, Error)\n>> +\t\t\t<< \"Pipeline with \" << videos.size()\n>> +\t\t\t<< \" video capture nodes is not supported\";\n>> +\t\treturn false;\n>> +\t}\n>> +\n>> +\t/* Locate and open the capture video node. */\n>> +\tvideo_ = new V4L2VideoDevice(videos[0]);\n>> +\tif (video_->open() < 0)\n>> +\t\treturn false;\n>> +\n>> +\tif (video_->caps().isMultiplanar()) {\n>> +\t\tLOG(SimplePipeline, Error)\n>> +\t\t\t<< \"V4L2 multiplanar devices are not supported\";\n>> +\t\treturn false;\n>> +\t}\n>> +\n>> +\tvideo_->bufferReady.connect(this, &SimplePipelineHandler::bufferReady);\n>> +\n>> +\t/*\n>> +\t * Create one camera data instance for each sensor and gather all\n>> +\t * entities in all pipelines.\n>> +\t */\n>> +\tstd::vector> pipelines;\n>> +\tstd::set entities;\n>> +\n>> +\tpipelines.reserve(sensors.size());\n>> +\n>> +\tfor (MediaEntity *sensor : sensors) {\n>> +\t\tstd::unique_ptr data =\n>> +\t\t\tstd::make_unique(this, sensor,\n>> +\t\t\t\t\t\t\t videos[0]);\n>> +\t\tif (!data->isValid()) {\n>> +\t\t\tLOG(SimplePipeline, Error)\n>> +\t\t\t\t<< \"No valid pipeline for sensor '\"\n>> +\t\t\t\t<< sensor->name() << \"', skipping\";\n>> +\t\t\tcontinue;\n>> +\t\t}\n>> +\n>> +\t\tfor (SimpleCameraData::Entity &entity : data->entities_)\n>> +\t\t\tentities.insert(entity.entity);\n>> +\n>> +\t\tpipelines.push_back(std::move(data));\n>> +\t}\n>> +\n>> +\tif (entities.empty())\n>> +\t\treturn false;\n>> +\n>> +\t/* Create and open V4L2Subdev instances for all the entities. */\n>> +\tfor (MediaEntity *entity : entities) {\n> \n> \t\t/* Create a new map entry with the entity, and a\n> \t\t * constructed V4L2Subdevice() using the entity for the\n> \t\t * default constructor. */\n> \n> or some such? (Otherwise, emplacing the entity twice looks really odd\n> and obfuscated)\n> \n>> +\t\tauto elem = subdevs_.emplace(std::piecewise_construct,\n>> +\t\t\t\t\t std::forward_as_tuple(entity),\n>> +\t\t\t\t\t std::forward_as_tuple(entity));\n> \n> \n> Is a piecewise_construct required here?\n> \n> \tauto elem = subdevs_.emplace(entity, entity);\n> \n> should do the trick, (and at least compiles).\n> \n> \n>> +\t\tV4L2Subdevice *subdev = &elem.first->second;\n> \n> The first second. Well I really do love pairs :-)\n> \n>> +\t\tint ret = subdev->open();\n>> +\t\tif (ret < 0) {\n>> +\t\t\tLOG(SimplePipeline, Error)\n>> +\t\t\t\t<< \"Failed to open \" << subdev->deviceNode()\n>> +\t\t\t\t<< \": \" << strerror(-ret);\n>> +\t\t\treturn false;\n>> +\t\t}\n>> +\t}\n>> +\n>> +\t/* Initialize each pipeline and register a corresponding camera. */\n>> +\tfor (std::unique_ptr &data : pipelines) {\n>> +\t\tint ret = data->init();\n>> +\t\tif (ret < 0)\n>> +\t\t\tcontinue;\n>> +\n>> +\t\tstd::shared_ptr camera =\n>> +\t\t\tCamera::create(this, data->sensor_->entity()->name(),\n>> +\t\t\t\t data->streams());\n>> +\t\tregisterCamera(std::move(camera), std::move(data));\n>> +\t}\n>> +\n>> +\treturn true;\n>> +}\n>> +\n>> +V4L2Subdevice *SimplePipelineHandler::subdev(const MediaEntity *entity)\n>> +{\n>> +\tauto iter = subdevs_.find(entity);\n>> +\tif (iter == subdevs_.end())\n>> +\t\treturn nullptr;\n>> +\n>> +\treturn &iter->second;\n>> +}\n>> +\n>> +/* -----------------------------------------------------------------------------\n>> + * Buffer Handling\n>> + */\n>> +\n>> +void SimplePipelineHandler::bufferReady(FrameBuffer *buffer)\n>> +{\n>> +\tASSERT(activeCamera_);\n>> +\tRequest *request = buffer->request();\n>> +\tcompleteBuffer(activeCamera_, request, buffer);\n>> +\tcompleteRequest(activeCamera_, request);\n>> +}\n>> +\n>> +REGISTER_PIPELINE_HANDLER(SimplePipelineHandler);\n>> +\n>> +} /* namespace libcamera */\n>>\n>","headers":{"Return-Path":"","Received":["from mail-lf1-x134.google.com (mail-lf1-x134.google.com\n\t[IPv6:2a00:1450:4864:20::134])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 563C9600FA\n\tfor ;\n\tFri, 3 Apr 2020 15:13:39 +0200 (CEST)","by mail-lf1-x134.google.com with SMTP id n20so5706692lfl.10\n\tfor ;\n\tFri, 03 Apr 2020 06:13:39 -0700 (PDT)","from [192.168.118.216] (37-144-159-139.broadband.corbina.ru.\n\t[37.144.159.139]) by smtp.gmail.com with ESMTPSA id\n\tv21sm5634752lji.81.2020.04.03.06.13.36\n\t(version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);\n\tFri, 03 Apr 2020 06:13:36 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key; \n\tunprotected) header.d=linaro.org\n\theader.i=@linaro.org header.b=\"cxJP1RR5\"; \n\tdkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=subject:to:cc:references:from:message-id:date:user-agent\n\t:mime-version:in-reply-to:content-language:content-transfer-encoding; \n\tbh=v8hEcKXMqtzhcZJtghZ2EDb0qBzQltxy7JQdiJEO38I=;\n\tb=cxJP1RR5EJDBWrYN49nP6ap4D11kSfaRDwzPWn/hl0MPDilX70dFcFtl8IlaI7iuVc\n\tddQKmMzNqSBq6qcH10eHfUGXtaeHSXZiw+P8E+sbs8YXCHJf74bSadp3xb20km2yLf/l\n\tLWV82ZH1zc3zi2k07uSk+J8sde3xWm3fl7xu5iDfrY8EqmujlJYyNhqM+U/UVT24Jj8u\n\trowsHxp43haLQcqlfyseQSXvFojrtQp+XdCWLdz5BbSur1nDSd4UNX789k2vcpgrntwO\n\tB+2LsQndzXQ/1/ZT7T7BD1aZNzbF/i8HD0T1HmX08MNNrbtaQ8QBufzuOlirLSqtUpVI\n\tDOgQ==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:subject:to:cc:references:from:message-id:date\n\t:user-agent:mime-version:in-reply-to:content-language\n\t:content-transfer-encoding;\n\tbh=v8hEcKXMqtzhcZJtghZ2EDb0qBzQltxy7JQdiJEO38I=;\n\tb=fGYm7K+QZhkZPOp61ucNBU1hXY30KPqMj66rm508X1yUe45cj0FeNORT+A24kmZvcN\n\tTVur3E/9fsFdN+egJfIa6MrA2KTExRPlJpH8BDbaCcNlDXRoyKF3Fv3+LF5yGNxAijDi\n\tqtZyUHhIBaZ8YKL4+3krvQIUp/CXzpEt2CDQNhrHxnrB034lvuM2nlWt6Jpa960jr1Z1\n\tA0EKOnzuHqQvdsfEbjN/KUVQSKDVjk1zjhGETd/tdv3ViHivA70tXWT3VSVxaVkXzs3r\n\txyS5hyWlyPsR/wKG7ZV1cOXQRUW6KdSKoD0wCKM7OX80eMy28U6XPnCixgZlz4APRrAs\n\tjXEQ==","X-Gm-Message-State":"AGi0PuZAm/pb4otAzvUVDiGfJS+wewJMRJHeCZdKN9QaUNVv9SDdgg80\n\t1QCsIaUOGIDy010eBrE2qcNM9Q==","X-Google-Smtp-Source":"APiQypLllwnoyM1pbRzudkoYzU1rfXXtWP2Xy0QYiUpzYFu/jGfrx1vYrC4CotRs/2zfTDMMF91USQ==","X-Received":"by 2002:a19:9141:: with SMTP id y1mr5373614lfj.168.1585919618147;\n\tFri, 03 Apr 2020 06:13:38 -0700 (PDT)","To":"kieran.bingham@ideasonboard.com,\n\tLaurent Pinchart ,\n\tlibcamera-devel@lists.libcamera.org","Cc":"Martijn Braam ,\n\tBenjamin GAIGNARD ","References":"<20200320014839.14683-1-laurent.pinchart@ideasonboard.com>\n\t<20200320014839.14683-10-laurent.pinchart@ideasonboard.com>\n\t<43484c6b-44ac-4bd7-88e4-b46070840315@ideasonboard.com>","From":"Andrey Konovalov ","Message-ID":"<0519333c-3d17-325a-704b-664e475f556c@linaro.org>","Date":"Fri, 3 Apr 2020 16:13:35 +0300","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101\n\tThunderbird/68.4.1","MIME-Version":"1.0","In-Reply-To":"<43484c6b-44ac-4bd7-88e4-b46070840315@ideasonboard.com>","Content-Type":"text/plain; charset=utf-8; format=flowed","Content-Language":"en-US","Content-Transfer-Encoding":"7bit","Subject":"Re: [libcamera-devel] [PATCH v3 09/11] libcamera: pipeline: Add a\n\tsimple pipeline handler","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","X-List-Received-Date":"Fri, 03 Apr 2020 13:13:39 -0000"}},{"id":4372,"web_url":"https://patchwork.libcamera.org/comment/4372/","msgid":"<053e8bb2-1d50-1aa5-b09a-fdb6718e3a41@linaro.org>","date":"2020-04-03T13:55:32","subject":"Re: [libcamera-devel] [PATCH v3 09/11] libcamera: pipeline: Add a\n\tsimple pipeline handler","submitter":{"id":25,"url":"https://patchwork.libcamera.org/api/people/25/","name":"Andrey Konovalov","email":"andrey.konovalov@linaro.org"},"content":"Hi Laurent,\n\nOn 20.03.2020 04:48, Laurent Pinchart wrote:\n> From: Martijn Braam \n> \n> This new pipeline handler aims at supporting any simple device without\n> requiring any device-specific code. Simple devices are currently defined\n> as a graph made of one or multiple camera sensors and a single video\n> node, with each sensor connected to the video node through a linear\n> pipeline.\n> \n> The simple pipeline handler will automatically parse the media graph,\n> enumerate sensors, build supported stream configurations, and configure\n> the pipeline, without any device-specific knowledge. It doesn't support\n> configuration of any processing in the pipeline at the moment, but may\n> be extended to support simple processing such as format conversion or\n> scaling in the future.\n> \n> The only device-specific information in the pipeline handler is the list\n> of supported drivers, required for device matching. We may be able to\n> remove this in the future by matching with the simple pipeline handler\n> as a last resort option, after all other pipeline handlers have been\n> tried.\n> \n> Signed-off-by: Martijn Braam \n> Signed-off-by: Laurent Pinchart \n> ---\n> Changes since v2:\n> \n> - Log an error when setupFormats() fail\n> - Propagate getFormat() and setFormat() errors to the caller of\n> setupFormats()\n> - Reorder variable declarations in validate()\n> - Add \\todo comment related to the selection of the default format\n> - Use log Error instead of Info if pipeline isn't valid\n> - Rebase on top of V4L2PixelFormat\n> \n> Changes since v1:\n> \n> - Rebase on top of buffer API rework\n> - Expose stream formats\n> - Rework camera data config\n> ---\n> src/libcamera/pipeline/meson.build | 1 +\n> src/libcamera/pipeline/simple/meson.build | 3 +\n> src/libcamera/pipeline/simple/simple.cpp | 710 ++++++++++++++++++++++\n> 3 files changed, 714 insertions(+)\n> create mode 100644 src/libcamera/pipeline/simple/meson.build\n> create mode 100644 src/libcamera/pipeline/simple/simple.cpp\n> \n> diff --git a/src/libcamera/pipeline/meson.build b/src/libcamera/pipeline/meson.build\n> index 0d466225a72e..606ba31a0319 100644\n> --- a/src/libcamera/pipeline/meson.build\n> +++ b/src/libcamera/pipeline/meson.build\n> @@ -5,3 +5,4 @@ libcamera_sources += files([\n> \n> subdir('ipu3')\n> subdir('rkisp1')\n> +subdir('simple')\n> diff --git a/src/libcamera/pipeline/simple/meson.build b/src/libcamera/pipeline/simple/meson.build\n> new file mode 100644\n> index 000000000000..4945a3e173cf\n> --- /dev/null\n> +++ b/src/libcamera/pipeline/simple/meson.build\n> @@ -0,0 +1,3 @@\n> +libcamera_sources += files([\n> + 'simple.cpp',\n> +])\n> diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp\n> new file mode 100644\n> index 000000000000..545a99fe31ca\n> --- /dev/null\n> +++ b/src/libcamera/pipeline/simple/simple.cpp\n> @@ -0,0 +1,710 @@\n> +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n> +/*\n> + * Copyright (C) 2020, Laurent Pinchart\n> + * Copyright (C) 2019, Martijn Braam\n> + *\n> + * simple.cpp - Pipeline handler for simple pipelines\n> + */\n> +\n> +#include \n> +#include \n> +#include \n> +#include \n> +#include \n> +#include \n> +#include \n> +#include \n> +#include \n> +#include \n> +\n> +#include \n> +\n> +#include \n> +#include \n> +#include \n> +\n> +#include \"camera_sensor.h\"\n> +#include \"device_enumerator.h\"\n> +#include \"log.h\"\n> +#include \"media_device.h\"\n> +#include \"pipeline_handler.h\"\n> +#include \"v4l2_subdevice.h\"\n> +#include \"v4l2_videodevice.h\"\n> +\n> +namespace libcamera {\n> +\n> +LOG_DEFINE_CATEGORY(SimplePipeline)\n> +\n> +class SimplePipelineHandler;\n> +\n> +class SimpleCameraData : public CameraData\n> +{\n> +public:\n> +\tSimpleCameraData(PipelineHandler *pipe, MediaEntity *sensor,\n> +\t\t\t MediaEntity *video);\n> +\n> +\tbool isValid() const { return sensor_ != nullptr; }\n> +\tstd::set streams() { return { &stream_ }; }\n> +\n> +\tint init();\n> +\tint setupLinks();\n> +\tint setupFormats(V4L2SubdeviceFormat *format,\n> +\t\t\t V4L2Subdevice::Whence whence);\n> +\n> +\tstruct Entity {\n> +\t\tMediaEntity *entity;\n> +\t\tMediaLink *link;\n> +\t};\n> +\n> +\tstruct Configuration {\n> +\t\tuint32_t code;\n> +\t\tPixelFormat pixelFormat;\n> +\t\tSize size;\n> +\t};\n> +\n> +\tStream stream_;\n> +\tstd::unique_ptr sensor_;\n> +\tstd::list entities_;\n> +\n> +\tstd::vector configs_;\n> +\tstd::map formats_;\n> +};\n> +\n> +class SimpleCameraConfiguration : public CameraConfiguration\n> +{\n> +public:\n> +\tSimpleCameraConfiguration(Camera *camera, SimpleCameraData *data);\n> +\n> +\tStatus validate() override;\n> +\n> +\tconst V4L2SubdeviceFormat &sensorFormat() { return sensorFormat_; }\n> +\n> +private:\n> +\t/*\n> +\t * The SimpleCameraData instance is guaranteed to be valid as long as\n> +\t * the corresponding Camera instance is valid. In order to borrow a\n> +\t * reference to the camera data, store a new reference to the camera.\n> +\t */\n> +\tstd::shared_ptr camera_;\n> +\tconst SimpleCameraData *data_;\n> +\n> +\tV4L2SubdeviceFormat sensorFormat_;\n> +};\n> +\n> +class SimplePipelineHandler : public PipelineHandler\n> +{\n> +public:\n> +\tSimplePipelineHandler(CameraManager *manager);\n> +\t~SimplePipelineHandler();\n> +\n> +\tCameraConfiguration *generateConfiguration(Camera *camera,\n> +\t\t\t\t\t\t const StreamRoles &roles) override;\n> +\tint configure(Camera *camera, CameraConfiguration *config) override;\n> +\n> +\tint exportFrameBuffers(Camera *camera, Stream *stream,\n> +\t\t\t std::vector> *buffers) override;\n> +\n> +\tint start(Camera *camera) override;\n> +\tvoid stop(Camera *camera) override;\n> +\n> +\tbool match(DeviceEnumerator *enumerator) override;\n> +\n> +\tV4L2VideoDevice *video() { return video_; }\n> +\tV4L2Subdevice *subdev(const MediaEntity *entity);\n> +\n> +protected:\n> +\tint queueRequestDevice(Camera *camera, Request *request) override;\n> +\n> +private:\n> +\tSimpleCameraData *cameraData(const Camera *camera)\n> +\t{\n> +\t\treturn static_cast(\n> +\t\t\tPipelineHandler::cameraData(camera));\n> +\t}\n> +\n> +\tint initLinks();\n> +\n> +\tint createCamera(MediaEntity *sensor);\n> +\n> +\tvoid bufferReady(FrameBuffer *buffer);\n> +\n> +\tMediaDevice *media_;\n> +\tV4L2VideoDevice *video_;\n> +\tstd::map subdevs_;\n> +\n> +\tCamera *activeCamera_;\n> +};\n> +\n> +/* -----------------------------------------------------------------------------\n> + * Camera Data\n> + */\n> +SimpleCameraData::SimpleCameraData(PipelineHandler *pipe, MediaEntity *sensor,\n> +\t\t\t\t MediaEntity *video)\n> +\t: CameraData(pipe)\n> +{\n> +\tint ret;\n> +\n> +\t/*\n> +\t * Walk the pipeline towards the video node and store all entities\n> +\t * along the way.\n> +\t */\n> +\tMediaEntity *source = sensor;\n> +\n> +\twhile (source) {\n> +\t\t/* If we have reached the video node, we're done. */\n> +\t\tif (source == video)\n> +\t\t\tbreak;\n> +\n> +\t\t/* Use the first output pad that has links. */\n> +\t\tMediaPad *sourcePad = nullptr;\n> +\t\tfor (MediaPad *pad : source->pads()) {\n> +\t\t\tif ((pad->flags() & MEDIA_PAD_FL_SOURCE) &&\n> +\t\t\t !pad->links().empty()) {\n> +\t\t\t\tsourcePad = pad;\n> +\t\t\t\tbreak;\n> +\t\t\t}\n> +\t\t}\n> +\n> +\t\tif (!sourcePad)\n> +\t\t\treturn;\n> +\n> +\t\t/* Use the first link that isn't immutable and disabled. */\n> +\t\tMediaLink *sourceLink = nullptr;\n> +\t\tfor (MediaLink *link : sourcePad->links()) {\n> +\t\t\tif ((link->flags() & MEDIA_LNK_FL_ENABLED) ||\n> +\t\t\t !(link->flags() & MEDIA_LNK_FL_IMMUTABLE)) {\n> +\t\t\t\tsourceLink = link;\n> +\t\t\t\tbreak;\n> +\t\t\t}\n> +\t\t}\n> +\n> +\t\tif (!sourceLink)\n> +\t\t\treturn;\n> +\n> +\t\tentities_.push_back({ source, sourceLink });\n> +\n> +\t\tsource = sourceLink->sink()->entity();\n> +\n> +\t\t/* Avoid infinite loops. */\n> +\t\tauto iter = std::find_if(entities_.begin(), entities_.end(),\n> +\t\t\t\t\t [&](const Entity &entity) {\n> +\t\t\t\t\t\t return entity.entity == source;\n> +\t\t\t\t\t });\n> +\t\tif (iter != entities_.end()) {\n> +\t\t\tLOG(SimplePipeline, Info) << \"Loop detected in pipeline\";\n> +\t\t\treturn;\n> +\t\t}\n> +\t}\n> +\n> +\t/* We have a valid pipeline, create the camera sensor. */\n> +\tsensor_ = std::make_unique(sensor);\n> +\tret = sensor_->init();\n> +\tif (ret) {\n> +\t\tsensor_.reset();\n> +\t\treturn;\n> +\t}\n> +}\n> +\n> +int SimpleCameraData::init()\n> +{\n> +\tSimplePipelineHandler *pipe = static_cast(pipe_);\n> +\tV4L2VideoDevice *video = pipe->video();\n> +\tint ret;\n> +\n> +\t/*\n> +\t * Enumerate the possible pipeline configurations. For each media bus\n> +\t * format supported by the sensor, propagate the formats through the\n> +\t * pipeline, and enumerate the corresponding possible V4L2 pixel\n> +\t * formats on the video node.\n> +\t */\n> +\tfor (unsigned int code : sensor_->mbusCodes()) {\n> +\t\tV4L2SubdeviceFormat format{ code, sensor_->resolution() };\n> +\n> +\t\t/*\n> +\t\t * Setup links first as some subdev drivers take active links\n> +\t\t * into account to propaget TRY formats. So is life :-(\n> +\t\t */\n> +\t\tret = setupLinks();\n> +\t\tif (ret < 0)\n> +\t\t\treturn ret;\n> +\n> +\t\tret = setupFormats(&format, V4L2Subdevice::TryFormat);\n> +\t\tif (ret < 0) {\n> +\t\t\tLOG(SimplePipeline, Error)\n> +\t\t\t\t<< \"Failed to setup pipeline for media bus code \"\n> +\t\t\t\t<< utils::hex(code, 4);\n> +\t\t\treturn ret;\n> +\t\t}\n> +\n> +\t\tstd::map> videoFormats =\n> +\t\t\tvideo->formats(format.mbus_code);\n> +\n> +\t\tLOG(SimplePipeline, Debug)\n> +\t\t\t<< \"Adding configuration for \" << format.size.toString()\n> +\t\t\t<< \" in pixel formats [ \"\n> +\t\t\t<< utils::join(videoFormats, \", \",\n> +\t\t\t\t [](const auto &f) {\n> +\t\t\t\t\t return f.first.toString();\n> +\t\t\t\t })\n> +\t\t\t<< \" ]\";\n> +\n> +\t\t/*\n> +\t\t * Store the configuration in the formats_ map, mapping\n> +\t\t * PixelFormat to configuration. Any previously stored value is\n> +\t\t * overwritten, as the pipeline handler currently doesn't care\n> +\t\t * about how a particular PixelFormat is achieved.\n> +\t\t */\n> +\t\tfor (const auto &videoFormat : videoFormats) {\n> +\t\t\tPixelFormat pixelFormat = video->toPixelFormat(videoFormat.first);\n> +\t\t\tif (!pixelFormat)\n> +\t\t\t\tcontinue;\n> +\n> +\t\t\tConfiguration config;\n> +\t\t\tconfig.code = code;\n> +\t\t\tconfig.pixelFormat = pixelFormat;\n> +\t\t\tconfig.size = format.size;\n> +\n> +\t\t\tformats_[pixelFormat] = config;\n> +\t\t}\n> +\t}\n> +\n> +\tif (formats_.empty()) {\n> +\t\tLOG(SimplePipeline, Error) << \"No valid configuration found\";\n> +\t\treturn -EINVAL;\n> +\t}\n> +\n> +\treturn 0;\n> +}\n> +\n> +int SimpleCameraData::setupLinks()\n> +{\n> +\tint ret;\n> +\n> +\t/*\n> +\t * Configure all links along the pipeline. Some entities may not allow\n> +\t * multiple sink links to be enabled together, even on different sink\n> +\t * pads. We must thus start by disabling all sink links (but the one we\n> +\t * want to enable) before enabling the pipeline link.\n> +\t */\n> +\tfor (SimpleCameraData::Entity &e : entities_) {\n> +\t\tfor (MediaPad *pad : e.link->sink()->entity()->pads()) {\n> +\t\t\tfor (MediaLink *link : pad->links()) {\n> +\t\t\t\tif (link == e.link)\n> +\t\t\t\t\tcontinue;\n> +\n> +\t\t\t\tif ((link->flags() & MEDIA_LNK_FL_ENABLED) &&\n> +\t\t\t\t !(link->flags() & MEDIA_LNK_FL_IMMUTABLE)) {\n> +\t\t\t\t\tret = link->setEnabled(false);\n> +\t\t\t\t\tif (ret < 0)\n> +\t\t\t\t\t\treturn ret;\n> +\t\t\t\t}\n> +\t\t\t}\n> +\t\t}\n> +\n> +\t\tif (!(e.link->flags() & MEDIA_LNK_FL_ENABLED)) {\n> +\t\t\tret = e.link->setEnabled(true);\n> +\t\t\tif (ret < 0)\n> +\t\t\t\treturn ret;\n> +\t\t}\n> +\t}\n> +\n> +\treturn 0;\n> +}\n> +\n> +int SimpleCameraData::setupFormats(V4L2SubdeviceFormat *format,\n> +\t\t\t\t V4L2Subdevice::Whence whence)\n> +{\n> +\tSimplePipelineHandler *pipe = static_cast(pipe_);\n> +\tint ret;\n> +\n> +\t/*\n> +\t * Configure the format on the sensor output and propagate it through\n> +\t * the pipeline.\n> +\t */\n> +\tret = sensor_->setFormat(format);\n> +\tif (ret < 0)\n> +\t\treturn ret;\n> +\n> +\tfor (const Entity &e : entities_) {\n> +\t\tMediaLink *link = e.link;\n> +\t\tMediaPad *source = link->source();\n> +\t\tMediaPad *sink = link->sink();\n> +\n> +\t\tif (source->entity() != sensor_->entity()) {\n> +\t\t\tV4L2Subdevice *subdev = pipe->subdev(source->entity());\n> +\t\t\tret = subdev->getFormat(source->index(), format, whence);\n> +\t\t\tif (ret < 0)\n> +\t\t\t\treturn ret;\n> +\t\t}\n> +\n> +\t\tif (sink->entity()->function() != MEDIA_ENT_F_IO_V4L) {\n> +\t\t\tV4L2Subdevice *subdev = pipe->subdev(sink->entity());\n> +\t\t\tret = subdev->setFormat(sink->index(), format, whence);\n> +\t\t\tif (ret < 0)\n> +\t\t\t\treturn ret;\n> +\t\t}\n> +\n> +\t\tLOG(SimplePipeline, Debug)\n> +\t\t\t<< \"Link '\" << source->entity()->name()\n> +\t\t\t<< \"':\" << source->index()\n> +\t\t\t<< \" -> '\" << sink->entity()->name()\n> +\t\t\t<< \"':\" << sink->index()\n> +\t\t\t<< \" configured with format \" << format->toString();\n> +\t}\n> +\n> +\treturn 0;\n> +}\n> +\n> +/* -----------------------------------------------------------------------------\n> + * Camera Configuration\n> + */\n> +\n> +SimpleCameraConfiguration::SimpleCameraConfiguration(Camera *camera,\n> +\t\t\t\t\t\t SimpleCameraData *data)\n> +\t: CameraConfiguration(), camera_(camera->shared_from_this()),\n> +\t data_(data)\n> +{\n> +}\n> +\n> +CameraConfiguration::Status SimpleCameraConfiguration::validate()\n> +{\n> +\tStatus status = Valid;\n> +\n> +\tif (config_.empty())\n> +\t\treturn Invalid;\n> +\n> +\t/* Cap the number of entries to the available streams. */\n> +\tif (config_.size() > 1) {\n> +\t\tconfig_.resize(1);\n> +\t\tstatus = Adjusted;\n> +\t}\n> +\n> +\tStreamConfiguration &cfg = config_[0];\n> +\n> +\t/* Adjust the pixel format. */\n> +\tauto it = data_->formats_.find(cfg.pixelFormat);\n> +\tif (it == data_->formats_.end())\n> +\t\tit = data_->formats_.begin();\n> +\n> +\tPixelFormat pixelFormat = it->first;\n> +\tif (cfg.pixelFormat != pixelFormat) {\n> +\t\tLOG(SimplePipeline, Debug) << \"Adjusting pixel format\";\n> +\t\tcfg.pixelFormat = pixelFormat;\n> +\t\tstatus = Adjusted;\n> +\t}\n> +\n> +\tconst SimpleCameraData::Configuration &pipeConfig = it->second;\n> +\tif (cfg.size != pipeConfig.size) {\n> +\t\tLOG(SimplePipeline, Debug)\n> +\t\t\t<< \"Adjusting size from \" << cfg.size.toString()\n> +\t\t\t<< \" to \" << pipeConfig.size.toString();\n> +\t\tcfg.size = pipeConfig.size;\n> +\t\tstatus = Adjusted;\n> +\t}\n> +\n> +\tcfg.bufferCount = 3;\n> +\n> +\treturn status;\n> +}\n> +\n> +/* -----------------------------------------------------------------------------\n> + * Pipeline Handler\n> + */\n> +\n> +SimplePipelineHandler::SimplePipelineHandler(CameraManager *manager)\n> +\t: PipelineHandler(manager), video_(nullptr)\n> +{\n> +}\n> +\n> +SimplePipelineHandler::~SimplePipelineHandler()\n> +{\n> +\tdelete video_;\n> +}\n> +\n> +CameraConfiguration *SimplePipelineHandler::generateConfiguration(Camera *camera,\n> +\t\t\t\t\t\t\t\t const StreamRoles &roles)\n> +{\n> +\tSimpleCameraData *data = cameraData(camera);\n> +\tCameraConfiguration *config =\n> +\t\tnew SimpleCameraConfiguration(camera, data);\n> +\n> +\tif (roles.empty())\n> +\t\treturn config;\n> +\n> +\t/* Create the formats map. */\n> +\tstd::map> formats;\n> +\tstd::transform(data->formats_.begin(), data->formats_.end(),\n> +\t\t std::inserter(formats, formats.end()),\n> +\t\t [](const auto &format) -> decltype(formats)::value_type {\n> +\t\t\t const PixelFormat &pixelFormat = format.first;\n> +\t\t\t const Size &size = format.second.size;\n> +\t\t\t return { pixelFormat, { size } };\n> +\t\t });\n> +\n> +\t/*\n> +\t * Create the stream configuration. Take the first entry in the formats\n> +\t * map as the default, for lack of a better option.\n> +\t *\n> +\t * \\todo Implement a better way to pick the default format\n> +\t */\n> +\tStreamConfiguration cfg{ StreamFormats{ formats } };\n> +\tcfg.pixelFormat = formats.begin()->first;\n> +\tcfg.size = formats.begin()->second[0].max;\n> +\n> +\tconfig->addConfiguration(cfg);\n> +\n> +\tconfig->validate();\n> +\n> +\treturn config;\n> +}\n> +\n> +int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c)\n> +{\n> +\tSimpleCameraConfiguration *config =\n> +\t\tstatic_cast(c);\n> +\tSimpleCameraData *data = cameraData(camera);\n> +\tStreamConfiguration &cfg = config->at(0);\n> +\tint ret;\n> +\n> +\t/*\n> +\t * Configure links on the pipeline and propagate formats from the\n> +\t * sensor to the video node.\n> +\t */\n> +\tret = data->setupLinks();\n> +\tif (ret < 0)\n> +\t\treturn ret;\n> +\n> +\tconst SimpleCameraData::Configuration &pipeConfig =\n> +\t\tdata->formats_[cfg.pixelFormat];\n> +\n> +\tV4L2SubdeviceFormat format{ pipeConfig.code, data->sensor_->resolution() };\n> +\n> +\tret = data->setupFormats(&format, V4L2Subdevice::ActiveFormat);\n> +\tif (ret < 0)\n> +\t\treturn ret;\n> +\n> +\t/* Configure the video node. */\n> +\tV4L2PixelFormat videoFormat = video_->toV4L2PixelFormat(cfg.pixelFormat);\n> +\n> +\tV4L2DeviceFormat outputFormat = {};\n> +\toutputFormat.fourcc = videoFormat;\n> +\toutputFormat.size = cfg.size;\n> +\n> +\tret = video_->setFormat(&outputFormat);\n> +\tif (ret)\n> +\t\treturn ret;\n> +\n> +\tif (outputFormat.size != cfg.size || outputFormat.fourcc != videoFormat) {\n> +\t\tLOG(SimplePipeline, Error)\n> +\t\t\t<< \"Unable to configure capture in \" << cfg.toString();\n> +\t\treturn -EINVAL;\n> +\t}\n> +\n> +\tcfg.setStream(&data->stream_);\n> +\n> +\treturn 0;\n> +}\n> +\n> +int SimplePipelineHandler::exportFrameBuffers(Camera *camera, Stream *stream,\n> +\t\t\t\t\t std::vector> *buffers)\n> +{\n> +\tunsigned int count = stream->configuration().bufferCount;\n> +\n> +\treturn video_->exportBuffers(count, buffers);\n> +}\n> +\n> +int SimplePipelineHandler::start(Camera *camera)\n> +{\n> +\tSimpleCameraData *data = cameraData(camera);\n> +\tunsigned int count = data->stream_.configuration().bufferCount;\n> +\n> +\tint ret = video_->importBuffers(count);\n> +\tif (ret < 0)\n> +\t\treturn ret;\n> +\n> +\tret = video_->streamOn();\n> +\tif (ret < 0) {\n> +\t\tvideo_->releaseBuffers();\n> +\t\treturn ret;\n> +\t}\n> +\n> +\tactiveCamera_ = camera;\n> +\n> +\treturn 0;\n> +}\n> +\n> +void SimplePipelineHandler::stop(Camera *camera)\n> +{\n> +\tvideo_->streamOff();\n> +\tvideo_->releaseBuffers();\n> +\tactiveCamera_ = nullptr;\n> +}\n> +\n> +int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request)\n> +{\n> +\tSimpleCameraData *data = cameraData(camera);\n> +\tStream *stream = &data->stream_;\n> +\n> +\tFrameBuffer *buffer = request->findBuffer(stream);\n> +\tif (!buffer) {\n> +\t\tLOG(SimplePipeline, Error)\n> +\t\t\t<< \"Attempt to queue request with invalid stream\";\n> +\t\treturn -ENOENT;\n> +\t}\n> +\n> +\treturn video_->queueBuffer(buffer);\n> +}\n> +\n> +/* -----------------------------------------------------------------------------\n> + * Match and Setup\n> + */\n> +\n> +bool SimplePipelineHandler::match(DeviceEnumerator *enumerator)\n> +{\n> +\tstatic const char * const drivers[] = {\n> +\t\t\"imx7-csi\",\n> +\t\t\"sun6i-csi\",\n> +\t};\n> +\n> +\tfor (const char *driver : drivers) {\n> +\t\tDeviceMatch dm(driver);\n> +\t\tmedia_ = acquireMediaDevice(enumerator, dm);\n> +\t\tif (media_)\n> +\t\t\tbreak;\n> +\t}\n> +\n> +\tif (!media_)\n> +\t\treturn false;\n> +\n> +\t/*\n> +\t * Locate sensors and video nodes. We only support pipelines with at\n> +\t * least one sensor and exactly one video captude node.\n> +\t */\n> +\tstd::vector sensors;\n> +\tstd::vector videos;\n> +\n> +\tfor (MediaEntity *entity : media_->entities()) {\n> +\t\tswitch (entity->function()) {\n> +\t\tcase MEDIA_ENT_F_CAM_SENSOR:\n> +\t\t\tsensors.push_back(entity);\n> +\t\t\tbreak;\n> +\n> +\t\tcase MEDIA_ENT_F_IO_V4L:\n> +\t\t\tif (entity->pads().size() == 1 &&\n> +\t\t\t (entity->pads()[0]->flags() & MEDIA_PAD_FL_SINK))\n> +\t\t\t\tvideos.push_back(entity);\n> +\t\t\tbreak;\n> +\n> +\t\tdefault:\n> +\t\t\tbreak;\n> +\t\t}\n> +\t}\n> +\n> +\tif (sensors.empty()) {\n> +\t\tLOG(SimplePipeline, Error) << \"No sensor found\";\n> +\t\treturn false;\n> +\t}\n> +\n> +\tif (videos.size() != 1) {\n> +\t\tLOG(SimplePipeline, Error)\n> +\t\t\t<< \"Pipeline with \" << videos.size()\n> +\t\t\t<< \" video capture nodes is not supported\";\n> +\t\treturn false;\n\nCould these Error be changed to Warning, and the \"return false;\" statement be dropped?\n\nI agree that there is no code yet to handle multiple video capture nodes, but this change\nallows me to use this simple pipeline handler with db410c board and one sensor connected\nto the first CSI interface (msm_csiphy0). So it can work in some simple configurations\nwith multiple video capture nodes already.\n\n> +\t}\n> +\n> +\t/* Locate and open the capture video node. */\n> +\tvideo_ = new V4L2VideoDevice(videos[0]);\n> +\tif (video_->open() < 0)\n> +\t\treturn false;\n> +\n> +\tif (video_->caps().isMultiplanar()) {\n> +\t\tLOG(SimplePipeline, Error)\n> +\t\t\t<< \"V4L2 multiplanar devices are not supported\";\n> +\t\treturn false;\n\nSame here. For the same reasons.\n\nThanks,\nAndrey\n\n> +\t}\n> +\n> +\tvideo_->bufferReady.connect(this, &SimplePipelineHandler::bufferReady);\n> +\n> +\t/*\n> +\t * Create one camera data instance for each sensor and gather all\n> +\t * entities in all pipelines.\n> +\t */\n> +\tstd::vector> pipelines;\n> +\tstd::set entities;\n> +\n> +\tpipelines.reserve(sensors.size());\n> +\n> +\tfor (MediaEntity *sensor : sensors) {\n> +\t\tstd::unique_ptr data =\n> +\t\t\tstd::make_unique(this, sensor,\n> +\t\t\t\t\t\t\t videos[0]);\n> +\t\tif (!data->isValid()) {\n> +\t\t\tLOG(SimplePipeline, Error)\n> +\t\t\t\t<< \"No valid pipeline for sensor '\"\n> +\t\t\t\t<< sensor->name() << \"', skipping\";\n> +\t\t\tcontinue;\n> +\t\t}\n> +\n> +\t\tfor (SimpleCameraData::Entity &entity : data->entities_)\n> +\t\t\tentities.insert(entity.entity);\n> +\n> +\t\tpipelines.push_back(std::move(data));\n> +\t}\n> +\n> +\tif (entities.empty())\n> +\t\treturn false;\n> +\n> +\t/* Create and open V4L2Subdev instances for all the entities. */\n> +\tfor (MediaEntity *entity : entities) {\n> +\t\tauto elem = subdevs_.emplace(std::piecewise_construct,\n> +\t\t\t\t\t std::forward_as_tuple(entity),\n> +\t\t\t\t\t std::forward_as_tuple(entity));\n> +\t\tV4L2Subdevice *subdev = &elem.first->second;\n> +\t\tint ret = subdev->open();\n> +\t\tif (ret < 0) {\n> +\t\t\tLOG(SimplePipeline, Error)\n> +\t\t\t\t<< \"Failed to open \" << subdev->deviceNode()\n> +\t\t\t\t<< \": \" << strerror(-ret);\n> +\t\t\treturn false;\n> +\t\t}\n> +\t}\n> +\n> +\t/* Initialize each pipeline and register a corresponding camera. */\n> +\tfor (std::unique_ptr &data : pipelines) {\n> +\t\tint ret = data->init();\n> +\t\tif (ret < 0)\n> +\t\t\tcontinue;\n> +\n> +\t\tstd::shared_ptr camera =\n> +\t\t\tCamera::create(this, data->sensor_->entity()->name(),\n> +\t\t\t\t data->streams());\n> +\t\tregisterCamera(std::move(camera), std::move(data));\n> +\t}\n> +\n> +\treturn true;\n> +}\n> +\n> +V4L2Subdevice *SimplePipelineHandler::subdev(const MediaEntity *entity)\n> +{\n> +\tauto iter = subdevs_.find(entity);\n> +\tif (iter == subdevs_.end())\n> +\t\treturn nullptr;\n> +\n> +\treturn &iter->second;\n> +}\n> +\n> +/* -----------------------------------------------------------------------------\n> + * Buffer Handling\n> + */\n> +\n> +void SimplePipelineHandler::bufferReady(FrameBuffer *buffer)\n> +{\n> +\tASSERT(activeCamera_);\n> +\tRequest *request = buffer->request();\n> +\tcompleteBuffer(activeCamera_, request, buffer);\n> +\tcompleteRequest(activeCamera_, request);\n> +}\n> +\n> +REGISTER_PIPELINE_HANDLER(SimplePipelineHandler);\n> +\n> +} /* namespace libcamera */\n>","headers":{"Return-Path":"","Received":["from mail-lj1-x241.google.com (mail-lj1-x241.google.com\n\t[IPv6:2a00:1450:4864:20::241])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id E07B8600FA\n\tfor ;\n\tFri, 3 Apr 2020 15:55:35 +0200 (CEST)","by mail-lj1-x241.google.com with SMTP id p14so7002234lji.11\n\tfor ;\n\tFri, 03 Apr 2020 06:55:35 -0700 (PDT)","from [192.168.118.216] (37-144-159-139.broadband.corbina.ru.\n\t[37.144.159.139]) by smtp.gmail.com with ESMTPSA id\n\td19sm4904568lji.95.2020.04.03.06.55.33\n\t(version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);\n\tFri, 03 Apr 2020 06:55:33 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key; \n\tunprotected) header.d=linaro.org\n\theader.i=@linaro.org header.b=\"cxplNwPG\"; \n\tdkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=subject:to:cc:references:from:message-id:date:user-agent\n\t:mime-version:in-reply-to:content-language:content-transfer-encoding; \n\tbh=w5q1o3GP+B+Byma0Qoeh3aSKiRc9SFUJ7n47zMOd2pM=;\n\tb=cxplNwPG8y/1YoRv30mVWo2bttF5RqIka8JGXAbqgwT4mIvn096aM4RJ2nMVZGyhW+\n\tPN/WT4awWhztSRg+JQMNwD513Ivu1tshcaHZ6dQrcrbKLFzCAhrcCYzavoKrzglHdnyJ\n\t3rn9OOFmzOypMmj6rfRzeJNyBZue7ux4JNlMeICh3YCZ9zyc1321+4A2dbhvrsUMzbVK\n\tOMlsEJiEINRy6fhZs2oKD5LA7fQfV7qkhGkEZFflcLpknsyGnm9rsNEuSuFZI0CyKH8n\n\teN6A80Ai0LHFeZeJdAW1dqCdY5u6qynRWFFz1fbOpgW+S/xHZ61oseNZ9115jMC9GIap\n\t7nvg==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:subject:to:cc:references:from:message-id:date\n\t:user-agent:mime-version:in-reply-to:content-language\n\t:content-transfer-encoding;\n\tbh=w5q1o3GP+B+Byma0Qoeh3aSKiRc9SFUJ7n47zMOd2pM=;\n\tb=XRGh/r+mGpGg9TMne8cbaX7UymmXa5I79ttmdgeIEDujG5e/hig99el2+pJRHpi1Yz\n\taPXooD/5NaIdIljFtLzT8yptvsvYwa0bqYoPk3CPtSAGuygkzrJ2qkr9IoOygLMcsA78\n\tDO3giflLZJPIkFvdDY6x3HaaNAT2NxQHvQSpBFi/OpbH8C/fBsh5KxgQiXxEevw740Rs\n\tdLbgETCY2g/rU4YKeGl+Hf0nFPODq1Vn9WzKQyHrAHdLqkARcBFUntLBFHH6oQXOIg5B\n\tfxczeoy0eAkEQ+5sq5/YpCilQG+xPAAZ+6N67BD2jNWUL+diJyTwVdp7sBn+c7uiGkuw\n\tOLsg==","X-Gm-Message-State":"AGi0PuZDuK0Qlj8q648XWphnHT9F3xbvfCtvotiu+p8+fJqyh460F9bk\n\tAIs8suncgrkX0hGq4W2upaS+RQ==","X-Google-Smtp-Source":"APiQypLlRd6FSAnTLiNJ4ryJVAvWorgYdNjaDJlXpcAFUaWUUhm4+EUogg1nUOehwH2MaIHAwHSZ7A==","X-Received":"by 2002:a2e:9d85:: with SMTP id c5mr4880025ljj.168.1585922134915;\n\tFri, 03 Apr 2020 06:55:34 -0700 (PDT)","To":"Laurent Pinchart ,\n\tlibcamera-devel@lists.libcamera.org","Cc":"Martijn Braam ,\n\tBenjamin GAIGNARD ","References":"<20200320014839.14683-1-laurent.pinchart@ideasonboard.com>\n\t<20200320014839.14683-10-laurent.pinchart@ideasonboard.com>","From":"Andrey Konovalov ","Message-ID":"<053e8bb2-1d50-1aa5-b09a-fdb6718e3a41@linaro.org>","Date":"Fri, 3 Apr 2020 16:55:32 +0300","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101\n\tThunderbird/68.4.1","MIME-Version":"1.0","In-Reply-To":"<20200320014839.14683-10-laurent.pinchart@ideasonboard.com>","Content-Type":"text/plain; charset=utf-8; format=flowed","Content-Language":"en-US","Content-Transfer-Encoding":"7bit","Subject":"Re: [libcamera-devel] [PATCH v3 09/11] libcamera: pipeline: Add a\n\tsimple pipeline handler","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","X-List-Received-Date":"Fri, 03 Apr 2020 13:55:36 -0000"}},{"id":4378,"web_url":"https://patchwork.libcamera.org/comment/4378/","msgid":"<20200403235536.GD9690@pendragon.ideasonboard.com>","date":"2020-04-03T23:55:36","subject":"Re: [libcamera-devel] [PATCH v3 09/11] libcamera: pipeline: Add a\n\tsimple pipeline handler","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Kieran,\n\nOn Tue, Mar 31, 2020 at 12:45:14PM +0100, Kieran Bingham wrote:\n> On 20/03/2020 01:48, Laurent Pinchart wrote:\n> > From: Martijn Braam \n> > \n> > This new pipeline handler aims at supporting any simple device without\n> > requiring any device-specific code. Simple devices are currently defined\n> > as a graph made of one or multiple camera sensors and a single video\n> > node, with each sensor connected to the video node through a linear\n> > pipeline.\n> > \n> > The simple pipeline handler will automatically parse the media graph,\n> > enumerate sensors, build supported stream configurations, and configure\n> > the pipeline, without any device-specific knowledge. It doesn't support\n> > configuration of any processing in the pipeline at the moment, but may\n> > be extended to support simple processing such as format conversion or\n> > scaling in the future.\n> > \n> > The only device-specific information in the pipeline handler is the list\n> > of supported drivers, required for device matching. We may be able to\n> > remove this in the future by matching with the simple pipeline handler\n> > as a last resort option, after all other pipeline handlers have been\n> > tried.\n> > \n> > Signed-off-by: Martijn Braam \n> > Signed-off-by: Laurent Pinchart \n> \n> Reviewed-by: Kieran Bingham \n> \n> Some comments and discussion points below, but I think we should work\n> towards trying to get this integrated so it can actually be used (which\n> is where we'll find the nuances and corner cases as people test it)\n\n:-)\n\nPlease see below for a few answers.\n\n> > ---\n> > Changes since v2:\n> > \n> > - Log an error when setupFormats() fail\n> > - Propagate getFormat() and setFormat() errors to the caller of\n> > setupFormats()\n> > - Reorder variable declarations in validate()\n> > - Add \\todo comment related to the selection of the default format\n> > - Use log Error instead of Info if pipeline isn't valid\n> > - Rebase on top of V4L2PixelFormat\n> > \n> > Changes since v1:\n> > \n> > - Rebase on top of buffer API rework\n> > - Expose stream formats\n> > - Rework camera data config\n> > ---\n> > src/libcamera/pipeline/meson.build | 1 +\n> > src/libcamera/pipeline/simple/meson.build | 3 +\n> > src/libcamera/pipeline/simple/simple.cpp | 710 ++++++++++++++++++++++\n> > 3 files changed, 714 insertions(+)\n> > create mode 100644 src/libcamera/pipeline/simple/meson.build\n> > create mode 100644 src/libcamera/pipeline/simple/simple.cpp\n> > \n> > diff --git a/src/libcamera/pipeline/meson.build b/src/libcamera/pipeline/meson.build\n> > index 0d466225a72e..606ba31a0319 100644\n> > --- a/src/libcamera/pipeline/meson.build\n> > +++ b/src/libcamera/pipeline/meson.build\n> > @@ -5,3 +5,4 @@ libcamera_sources += files([\n> > \n> > subdir('ipu3')\n> > subdir('rkisp1')\n> > +subdir('simple')\n> > diff --git a/src/libcamera/pipeline/simple/meson.build b/src/libcamera/pipeline/simple/meson.build\n> > new file mode 100644\n> > index 000000000000..4945a3e173cf\n> > --- /dev/null\n> > +++ b/src/libcamera/pipeline/simple/meson.build\n> > @@ -0,0 +1,3 @@\n> > +libcamera_sources += files([\n> > + 'simple.cpp',\n> > +])\n> > diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp\n> > new file mode 100644\n> > index 000000000000..545a99fe31ca\n> > --- /dev/null\n> > +++ b/src/libcamera/pipeline/simple/simple.cpp\n> > @@ -0,0 +1,710 @@\n> > +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n> > +/*\n> > + * Copyright (C) 2020, Laurent Pinchart\n> > + * Copyright (C) 2019, Martijn Braam\n> > + *\n> > + * simple.cpp - Pipeline handler for simple pipelines\n> > + */\n> > +\n> > +#include \n> > +#include \n> > +#include \n> > +#include \n> > +#include \n> > +#include \n> > +#include \n> > +#include \n> > +#include \n> > +#include \n> > +\n> > +#include \n> > +\n> > +#include \n> > +#include \n> > +#include \n> > +\n> > +#include \"camera_sensor.h\"\n> > +#include \"device_enumerator.h\"\n> > +#include \"log.h\"\n> > +#include \"media_device.h\"\n> > +#include \"pipeline_handler.h\"\n> > +#include \"v4l2_subdevice.h\"\n> > +#include \"v4l2_videodevice.h\"\n> > +\n> > +namespace libcamera {\n> > +\n> > +LOG_DEFINE_CATEGORY(SimplePipeline)\n> > +\n> > +class SimplePipelineHandler;\n> > +\n> > +class SimpleCameraData : public CameraData\n> > +{\n> > +public:\n> > +\tSimpleCameraData(PipelineHandler *pipe, MediaEntity *sensor,\n> > +\t\t\t MediaEntity *video);\n> > +\n> > +\tbool isValid() const { return sensor_ != nullptr; }\n> > +\tstd::set streams() { return { &stream_ }; }\n> > +\n> > +\tint init();\n> > +\tint setupLinks();\n> > +\tint setupFormats(V4L2SubdeviceFormat *format,\n> > +\t\t\t V4L2Subdevice::Whence whence);\n> > +\n> > +\tstruct Entity {\n> > +\t\tMediaEntity *entity;\n> > +\t\tMediaLink *link;\n> > +\t};\n> > +\n> > +\tstruct Configuration {\n> > +\t\tuint32_t code;\n> > +\t\tPixelFormat pixelFormat;\n> > +\t\tSize size;\n> > +\t};\n> > +\n> > +\tStream stream_;\n> > +\tstd::unique_ptr sensor_;\n> > +\tstd::list entities_;\n> > +\n> > +\tstd::vector configs_;\n> > +\tstd::map formats_;\n> > +};\n> > +\n> > +class SimpleCameraConfiguration : public CameraConfiguration\n> > +{\n> > +public:\n> > +\tSimpleCameraConfiguration(Camera *camera, SimpleCameraData *data);\n> > +\n> > +\tStatus validate() override;\n> > +\n> > +\tconst V4L2SubdeviceFormat &sensorFormat() { return sensorFormat_; }\n> > +\n> > +private:\n> > +\t/*\n> > +\t * The SimpleCameraData instance is guaranteed to be valid as long as\n> > +\t * the corresponding Camera instance is valid. In order to borrow a\n> > +\t * reference to the camera data, store a new reference to the camera.\n> > +\t */\n> > +\tstd::shared_ptr camera_;\n> > +\tconst SimpleCameraData *data_;\n> > +\n> > +\tV4L2SubdeviceFormat sensorFormat_;\n> > +};\n> > +\n> > +class SimplePipelineHandler : public PipelineHandler\n> > +{\n> > +public:\n> > +\tSimplePipelineHandler(CameraManager *manager);\n> > +\t~SimplePipelineHandler();\n> > +\n> > +\tCameraConfiguration *generateConfiguration(Camera *camera,\n> > +\t\t\t\t\t\t const StreamRoles &roles) override;\n> > +\tint configure(Camera *camera, CameraConfiguration *config) override;\n> > +\n> > +\tint exportFrameBuffers(Camera *camera, Stream *stream,\n> > +\t\t\t std::vector> *buffers) override;\n> > +\n> > +\tint start(Camera *camera) override;\n> > +\tvoid stop(Camera *camera) override;\n> > +\n> > +\tbool match(DeviceEnumerator *enumerator) override;\n> > +\n> > +\tV4L2VideoDevice *video() { return video_; }\n> > +\tV4L2Subdevice *subdev(const MediaEntity *entity);\n> > +\n> > +protected:\n> > +\tint queueRequestDevice(Camera *camera, Request *request) override;\n> > +\n> > +private:\n> > +\tSimpleCameraData *cameraData(const Camera *camera)\n> > +\t{\n> > +\t\treturn static_cast(\n> > +\t\t\tPipelineHandler::cameraData(camera));\n> > +\t}\n> > +\n> > +\tint initLinks();\n> > +\n> > +\tint createCamera(MediaEntity *sensor);\n> > +\n> > +\tvoid bufferReady(FrameBuffer *buffer);\n> > +\n> > +\tMediaDevice *media_;\n> > +\tV4L2VideoDevice *video_;\n> > +\tstd::map subdevs_;\n> > +\n> > +\tCamera *activeCamera_;\n> > +};\n> > +\n> > +/* -----------------------------------------------------------------------------\n> > + * Camera Data\n> > + */\n> > +SimpleCameraData::SimpleCameraData(PipelineHandler *pipe, MediaEntity *sensor,\n> > +\t\t\t\t MediaEntity *video)\n> > +\t: CameraData(pipe)\n> > +{\n> > +\tint ret;\n> > +\n> > +\t/*\n> > +\t * Walk the pipeline towards the video node and store all entities\n> > +\t * along the way.\n> > +\t */\n> > +\tMediaEntity *source = sensor;\n> > +\n> > +\twhile (source) {\n> > +\t\t/* If we have reached the video node, we're done. */\n> > +\t\tif (source == video)\n> > +\t\t\tbreak;\n> > +\n> > +\t\t/* Use the first output pad that has links. */\n> > +\t\tMediaPad *sourcePad = nullptr;\n> > +\t\tfor (MediaPad *pad : source->pads()) {\n> > +\t\t\tif ((pad->flags() & MEDIA_PAD_FL_SOURCE) &&\n> > +\t\t\t !pad->links().empty()) {\n> > +\t\t\t\tsourcePad = pad;\n> > +\t\t\t\tbreak;\n> > +\t\t\t}\n> > +\t\t}\n> > +\n> > +\t\tif (!sourcePad)\n> > +\t\t\treturn;\n> > +\n> > +\t\t/* Use the first link that isn't immutable and disabled. */\n> > +\t\tMediaLink *sourceLink = nullptr;\n> > +\t\tfor (MediaLink *link : sourcePad->links()) {\n> > +\t\t\tif ((link->flags() & MEDIA_LNK_FL_ENABLED) ||\n> > +\t\t\t !(link->flags() & MEDIA_LNK_FL_IMMUTABLE)) {\n> > +\t\t\t\tsourceLink = link;\n> > +\t\t\t\tbreak;\n> > +\t\t\t}\n> > +\t\t}\n> > +\n> > +\t\tif (!sourceLink)\n> > +\t\t\treturn;\n> > +\n> > +\t\tentities_.push_back({ source, sourceLink });\n> > +\n> > +\t\tsource = sourceLink->sink()->entity();\n> > +\n> > +\t\t/* Avoid infinite loops. */\n> > +\t\tauto iter = std::find_if(entities_.begin(), entities_.end(),\n> > +\t\t\t\t\t [&](const Entity &entity) {\n> > +\t\t\t\t\t\t return entity.entity == source;\n> > +\t\t\t\t\t });\n> > +\t\tif (iter != entities_.end()) {\n> > +\t\t\tLOG(SimplePipeline, Info) << \"Loop detected in pipeline\";\n> > +\t\t\treturn;\n> > +\t\t}\n> > +\t}\n> > +\n> > +\t/* We have a valid pipeline, create the camera sensor. */\n> > +\tsensor_ = std::make_unique(sensor);\n> > +\tret = sensor_->init();\n> > +\tif (ret) {\n> > +\t\tsensor_.reset();\n> > +\t\treturn;\n> > +\t}\n> > +}\n> > +\n> > +int SimpleCameraData::init()\n> > +{\n> > +\tSimplePipelineHandler *pipe = static_cast(pipe_);\n> > +\tV4L2VideoDevice *video = pipe->video();\n> > +\tint ret;\n> > +\n> > +\t/*\n> > +\t * Enumerate the possible pipeline configurations. For each media bus\n> > +\t * format supported by the sensor, propagate the formats through the\n> > +\t * pipeline, and enumerate the corresponding possible V4L2 pixel\n> > +\t * formats on the video node.\n> > +\t */\n> > +\tfor (unsigned int code : sensor_->mbusCodes()) {\n> > +\t\tV4L2SubdeviceFormat format{ code, sensor_->resolution() };\n> > +\n> > +\t\t/*\n> > +\t\t * Setup links first as some subdev drivers take active links\n> > +\t\t * into account to propaget TRY formats. So is life :-(\n> \n> /propaget/propagate/\n> \n> /So is life/Such is life/\n> \n> > +\t\t */\n> > +\t\tret = setupLinks();\n> > +\t\tif (ret < 0)\n> > +\t\t\treturn ret;\n> > +\n> > +\t\tret = setupFormats(&format, V4L2Subdevice::TryFormat);\n> > +\t\tif (ret < 0) {\n> > +\t\t\tLOG(SimplePipeline, Error)\n> > +\t\t\t\t<< \"Failed to setup pipeline for media bus code \"\n> > +\t\t\t\t<< utils::hex(code, 4);\n> \n> Oh dear, seems we need mbusCode.toString() too! hehe\n> \n> > +\t\t\treturn ret;\n> \n> Andrey states that this return prevents iterating potentially successful\n> mbus-codes, and should instead 'continue'...\n\nI'll reply to that in a separate e-mail/\n\n> > +\t\t}\n> > +\n> > +\t\tstd::map> videoFormats =\n> > +\t\t\tvideo->formats(format.mbus_code);\n> > +\n> > +\t\tLOG(SimplePipeline, Debug)\n> > +\t\t\t<< \"Adding configuration for \" << format.size.toString()\n> > +\t\t\t<< \" in pixel formats [ \"\n> > +\t\t\t<< utils::join(videoFormats, \", \",\n> > +\t\t\t\t [](const auto &f) {\n> > +\t\t\t\t\t return f.first.toString();\n> > +\t\t\t\t })\n> \n> Aha, I kinda like the utils::join!\n> \n> > +\t\t\t<< \" ]\";\n> > +\n> > +\t\t/*\n> > +\t\t * Store the configuration in the formats_ map, mapping\n> > +\t\t * PixelFormat to configuration. Any previously stored value is\n> \n> /to configuration/to the current configuration/ ?\n\nI'll write \"mapping the PixelFormat to the corresponding configuration\".\n\n> > +\t\t * overwritten, as the pipeline handler currently doesn't care\n> > +\t\t * about how a particular PixelFormat is achieved.\n> \n> As long as there's an output, I agree. If devices have specific\n> constraints such as better performance or such with a particular\n> mbus-code, then they need to create a platform specific pipeline handler\n> to deal with the nuances.\n> \n> Maybe there might be tuning hints or something later, but certainly not now.\n\nExactly my thoughts :-)\n\n> > +\t\t */\n> > +\t\tfor (const auto &videoFormat : videoFormats) {\n> > +\t\t\tPixelFormat pixelFormat = video->toPixelFormat(videoFormat.first);\n> > +\t\t\tif (!pixelFormat)\n> > +\t\t\t\tcontinue;\n> > +\n> > +\t\t\tConfiguration config;\n> > +\t\t\tconfig.code = code;\n> > +\t\t\tconfig.pixelFormat = pixelFormat;\n> > +\t\t\tconfig.size = format.size;\n> > +\n> > +\t\t\tformats_[pixelFormat] = config;\n> > +\t\t}\n> > +\t}\n> > +\n> > +\tif (formats_.empty()) {\n> > +\t\tLOG(SimplePipeline, Error) << \"No valid configuration found\";\n> > +\t\treturn -EINVAL;\n> > +\t}\n> > +\n> > +\treturn 0;\n> > +}\n> > +\n> > +int SimpleCameraData::setupLinks()\n> > +{\n> > +\tint ret;\n> > +\n> > +\t/*\n> > +\t * Configure all links along the pipeline. Some entities may not allow\n> > +\t * multiple sink links to be enabled together, even on different sink\n> > +\t * pads. We must thus start by disabling all sink links (but the one we\n> > +\t * want to enable) before enabling the pipeline link.\n> \n> Eeek - this sounds like it could have potential system effects,\n> (disabling existing configurations) but I think this is the\n> 'Such-is-life' comment I read earlier.\n\nPossibly, but the pipeline handler owns the whole media device, so it\nshouldn't be a real issue.\n\n> > +\t */\n> > +\tfor (SimpleCameraData::Entity &e : entities_) {\n> > +\t\tfor (MediaPad *pad : e.link->sink()->entity()->pads()) {\n> \n> For readability, I would be tempted to pull e.link->sink()->entity() to\n> a local var called Entity &remote, or Entity &endpoint, or such,\n> essentially to make this line read as:\n> \n> for (MediaPad *pad : remote->pads()) {\n\nDone with MediaEntity *remote = ...;\n\n> > +\t\t\tfor (MediaLink *link : pad->links()) {\n> > +\t\t\t\tif (link == e.link)\n> > +\t\t\t\t\tcontinue;\n> > +\n> > +\t\t\t\tif ((link->flags() & MEDIA_LNK_FL_ENABLED) &&\n> > +\t\t\t\t !(link->flags() & MEDIA_LNK_FL_IMMUTABLE)) {\n> > +\t\t\t\t\tret = link->setEnabled(false);\n> > +\t\t\t\t\tif (ret < 0)\n> > +\t\t\t\t\t\treturn ret;\n> > +\t\t\t\t}\n> > +\t\t\t}\n> > +\t\t}\n> > +\n> \n> I'm not going to try it out, but would this be easier / clearer if we\n> simply disabled *all* links, then explicitly enable links on our pipeline?\n> \n> (only a devils-advocate discussion point, not something that's needed).\n\nI considered it after writing this code, and I thought the more complex\napproach could be useful in the future, so I wasn't enclined to throw it\naway :-)\n\n> > +\t\tif (!(e.link->flags() & MEDIA_LNK_FL_ENABLED)) {\n> > +\t\t\tret = e.link->setEnabled(true);\n> > +\t\t\tif (ret < 0)\n> > +\t\t\t\treturn ret;\n> > +\t\t}\n> > +\t}\n> > +\n> > +\treturn 0;\n> > +}\n> > +\n> > +int SimpleCameraData::setupFormats(V4L2SubdeviceFormat *format,\n> > +\t\t\t\t V4L2Subdevice::Whence whence)\n> > +{\n> > +\tSimplePipelineHandler *pipe = static_cast(pipe_);\n> > +\tint ret;\n> > +\n> > +\t/*\n> > +\t * Configure the format on the sensor output and propagate it through\n> > +\t * the pipeline.\n> > +\t */\n> > +\tret = sensor_->setFormat(format);\n> > +\tif (ret < 0)\n> > +\t\treturn ret;\n> > +\n> > +\tfor (const Entity &e : entities_) {\n> > +\t\tMediaLink *link = e.link;\n> > +\t\tMediaPad *source = link->source();\n> > +\t\tMediaPad *sink = link->sink();\n> > +\n> > +\t\tif (source->entity() != sensor_->entity()) {\n> > +\t\t\tV4L2Subdevice *subdev = pipe->subdev(source->entity());\n> > +\t\t\tret = subdev->getFormat(source->index(), format, whence);\n> > +\t\t\tif (ret < 0)\n> > +\t\t\t\treturn ret;\n> > +\t\t}\n> > +\n> > +\t\tif (sink->entity()->function() != MEDIA_ENT_F_IO_V4L) {\n> > +\t\t\tV4L2Subdevice *subdev = pipe->subdev(sink->entity());\n> > +\t\t\tret = subdev->setFormat(sink->index(), format, whence);\n> > +\t\t\tif (ret < 0)\n> > +\t\t\t\treturn ret;\n> > +\t\t}\n> > +\n> > +\t\tLOG(SimplePipeline, Debug)\n> > +\t\t\t<< \"Link '\" << source->entity()->name()\n> > +\t\t\t<< \"':\" << source->index()\n> > +\t\t\t<< \" -> '\" << sink->entity()->name()\n> > +\t\t\t<< \"':\" << sink->index()\n> > +\t\t\t<< \" configured with format \" << format->toString();\n> \n> I wonder if we can get the whole pipeline configuration string to a\n> single line... but not required now.\n> \n> > +\t}\n> > +\n> > +\treturn 0;\n> > +}\n> > +\n> > +/* -----------------------------------------------------------------------------\n> > + * Camera Configuration\n> > + */\n> > +\n> > +SimpleCameraConfiguration::SimpleCameraConfiguration(Camera *camera,\n> > +\t\t\t\t\t\t SimpleCameraData *data)\n> > +\t: CameraConfiguration(), camera_(camera->shared_from_this()),\n> > +\t data_(data)\n> > +{\n> > +}\n> > +\n> > +CameraConfiguration::Status SimpleCameraConfiguration::validate()\n> > +{\n> > +\tStatus status = Valid;\n> > +\n> > +\tif (config_.empty())\n> > +\t\treturn Invalid;\n> > +\n> > +\t/* Cap the number of entries to the available streams. */\n> > +\tif (config_.size() > 1) {\n> > +\t\tconfig_.resize(1);\n> > +\t\tstatus = Adjusted;\n> > +\t}\n> > +\n> > +\tStreamConfiguration &cfg = config_[0];\n> > +\n> > +\t/* Adjust the pixel format. */\n> > +\tauto it = data_->formats_.find(cfg.pixelFormat);\n> > +\tif (it == data_->formats_.end())\n> > +\t\tit = data_->formats_.begin();\n> > +\n> > +\tPixelFormat pixelFormat = it->first;\n> > +\tif (cfg.pixelFormat != pixelFormat) {\n> > +\t\tLOG(SimplePipeline, Debug) << \"Adjusting pixel format\";\n> > +\t\tcfg.pixelFormat = pixelFormat;\n> > +\t\tstatus = Adjusted;\n> > +\t}\n> > +\n> > +\tconst SimpleCameraData::Configuration &pipeConfig = it->second;\n> > +\tif (cfg.size != pipeConfig.size) {\n> > +\t\tLOG(SimplePipeline, Debug)\n> > +\t\t\t<< \"Adjusting size from \" << cfg.size.toString()\n> > +\t\t\t<< \" to \" << pipeConfig.size.toString();\n> > +\t\tcfg.size = pipeConfig.size;\n> > +\t\tstatus = Adjusted;\n> > +\t}\n> > +\n> > +\tcfg.bufferCount = 3;\n> > +\n> > +\treturn status;\n> > +}\n> > +\n> > +/* -----------------------------------------------------------------------------\n> > + * Pipeline Handler\n> > + */\n> > +\n> > +SimplePipelineHandler::SimplePipelineHandler(CameraManager *manager)\n> > +\t: PipelineHandler(manager), video_(nullptr)\n> > +{\n> > +}\n> > +\n> > +SimplePipelineHandler::~SimplePipelineHandler()\n> > +{\n> > +\tdelete video_;\n> > +}\n> > +\n> > +CameraConfiguration *SimplePipelineHandler::generateConfiguration(Camera *camera,\n> > +\t\t\t\t\t\t\t\t const StreamRoles &roles)\n> > +{\n> > +\tSimpleCameraData *data = cameraData(camera);\n> > +\tCameraConfiguration *config =\n> > +\t\tnew SimpleCameraConfiguration(camera, data);\n> > +\n> > +\tif (roles.empty())\n> > +\t\treturn config;\n> > +\n> > +\t/* Create the formats map. */\n> > +\tstd::map> formats;\n> > +\tstd::transform(data->formats_.begin(), data->formats_.end(),\n> > +\t\t std::inserter(formats, formats.end()),\n> > +\t\t [](const auto &format) -> decltype(formats)::value_type {\n> > +\t\t\t const PixelFormat &pixelFormat = format.first;\n> > +\t\t\t const Size &size = format.second.size;\n> > +\t\t\t return { pixelFormat, { size } };\n> > +\t\t });\n> \n> This is quite unfriendly to the reader :-( ... not that I have a better\n> alternative ... but eugh\n> \n> > +\n> > +\t/*\n> > +\t * Create the stream configuration. Take the first entry in the formats\n> > +\t * map as the default, for lack of a better option.\n> > +\t *\n> > +\t * \\todo Implement a better way to pick the default format\n> \n> Is there one? It's a generic configuration for a pipeline handler.\n> \n> What might be better as a general case, and presumably any logic added\n> here would potentially be applicable to most pipeline handlers too.\n> \n> The biggest Size? The smallest Size? or the nearest size to a list of\n> predefined sizes for a given role?\n\nMaybe something like that, or maybe asking kernel drivers to sort\nformats according to some priority rules, or something else. I'm not\nsure what it should be, but I wanted to record the fact that we should\nlook at it.\n\n> > +\t */\n> > +\tStreamConfiguration cfg{ StreamFormats{ formats } };\n> > +\tcfg.pixelFormat = formats.begin()->first;\n> > +\tcfg.size = formats.begin()->second[0].max;\n> > +\n> > +\tconfig->addConfiguration(cfg);\n> > +\n> > +\tconfig->validate();\n> > +\n> > +\treturn config;\n> > +}\n> > +\n> > +int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c)\n> > +{\n> > +\tSimpleCameraConfiguration *config =\n> > +\t\tstatic_cast(c);\n> > +\tSimpleCameraData *data = cameraData(camera);\n> > +\tStreamConfiguration &cfg = config->at(0);\n> > +\tint ret;\n> > +\n> > +\t/*\n> > +\t * Configure links on the pipeline and propagate formats from the\n> > +\t * sensor to the video node.\n> > +\t */\n> > +\tret = data->setupLinks();\n> > +\tif (ret < 0)\n> > +\t\treturn ret;\n> > +\n> > +\tconst SimpleCameraData::Configuration &pipeConfig =\n> > +\t\tdata->formats_[cfg.pixelFormat];\n> > +\n> > +\tV4L2SubdeviceFormat format{ pipeConfig.code, data->sensor_->resolution() };\n> > +\n> > +\tret = data->setupFormats(&format, V4L2Subdevice::ActiveFormat);\n> > +\tif (ret < 0)\n> > +\t\treturn ret;\n> > +\n> > +\t/* Configure the video node. */\n> > +\tV4L2PixelFormat videoFormat = video_->toV4L2PixelFormat(cfg.pixelFormat);\n> > +\n> > +\tV4L2DeviceFormat outputFormat = {};\n> > +\toutputFormat.fourcc = videoFormat;\n> > +\toutputFormat.size = cfg.size;\n> > +\n> > +\tret = video_->setFormat(&outputFormat);\n> > +\tif (ret)\n> > +\t\treturn ret;\n> > +\n> > +\tif (outputFormat.size != cfg.size || outputFormat.fourcc != videoFormat) {\n> > +\t\tLOG(SimplePipeline, Error)\n> > +\t\t\t<< \"Unable to configure capture in \" << cfg.toString();\n> > +\t\treturn -EINVAL;\n> > +\t}\n> > +\n> > +\tcfg.setStream(&data->stream_);\n> > +\n> > +\treturn 0;\n> > +}\n> > +\n> > +int SimplePipelineHandler::exportFrameBuffers(Camera *camera, Stream *stream,\n> > +\t\t\t\t\t std::vector> *buffers)\n> > +{\n> > +\tunsigned int count = stream->configuration().bufferCount;\n> > +\n> > +\treturn video_->exportBuffers(count, buffers);\n> > +}\n> > +\n> > +int SimplePipelineHandler::start(Camera *camera)\n> > +{\n> > +\tSimpleCameraData *data = cameraData(camera);\n> > +\tunsigned int count = data->stream_.configuration().bufferCount;\n> > +\n> > +\tint ret = video_->importBuffers(count);\n> > +\tif (ret < 0)\n> > +\t\treturn ret;\n> > +\n> > +\tret = video_->streamOn();\n> > +\tif (ret < 0) {\n> > +\t\tvideo_->releaseBuffers();\n> \n> I wonder if video_->importBuffers() should be something that now happens\n> at video_->streamOn()?\n> \n> I presume that it is now more of an operation to set up V4L2 with the\n> internal buffer structures for passing the information on 'which' buffer\n> is in use to V4L2...\n> \n> Every video stream will have to do this?\n> \n> Otherwise, if conceptually we don't want to mis-manage buffer handling\n> in V4L2VideoDevice::streamOn() we might want to have a V4L2Stream object\n> to do so ...\n> \n> (clearly those are just V4L2VideoDevice API discussion points, and not\n> actions on this patch)\n\nGood questions, I hadn't thought about that, and it could indeed make\nsense. The operations are separate in V4L2 so one can argue that\nV4L2VideoDevice should just be a thin wrapper and separate them, but if\nthey're always used the same way, it can make sense to refactor that.\nLet's keep it in mind. Niklas had proposed a V4L2Stream a while ago I\nthink, which I didn't like, but that was because of how it tried to\nexpose things automatically to the Camera object. If it's just an\ninternal helper, it's an idea we could explore.\n\n> > +\t\treturn ret;\n> > +\t}\n> > +\n> > +\tactiveCamera_ = camera;\n> \n> Will simple pipeline handler support more than one camera?\n\nI'm not sure yet, but I wouldn't rule it out completely. I also think it\ncould make sense, at some point, to refactor this, and extract code to\nan object to model a sensor-to-videodev pipeline. That could be used by\nmore complex pipeline handlers that have an inline ISP with little\nprocessing coupled with an offline ISP.\n\n> The activeCamera_ feels like a way of supporting more than one camera on\n> a pipeline, which could otherwise have been a bool cameraActive to\n> represent it's state. In fact I'm not sure it's even needed to do\n> that... The only actual check is an assert to ensure that it's set\n> before using it in the bufferReady() callbacks, which could likely have\n> accessed the camera directly?\n> \n> But if it's worth keeping this as it is to keep it close to other\n> pipeline handlers then so be it.\n> \n> \n> Looks like it is used actaully, as the Camera is only created and\n> registered with the CameraManager, so we don't really have direct access\n> to the Camera within the PipelineHandler, and we have to be given it..\n> (even though /we/ created it :-D)\n\nYes, and I'm thinking it could make sense to store it automatically in\nthe base CameraData class. That's something we should look at when we'll\nrefactor pipeline handlers.\n\n> > +\n> > +\treturn 0;\n> > +}\n> > +\n> > +void SimplePipelineHandler::stop(Camera *camera)\n> > +{\n> > +\tvideo_->streamOff();\n> > +\tvideo_->releaseBuffers();\n> > +\tactiveCamera_ = nullptr;\n> > +}\n> > +\n> > +int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request)\n> > +{\n> > +\tSimpleCameraData *data = cameraData(camera);\n> > +\tStream *stream = &data->stream_;\n> > +\n> > +\tFrameBuffer *buffer = request->findBuffer(stream);\n> > +\tif (!buffer) {\n> > +\t\tLOG(SimplePipeline, Error)\n> > +\t\t\t<< \"Attempt to queue request with invalid stream\";\n> > +\t\treturn -ENOENT;\n> > +\t}\n> > +\n> > +\treturn video_->queueBuffer(buffer);\n> > +}\n> > +\n> > +/* -----------------------------------------------------------------------------\n> > + * Match and Setup\n> > + */\n> > +\n> > +bool SimplePipelineHandler::match(DeviceEnumerator *enumerator)\n> > +{\n> > +\tstatic const char * const drivers[] = {\n> > +\t\t\"imx7-csi\",\n> > +\t\t\"sun6i-csi\",\n> > +\t};\n> > +\n> > +\tfor (const char *driver : drivers) {\n> > +\t\tDeviceMatch dm(driver);\n> > +\t\tmedia_ = acquireMediaDevice(enumerator, dm);\n> > +\t\tif (media_)\n> > +\t\t\tbreak;\n> > +\t}\n> > +\n> > +\tif (!media_)\n> > +\t\treturn false;\n> > +\n> > +\t/*\n> > +\t * Locate sensors and video nodes. We only support pipelines with at\n> > +\t * least one sensor and exactly one video captude node.\n> \n> /captude/capture/\n> \n> > +\t */\n> > +\tstd::vector sensors;\n> > +\tstd::vector videos;\n> > +\n> > +\tfor (MediaEntity *entity : media_->entities()) {\n> > +\t\tswitch (entity->function()) {\n> > +\t\tcase MEDIA_ENT_F_CAM_SENSOR:\n> > +\t\t\tsensors.push_back(entity);\n> > +\t\t\tbreak;\n> > +\n> > +\t\tcase MEDIA_ENT_F_IO_V4L:\n> > +\t\t\tif (entity->pads().size() == 1 &&\n> > +\t\t\t (entity->pads()[0]->flags() & MEDIA_PAD_FL_SINK))\n> > +\t\t\t\tvideos.push_back(entity);\n> > +\t\t\tbreak;\n> > +\n> > +\t\tdefault:\n> > +\t\t\tbreak;\n> > +\t\t}\n> > +\t}\n> > +\n> > +\tif (sensors.empty()) {\n> > +\t\tLOG(SimplePipeline, Error) << \"No sensor found\";\n> > +\t\treturn false;\n> > +\t}\n> > +\n> > +\tif (videos.size() != 1) {\n> > +\t\tLOG(SimplePipeline, Error)\n> > +\t\t\t<< \"Pipeline with \" << videos.size()\n> > +\t\t\t<< \" video capture nodes is not supported\";\n> > +\t\treturn false;\n> > +\t}\n> > +\n> > +\t/* Locate and open the capture video node. */\n> > +\tvideo_ = new V4L2VideoDevice(videos[0]);\n> > +\tif (video_->open() < 0)\n> > +\t\treturn false;\n> > +\n> > +\tif (video_->caps().isMultiplanar()) {\n> > +\t\tLOG(SimplePipeline, Error)\n> > +\t\t\t<< \"V4L2 multiplanar devices are not supported\";\n> > +\t\treturn false;\n> > +\t}\n> > +\n> > +\tvideo_->bufferReady.connect(this, &SimplePipelineHandler::bufferReady);\n> > +\n> > +\t/*\n> > +\t * Create one camera data instance for each sensor and gather all\n> > +\t * entities in all pipelines.\n> > +\t */\n> > +\tstd::vector> pipelines;\n> > +\tstd::set entities;\n> > +\n> > +\tpipelines.reserve(sensors.size());\n> > +\n> > +\tfor (MediaEntity *sensor : sensors) {\n> > +\t\tstd::unique_ptr data =\n> > +\t\t\tstd::make_unique(this, sensor,\n> > +\t\t\t\t\t\t\t videos[0]);\n> > +\t\tif (!data->isValid()) {\n> > +\t\t\tLOG(SimplePipeline, Error)\n> > +\t\t\t\t<< \"No valid pipeline for sensor '\"\n> > +\t\t\t\t<< sensor->name() << \"', skipping\";\n> > +\t\t\tcontinue;\n> > +\t\t}\n> > +\n> > +\t\tfor (SimpleCameraData::Entity &entity : data->entities_)\n> > +\t\t\tentities.insert(entity.entity);\n> > +\n> > +\t\tpipelines.push_back(std::move(data));\n> > +\t}\n> > +\n> > +\tif (entities.empty())\n> > +\t\treturn false;\n> > +\n> > +\t/* Create and open V4L2Subdev instances for all the entities. */\n> > +\tfor (MediaEntity *entity : entities) {\n> \n> \t\t/* Create a new map entry with the entity, and a\n> \t\t * constructed V4L2Subdevice() using the entity for the\n> \t\t * default constructor. */\n> \n> or some such? (Otherwise, emplacing the entity twice looks really odd\n> and obfuscated)\n\nIt's not the default constructor, it's the constructor taking a\nMediaEntity. I'll see if I can find a good way to express this without\njust documenting the C++ idioms :-)\n\n> > +\t\tauto elem = subdevs_.emplace(std::piecewise_construct,\n> > +\t\t\t\t\t std::forward_as_tuple(entity),\n> > +\t\t\t\t\t std::forward_as_tuple(entity));\n> \n> \n> Is a piecewise_construct required here?\n> \n> \tauto elem = subdevs_.emplace(entity, entity);\n> \n> should do the trick, (and at least compiles).\n\nIt's not required, but it's more efficient in this case (see commit\n38dd90307ab2).\n\n> > +\t\tV4L2Subdevice *subdev = &elem.first->second;\n> \n> The first second. Well I really do love pairs :-)\n\n:-) I wish the value type for maps wasn't a std::pair but a\nstd::map_entry that would inherit from std::pair and expose key and\nvalud aliases for first and second.\n\n> > +\t\tint ret = subdev->open();\n> > +\t\tif (ret < 0) {\n> > +\t\t\tLOG(SimplePipeline, Error)\n> > +\t\t\t\t<< \"Failed to open \" << subdev->deviceNode()\n> > +\t\t\t\t<< \": \" << strerror(-ret);\n> > +\t\t\treturn false;\n> > +\t\t}\n> > +\t}\n> > +\n> > +\t/* Initialize each pipeline and register a corresponding camera. */\n> > +\tfor (std::unique_ptr &data : pipelines) {\n> > +\t\tint ret = data->init();\n> > +\t\tif (ret < 0)\n> > +\t\t\tcontinue;\n> > +\n> > +\t\tstd::shared_ptr camera =\n> > +\t\t\tCamera::create(this, data->sensor_->entity()->name(),\n> > +\t\t\t\t data->streams());\n> > +\t\tregisterCamera(std::move(camera), std::move(data));\n> > +\t}\n> > +\n> > +\treturn true;\n> > +}\n> > +\n> > +V4L2Subdevice *SimplePipelineHandler::subdev(const MediaEntity *entity)\n> > +{\n> > +\tauto iter = subdevs_.find(entity);\n> > +\tif (iter == subdevs_.end())\n> > +\t\treturn nullptr;\n> > +\n> > +\treturn &iter->second;\n> > +}\n> > +\n> > +/* -----------------------------------------------------------------------------\n> > + * Buffer Handling\n> > + */\n> > +\n> > +void SimplePipelineHandler::bufferReady(FrameBuffer *buffer)\n> > +{\n> > +\tASSERT(activeCamera_);\n> > +\tRequest *request = buffer->request();\n> > +\tcompleteBuffer(activeCamera_, request, buffer);\n> > +\tcompleteRequest(activeCamera_, request);\n> > +}\n> > +\n> > +REGISTER_PIPELINE_HANDLER(SimplePipelineHandler);\n> > +\n> > +} /* namespace libcamera */","headers":{"Return-Path":"","Received":["from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 79D03605D1\n\tfor ;\n\tSat, 4 Apr 2020 01:55:46 +0200 (CEST)","from pendragon.ideasonboard.com (81-175-216-236.bb.dnainternet.fi\n\t[81.175.216.236])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id A1060321;\n\tSat, 4 Apr 2020 01:55:45 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ideasonboard.com\n\theader.i=@ideasonboard.com\n\theader.b=\"Z5RdiPHn\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1585958146;\n\tbh=mfO1j0d/Et7NmiMMsVT2kr6bTmVl/kGXxM8hA5RCx8c=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=Z5RdiPHnySQ3vWxvkH8Av0AZQlSGCDbB89B0oTGzoYlDQirhQwCZPUiNRsBVdcJJL\n\trBg3xATtiQfLmY/pVeT+h/3c4jBnIKhYGBr/B5kp6uVRbf++UIft29YoupGTDDYRyM\n\tPIwJ0iC34QQm+WagGcxvoXc2G0BX8ynIqIqWs0+E=","Date":"Sat, 4 Apr 2020 02:55:36 +0300","From":"Laurent Pinchart ","To":"Kieran Bingham ","Cc":"libcamera-devel@lists.libcamera.org, Martijn Braam , \n\tBenjamin GAIGNARD ","Message-ID":"<20200403235536.GD9690@pendragon.ideasonboard.com>","References":"<20200320014839.14683-1-laurent.pinchart@ideasonboard.com>\n\t<20200320014839.14683-10-laurent.pinchart@ideasonboard.com>\n\t<43484c6b-44ac-4bd7-88e4-b46070840315@ideasonboard.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<43484c6b-44ac-4bd7-88e4-b46070840315@ideasonboard.com>","Subject":"Re: [libcamera-devel] [PATCH v3 09/11] libcamera: pipeline: Add a\n\tsimple pipeline handler","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","X-List-Received-Date":"Fri, 03 Apr 2020 23:55:46 -0000"}},{"id":4379,"web_url":"https://patchwork.libcamera.org/comment/4379/","msgid":"<20200404000825.GE9690@pendragon.ideasonboard.com>","date":"2020-04-04T00:08:25","subject":"Re: [libcamera-devel] [PATCH v3 09/11] libcamera: pipeline: Add a\n\tsimple pipeline handler","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Andrey,\n\nOn Fri, Apr 03, 2020 at 04:55:32PM +0300, Andrey Konovalov wrote:\n> On 20.03.2020 04:48, Laurent Pinchart wrote:\n> > From: Martijn Braam \n> > \n> > This new pipeline handler aims at supporting any simple device without\n> > requiring any device-specific code. Simple devices are currently defined\n> > as a graph made of one or multiple camera sensors and a single video\n> > node, with each sensor connected to the video node through a linear\n> > pipeline.\n> > \n> > The simple pipeline handler will automatically parse the media graph,\n> > enumerate sensors, build supported stream configurations, and configure\n> > the pipeline, without any device-specific knowledge. It doesn't support\n> > configuration of any processing in the pipeline at the moment, but may\n> > be extended to support simple processing such as format conversion or\n> > scaling in the future.\n> > \n> > The only device-specific information in the pipeline handler is the list\n> > of supported drivers, required for device matching. We may be able to\n> > remove this in the future by matching with the simple pipeline handler\n> > as a last resort option, after all other pipeline handlers have been\n> > tried.\n> > \n> > Signed-off-by: Martijn Braam \n> > Signed-off-by: Laurent Pinchart \n> > ---\n> > Changes since v2:\n> > \n> > - Log an error when setupFormats() fail\n> > - Propagate getFormat() and setFormat() errors to the caller of\n> > setupFormats()\n> > - Reorder variable declarations in validate()\n> > - Add \\todo comment related to the selection of the default format\n> > - Use log Error instead of Info if pipeline isn't valid\n> > - Rebase on top of V4L2PixelFormat\n> > \n> > Changes since v1:\n> > \n> > - Rebase on top of buffer API rework\n> > - Expose stream formats\n> > - Rework camera data config\n> > ---\n> > src/libcamera/pipeline/meson.build | 1 +\n> > src/libcamera/pipeline/simple/meson.build | 3 +\n> > src/libcamera/pipeline/simple/simple.cpp | 710 ++++++++++++++++++++++\n> > 3 files changed, 714 insertions(+)\n> > create mode 100644 src/libcamera/pipeline/simple/meson.build\n> > create mode 100644 src/libcamera/pipeline/simple/simple.cpp\n> > \n> > diff --git a/src/libcamera/pipeline/meson.build b/src/libcamera/pipeline/meson.build\n> > index 0d466225a72e..606ba31a0319 100644\n> > --- a/src/libcamera/pipeline/meson.build\n> > +++ b/src/libcamera/pipeline/meson.build\n> > @@ -5,3 +5,4 @@ libcamera_sources += files([\n> > \n> > subdir('ipu3')\n> > subdir('rkisp1')\n> > +subdir('simple')\n> > diff --git a/src/libcamera/pipeline/simple/meson.build b/src/libcamera/pipeline/simple/meson.build\n> > new file mode 100644\n> > index 000000000000..4945a3e173cf\n> > --- /dev/null\n> > +++ b/src/libcamera/pipeline/simple/meson.build\n> > @@ -0,0 +1,3 @@\n> > +libcamera_sources += files([\n> > + 'simple.cpp',\n> > +])\n> > diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp\n> > new file mode 100644\n> > index 000000000000..545a99fe31ca\n> > --- /dev/null\n> > +++ b/src/libcamera/pipeline/simple/simple.cpp\n> > @@ -0,0 +1,710 @@\n> > +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n> > +/*\n> > + * Copyright (C) 2020, Laurent Pinchart\n> > + * Copyright (C) 2019, Martijn Braam\n> > + *\n> > + * simple.cpp - Pipeline handler for simple pipelines\n> > + */\n> > +\n> > +#include \n> > +#include \n> > +#include \n> > +#include \n> > +#include \n> > +#include \n> > +#include \n> > +#include \n> > +#include \n> > +#include \n> > +\n> > +#include \n> > +\n> > +#include \n> > +#include \n> > +#include \n> > +\n> > +#include \"camera_sensor.h\"\n> > +#include \"device_enumerator.h\"\n> > +#include \"log.h\"\n> > +#include \"media_device.h\"\n> > +#include \"pipeline_handler.h\"\n> > +#include \"v4l2_subdevice.h\"\n> > +#include \"v4l2_videodevice.h\"\n> > +\n> > +namespace libcamera {\n> > +\n> > +LOG_DEFINE_CATEGORY(SimplePipeline)\n> > +\n> > +class SimplePipelineHandler;\n> > +\n> > +class SimpleCameraData : public CameraData\n> > +{\n> > +public:\n> > +\tSimpleCameraData(PipelineHandler *pipe, MediaEntity *sensor,\n> > +\t\t\t MediaEntity *video);\n> > +\n> > +\tbool isValid() const { return sensor_ != nullptr; }\n> > +\tstd::set streams() { return { &stream_ }; }\n> > +\n> > +\tint init();\n> > +\tint setupLinks();\n> > +\tint setupFormats(V4L2SubdeviceFormat *format,\n> > +\t\t\t V4L2Subdevice::Whence whence);\n> > +\n> > +\tstruct Entity {\n> > +\t\tMediaEntity *entity;\n> > +\t\tMediaLink *link;\n> > +\t};\n> > +\n> > +\tstruct Configuration {\n> > +\t\tuint32_t code;\n> > +\t\tPixelFormat pixelFormat;\n> > +\t\tSize size;\n> > +\t};\n> > +\n> > +\tStream stream_;\n> > +\tstd::unique_ptr sensor_;\n> > +\tstd::list entities_;\n> > +\n> > +\tstd::vector configs_;\n> > +\tstd::map formats_;\n> > +};\n> > +\n> > +class SimpleCameraConfiguration : public CameraConfiguration\n> > +{\n> > +public:\n> > +\tSimpleCameraConfiguration(Camera *camera, SimpleCameraData *data);\n> > +\n> > +\tStatus validate() override;\n> > +\n> > +\tconst V4L2SubdeviceFormat &sensorFormat() { return sensorFormat_; }\n> > +\n> > +private:\n> > +\t/*\n> > +\t * The SimpleCameraData instance is guaranteed to be valid as long as\n> > +\t * the corresponding Camera instance is valid. In order to borrow a\n> > +\t * reference to the camera data, store a new reference to the camera.\n> > +\t */\n> > +\tstd::shared_ptr camera_;\n> > +\tconst SimpleCameraData *data_;\n> > +\n> > +\tV4L2SubdeviceFormat sensorFormat_;\n> > +};\n> > +\n> > +class SimplePipelineHandler : public PipelineHandler\n> > +{\n> > +public:\n> > +\tSimplePipelineHandler(CameraManager *manager);\n> > +\t~SimplePipelineHandler();\n> > +\n> > +\tCameraConfiguration *generateConfiguration(Camera *camera,\n> > +\t\t\t\t\t\t const StreamRoles &roles) override;\n> > +\tint configure(Camera *camera, CameraConfiguration *config) override;\n> > +\n> > +\tint exportFrameBuffers(Camera *camera, Stream *stream,\n> > +\t\t\t std::vector> *buffers) override;\n> > +\n> > +\tint start(Camera *camera) override;\n> > +\tvoid stop(Camera *camera) override;\n> > +\n> > +\tbool match(DeviceEnumerator *enumerator) override;\n> > +\n> > +\tV4L2VideoDevice *video() { return video_; }\n> > +\tV4L2Subdevice *subdev(const MediaEntity *entity);\n> > +\n> > +protected:\n> > +\tint queueRequestDevice(Camera *camera, Request *request) override;\n> > +\n> > +private:\n> > +\tSimpleCameraData *cameraData(const Camera *camera)\n> > +\t{\n> > +\t\treturn static_cast(\n> > +\t\t\tPipelineHandler::cameraData(camera));\n> > +\t}\n> > +\n> > +\tint initLinks();\n> > +\n> > +\tint createCamera(MediaEntity *sensor);\n> > +\n> > +\tvoid bufferReady(FrameBuffer *buffer);\n> > +\n> > +\tMediaDevice *media_;\n> > +\tV4L2VideoDevice *video_;\n> > +\tstd::map subdevs_;\n> > +\n> > +\tCamera *activeCamera_;\n> > +};\n> > +\n> > +/* -----------------------------------------------------------------------------\n> > + * Camera Data\n> > + */\n> > +SimpleCameraData::SimpleCameraData(PipelineHandler *pipe, MediaEntity *sensor,\n> > +\t\t\t\t MediaEntity *video)\n> > +\t: CameraData(pipe)\n> > +{\n> > +\tint ret;\n> > +\n> > +\t/*\n> > +\t * Walk the pipeline towards the video node and store all entities\n> > +\t * along the way.\n> > +\t */\n> > +\tMediaEntity *source = sensor;\n> > +\n> > +\twhile (source) {\n> > +\t\t/* If we have reached the video node, we're done. */\n> > +\t\tif (source == video)\n> > +\t\t\tbreak;\n> > +\n> > +\t\t/* Use the first output pad that has links. */\n> > +\t\tMediaPad *sourcePad = nullptr;\n> > +\t\tfor (MediaPad *pad : source->pads()) {\n> > +\t\t\tif ((pad->flags() & MEDIA_PAD_FL_SOURCE) &&\n> > +\t\t\t !pad->links().empty()) {\n> > +\t\t\t\tsourcePad = pad;\n> > +\t\t\t\tbreak;\n> > +\t\t\t}\n> > +\t\t}\n> > +\n> > +\t\tif (!sourcePad)\n> > +\t\t\treturn;\n> > +\n> > +\t\t/* Use the first link that isn't immutable and disabled. */\n> > +\t\tMediaLink *sourceLink = nullptr;\n> > +\t\tfor (MediaLink *link : sourcePad->links()) {\n> > +\t\t\tif ((link->flags() & MEDIA_LNK_FL_ENABLED) ||\n> > +\t\t\t !(link->flags() & MEDIA_LNK_FL_IMMUTABLE)) {\n> > +\t\t\t\tsourceLink = link;\n> > +\t\t\t\tbreak;\n> > +\t\t\t}\n> > +\t\t}\n> > +\n> > +\t\tif (!sourceLink)\n> > +\t\t\treturn;\n> > +\n> > +\t\tentities_.push_back({ source, sourceLink });\n> > +\n> > +\t\tsource = sourceLink->sink()->entity();\n> > +\n> > +\t\t/* Avoid infinite loops. */\n> > +\t\tauto iter = std::find_if(entities_.begin(), entities_.end(),\n> > +\t\t\t\t\t [&](const Entity &entity) {\n> > +\t\t\t\t\t\t return entity.entity == source;\n> > +\t\t\t\t\t });\n> > +\t\tif (iter != entities_.end()) {\n> > +\t\t\tLOG(SimplePipeline, Info) << \"Loop detected in pipeline\";\n> > +\t\t\treturn;\n> > +\t\t}\n> > +\t}\n> > +\n> > +\t/* We have a valid pipeline, create the camera sensor. */\n> > +\tsensor_ = std::make_unique(sensor);\n> > +\tret = sensor_->init();\n> > +\tif (ret) {\n> > +\t\tsensor_.reset();\n> > +\t\treturn;\n> > +\t}\n> > +}\n> > +\n> > +int SimpleCameraData::init()\n> > +{\n> > +\tSimplePipelineHandler *pipe = static_cast(pipe_);\n> > +\tV4L2VideoDevice *video = pipe->video();\n> > +\tint ret;\n> > +\n> > +\t/*\n> > +\t * Enumerate the possible pipeline configurations. For each media bus\n> > +\t * format supported by the sensor, propagate the formats through the\n> > +\t * pipeline, and enumerate the corresponding possible V4L2 pixel\n> > +\t * formats on the video node.\n> > +\t */\n> > +\tfor (unsigned int code : sensor_->mbusCodes()) {\n> > +\t\tV4L2SubdeviceFormat format{ code, sensor_->resolution() };\n> > +\n> > +\t\t/*\n> > +\t\t * Setup links first as some subdev drivers take active links\n> > +\t\t * into account to propaget TRY formats. So is life :-(\n> > +\t\t */\n> > +\t\tret = setupLinks();\n> > +\t\tif (ret < 0)\n> > +\t\t\treturn ret;\n> > +\n> > +\t\tret = setupFormats(&format, V4L2Subdevice::TryFormat);\n> > +\t\tif (ret < 0) {\n> > +\t\t\tLOG(SimplePipeline, Error)\n> > +\t\t\t\t<< \"Failed to setup pipeline for media bus code \"\n> > +\t\t\t\t<< utils::hex(code, 4);\n> > +\t\t\treturn ret;\n> > +\t\t}\n> > +\n> > +\t\tstd::map> videoFormats =\n> > +\t\t\tvideo->formats(format.mbus_code);\n> > +\n> > +\t\tLOG(SimplePipeline, Debug)\n> > +\t\t\t<< \"Adding configuration for \" << format.size.toString()\n> > +\t\t\t<< \" in pixel formats [ \"\n> > +\t\t\t<< utils::join(videoFormats, \", \",\n> > +\t\t\t\t [](const auto &f) {\n> > +\t\t\t\t\t return f.first.toString();\n> > +\t\t\t\t })\n> > +\t\t\t<< \" ]\";\n> > +\n> > +\t\t/*\n> > +\t\t * Store the configuration in the formats_ map, mapping\n> > +\t\t * PixelFormat to configuration. Any previously stored value is\n> > +\t\t * overwritten, as the pipeline handler currently doesn't care\n> > +\t\t * about how a particular PixelFormat is achieved.\n> > +\t\t */\n> > +\t\tfor (const auto &videoFormat : videoFormats) {\n> > +\t\t\tPixelFormat pixelFormat = video->toPixelFormat(videoFormat.first);\n> > +\t\t\tif (!pixelFormat)\n> > +\t\t\t\tcontinue;\n> > +\n> > +\t\t\tConfiguration config;\n> > +\t\t\tconfig.code = code;\n> > +\t\t\tconfig.pixelFormat = pixelFormat;\n> > +\t\t\tconfig.size = format.size;\n> > +\n> > +\t\t\tformats_[pixelFormat] = config;\n> > +\t\t}\n> > +\t}\n> > +\n> > +\tif (formats_.empty()) {\n> > +\t\tLOG(SimplePipeline, Error) << \"No valid configuration found\";\n> > +\t\treturn -EINVAL;\n> > +\t}\n> > +\n> > +\treturn 0;\n> > +}\n> > +\n> > +int SimpleCameraData::setupLinks()\n> > +{\n> > +\tint ret;\n> > +\n> > +\t/*\n> > +\t * Configure all links along the pipeline. Some entities may not allow\n> > +\t * multiple sink links to be enabled together, even on different sink\n> > +\t * pads. We must thus start by disabling all sink links (but the one we\n> > +\t * want to enable) before enabling the pipeline link.\n> > +\t */\n> > +\tfor (SimpleCameraData::Entity &e : entities_) {\n> > +\t\tfor (MediaPad *pad : e.link->sink()->entity()->pads()) {\n> > +\t\t\tfor (MediaLink *link : pad->links()) {\n> > +\t\t\t\tif (link == e.link)\n> > +\t\t\t\t\tcontinue;\n> > +\n> > +\t\t\t\tif ((link->flags() & MEDIA_LNK_FL_ENABLED) &&\n> > +\t\t\t\t !(link->flags() & MEDIA_LNK_FL_IMMUTABLE)) {\n> > +\t\t\t\t\tret = link->setEnabled(false);\n> > +\t\t\t\t\tif (ret < 0)\n> > +\t\t\t\t\t\treturn ret;\n> > +\t\t\t\t}\n> > +\t\t\t}\n> > +\t\t}\n> > +\n> > +\t\tif (!(e.link->flags() & MEDIA_LNK_FL_ENABLED)) {\n> > +\t\t\tret = e.link->setEnabled(true);\n> > +\t\t\tif (ret < 0)\n> > +\t\t\t\treturn ret;\n> > +\t\t}\n> > +\t}\n> > +\n> > +\treturn 0;\n> > +}\n> > +\n> > +int SimpleCameraData::setupFormats(V4L2SubdeviceFormat *format,\n> > +\t\t\t\t V4L2Subdevice::Whence whence)\n> > +{\n> > +\tSimplePipelineHandler *pipe = static_cast(pipe_);\n> > +\tint ret;\n> > +\n> > +\t/*\n> > +\t * Configure the format on the sensor output and propagate it through\n> > +\t * the pipeline.\n> > +\t */\n> > +\tret = sensor_->setFormat(format);\n> > +\tif (ret < 0)\n> > +\t\treturn ret;\n> > +\n> > +\tfor (const Entity &e : entities_) {\n> > +\t\tMediaLink *link = e.link;\n> > +\t\tMediaPad *source = link->source();\n> > +\t\tMediaPad *sink = link->sink();\n> > +\n> > +\t\tif (source->entity() != sensor_->entity()) {\n> > +\t\t\tV4L2Subdevice *subdev = pipe->subdev(source->entity());\n> > +\t\t\tret = subdev->getFormat(source->index(), format, whence);\n> > +\t\t\tif (ret < 0)\n> > +\t\t\t\treturn ret;\n> > +\t\t}\n> > +\n> > +\t\tif (sink->entity()->function() != MEDIA_ENT_F_IO_V4L) {\n> > +\t\t\tV4L2Subdevice *subdev = pipe->subdev(sink->entity());\n> > +\t\t\tret = subdev->setFormat(sink->index(), format, whence);\n> > +\t\t\tif (ret < 0)\n> > +\t\t\t\treturn ret;\n> > +\t\t}\n> > +\n> > +\t\tLOG(SimplePipeline, Debug)\n> > +\t\t\t<< \"Link '\" << source->entity()->name()\n> > +\t\t\t<< \"':\" << source->index()\n> > +\t\t\t<< \" -> '\" << sink->entity()->name()\n> > +\t\t\t<< \"':\" << sink->index()\n> > +\t\t\t<< \" configured with format \" << format->toString();\n> > +\t}\n> > +\n> > +\treturn 0;\n> > +}\n> > +\n> > +/* -----------------------------------------------------------------------------\n> > + * Camera Configuration\n> > + */\n> > +\n> > +SimpleCameraConfiguration::SimpleCameraConfiguration(Camera *camera,\n> > +\t\t\t\t\t\t SimpleCameraData *data)\n> > +\t: CameraConfiguration(), camera_(camera->shared_from_this()),\n> > +\t data_(data)\n> > +{\n> > +}\n> > +\n> > +CameraConfiguration::Status SimpleCameraConfiguration::validate()\n> > +{\n> > +\tStatus status = Valid;\n> > +\n> > +\tif (config_.empty())\n> > +\t\treturn Invalid;\n> > +\n> > +\t/* Cap the number of entries to the available streams. */\n> > +\tif (config_.size() > 1) {\n> > +\t\tconfig_.resize(1);\n> > +\t\tstatus = Adjusted;\n> > +\t}\n> > +\n> > +\tStreamConfiguration &cfg = config_[0];\n> > +\n> > +\t/* Adjust the pixel format. */\n> > +\tauto it = data_->formats_.find(cfg.pixelFormat);\n> > +\tif (it == data_->formats_.end())\n> > +\t\tit = data_->formats_.begin();\n> > +\n> > +\tPixelFormat pixelFormat = it->first;\n> > +\tif (cfg.pixelFormat != pixelFormat) {\n> > +\t\tLOG(SimplePipeline, Debug) << \"Adjusting pixel format\";\n> > +\t\tcfg.pixelFormat = pixelFormat;\n> > +\t\tstatus = Adjusted;\n> > +\t}\n> > +\n> > +\tconst SimpleCameraData::Configuration &pipeConfig = it->second;\n> > +\tif (cfg.size != pipeConfig.size) {\n> > +\t\tLOG(SimplePipeline, Debug)\n> > +\t\t\t<< \"Adjusting size from \" << cfg.size.toString()\n> > +\t\t\t<< \" to \" << pipeConfig.size.toString();\n> > +\t\tcfg.size = pipeConfig.size;\n> > +\t\tstatus = Adjusted;\n> > +\t}\n> > +\n> > +\tcfg.bufferCount = 3;\n> > +\n> > +\treturn status;\n> > +}\n> > +\n> > +/* -----------------------------------------------------------------------------\n> > + * Pipeline Handler\n> > + */\n> > +\n> > +SimplePipelineHandler::SimplePipelineHandler(CameraManager *manager)\n> > +\t: PipelineHandler(manager), video_(nullptr)\n> > +{\n> > +}\n> > +\n> > +SimplePipelineHandler::~SimplePipelineHandler()\n> > +{\n> > +\tdelete video_;\n> > +}\n> > +\n> > +CameraConfiguration *SimplePipelineHandler::generateConfiguration(Camera *camera,\n> > +\t\t\t\t\t\t\t\t const StreamRoles &roles)\n> > +{\n> > +\tSimpleCameraData *data = cameraData(camera);\n> > +\tCameraConfiguration *config =\n> > +\t\tnew SimpleCameraConfiguration(camera, data);\n> > +\n> > +\tif (roles.empty())\n> > +\t\treturn config;\n> > +\n> > +\t/* Create the formats map. */\n> > +\tstd::map> formats;\n> > +\tstd::transform(data->formats_.begin(), data->formats_.end(),\n> > +\t\t std::inserter(formats, formats.end()),\n> > +\t\t [](const auto &format) -> decltype(formats)::value_type {\n> > +\t\t\t const PixelFormat &pixelFormat = format.first;\n> > +\t\t\t const Size &size = format.second.size;\n> > +\t\t\t return { pixelFormat, { size } };\n> > +\t\t });\n> > +\n> > +\t/*\n> > +\t * Create the stream configuration. Take the first entry in the formats\n> > +\t * map as the default, for lack of a better option.\n> > +\t *\n> > +\t * \\todo Implement a better way to pick the default format\n> > +\t */\n> > +\tStreamConfiguration cfg{ StreamFormats{ formats } };\n> > +\tcfg.pixelFormat = formats.begin()->first;\n> > +\tcfg.size = formats.begin()->second[0].max;\n> > +\n> > +\tconfig->addConfiguration(cfg);\n> > +\n> > +\tconfig->validate();\n> > +\n> > +\treturn config;\n> > +}\n> > +\n> > +int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c)\n> > +{\n> > +\tSimpleCameraConfiguration *config =\n> > +\t\tstatic_cast(c);\n> > +\tSimpleCameraData *data = cameraData(camera);\n> > +\tStreamConfiguration &cfg = config->at(0);\n> > +\tint ret;\n> > +\n> > +\t/*\n> > +\t * Configure links on the pipeline and propagate formats from the\n> > +\t * sensor to the video node.\n> > +\t */\n> > +\tret = data->setupLinks();\n> > +\tif (ret < 0)\n> > +\t\treturn ret;\n> > +\n> > +\tconst SimpleCameraData::Configuration &pipeConfig =\n> > +\t\tdata->formats_[cfg.pixelFormat];\n> > +\n> > +\tV4L2SubdeviceFormat format{ pipeConfig.code, data->sensor_->resolution() };\n> > +\n> > +\tret = data->setupFormats(&format, V4L2Subdevice::ActiveFormat);\n> > +\tif (ret < 0)\n> > +\t\treturn ret;\n> > +\n> > +\t/* Configure the video node. */\n> > +\tV4L2PixelFormat videoFormat = video_->toV4L2PixelFormat(cfg.pixelFormat);\n> > +\n> > +\tV4L2DeviceFormat outputFormat = {};\n> > +\toutputFormat.fourcc = videoFormat;\n> > +\toutputFormat.size = cfg.size;\n> > +\n> > +\tret = video_->setFormat(&outputFormat);\n> > +\tif (ret)\n> > +\t\treturn ret;\n> > +\n> > +\tif (outputFormat.size != cfg.size || outputFormat.fourcc != videoFormat) {\n> > +\t\tLOG(SimplePipeline, Error)\n> > +\t\t\t<< \"Unable to configure capture in \" << cfg.toString();\n> > +\t\treturn -EINVAL;\n> > +\t}\n> > +\n> > +\tcfg.setStream(&data->stream_);\n> > +\n> > +\treturn 0;\n> > +}\n> > +\n> > +int SimplePipelineHandler::exportFrameBuffers(Camera *camera, Stream *stream,\n> > +\t\t\t\t\t std::vector> *buffers)\n> > +{\n> > +\tunsigned int count = stream->configuration().bufferCount;\n> > +\n> > +\treturn video_->exportBuffers(count, buffers);\n> > +}\n> > +\n> > +int SimplePipelineHandler::start(Camera *camera)\n> > +{\n> > +\tSimpleCameraData *data = cameraData(camera);\n> > +\tunsigned int count = data->stream_.configuration().bufferCount;\n> > +\n> > +\tint ret = video_->importBuffers(count);\n> > +\tif (ret < 0)\n> > +\t\treturn ret;\n> > +\n> > +\tret = video_->streamOn();\n> > +\tif (ret < 0) {\n> > +\t\tvideo_->releaseBuffers();\n> > +\t\treturn ret;\n> > +\t}\n> > +\n> > +\tactiveCamera_ = camera;\n> > +\n> > +\treturn 0;\n> > +}\n> > +\n> > +void SimplePipelineHandler::stop(Camera *camera)\n> > +{\n> > +\tvideo_->streamOff();\n> > +\tvideo_->releaseBuffers();\n> > +\tactiveCamera_ = nullptr;\n> > +}\n> > +\n> > +int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request)\n> > +{\n> > +\tSimpleCameraData *data = cameraData(camera);\n> > +\tStream *stream = &data->stream_;\n> > +\n> > +\tFrameBuffer *buffer = request->findBuffer(stream);\n> > +\tif (!buffer) {\n> > +\t\tLOG(SimplePipeline, Error)\n> > +\t\t\t<< \"Attempt to queue request with invalid stream\";\n> > +\t\treturn -ENOENT;\n> > +\t}\n> > +\n> > +\treturn video_->queueBuffer(buffer);\n> > +}\n> > +\n> > +/* -----------------------------------------------------------------------------\n> > + * Match and Setup\n> > + */\n> > +\n> > +bool SimplePipelineHandler::match(DeviceEnumerator *enumerator)\n> > +{\n> > +\tstatic const char * const drivers[] = {\n> > +\t\t\"imx7-csi\",\n> > +\t\t\"sun6i-csi\",\n> > +\t};\n> > +\n> > +\tfor (const char *driver : drivers) {\n> > +\t\tDeviceMatch dm(driver);\n> > +\t\tmedia_ = acquireMediaDevice(enumerator, dm);\n> > +\t\tif (media_)\n> > +\t\t\tbreak;\n> > +\t}\n> > +\n> > +\tif (!media_)\n> > +\t\treturn false;\n> > +\n> > +\t/*\n> > +\t * Locate sensors and video nodes. We only support pipelines with at\n> > +\t * least one sensor and exactly one video captude node.\n> > +\t */\n> > +\tstd::vector sensors;\n> > +\tstd::vector videos;\n> > +\n> > +\tfor (MediaEntity *entity : media_->entities()) {\n> > +\t\tswitch (entity->function()) {\n> > +\t\tcase MEDIA_ENT_F_CAM_SENSOR:\n> > +\t\t\tsensors.push_back(entity);\n> > +\t\t\tbreak;\n> > +\n> > +\t\tcase MEDIA_ENT_F_IO_V4L:\n> > +\t\t\tif (entity->pads().size() == 1 &&\n> > +\t\t\t (entity->pads()[0]->flags() & MEDIA_PAD_FL_SINK))\n> > +\t\t\t\tvideos.push_back(entity);\n> > +\t\t\tbreak;\n> > +\n> > +\t\tdefault:\n> > +\t\t\tbreak;\n> > +\t\t}\n> > +\t}\n> > +\n> > +\tif (sensors.empty()) {\n> > +\t\tLOG(SimplePipeline, Error) << \"No sensor found\";\n> > +\t\treturn false;\n> > +\t}\n> > +\n> > +\tif (videos.size() != 1) {\n> > +\t\tLOG(SimplePipeline, Error)\n> > +\t\t\t<< \"Pipeline with \" << videos.size()\n> > +\t\t\t<< \" video capture nodes is not supported\";\n> > +\t\treturn false;\n> \n> Could these Error be changed to Warning, and the \"return false;\" statement be dropped?\n> \n> I agree that there is no code yet to handle multiple video capture nodes, but this change\n> allows me to use this simple pipeline handler with db410c board and one sensor connected\n> to the first CSI interface (msm_csiphy0). So it can work in some simple configurations\n> with multiple video capture nodes already.\n\nWe can do that, but I don't think it's the right fix. There are little\nguarantees regarding the order in which the video nodes are added to the\nvideos vector. I think we should instead pass the videos list to the\nSimpleCameraData constructor, and get it to pick the video node\ncorresponding to the pipeline output. I can try to implement a patch for\nthat (but you could beat me to it :-)), but I'd like to do so on top of\nthis one.\n\n> > +\t}\n> > +\n> > +\t/* Locate and open the capture video node. */\n> > +\tvideo_ = new V4L2VideoDevice(videos[0]);\n> > +\tif (video_->open() < 0)\n> > +\t\treturn false;\n> > +\n> > +\tif (video_->caps().isMultiplanar()) {\n> > +\t\tLOG(SimplePipeline, Error)\n> > +\t\t\t<< \"V4L2 multiplanar devices are not supported\";\n> > +\t\treturn false;\n> \n> Same here. For the same reasons.\n\nDo you mean that your multiplanar device works with this series ?\n\n> > +\t}\n> > +\n> > +\tvideo_->bufferReady.connect(this, &SimplePipelineHandler::bufferReady);\n> > +\n> > +\t/*\n> > +\t * Create one camera data instance for each sensor and gather all\n> > +\t * entities in all pipelines.\n> > +\t */\n> > +\tstd::vector> pipelines;\n> > +\tstd::set entities;\n> > +\n> > +\tpipelines.reserve(sensors.size());\n> > +\n> > +\tfor (MediaEntity *sensor : sensors) {\n> > +\t\tstd::unique_ptr data =\n> > +\t\t\tstd::make_unique(this, sensor,\n> > +\t\t\t\t\t\t\t videos[0]);\n> > +\t\tif (!data->isValid()) {\n> > +\t\t\tLOG(SimplePipeline, Error)\n> > +\t\t\t\t<< \"No valid pipeline for sensor '\"\n> > +\t\t\t\t<< sensor->name() << \"', skipping\";\n> > +\t\t\tcontinue;\n> > +\t\t}\n> > +\n> > +\t\tfor (SimpleCameraData::Entity &entity : data->entities_)\n> > +\t\t\tentities.insert(entity.entity);\n> > +\n> > +\t\tpipelines.push_back(std::move(data));\n> > +\t}\n> > +\n> > +\tif (entities.empty())\n> > +\t\treturn false;\n> > +\n> > +\t/* Create and open V4L2Subdev instances for all the entities. */\n> > +\tfor (MediaEntity *entity : entities) {\n> > +\t\tauto elem = subdevs_.emplace(std::piecewise_construct,\n> > +\t\t\t\t\t std::forward_as_tuple(entity),\n> > +\t\t\t\t\t std::forward_as_tuple(entity));\n> > +\t\tV4L2Subdevice *subdev = &elem.first->second;\n> > +\t\tint ret = subdev->open();\n> > +\t\tif (ret < 0) {\n> > +\t\t\tLOG(SimplePipeline, Error)\n> > +\t\t\t\t<< \"Failed to open \" << subdev->deviceNode()\n> > +\t\t\t\t<< \": \" << strerror(-ret);\n> > +\t\t\treturn false;\n> > +\t\t}\n> > +\t}\n> > +\n> > +\t/* Initialize each pipeline and register a corresponding camera. */\n> > +\tfor (std::unique_ptr &data : pipelines) {\n> > +\t\tint ret = data->init();\n> > +\t\tif (ret < 0)\n> > +\t\t\tcontinue;\n> > +\n> > +\t\tstd::shared_ptr camera =\n> > +\t\t\tCamera::create(this, data->sensor_->entity()->name(),\n> > +\t\t\t\t data->streams());\n> > +\t\tregisterCamera(std::move(camera), std::move(data));\n> > +\t}\n> > +\n> > +\treturn true;\n> > +}\n> > +\n> > +V4L2Subdevice *SimplePipelineHandler::subdev(const MediaEntity *entity)\n> > +{\n> > +\tauto iter = subdevs_.find(entity);\n> > +\tif (iter == subdevs_.end())\n> > +\t\treturn nullptr;\n> > +\n> > +\treturn &iter->second;\n> > +}\n> > +\n> > +/* -----------------------------------------------------------------------------\n> > + * Buffer Handling\n> > + */\n> > +\n> > +void SimplePipelineHandler::bufferReady(FrameBuffer *buffer)\n> > +{\n> > +\tASSERT(activeCamera_);\n> > +\tRequest *request = buffer->request();\n> > +\tcompleteBuffer(activeCamera_, request, buffer);\n> > +\tcompleteRequest(activeCamera_, request);\n> > +}\n> > +\n> > +REGISTER_PIPELINE_HANDLER(SimplePipelineHandler);\n> > +\n> > +} /* namespace libcamera */","headers":{"Return-Path":"","Received":["from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 19D2E605D1\n\tfor ;\n\tSat, 4 Apr 2020 02:08:35 +0200 (CEST)","from pendragon.ideasonboard.com (81-175-216-236.bb.dnainternet.fi\n\t[81.175.216.236])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 55D87321;\n\tSat, 4 Apr 2020 02:08:34 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ideasonboard.com\n\theader.i=@ideasonboard.com\n\theader.b=\"RIRhmYmf\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1585958914;\n\tbh=UvSTWNEdKfBYO2JFeqhEbJFp7iWgDB6aSQS3BMJJLW4=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=RIRhmYmf5xmfPsqoAlPoMQ6GNhEAy0UmtC2urTA9YXWp+O3yV/vb6Kpf+2tHVscLD\n\tGa4w3y1ddFr30/KJpx+yaPY4otAz11KSMkeMeF67U1Bz+jcGaTGd9++ABpgwo0RJCu\n\ti4xGbpex+BGcr+p0UpiPlC6fbmOGbChsww7BBaew=","Date":"Sat, 4 Apr 2020 03:08:25 +0300","From":"Laurent Pinchart ","To":"Andrey Konovalov ","Cc":"libcamera-devel@lists.libcamera.org, Martijn Braam , \n\tBenjamin GAIGNARD ","Message-ID":"<20200404000825.GE9690@pendragon.ideasonboard.com>","References":"<20200320014839.14683-1-laurent.pinchart@ideasonboard.com>\n\t<20200320014839.14683-10-laurent.pinchart@ideasonboard.com>\n\t<053e8bb2-1d50-1aa5-b09a-fdb6718e3a41@linaro.org>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<053e8bb2-1d50-1aa5-b09a-fdb6718e3a41@linaro.org>","Subject":"Re: [libcamera-devel] [PATCH v3 09/11] libcamera: pipeline: Add a\n\tsimple pipeline handler","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","X-List-Received-Date":"Sat, 04 Apr 2020 00:08:35 -0000"}},{"id":4380,"web_url":"https://patchwork.libcamera.org/comment/4380/","msgid":"<20200404001202.GF9690@pendragon.ideasonboard.com>","date":"2020-04-04T00:12:02","subject":"Re: [libcamera-devel] [PATCH v3 09/11] libcamera: pipeline: Add a\n\tsimple pipeline handler","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Andrey,\n\nOn Fri, Apr 03, 2020 at 04:13:35PM +0300, Andrey Konovalov wrote:\n> On 31.03.2020 14:45, Kieran Bingham wrote:\n> > On 20/03/2020 01:48, Laurent Pinchart wrote:\n> >> From: Martijn Braam \n> >>\n> >> This new pipeline handler aims at supporting any simple device without\n> >> requiring any device-specific code. Simple devices are currently defined\n> >> as a graph made of one or multiple camera sensors and a single video\n> >> node, with each sensor connected to the video node through a linear\n> >> pipeline.\n> >>\n> >> The simple pipeline handler will automatically parse the media graph,\n> >> enumerate sensors, build supported stream configurations, and configure\n> >> the pipeline, without any device-specific knowledge. It doesn't support\n> >> configuration of any processing in the pipeline at the moment, but may\n> >> be extended to support simple processing such as format conversion or\n> >> scaling in the future.\n> >>\n> >> The only device-specific information in the pipeline handler is the list\n> >> of supported drivers, required for device matching. We may be able to\n> >> remove this in the future by matching with the simple pipeline handler\n> >> as a last resort option, after all other pipeline handlers have been\n> >> tried.\n> >>\n> >> Signed-off-by: Martijn Braam \n> >> Signed-off-by: Laurent Pinchart \n> > \n> > Reviewed-by: Kieran Bingham \n> > \n> > Some comments and discussion points below, but I think we should work\n> > towards trying to get this integrated so it can actually be used (which\n> > is where we'll find the nuances and corner cases as people test it)\n> > \n> >> ---\n> >> Changes since v2:\n> >>\n> >> - Log an error when setupFormats() fail\n> >> - Propagate getFormat() and setFormat() errors to the caller of\n> >> setupFormats()\n> >> - Reorder variable declarations in validate()\n> >> - Add \\todo comment related to the selection of the default format\n> >> - Use log Error instead of Info if pipeline isn't valid\n> >> - Rebase on top of V4L2PixelFormat\n> >>\n> >> Changes since v1:\n> >>\n> >> - Rebase on top of buffer API rework\n> >> - Expose stream formats\n> >> - Rework camera data config\n> >> ---\n> >> src/libcamera/pipeline/meson.build | 1 +\n> >> src/libcamera/pipeline/simple/meson.build | 3 +\n> >> src/libcamera/pipeline/simple/simple.cpp | 710 ++++++++++++++++++++++\n> >> 3 files changed, 714 insertions(+)\n> >> create mode 100644 src/libcamera/pipeline/simple/meson.build\n> >> create mode 100644 src/libcamera/pipeline/simple/simple.cpp\n> >>\n> >> diff --git a/src/libcamera/pipeline/meson.build b/src/libcamera/pipeline/meson.build\n> >> index 0d466225a72e..606ba31a0319 100644\n> >> --- a/src/libcamera/pipeline/meson.build\n> >> +++ b/src/libcamera/pipeline/meson.build\n> >> @@ -5,3 +5,4 @@ libcamera_sources += files([\n> >> \n> >> subdir('ipu3')\n> >> subdir('rkisp1')\n> >> +subdir('simple')\n> >> diff --git a/src/libcamera/pipeline/simple/meson.build b/src/libcamera/pipeline/simple/meson.build\n> >> new file mode 100644\n> >> index 000000000000..4945a3e173cf\n> >> --- /dev/null\n> >> +++ b/src/libcamera/pipeline/simple/meson.build\n> >> @@ -0,0 +1,3 @@\n> >> +libcamera_sources += files([\n> >> + 'simple.cpp',\n> >> +])\n> >> diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp\n> >> new file mode 100644\n> \n> \n> \n> >> +int SimpleCameraData::init()\n> >> +{\n> >> +\tSimplePipelineHandler *pipe = static_cast(pipe_);\n> >> +\tV4L2VideoDevice *video = pipe->video();\n> >> +\tint ret;\n> >> +\n> >> +\t/*\n> >> +\t * Enumerate the possible pipeline configurations. For each media bus\n> >> +\t * format supported by the sensor, propagate the formats through the\n> >> +\t * pipeline, and enumerate the corresponding possible V4L2 pixel\n> >> +\t * formats on the video node.\n> >> +\t */\n> >> +\tfor (unsigned int code : sensor_->mbusCodes()) {\n> >> +\t\tV4L2SubdeviceFormat format{ code, sensor_->resolution() };\n> >> +\n> >> +\t\t/*\n> >> +\t\t * Setup links first as some subdev drivers take active links\n> >> +\t\t * into account to propaget TRY formats. So is life :-(\n> > \n> > /propaget/propagate/\n> > \n> > /So is life/Such is life/\n> > \n> > \n> >> +\t\t */\n> >> +\t\tret = setupLinks();\n> >> +\t\tif (ret < 0)\n> >> +\t\t\treturn ret;\n> >> +\n> >> +\t\tret = setupFormats(&format, V4L2Subdevice::TryFormat);\n> >> +\t\tif (ret < 0) {\n> >> +\t\t\tLOG(SimplePipeline, Error)\n> >> +\t\t\t\t<< \"Failed to setup pipeline for media bus code \"\n> >> +\t\t\t\t<< utils::hex(code, 4);\n> > \n> > Oh dear, seems we need mbusCode.toString() too! hehe\n> > \n> >> +\t\t\treturn ret;\n> > \n> > Andrey states that this return prevents iterating potentially successful\n> > mbus-codes, and should instead 'continue'...\n> \n> I've taken a deeper look at the ov5640 + db410c logs I've received, and the above\n> return *could* be a potential problem, but definitely not in this (ov5640 + db410c)\n> case.\n> \n> We only saw SimpleCameraData::setupFormats() returning error due to failing\n> sensor_->setFormat(format) call.\n> This happened if before the cam is started (and the stream cfg isn't set to a\n> particular width*height) the sensor is configured for the frame rate greater than 15 fps.\n> In particular, in the ov5640's driver probe function the initial mode is set to\n> \"YUV422 UYVY VGA@30fps\".\n> ov5640_enum_frame_size() doesn't consider the current fps at all, so the maximum\n> frame size reported is 2592x1944. But it only works at 15fps max, and an attempt\n> to set the sensor format to a one with the maximum possible frame size of 2592x1944\n> results in -EINVAL returned by SUBDEV_S_FMT.\n> (So in our ov5640 + db410c case replacing the \"return\" with \"continue\" does let the pipeline\n> handler to try all the other media bus codes if the first one fails, but they all fail the\n> same way as the first one does.)\n> \n> If this issue with the frame rate is worked around in some way, the enumeration\n> of the possible pipeline configurations completes kind of OK: a good configuration\n> is selected in the end. But during the enumeration all the configurations, the\n> broken ones included are considered as the working ones:\n> \n> The links are set up this way (\"entity name\":pad_number]):\n> \n> \"ov5640 4-0076\":0 -> \"msm_csiphy0\":0,\"msm_csiphy0\":1 -> \"msm_csid0\":0,\"msm_csid0\":1 ->\n> \t-> \"msm_ispif0\":0,\"msm_ispif0\":1 -> \"msm_vfe0_rdi0\":0\n> \n> \n> SimpleCameraData::setupFormats() creates a broken pipeline when it tries the sensor format\n> not supported by the rest of the entities. This results in e.g. the following:\n> \n> \"ov5640 4-0076\":0[MEDIA_BUS_FMT_RGB565_2X8_BE] -> [MEDIA_BUS_FMT_UYVY8_2X8]\"msm_csiphy0\":0,\n> \"msm_csiphy0\":1[MEDIA_BUS_FMT_UYVY8_2X8] -> [MEDIA_BUS_FMT_UYVY8_2X8]\"msm_csid0\":0, etc\n> \n> I.e. an attempt to set fmt on \"msm_csiphy0\":0 to MEDIA_BUS_FMT_RGB565_2X8_BE results in the\n> format set to MEDIA_BUS_FMT_UYVY8_2X8 and no error is returned here (and down the pipeline\n> as MEDIA_BUS_FMT_UYVY8_2X8 is set OK for all the other pads). So we get the first link misconfigured,\n> but the result is considered as a good pipeline as no error was returned while the format\n> from the sensor output was propagated from the sensor output down to \"msm_vfe0_rdi0\":0.\n> And the broken configuration is stored in the formats_ map (in the end, it works in our\n> ov5640+db410c case as the broken configurations are rewritten with the valid ones later).\n> \n> Who/what is to ensure that for each of the links, the source and the sink pads of the same link\n> are configured to the same format?\n\nThe kernel is responsible for validating that at stream on time, but I\nthink we should also verify it in setupFormats(). Would you like to send\na patch on top of this series ?\n\n> >> +\t\t}\n> >> +\n> >> +\t\tstd::map> videoFormats =\n> >> +\t\t\tvideo->formats(format.mbus_code);\n> >> +\n> >> +\t\tLOG(SimplePipeline, Debug)\n> >> +\t\t\t<< \"Adding configuration for \" << format.size.toString()\n> >> +\t\t\t<< \" in pixel formats [ \"\n> >> +\t\t\t<< utils::join(videoFormats, \", \",\n> >> +\t\t\t\t [](const auto &f) {\n> >> +\t\t\t\t\t return f.first.toString();\n> >> +\t\t\t\t })\n> > \n> > Aha, I kinda like the utils::join!\n> > \n> >> +\t\t\t<< \" ]\";\n> >> +\n> >> +\t\t/*\n> >> +\t\t * Store the configuration in the formats_ map, mapping\n> >> +\t\t * PixelFormat to configuration. Any previously stored value is\n> > \n> > /to configuration/to the current configuration/ ?\n> > \n> >> +\t\t * overwritten, as the pipeline handler currently doesn't care\n> >> +\t\t * about how a particular PixelFormat is achieved.\n> > \n> > As long as there's an output, I agree. If devices have specific\n> > constraints such as better performance or such with a particular\n> > mbus-code, then they need to create a platform specific pipeline handler\n> > to deal with the nuances.\n> > \n> > Maybe there might be tuning hints or something later, but certainly not now.\n> > \n> > \n> >> +\t\t */\n> >> +\t\tfor (const auto &videoFormat : videoFormats) {\n> >> +\t\t\tPixelFormat pixelFormat = video->toPixelFormat(videoFormat.first);\n> >> +\t\t\tif (!pixelFormat)\n> >> +\t\t\t\tcontinue;\n> >> +\n> >> +\t\t\tConfiguration config;\n> >> +\t\t\tconfig.code = code;\n> >> +\t\t\tconfig.pixelFormat = pixelFormat;\n> >> +\t\t\tconfig.size = format.size;\n> >> +\n> >> +\t\t\tformats_[pixelFormat] = config;\n> >> +\t\t}\n> >> +\t}\n> >> +\n> >> +\tif (formats_.empty()) {\n> >> +\t\tLOG(SimplePipeline, Error) << \"No valid configuration found\";\n> >> +\t\treturn -EINVAL;\n> >> +\t}\n> >> +\n> >> +\treturn 0;\n> >> +}\n\n[snip]","headers":{"Return-Path":"","Received":["from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id C4F16605D1\n\tfor ;\n\tSat, 4 Apr 2020 02:12:11 +0200 (CEST)","from pendragon.ideasonboard.com (81-175-216-236.bb.dnainternet.fi\n\t[81.175.216.236])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 087D9321;\n\tSat, 4 Apr 2020 02:12:10 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ideasonboard.com\n\theader.i=@ideasonboard.com\n\theader.b=\"fbXpqaiS\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1585959131;\n\tbh=wcYjwZnf1lSx5mGc5Rs/SJRD3zgQ14meYjJs27ryxkk=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=fbXpqaiSH0uh1PODyFAsM/dglRmsImw62DzYug43Gv56TBAzT/tQOcL7hpJBSXbIn\n\tNcG6/G9WzEyc6EPZYyPMc/2vEkTw4cG2VgAzOedCjPaQDrTbeNF2X/Dh7oW3UbTwo0\n\tuGU5c8dVHoPtIF/0eitXW22q6v8Ci7oPNaCzE7rY=","Date":"Sat, 4 Apr 2020 03:12:02 +0300","From":"Laurent Pinchart ","To":"Andrey Konovalov ","Cc":"kieran.bingham@ideasonboard.com, libcamera-devel@lists.libcamera.org,\n\tMartijn Braam ,\n\tBenjamin GAIGNARD ","Message-ID":"<20200404001202.GF9690@pendragon.ideasonboard.com>","References":"<20200320014839.14683-1-laurent.pinchart@ideasonboard.com>\n\t<20200320014839.14683-10-laurent.pinchart@ideasonboard.com>\n\t<43484c6b-44ac-4bd7-88e4-b46070840315@ideasonboard.com>\n\t<0519333c-3d17-325a-704b-664e475f556c@linaro.org>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<0519333c-3d17-325a-704b-664e475f556c@linaro.org>","Subject":"Re: [libcamera-devel] [PATCH v3 09/11] libcamera: pipeline: Add a\n\tsimple pipeline handler","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","X-List-Received-Date":"Sat, 04 Apr 2020 00:12:12 -0000"}},{"id":4426,"web_url":"https://patchwork.libcamera.org/comment/4426/","msgid":"<2e0132f7-d6c1-4434-ec91-40cd7ef76ffb@linaro.org>","date":"2020-04-08T17:02:53","subject":"Re: [libcamera-devel] [PATCH v3 09/11] libcamera: pipeline: Add a\n\tsimple pipeline handler","submitter":{"id":25,"url":"https://patchwork.libcamera.org/api/people/25/","name":"Andrey Konovalov","email":"andrey.konovalov@linaro.org"},"content":"Hi Laurent,\n\nOn 04.04.2020 03:08, Laurent Pinchart wrote:\n> Hi Andrey,\n> \n> On Fri, Apr 03, 2020 at 04:55:32PM +0300, Andrey Konovalov wrote:\n>> On 20.03.2020 04:48, Laurent Pinchart wrote:\n>>> From: Martijn Braam \n>>>\n>>> This new pipeline handler aims at supporting any simple device without\n>>> requiring any device-specific code. Simple devices are currently defined\n>>> as a graph made of one or multiple camera sensors and a single video\n>>> node, with each sensor connected to the video node through a linear\n>>> pipeline.\n>>>\n>>> The simple pipeline handler will automatically parse the media graph,\n>>> enumerate sensors, build supported stream configurations, and configure\n>>> the pipeline, without any device-specific knowledge. It doesn't support\n>>> configuration of any processing in the pipeline at the moment, but may\n>>> be extended to support simple processing such as format conversion or\n>>> scaling in the future.\n>>>\n>>> The only device-specific information in the pipeline handler is the list\n>>> of supported drivers, required for device matching. We may be able to\n>>> remove this in the future by matching with the simple pipeline handler\n>>> as a last resort option, after all other pipeline handlers have been\n>>> tried.\n>>>\n>>> Signed-off-by: Martijn Braam \n>>> Signed-off-by: Laurent Pinchart \n>>> ---\n>>> Changes since v2:\n>>>\n>>> - Log an error when setupFormats() fail\n>>> - Propagate getFormat() and setFormat() errors to the caller of\n>>> setupFormats()\n>>> - Reorder variable declarations in validate()\n>>> - Add \\todo comment related to the selection of the default format\n>>> - Use log Error instead of Info if pipeline isn't valid\n>>> - Rebase on top of V4L2PixelFormat\n>>>\n>>> Changes since v1:\n>>>\n>>> - Rebase on top of buffer API rework\n>>> - Expose stream formats\n>>> - Rework camera data config\n>>> ---\n>>> src/libcamera/pipeline/meson.build | 1 +\n>>> src/libcamera/pipeline/simple/meson.build | 3 +\n>>> src/libcamera/pipeline/simple/simple.cpp | 710 ++++++++++++++++++++++\n>>> 3 files changed, 714 insertions(+)\n>>> create mode 100644 src/libcamera/pipeline/simple/meson.build\n>>> create mode 100644 src/libcamera/pipeline/simple/simple.cpp\n>>>\n>>> diff --git a/src/libcamera/pipeline/meson.build b/src/libcamera/pipeline/meson.build\n>>> index 0d466225a72e..606ba31a0319 100644\n>>> --- a/src/libcamera/pipeline/meson.build\n>>> +++ b/src/libcamera/pipeline/meson.build\n>>> @@ -5,3 +5,4 @@ libcamera_sources += files([\n>>> \n>>> subdir('ipu3')\n>>> subdir('rkisp1')\n>>> +subdir('simple')\n>>> diff --git a/src/libcamera/pipeline/simple/meson.build b/src/libcamera/pipeline/simple/meson.build\n>>> new file mode 100644\n>>> index 000000000000..4945a3e173cf\n>>> --- /dev/null\n>>> +++ b/src/libcamera/pipeline/simple/meson.build\n>>> @@ -0,0 +1,3 @@\n>>> +libcamera_sources += files([\n>>> + 'simple.cpp',\n>>> +])\n>>> diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp\n>>> new file mode 100644\n>>> index 000000000000..545a99fe31ca\n>>> --- /dev/null\n>>> +++ b/src/libcamera/pipeline/simple/simple.cpp\n>>> @@ -0,0 +1,710 @@\n>>> +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n>>> +/*\n>>> + * Copyright (C) 2020, Laurent Pinchart\n>>> + * Copyright (C) 2019, Martijn Braam\n>>> + *\n>>> + * simple.cpp - Pipeline handler for simple pipelines\n>>> + */\n>>> +\n>>> +#include \n>>> +#include \n>>> +#include \n>>> +#include \n>>> +#include \n>>> +#include \n>>> +#include \n>>> +#include \n>>> +#include \n>>> +#include \n>>> +\n>>> +#include \n>>> +\n>>> +#include \n>>> +#include \n>>> +#include \n>>> +\n>>> +#include \"camera_sensor.h\"\n>>> +#include \"device_enumerator.h\"\n>>> +#include \"log.h\"\n>>> +#include \"media_device.h\"\n>>> +#include \"pipeline_handler.h\"\n>>> +#include \"v4l2_subdevice.h\"\n>>> +#include \"v4l2_videodevice.h\"\n>>> +\n>>> +namespace libcamera {\n>>> +\n>>> +LOG_DEFINE_CATEGORY(SimplePipeline)\n>>> +\n>>> +class SimplePipelineHandler;\n>>> +\n>>> +class SimpleCameraData : public CameraData\n>>> +{\n>>> +public:\n>>> +\tSimpleCameraData(PipelineHandler *pipe, MediaEntity *sensor,\n>>> +\t\t\t MediaEntity *video);\n>>> +\n>>> +\tbool isValid() const { return sensor_ != nullptr; }\n>>> +\tstd::set streams() { return { &stream_ }; }\n>>> +\n>>> +\tint init();\n>>> +\tint setupLinks();\n>>> +\tint setupFormats(V4L2SubdeviceFormat *format,\n>>> +\t\t\t V4L2Subdevice::Whence whence);\n>>> +\n>>> +\tstruct Entity {\n>>> +\t\tMediaEntity *entity;\n>>> +\t\tMediaLink *link;\n>>> +\t};\n>>> +\n>>> +\tstruct Configuration {\n>>> +\t\tuint32_t code;\n>>> +\t\tPixelFormat pixelFormat;\n>>> +\t\tSize size;\n>>> +\t};\n>>> +\n>>> +\tStream stream_;\n>>> +\tstd::unique_ptr sensor_;\n>>> +\tstd::list entities_;\n>>> +\n>>> +\tstd::vector configs_;\n>>> +\tstd::map formats_;\n>>> +};\n>>> +\n>>> +class SimpleCameraConfiguration : public CameraConfiguration\n>>> +{\n>>> +public:\n>>> +\tSimpleCameraConfiguration(Camera *camera, SimpleCameraData *data);\n>>> +\n>>> +\tStatus validate() override;\n>>> +\n>>> +\tconst V4L2SubdeviceFormat &sensorFormat() { return sensorFormat_; }\n>>> +\n>>> +private:\n>>> +\t/*\n>>> +\t * The SimpleCameraData instance is guaranteed to be valid as long as\n>>> +\t * the corresponding Camera instance is valid. In order to borrow a\n>>> +\t * reference to the camera data, store a new reference to the camera.\n>>> +\t */\n>>> +\tstd::shared_ptr camera_;\n>>> +\tconst SimpleCameraData *data_;\n>>> +\n>>> +\tV4L2SubdeviceFormat sensorFormat_;\n>>> +};\n>>> +\n>>> +class SimplePipelineHandler : public PipelineHandler\n>>> +{\n>>> +public:\n>>> +\tSimplePipelineHandler(CameraManager *manager);\n>>> +\t~SimplePipelineHandler();\n>>> +\n>>> +\tCameraConfiguration *generateConfiguration(Camera *camera,\n>>> +\t\t\t\t\t\t const StreamRoles &roles) override;\n>>> +\tint configure(Camera *camera, CameraConfiguration *config) override;\n>>> +\n>>> +\tint exportFrameBuffers(Camera *camera, Stream *stream,\n>>> +\t\t\t std::vector> *buffers) override;\n>>> +\n>>> +\tint start(Camera *camera) override;\n>>> +\tvoid stop(Camera *camera) override;\n>>> +\n>>> +\tbool match(DeviceEnumerator *enumerator) override;\n>>> +\n>>> +\tV4L2VideoDevice *video() { return video_; }\n>>> +\tV4L2Subdevice *subdev(const MediaEntity *entity);\n>>> +\n>>> +protected:\n>>> +\tint queueRequestDevice(Camera *camera, Request *request) override;\n>>> +\n>>> +private:\n>>> +\tSimpleCameraData *cameraData(const Camera *camera)\n>>> +\t{\n>>> +\t\treturn static_cast(\n>>> +\t\t\tPipelineHandler::cameraData(camera));\n>>> +\t}\n>>> +\n>>> +\tint initLinks();\n>>> +\n>>> +\tint createCamera(MediaEntity *sensor);\n>>> +\n>>> +\tvoid bufferReady(FrameBuffer *buffer);\n>>> +\n>>> +\tMediaDevice *media_;\n>>> +\tV4L2VideoDevice *video_;\n>>> +\tstd::map subdevs_;\n>>> +\n>>> +\tCamera *activeCamera_;\n>>> +};\n>>> +\n>>> +/* -----------------------------------------------------------------------------\n>>> + * Camera Data\n>>> + */\n>>> +SimpleCameraData::SimpleCameraData(PipelineHandler *pipe, MediaEntity *sensor,\n>>> +\t\t\t\t MediaEntity *video)\n>>> +\t: CameraData(pipe)\n>>> +{\n>>> +\tint ret;\n>>> +\n>>> +\t/*\n>>> +\t * Walk the pipeline towards the video node and store all entities\n>>> +\t * along the way.\n>>> +\t */\n>>> +\tMediaEntity *source = sensor;\n>>> +\n>>> +\twhile (source) {\n>>> +\t\t/* If we have reached the video node, we're done. */\n>>> +\t\tif (source == video)\n>>> +\t\t\tbreak;\n>>> +\n>>> +\t\t/* Use the first output pad that has links. */\n>>> +\t\tMediaPad *sourcePad = nullptr;\n>>> +\t\tfor (MediaPad *pad : source->pads()) {\n>>> +\t\t\tif ((pad->flags() & MEDIA_PAD_FL_SOURCE) &&\n>>> +\t\t\t !pad->links().empty()) {\n>>> +\t\t\t\tsourcePad = pad;\n>>> +\t\t\t\tbreak;\n>>> +\t\t\t}\n>>> +\t\t}\n>>> +\n>>> +\t\tif (!sourcePad)\n>>> +\t\t\treturn;\n>>> +\n>>> +\t\t/* Use the first link that isn't immutable and disabled. */\n>>> +\t\tMediaLink *sourceLink = nullptr;\n>>> +\t\tfor (MediaLink *link : sourcePad->links()) {\n>>> +\t\t\tif ((link->flags() & MEDIA_LNK_FL_ENABLED) ||\n>>> +\t\t\t !(link->flags() & MEDIA_LNK_FL_IMMUTABLE)) {\n>>> +\t\t\t\tsourceLink = link;\n>>> +\t\t\t\tbreak;\n>>> +\t\t\t}\n>>> +\t\t}\n>>> +\n>>> +\t\tif (!sourceLink)\n>>> +\t\t\treturn;\n>>> +\n>>> +\t\tentities_.push_back({ source, sourceLink });\n>>> +\n>>> +\t\tsource = sourceLink->sink()->entity();\n>>> +\n>>> +\t\t/* Avoid infinite loops. */\n>>> +\t\tauto iter = std::find_if(entities_.begin(), entities_.end(),\n>>> +\t\t\t\t\t [&](const Entity &entity) {\n>>> +\t\t\t\t\t\t return entity.entity == source;\n>>> +\t\t\t\t\t });\n>>> +\t\tif (iter != entities_.end()) {\n>>> +\t\t\tLOG(SimplePipeline, Info) << \"Loop detected in pipeline\";\n>>> +\t\t\treturn;\n>>> +\t\t}\n>>> +\t}\n>>> +\n>>> +\t/* We have a valid pipeline, create the camera sensor. */\n>>> +\tsensor_ = std::make_unique(sensor);\n>>> +\tret = sensor_->init();\n>>> +\tif (ret) {\n>>> +\t\tsensor_.reset();\n>>> +\t\treturn;\n>>> +\t}\n>>> +}\n>>> +\n>>> +int SimpleCameraData::init()\n>>> +{\n>>> +\tSimplePipelineHandler *pipe = static_cast(pipe_);\n>>> +\tV4L2VideoDevice *video = pipe->video();\n>>> +\tint ret;\n>>> +\n>>> +\t/*\n>>> +\t * Enumerate the possible pipeline configurations. For each media bus\n>>> +\t * format supported by the sensor, propagate the formats through the\n>>> +\t * pipeline, and enumerate the corresponding possible V4L2 pixel\n>>> +\t * formats on the video node.\n>>> +\t */\n>>> +\tfor (unsigned int code : sensor_->mbusCodes()) {\n>>> +\t\tV4L2SubdeviceFormat format{ code, sensor_->resolution() };\n>>> +\n>>> +\t\t/*\n>>> +\t\t * Setup links first as some subdev drivers take active links\n>>> +\t\t * into account to propaget TRY formats. So is life :-(\n>>> +\t\t */\n>>> +\t\tret = setupLinks();\n>>> +\t\tif (ret < 0)\n>>> +\t\t\treturn ret;\n>>> +\n>>> +\t\tret = setupFormats(&format, V4L2Subdevice::TryFormat);\n>>> +\t\tif (ret < 0) {\n>>> +\t\t\tLOG(SimplePipeline, Error)\n>>> +\t\t\t\t<< \"Failed to setup pipeline for media bus code \"\n>>> +\t\t\t\t<< utils::hex(code, 4);\n>>> +\t\t\treturn ret;\n>>> +\t\t}\n>>> +\n>>> +\t\tstd::map> videoFormats =\n>>> +\t\t\tvideo->formats(format.mbus_code);\n>>> +\n>>> +\t\tLOG(SimplePipeline, Debug)\n>>> +\t\t\t<< \"Adding configuration for \" << format.size.toString()\n>>> +\t\t\t<< \" in pixel formats [ \"\n>>> +\t\t\t<< utils::join(videoFormats, \", \",\n>>> +\t\t\t\t [](const auto &f) {\n>>> +\t\t\t\t\t return f.first.toString();\n>>> +\t\t\t\t })\n>>> +\t\t\t<< \" ]\";\n>>> +\n>>> +\t\t/*\n>>> +\t\t * Store the configuration in the formats_ map, mapping\n>>> +\t\t * PixelFormat to configuration. Any previously stored value is\n>>> +\t\t * overwritten, as the pipeline handler currently doesn't care\n>>> +\t\t * about how a particular PixelFormat is achieved.\n>>> +\t\t */\n>>> +\t\tfor (const auto &videoFormat : videoFormats) {\n>>> +\t\t\tPixelFormat pixelFormat = video->toPixelFormat(videoFormat.first);\n>>> +\t\t\tif (!pixelFormat)\n>>> +\t\t\t\tcontinue;\n>>> +\n>>> +\t\t\tConfiguration config;\n>>> +\t\t\tconfig.code = code;\n>>> +\t\t\tconfig.pixelFormat = pixelFormat;\n>>> +\t\t\tconfig.size = format.size;\n>>> +\n>>> +\t\t\tformats_[pixelFormat] = config;\n>>> +\t\t}\n>>> +\t}\n>>> +\n>>> +\tif (formats_.empty()) {\n>>> +\t\tLOG(SimplePipeline, Error) << \"No valid configuration found\";\n>>> +\t\treturn -EINVAL;\n>>> +\t}\n>>> +\n>>> +\treturn 0;\n>>> +}\n>>> +\n>>> +int SimpleCameraData::setupLinks()\n>>> +{\n>>> +\tint ret;\n>>> +\n>>> +\t/*\n>>> +\t * Configure all links along the pipeline. Some entities may not allow\n>>> +\t * multiple sink links to be enabled together, even on different sink\n>>> +\t * pads. We must thus start by disabling all sink links (but the one we\n>>> +\t * want to enable) before enabling the pipeline link.\n>>> +\t */\n>>> +\tfor (SimpleCameraData::Entity &e : entities_) {\n>>> +\t\tfor (MediaPad *pad : e.link->sink()->entity()->pads()) {\n>>> +\t\t\tfor (MediaLink *link : pad->links()) {\n>>> +\t\t\t\tif (link == e.link)\n>>> +\t\t\t\t\tcontinue;\n>>> +\n>>> +\t\t\t\tif ((link->flags() & MEDIA_LNK_FL_ENABLED) &&\n>>> +\t\t\t\t !(link->flags() & MEDIA_LNK_FL_IMMUTABLE)) {\n>>> +\t\t\t\t\tret = link->setEnabled(false);\n>>> +\t\t\t\t\tif (ret < 0)\n>>> +\t\t\t\t\t\treturn ret;\n>>> +\t\t\t\t}\n>>> +\t\t\t}\n>>> +\t\t}\n>>> +\n>>> +\t\tif (!(e.link->flags() & MEDIA_LNK_FL_ENABLED)) {\n>>> +\t\t\tret = e.link->setEnabled(true);\n>>> +\t\t\tif (ret < 0)\n>>> +\t\t\t\treturn ret;\n>>> +\t\t}\n>>> +\t}\n>>> +\n>>> +\treturn 0;\n>>> +}\n>>> +\n>>> +int SimpleCameraData::setupFormats(V4L2SubdeviceFormat *format,\n>>> +\t\t\t\t V4L2Subdevice::Whence whence)\n>>> +{\n>>> +\tSimplePipelineHandler *pipe = static_cast(pipe_);\n>>> +\tint ret;\n>>> +\n>>> +\t/*\n>>> +\t * Configure the format on the sensor output and propagate it through\n>>> +\t * the pipeline.\n>>> +\t */\n>>> +\tret = sensor_->setFormat(format);\n>>> +\tif (ret < 0)\n>>> +\t\treturn ret;\n>>> +\n>>> +\tfor (const Entity &e : entities_) {\n>>> +\t\tMediaLink *link = e.link;\n>>> +\t\tMediaPad *source = link->source();\n>>> +\t\tMediaPad *sink = link->sink();\n>>> +\n>>> +\t\tif (source->entity() != sensor_->entity()) {\n>>> +\t\t\tV4L2Subdevice *subdev = pipe->subdev(source->entity());\n>>> +\t\t\tret = subdev->getFormat(source->index(), format, whence);\n>>> +\t\t\tif (ret < 0)\n>>> +\t\t\t\treturn ret;\n>>> +\t\t}\n>>> +\n>>> +\t\tif (sink->entity()->function() != MEDIA_ENT_F_IO_V4L) {\n>>> +\t\t\tV4L2Subdevice *subdev = pipe->subdev(sink->entity());\n>>> +\t\t\tret = subdev->setFormat(sink->index(), format, whence);\n>>> +\t\t\tif (ret < 0)\n>>> +\t\t\t\treturn ret;\n>>> +\t\t}\n>>> +\n>>> +\t\tLOG(SimplePipeline, Debug)\n>>> +\t\t\t<< \"Link '\" << source->entity()->name()\n>>> +\t\t\t<< \"':\" << source->index()\n>>> +\t\t\t<< \" -> '\" << sink->entity()->name()\n>>> +\t\t\t<< \"':\" << sink->index()\n>>> +\t\t\t<< \" configured with format \" << format->toString();\n>>> +\t}\n>>> +\n>>> +\treturn 0;\n>>> +}\n>>> +\n>>> +/* -----------------------------------------------------------------------------\n>>> + * Camera Configuration\n>>> + */\n>>> +\n>>> +SimpleCameraConfiguration::SimpleCameraConfiguration(Camera *camera,\n>>> +\t\t\t\t\t\t SimpleCameraData *data)\n>>> +\t: CameraConfiguration(), camera_(camera->shared_from_this()),\n>>> +\t data_(data)\n>>> +{\n>>> +}\n>>> +\n>>> +CameraConfiguration::Status SimpleCameraConfiguration::validate()\n>>> +{\n>>> +\tStatus status = Valid;\n>>> +\n>>> +\tif (config_.empty())\n>>> +\t\treturn Invalid;\n>>> +\n>>> +\t/* Cap the number of entries to the available streams. */\n>>> +\tif (config_.size() > 1) {\n>>> +\t\tconfig_.resize(1);\n>>> +\t\tstatus = Adjusted;\n>>> +\t}\n>>> +\n>>> +\tStreamConfiguration &cfg = config_[0];\n>>> +\n>>> +\t/* Adjust the pixel format. */\n>>> +\tauto it = data_->formats_.find(cfg.pixelFormat);\n>>> +\tif (it == data_->formats_.end())\n>>> +\t\tit = data_->formats_.begin();\n>>> +\n>>> +\tPixelFormat pixelFormat = it->first;\n>>> +\tif (cfg.pixelFormat != pixelFormat) {\n>>> +\t\tLOG(SimplePipeline, Debug) << \"Adjusting pixel format\";\n>>> +\t\tcfg.pixelFormat = pixelFormat;\n>>> +\t\tstatus = Adjusted;\n>>> +\t}\n>>> +\n>>> +\tconst SimpleCameraData::Configuration &pipeConfig = it->second;\n>>> +\tif (cfg.size != pipeConfig.size) {\n>>> +\t\tLOG(SimplePipeline, Debug)\n>>> +\t\t\t<< \"Adjusting size from \" << cfg.size.toString()\n>>> +\t\t\t<< \" to \" << pipeConfig.size.toString();\n>>> +\t\tcfg.size = pipeConfig.size;\n>>> +\t\tstatus = Adjusted;\n>>> +\t}\n>>> +\n>>> +\tcfg.bufferCount = 3;\n>>> +\n>>> +\treturn status;\n>>> +}\n>>> +\n>>> +/* -----------------------------------------------------------------------------\n>>> + * Pipeline Handler\n>>> + */\n>>> +\n>>> +SimplePipelineHandler::SimplePipelineHandler(CameraManager *manager)\n>>> +\t: PipelineHandler(manager), video_(nullptr)\n>>> +{\n>>> +}\n>>> +\n>>> +SimplePipelineHandler::~SimplePipelineHandler()\n>>> +{\n>>> +\tdelete video_;\n>>> +}\n>>> +\n>>> +CameraConfiguration *SimplePipelineHandler::generateConfiguration(Camera *camera,\n>>> +\t\t\t\t\t\t\t\t const StreamRoles &roles)\n>>> +{\n>>> +\tSimpleCameraData *data = cameraData(camera);\n>>> +\tCameraConfiguration *config =\n>>> +\t\tnew SimpleCameraConfiguration(camera, data);\n>>> +\n>>> +\tif (roles.empty())\n>>> +\t\treturn config;\n>>> +\n>>> +\t/* Create the formats map. */\n>>> +\tstd::map> formats;\n>>> +\tstd::transform(data->formats_.begin(), data->formats_.end(),\n>>> +\t\t std::inserter(formats, formats.end()),\n>>> +\t\t [](const auto &format) -> decltype(formats)::value_type {\n>>> +\t\t\t const PixelFormat &pixelFormat = format.first;\n>>> +\t\t\t const Size &size = format.second.size;\n>>> +\t\t\t return { pixelFormat, { size } };\n>>> +\t\t });\n>>> +\n>>> +\t/*\n>>> +\t * Create the stream configuration. Take the first entry in the formats\n>>> +\t * map as the default, for lack of a better option.\n>>> +\t *\n>>> +\t * \\todo Implement a better way to pick the default format\n>>> +\t */\n>>> +\tStreamConfiguration cfg{ StreamFormats{ formats } };\n>>> +\tcfg.pixelFormat = formats.begin()->first;\n>>> +\tcfg.size = formats.begin()->second[0].max;\n>>> +\n>>> +\tconfig->addConfiguration(cfg);\n>>> +\n>>> +\tconfig->validate();\n>>> +\n>>> +\treturn config;\n>>> +}\n>>> +\n>>> +int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c)\n>>> +{\n>>> +\tSimpleCameraConfiguration *config =\n>>> +\t\tstatic_cast(c);\n>>> +\tSimpleCameraData *data = cameraData(camera);\n>>> +\tStreamConfiguration &cfg = config->at(0);\n>>> +\tint ret;\n>>> +\n>>> +\t/*\n>>> +\t * Configure links on the pipeline and propagate formats from the\n>>> +\t * sensor to the video node.\n>>> +\t */\n>>> +\tret = data->setupLinks();\n>>> +\tif (ret < 0)\n>>> +\t\treturn ret;\n>>> +\n>>> +\tconst SimpleCameraData::Configuration &pipeConfig =\n>>> +\t\tdata->formats_[cfg.pixelFormat];\n>>> +\n>>> +\tV4L2SubdeviceFormat format{ pipeConfig.code, data->sensor_->resolution() };\n>>> +\n>>> +\tret = data->setupFormats(&format, V4L2Subdevice::ActiveFormat);\n>>> +\tif (ret < 0)\n>>> +\t\treturn ret;\n>>> +\n>>> +\t/* Configure the video node. */\n>>> +\tV4L2PixelFormat videoFormat = video_->toV4L2PixelFormat(cfg.pixelFormat);\n>>> +\n>>> +\tV4L2DeviceFormat outputFormat = {};\n>>> +\toutputFormat.fourcc = videoFormat;\n>>> +\toutputFormat.size = cfg.size;\n>>> +\n>>> +\tret = video_->setFormat(&outputFormat);\n>>> +\tif (ret)\n>>> +\t\treturn ret;\n>>> +\n>>> +\tif (outputFormat.size != cfg.size || outputFormat.fourcc != videoFormat) {\n>>> +\t\tLOG(SimplePipeline, Error)\n>>> +\t\t\t<< \"Unable to configure capture in \" << cfg.toString();\n>>> +\t\treturn -EINVAL;\n>>> +\t}\n>>> +\n>>> +\tcfg.setStream(&data->stream_);\n>>> +\n>>> +\treturn 0;\n>>> +}\n>>> +\n>>> +int SimplePipelineHandler::exportFrameBuffers(Camera *camera, Stream *stream,\n>>> +\t\t\t\t\t std::vector> *buffers)\n>>> +{\n>>> +\tunsigned int count = stream->configuration().bufferCount;\n>>> +\n>>> +\treturn video_->exportBuffers(count, buffers);\n>>> +}\n>>> +\n>>> +int SimplePipelineHandler::start(Camera *camera)\n>>> +{\n>>> +\tSimpleCameraData *data = cameraData(camera);\n>>> +\tunsigned int count = data->stream_.configuration().bufferCount;\n>>> +\n>>> +\tint ret = video_->importBuffers(count);\n>>> +\tif (ret < 0)\n>>> +\t\treturn ret;\n>>> +\n>>> +\tret = video_->streamOn();\n>>> +\tif (ret < 0) {\n>>> +\t\tvideo_->releaseBuffers();\n>>> +\t\treturn ret;\n>>> +\t}\n>>> +\n>>> +\tactiveCamera_ = camera;\n>>> +\n>>> +\treturn 0;\n>>> +}\n>>> +\n>>> +void SimplePipelineHandler::stop(Camera *camera)\n>>> +{\n>>> +\tvideo_->streamOff();\n>>> +\tvideo_->releaseBuffers();\n>>> +\tactiveCamera_ = nullptr;\n>>> +}\n>>> +\n>>> +int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request)\n>>> +{\n>>> +\tSimpleCameraData *data = cameraData(camera);\n>>> +\tStream *stream = &data->stream_;\n>>> +\n>>> +\tFrameBuffer *buffer = request->findBuffer(stream);\n>>> +\tif (!buffer) {\n>>> +\t\tLOG(SimplePipeline, Error)\n>>> +\t\t\t<< \"Attempt to queue request with invalid stream\";\n>>> +\t\treturn -ENOENT;\n>>> +\t}\n>>> +\n>>> +\treturn video_->queueBuffer(buffer);\n>>> +}\n>>> +\n>>> +/* -----------------------------------------------------------------------------\n>>> + * Match and Setup\n>>> + */\n>>> +\n>>> +bool SimplePipelineHandler::match(DeviceEnumerator *enumerator)\n>>> +{\n>>> +\tstatic const char * const drivers[] = {\n>>> +\t\t\"imx7-csi\",\n>>> +\t\t\"sun6i-csi\",\n>>> +\t};\n>>> +\n>>> +\tfor (const char *driver : drivers) {\n>>> +\t\tDeviceMatch dm(driver);\n>>> +\t\tmedia_ = acquireMediaDevice(enumerator, dm);\n>>> +\t\tif (media_)\n>>> +\t\t\tbreak;\n>>> +\t}\n>>> +\n>>> +\tif (!media_)\n>>> +\t\treturn false;\n>>> +\n>>> +\t/*\n>>> +\t * Locate sensors and video nodes. We only support pipelines with at\n>>> +\t * least one sensor and exactly one video captude node.\n>>> +\t */\n>>> +\tstd::vector sensors;\n>>> +\tstd::vector videos;\n>>> +\n>>> +\tfor (MediaEntity *entity : media_->entities()) {\n>>> +\t\tswitch (entity->function()) {\n>>> +\t\tcase MEDIA_ENT_F_CAM_SENSOR:\n>>> +\t\t\tsensors.push_back(entity);\n>>> +\t\t\tbreak;\n>>> +\n>>> +\t\tcase MEDIA_ENT_F_IO_V4L:\n>>> +\t\t\tif (entity->pads().size() == 1 &&\n>>> +\t\t\t (entity->pads()[0]->flags() & MEDIA_PAD_FL_SINK))\n>>> +\t\t\t\tvideos.push_back(entity);\n>>> +\t\t\tbreak;\n>>> +\n>>> +\t\tdefault:\n>>> +\t\t\tbreak;\n>>> +\t\t}\n>>> +\t}\n>>> +\n>>> +\tif (sensors.empty()) {\n>>> +\t\tLOG(SimplePipeline, Error) << \"No sensor found\";\n>>> +\t\treturn false;\n>>> +\t}\n>>> +\n>>> +\tif (videos.size() != 1) {\n>>> +\t\tLOG(SimplePipeline, Error)\n>>> +\t\t\t<< \"Pipeline with \" << videos.size()\n>>> +\t\t\t<< \" video capture nodes is not supported\";\n>>> +\t\treturn false;\n>>\n>> Could these Error be changed to Warning, and the \"return false;\" statement be dropped?\n>>\n>> I agree that there is no code yet to handle multiple video capture nodes, but this change\n>> allows me to use this simple pipeline handler with db410c board and one sensor connected\n>> to the first CSI interface (msm_csiphy0). So it can work in some simple configurations\n>> with multiple video capture nodes already.\n> \n> We can do that, but I don't think it's the right fix. There are little\n> guarantees regarding the order in which the video nodes are added to the\n> videos vector.\n\nThat's true.\nOn db410c it worked just because the right video node appeared to get into videos[0]\nin the SimplePipelineHandler::match() (== by luck).\n\n> I think we should instead pass the videos list to the\n> SimpleCameraData constructor, and get it to pick the video node\n> corresponding to the pipeline output. I can try to implement a patch for\n> that (but you could beat me to it :-)),\n\nGreat, thanks!\n\n> but I'd like to do so on top of\n> this one.\n\nThat makes sense.\n\nThanks,\nAndrey\n\n>>> +\t}\n>>> +\n>>> +\t/* Locate and open the capture video node. */\n>>> +\tvideo_ = new V4L2VideoDevice(videos[0]);\n>>> +\tif (video_->open() < 0)\n>>> +\t\treturn false;\n>>> +\n>>> +\tif (video_->caps().isMultiplanar()) {\n>>> +\t\tLOG(SimplePipeline, Error)\n>>> +\t\t\t<< \"V4L2 multiplanar devices are not supported\";\n>>> +\t\treturn false;\n>>\n>> Same here. For the same reasons.\n> \n> Do you mean that your multiplanar device works with this series ?\n> \n>>> +\t}\n>>> +\n>>> +\tvideo_->bufferReady.connect(this, &SimplePipelineHandler::bufferReady);\n>>> +\n>>> +\t/*\n>>> +\t * Create one camera data instance for each sensor and gather all\n>>> +\t * entities in all pipelines.\n>>> +\t */\n>>> +\tstd::vector> pipelines;\n>>> +\tstd::set entities;\n>>> +\n>>> +\tpipelines.reserve(sensors.size());\n>>> +\n>>> +\tfor (MediaEntity *sensor : sensors) {\n>>> +\t\tstd::unique_ptr data =\n>>> +\t\t\tstd::make_unique(this, sensor,\n>>> +\t\t\t\t\t\t\t videos[0]);\n>>> +\t\tif (!data->isValid()) {\n>>> +\t\t\tLOG(SimplePipeline, Error)\n>>> +\t\t\t\t<< \"No valid pipeline for sensor '\"\n>>> +\t\t\t\t<< sensor->name() << \"', skipping\";\n>>> +\t\t\tcontinue;\n>>> +\t\t}\n>>> +\n>>> +\t\tfor (SimpleCameraData::Entity &entity : data->entities_)\n>>> +\t\t\tentities.insert(entity.entity);\n>>> +\n>>> +\t\tpipelines.push_back(std::move(data));\n>>> +\t}\n>>> +\n>>> +\tif (entities.empty())\n>>> +\t\treturn false;\n>>> +\n>>> +\t/* Create and open V4L2Subdev instances for all the entities. */\n>>> +\tfor (MediaEntity *entity : entities) {\n>>> +\t\tauto elem = subdevs_.emplace(std::piecewise_construct,\n>>> +\t\t\t\t\t std::forward_as_tuple(entity),\n>>> +\t\t\t\t\t std::forward_as_tuple(entity));\n>>> +\t\tV4L2Subdevice *subdev = &elem.first->second;\n>>> +\t\tint ret = subdev->open();\n>>> +\t\tif (ret < 0) {\n>>> +\t\t\tLOG(SimplePipeline, Error)\n>>> +\t\t\t\t<< \"Failed to open \" << subdev->deviceNode()\n>>> +\t\t\t\t<< \": \" << strerror(-ret);\n>>> +\t\t\treturn false;\n>>> +\t\t}\n>>> +\t}\n>>> +\n>>> +\t/* Initialize each pipeline and register a corresponding camera. */\n>>> +\tfor (std::unique_ptr &data : pipelines) {\n>>> +\t\tint ret = data->init();\n>>> +\t\tif (ret < 0)\n>>> +\t\t\tcontinue;\n>>> +\n>>> +\t\tstd::shared_ptr camera =\n>>> +\t\t\tCamera::create(this, data->sensor_->entity()->name(),\n>>> +\t\t\t\t data->streams());\n>>> +\t\tregisterCamera(std::move(camera), std::move(data));\n>>> +\t}\n>>> +\n>>> +\treturn true;\n>>> +}\n>>> +\n>>> +V4L2Subdevice *SimplePipelineHandler::subdev(const MediaEntity *entity)\n>>> +{\n>>> +\tauto iter = subdevs_.find(entity);\n>>> +\tif (iter == subdevs_.end())\n>>> +\t\treturn nullptr;\n>>> +\n>>> +\treturn &iter->second;\n>>> +}\n>>> +\n>>> +/* -----------------------------------------------------------------------------\n>>> + * Buffer Handling\n>>> + */\n>>> +\n>>> +void SimplePipelineHandler::bufferReady(FrameBuffer *buffer)\n>>> +{\n>>> +\tASSERT(activeCamera_);\n>>> +\tRequest *request = buffer->request();\n>>> +\tcompleteBuffer(activeCamera_, request, buffer);\n>>> +\tcompleteRequest(activeCamera_, request);\n>>> +}\n>>> +\n>>> +REGISTER_PIPELINE_HANDLER(SimplePipelineHandler);\n>>> +\n>>> +} /* namespace libcamera */\n>","headers":{"Return-Path":"","Received":["from mail-lf1-x141.google.com (mail-lf1-x141.google.com\n\t[IPv6:2a00:1450:4864:20::141])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 420F96279B\n\tfor ;\n\tWed, 8 Apr 2020 19:02:58 +0200 (CEST)","by mail-lf1-x141.google.com with SMTP id 131so5668240lfh.11\n\tfor ;\n\tWed, 08 Apr 2020 10:02:58 -0700 (PDT)","from [192.168.118.216] (37-144-159-139.broadband.corbina.ru.\n\t[37.144.159.139]) by smtp.gmail.com with ESMTPSA id\n\tt81sm8758534lff.52.2020.04.08.10.02.55\n\t(version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);\n\tWed, 08 Apr 2020 10:02:55 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key; \n\tunprotected) header.d=linaro.org\n\theader.i=@linaro.org header.b=\"Khmxs/RG\"; \n\tdkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=subject:to:cc:references:from:message-id:date:user-agent\n\t:mime-version:in-reply-to:content-language:content-transfer-encoding; \n\tbh=d/I7QahVsZTnth7ShX6nrZlpPZfv5bcuQo4R97LdJVY=;\n\tb=Khmxs/RGH0S5QD8tYyXw/pDnFq7TkXFM/Z5mvsFPyTYcl6W5MX80CTvUChuaaV0HL2\n\th8+5UjhQFOqp9h6m0MI+fho6xMap4nhBc32HBo29JZ9HWaN7UYX/b9nHR7ZtCmLpFaR/\n\tT3nqtprANjPeL7mBBL740sePqbWeQQx/S9fOCuaCkoH2NRSR8t8mvS1Haa+dlEbdNQ5Q\n\tAp5OFjtWotPGAFIJ62xRfue5fMvv7FMIAOHDsFTd8icSo9xd9TDsbu2Yb3IfEcgX4dUg\n\tK9JUMq1GymKrGekpW3MeGs6ZLQLSAhQSPGe4VnaurWHmTs8AMjHZETKiBRpiCJWNSbB5\n\thaZw==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:subject:to:cc:references:from:message-id:date\n\t:user-agent:mime-version:in-reply-to:content-language\n\t:content-transfer-encoding;\n\tbh=d/I7QahVsZTnth7ShX6nrZlpPZfv5bcuQo4R97LdJVY=;\n\tb=n4FV+VlvED2ByvBQfEDP8Ad6AnGPLVBQYEGl59GT5sRWXxy9v8M/3LD1J4IEIRJyNB\n\tqf5knBWUTHw1HoSTTbZ4YyCewKWMl77wGRp2b9UOAz84AIuqfWbxj7eXp9M7R7KPrdlP\n\trpEcZkyEjPpRlkZBnlA1E96vGwQIxbyHbGdMMLlShn2lHbQwMQ7vaeS+LsW3qxJGwWun\n\tyDKZD84l1u+zHzPAgKIUHy7qw3Z3bRg84EgGJgclXJCjAW2r3wgKZjfpylI59/tbUdGs\n\t7J66HYVsUuEAy2OBlKKAX9DC0evuuHFdVG6wI4FE7op3Zx7hY1ACQD9aC10OFpcpqVFH\n\tbzeQ==","X-Gm-Message-State":"AGi0PuZTdGn0RRDncR75EEcqJhk1Oy+zoCR7MAV0A684luOQBxP1fC2R\n\t78a8XcatfMnq0snU4+ox51TriA==","X-Google-Smtp-Source":"APiQypLndXj+f0r4WvXF0B6n7Nnw2JF+0qDT442DMXtosOgnn+M1whljmuWSMIl7m8r04dU/yDNL1A==","X-Received":"by 2002:a05:6512:1113:: with SMTP id\n\tl19mr5266314lfg.77.1586365377224; \n\tWed, 08 Apr 2020 10:02:57 -0700 (PDT)","To":"Laurent Pinchart ","Cc":"libcamera-devel@lists.libcamera.org, Martijn Braam , \n\tBenjamin GAIGNARD ","References":"<20200320014839.14683-1-laurent.pinchart@ideasonboard.com>\n\t<20200320014839.14683-10-laurent.pinchart@ideasonboard.com>\n\t<053e8bb2-1d50-1aa5-b09a-fdb6718e3a41@linaro.org>\n\t<20200404000825.GE9690@pendragon.ideasonboard.com>","From":"Andrey Konovalov ","Message-ID":"<2e0132f7-d6c1-4434-ec91-40cd7ef76ffb@linaro.org>","Date":"Wed, 8 Apr 2020 20:02:53 +0300","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101\n\tThunderbird/68.4.1","MIME-Version":"1.0","In-Reply-To":"<20200404000825.GE9690@pendragon.ideasonboard.com>","Content-Type":"text/plain; charset=utf-8; format=flowed","Content-Language":"en-US","Content-Transfer-Encoding":"7bit","Subject":"Re: [libcamera-devel] [PATCH v3 09/11] libcamera: pipeline: Add a\n\tsimple pipeline handler","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","X-List-Received-Date":"Wed, 08 Apr 2020 17:02:58 -0000"}},{"id":4428,"web_url":"https://patchwork.libcamera.org/comment/4428/","msgid":"<13e80003-3f82-9c62-50fe-1505a753c7d9@linaro.org>","date":"2020-04-08T17:40:08","subject":"Re: [libcamera-devel] [PATCH v3 09/11] libcamera: pipeline: Add a\n\tsimple pipeline handler","submitter":{"id":25,"url":"https://patchwork.libcamera.org/api/people/25/","name":"Andrey Konovalov","email":"andrey.konovalov@linaro.org"},"content":"Hi Laurent,\n\nOn 04.04.2020 03:12, Laurent Pinchart wrote:\n> Hi Andrey,\n> \n> On Fri, Apr 03, 2020 at 04:13:35PM +0300, Andrey Konovalov wrote:\n>> On 31.03.2020 14:45, Kieran Bingham wrote:\n>>> On 20/03/2020 01:48, Laurent Pinchart wrote:\n>>>> From: Martijn Braam \n>>>>\n>>>> This new pipeline handler aims at supporting any simple device without\n>>>> requiring any device-specific code. Simple devices are currently defined\n>>>> as a graph made of one or multiple camera sensors and a single video\n>>>> node, with each sensor connected to the video node through a linear\n>>>> pipeline.\n>>>>\n>>>> The simple pipeline handler will automatically parse the media graph,\n>>>> enumerate sensors, build supported stream configurations, and configure\n>>>> the pipeline, without any device-specific knowledge. It doesn't support\n>>>> configuration of any processing in the pipeline at the moment, but may\n>>>> be extended to support simple processing such as format conversion or\n>>>> scaling in the future.\n>>>>\n>>>> The only device-specific information in the pipeline handler is the list\n>>>> of supported drivers, required for device matching. We may be able to\n>>>> remove this in the future by matching with the simple pipeline handler\n>>>> as a last resort option, after all other pipeline handlers have been\n>>>> tried.\n>>>>\n>>>> Signed-off-by: Martijn Braam \n>>>> Signed-off-by: Laurent Pinchart \n>>>\n>>> Reviewed-by: Kieran Bingham \n>>>\n>>> Some comments and discussion points below, but I think we should work\n>>> towards trying to get this integrated so it can actually be used (which\n>>> is where we'll find the nuances and corner cases as people test it)\n>>>\n>>>> ---\n>>>> Changes since v2:\n>>>>\n>>>> - Log an error when setupFormats() fail\n>>>> - Propagate getFormat() and setFormat() errors to the caller of\n>>>> setupFormats()\n>>>> - Reorder variable declarations in validate()\n>>>> - Add \\todo comment related to the selection of the default format\n>>>> - Use log Error instead of Info if pipeline isn't valid\n>>>> - Rebase on top of V4L2PixelFormat\n>>>>\n>>>> Changes since v1:\n>>>>\n>>>> - Rebase on top of buffer API rework\n>>>> - Expose stream formats\n>>>> - Rework camera data config\n>>>> ---\n>>>> src/libcamera/pipeline/meson.build | 1 +\n>>>> src/libcamera/pipeline/simple/meson.build | 3 +\n>>>> src/libcamera/pipeline/simple/simple.cpp | 710 ++++++++++++++++++++++\n>>>> 3 files changed, 714 insertions(+)\n>>>> create mode 100644 src/libcamera/pipeline/simple/meson.build\n>>>> create mode 100644 src/libcamera/pipeline/simple/simple.cpp\n>>>>\n>>>> diff --git a/src/libcamera/pipeline/meson.build b/src/libcamera/pipeline/meson.build\n>>>> index 0d466225a72e..606ba31a0319 100644\n>>>> --- a/src/libcamera/pipeline/meson.build\n>>>> +++ b/src/libcamera/pipeline/meson.build\n>>>> @@ -5,3 +5,4 @@ libcamera_sources += files([\n>>>> \n>>>> subdir('ipu3')\n>>>> subdir('rkisp1')\n>>>> +subdir('simple')\n>>>> diff --git a/src/libcamera/pipeline/simple/meson.build b/src/libcamera/pipeline/simple/meson.build\n>>>> new file mode 100644\n>>>> index 000000000000..4945a3e173cf\n>>>> --- /dev/null\n>>>> +++ b/src/libcamera/pipeline/simple/meson.build\n>>>> @@ -0,0 +1,3 @@\n>>>> +libcamera_sources += files([\n>>>> + 'simple.cpp',\n>>>> +])\n>>>> diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp\n>>>> new file mode 100644\n>>\n>> \n>>\n>>>> +int SimpleCameraData::init()\n>>>> +{\n>>>> +\tSimplePipelineHandler *pipe = static_cast(pipe_);\n>>>> +\tV4L2VideoDevice *video = pipe->video();\n>>>> +\tint ret;\n>>>> +\n>>>> +\t/*\n>>>> +\t * Enumerate the possible pipeline configurations. For each media bus\n>>>> +\t * format supported by the sensor, propagate the formats through the\n>>>> +\t * pipeline, and enumerate the corresponding possible V4L2 pixel\n>>>> +\t * formats on the video node.\n>>>> +\t */\n>>>> +\tfor (unsigned int code : sensor_->mbusCodes()) {\n>>>> +\t\tV4L2SubdeviceFormat format{ code, sensor_->resolution() };\n>>>> +\n>>>> +\t\t/*\n>>>> +\t\t * Setup links first as some subdev drivers take active links\n>>>> +\t\t * into account to propaget TRY formats. So is life :-(\n>>>\n>>> /propaget/propagate/\n>>>\n>>> /So is life/Such is life/\n>>>\n>>>\n>>>> +\t\t */\n>>>> +\t\tret = setupLinks();\n>>>> +\t\tif (ret < 0)\n>>>> +\t\t\treturn ret;\n>>>> +\n>>>> +\t\tret = setupFormats(&format, V4L2Subdevice::TryFormat);\n>>>> +\t\tif (ret < 0) {\n>>>> +\t\t\tLOG(SimplePipeline, Error)\n>>>> +\t\t\t\t<< \"Failed to setup pipeline for media bus code \"\n>>>> +\t\t\t\t<< utils::hex(code, 4);\n>>>\n>>> Oh dear, seems we need mbusCode.toString() too! hehe\n>>>\n>>>> +\t\t\treturn ret;\n>>>\n>>> Andrey states that this return prevents iterating potentially successful\n>>> mbus-codes, and should instead 'continue'...\n>>\n>> I've taken a deeper look at the ov5640 + db410c logs I've received, and the above\n>> return *could* be a potential problem, but definitely not in this (ov5640 + db410c)\n>> case.\n>>\n>> We only saw SimpleCameraData::setupFormats() returning error due to failing\n>> sensor_->setFormat(format) call.\n>> This happened if before the cam is started (and the stream cfg isn't set to a\n>> particular width*height) the sensor is configured for the frame rate greater than 15 fps.\n>> In particular, in the ov5640's driver probe function the initial mode is set to\n>> \"YUV422 UYVY VGA@30fps\".\n>> ov5640_enum_frame_size() doesn't consider the current fps at all, so the maximum\n>> frame size reported is 2592x1944. But it only works at 15fps max, and an attempt\n>> to set the sensor format to a one with the maximum possible frame size of 2592x1944\n>> results in -EINVAL returned by SUBDEV_S_FMT.\n>> (So in our ov5640 + db410c case replacing the \"return\" with \"continue\" does let the pipeline\n>> handler to try all the other media bus codes if the first one fails, but they all fail the\n>> same way as the first one does.)\n>>\n>> If this issue with the frame rate is worked around in some way, the enumeration\n>> of the possible pipeline configurations completes kind of OK: a good configuration\n>> is selected in the end. But during the enumeration all the configurations, the\n>> broken ones included are considered as the working ones:\n>>\n>> The links are set up this way (\"entity name\":pad_number]):\n>>\n>> \"ov5640 4-0076\":0 -> \"msm_csiphy0\":0,\"msm_csiphy0\":1 -> \"msm_csid0\":0,\"msm_csid0\":1 ->\n>> \t-> \"msm_ispif0\":0,\"msm_ispif0\":1 -> \"msm_vfe0_rdi0\":0\n>>\n>>\n>> SimpleCameraData::setupFormats() creates a broken pipeline when it tries the sensor format\n>> not supported by the rest of the entities. This results in e.g. the following:\n>>\n>> \"ov5640 4-0076\":0[MEDIA_BUS_FMT_RGB565_2X8_BE] -> [MEDIA_BUS_FMT_UYVY8_2X8]\"msm_csiphy0\":0,\n>> \"msm_csiphy0\":1[MEDIA_BUS_FMT_UYVY8_2X8] -> [MEDIA_BUS_FMT_UYVY8_2X8]\"msm_csid0\":0, etc\n>>\n>> I.e. an attempt to set fmt on \"msm_csiphy0\":0 to MEDIA_BUS_FMT_RGB565_2X8_BE results in the\n>> format set to MEDIA_BUS_FMT_UYVY8_2X8 and no error is returned here (and down the pipeline\n>> as MEDIA_BUS_FMT_UYVY8_2X8 is set OK for all the other pads). So we get the first link misconfigured,\n>> but the result is considered as a good pipeline as no error was returned while the format\n>> from the sensor output was propagated from the sensor output down to \"msm_vfe0_rdi0\":0.\n>> And the broken configuration is stored in the formats_ map (in the end, it works in our\n>> ov5640+db410c case as the broken configurations are rewritten with the valid ones later).\n>>\n>> Who/what is to ensure that for each of the links, the source and the sink pads of the same link\n>> are configured to the same format?\n> \n> The kernel is responsible for validating that at stream on time, but I\n> think we should also verify it in setupFormats(). Would you like to send\n> a patch on top of this series ?\n\nI've got something which gets to the point when 'cam' starts streaming, but no buffer comes from the\nvideo device.\n\nThat is now SimpleCameraData::init() enumerates the possible pipeline configurations correctly (skips\nthe broken ones). But with the patch the StreamConfiguration now uses 'BA81' format which doesn't\nwork (though looks valid for the pipeline) instead of 'YUYV' before which worked OK. (The format is\nselected in SimplePipelineHandler::CreateConfiguration() by choosing the first entry in the formats\nmap.)\n\nI am looking into this now. But \"debugging by emails\" isn't the fastest process unfortunately (I don't\nhave ov5640 for db410c locally).\n\n\nThanks,\nAndrey\n\n>>>> +\t\t}\n>>>> +\n>>>> +\t\tstd::map> videoFormats =\n>>>> +\t\t\tvideo->formats(format.mbus_code);\n>>>> +\n>>>> +\t\tLOG(SimplePipeline, Debug)\n>>>> +\t\t\t<< \"Adding configuration for \" << format.size.toString()\n>>>> +\t\t\t<< \" in pixel formats [ \"\n>>>> +\t\t\t<< utils::join(videoFormats, \", \",\n>>>> +\t\t\t\t [](const auto &f) {\n>>>> +\t\t\t\t\t return f.first.toString();\n>>>> +\t\t\t\t })\n>>>\n>>> Aha, I kinda like the utils::join!\n>>>\n>>>> +\t\t\t<< \" ]\";\n>>>> +\n>>>> +\t\t/*\n>>>> +\t\t * Store the configuration in the formats_ map, mapping\n>>>> +\t\t * PixelFormat to configuration. Any previously stored value is\n>>>\n>>> /to configuration/to the current configuration/ ?\n>>>\n>>>> +\t\t * overwritten, as the pipeline handler currently doesn't care\n>>>> +\t\t * about how a particular PixelFormat is achieved.\n>>>\n>>> As long as there's an output, I agree. If devices have specific\n>>> constraints such as better performance or such with a particular\n>>> mbus-code, then they need to create a platform specific pipeline handler\n>>> to deal with the nuances.\n>>>\n>>> Maybe there might be tuning hints or something later, but certainly not now.\n>>>\n>>>\n>>>> +\t\t */\n>>>> +\t\tfor (const auto &videoFormat : videoFormats) {\n>>>> +\t\t\tPixelFormat pixelFormat = video->toPixelFormat(videoFormat.first);\n>>>> +\t\t\tif (!pixelFormat)\n>>>> +\t\t\t\tcontinue;\n>>>> +\n>>>> +\t\t\tConfiguration config;\n>>>> +\t\t\tconfig.code = code;\n>>>> +\t\t\tconfig.pixelFormat = pixelFormat;\n>>>> +\t\t\tconfig.size = format.size;\n>>>> +\n>>>> +\t\t\tformats_[pixelFormat] = config;\n>>>> +\t\t}\n>>>> +\t}\n>>>> +\n>>>> +\tif (formats_.empty()) {\n>>>> +\t\tLOG(SimplePipeline, Error) << \"No valid configuration found\";\n>>>> +\t\treturn -EINVAL;\n>>>> +\t}\n>>>> +\n>>>> +\treturn 0;\n>>>> +}\n> \n> [snip]\n>","headers":{"Return-Path":"","Received":["from mail-lf1-x142.google.com (mail-lf1-x142.google.com\n\t[IPv6:2a00:1450:4864:20::142])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 75D8B6279B\n\tfor ;\n\tWed, 8 Apr 2020 19:40:13 +0200 (CEST)","by mail-lf1-x142.google.com with SMTP id k28so5778486lfe.10\n\tfor ;\n\tWed, 08 Apr 2020 10:40:13 -0700 (PDT)","from [192.168.118.216] (37-144-159-139.broadband.corbina.ru.\n\t[37.144.159.139])\n\tby smtp.gmail.com with ESMTPSA id s8sm68149lfd.61.2020.04.08.10.40.11\n\t(version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);\n\tWed, 08 Apr 2020 10:40:12 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key; \n\tunprotected) header.d=linaro.org\n\theader.i=@linaro.org header.b=\"MdUNSH7r\"; \n\tdkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=subject:to:cc:references:from:message-id:date:user-agent\n\t:mime-version:in-reply-to:content-language:content-transfer-encoding; \n\tbh=qIbiZjyJ+7gIc+uuv35EA1bvWskLcXZrcCaNmpBBL9g=;\n\tb=MdUNSH7rR8qy43u/0iFaGn0mK8S0xJe5pSKhBnps+OSW7fRmksucMjxIomtabxT5dF\n\tEkdITIO6T/V5rEXMnglt6omPh3xe4oy+/+yuJHQkU0p7JEZDE8TjO5qYDnC8ZMwv9DgH\n\t/JKT7+Pw0puzChrWF1kSNlKX449DuX21eNKrD29J2Zd2yIMwac4z8bnzXFKWZ9e7gzRz\n\tdie9Wstj2T3qRcskKb85De7NNcefoQ9y3szNhd4wd2zd9R+x4DszrZ/idZsH4b/bJsS5\n\tuN0Yfyis3hvxERJdgwiFyrjq0NixKKHZPSsjUQFqc3dahsSDEksvqPld14ekYTjr1WyM\n\tRznA==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:subject:to:cc:references:from:message-id:date\n\t:user-agent:mime-version:in-reply-to:content-language\n\t:content-transfer-encoding;\n\tbh=qIbiZjyJ+7gIc+uuv35EA1bvWskLcXZrcCaNmpBBL9g=;\n\tb=EakCCGg+DneyU+dL+2HqDZee+NzalydJXkfhr1eRgZlBaxn6+kWn6BbOF1XgTCscBf\n\ttvjWhScKZKRKno/9ZQAZWBbtsw4CbOJNih51uwHmbJMqmJcYIi1kopsWuHKYXIYLrSdV\n\txTJU43aGuNAJ7w5Fq3I92gX8r6dk+RLZMgKgZJD3U366oc4bxyKgZFnr00X9v6Z5M7qy\n\t1XkRGZdcMTytyOa3BdrEnn2socbvlxlAoW/YY7Q8dRn/O7vK3VjXp4hXuh6QMWTAeXE2\n\tShA/chSMblEuwxlyLirGuK/yK1dIYoFrni5czVLgQtmujTdlqlGFzJ5WG6KVbTaujqkh\n\tS2oA==","X-Gm-Message-State":"AGi0PuZ0cfumLp9Xa0YhuSJTMAXAOneRH4sjjsJuG2CJ5o6x0SkrdIF1\n\tiTgkQYcZFp/O4qmNpww9Go19IwrT4P0=","X-Google-Smtp-Source":"APiQypKx3nImmf3mlTvXo2lafpOdyL9AOTgEYPUE7Y2qSIOp2VsXig1Fs+m+bkbUJ5Zdi8Jv+FIXKA==","X-Received":"by 2002:a19:ca13:: with SMTP id a19mr5256082lfg.68.1586367612687;\n\tWed, 08 Apr 2020 10:40:12 -0700 (PDT)","To":"Laurent Pinchart ","Cc":"kieran.bingham@ideasonboard.com, libcamera-devel@lists.libcamera.org,\n\tMartijn Braam ,\n\tBenjamin GAIGNARD ","References":"<20200320014839.14683-1-laurent.pinchart@ideasonboard.com>\n\t<20200320014839.14683-10-laurent.pinchart@ideasonboard.com>\n\t<43484c6b-44ac-4bd7-88e4-b46070840315@ideasonboard.com>\n\t<0519333c-3d17-325a-704b-664e475f556c@linaro.org>\n\t<20200404001202.GF9690@pendragon.ideasonboard.com>","From":"Andrey Konovalov ","Message-ID":"<13e80003-3f82-9c62-50fe-1505a753c7d9@linaro.org>","Date":"Wed, 8 Apr 2020 20:40:08 +0300","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101\n\tThunderbird/68.4.1","MIME-Version":"1.0","In-Reply-To":"<20200404001202.GF9690@pendragon.ideasonboard.com>","Content-Type":"text/plain; charset=utf-8; format=flowed","Content-Language":"en-US","Content-Transfer-Encoding":"7bit","Subject":"Re: [libcamera-devel] [PATCH v3 09/11] libcamera: pipeline: Add a\n\tsimple pipeline handler","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","X-List-Received-Date":"Wed, 08 Apr 2020 17:40:13 -0000"}},{"id":4429,"web_url":"https://patchwork.libcamera.org/comment/4429/","msgid":"<22ea7c4d-814c-2d59-e747-9c2698a53fdb@linaro.org>","date":"2020-04-08T17:52:00","subject":"Re: [libcamera-devel] [PATCH v3 09/11] libcamera: pipeline: Add a\n\tsimple pipeline handler","submitter":{"id":25,"url":"https://patchwork.libcamera.org/api/people/25/","name":"Andrey Konovalov","email":"andrey.konovalov@linaro.org"},"content":"Hi Laurent,\n\nOn 04.04.2020 03:08, Laurent Pinchart wrote:\n> Hi Andrey,\n> \n> On Fri, Apr 03, 2020 at 04:55:32PM +0300, Andrey Konovalov wrote:\n>> On 20.03.2020 04:48, Laurent Pinchart wrote:\n>>> From: Martijn Braam \n>>>\n>>> This new pipeline handler aims at supporting any simple device without\n>>> requiring any device-specific code. Simple devices are currently defined\n>>> as a graph made of one or multiple camera sensors and a single video\n>>> node, with each sensor connected to the video node through a linear\n>>> pipeline.\n>>>\n>>> The simple pipeline handler will automatically parse the media graph,\n>>> enumerate sensors, build supported stream configurations, and configure\n>>> the pipeline, without any device-specific knowledge. It doesn't support\n>>> configuration of any processing in the pipeline at the moment, but may\n>>> be extended to support simple processing such as format conversion or\n>>> scaling in the future.\n>>>\n>>> The only device-specific information in the pipeline handler is the list\n>>> of supported drivers, required for device matching. We may be able to\n>>> remove this in the future by matching with the simple pipeline handler\n>>> as a last resort option, after all other pipeline handlers have been\n>>> tried.\n>>>\n>>> Signed-off-by: Martijn Braam \n>>> Signed-off-by: Laurent Pinchart \n>>> ---\n>>> Changes since v2:\n>>>\n>>> - Log an error when setupFormats() fail\n>>> - Propagate getFormat() and setFormat() errors to the caller of\n>>> setupFormats()\n>>> - Reorder variable declarations in validate()\n>>> - Add \\todo comment related to the selection of the default format\n>>> - Use log Error instead of Info if pipeline isn't valid\n>>> - Rebase on top of V4L2PixelFormat\n>>>\n>>> Changes since v1:\n>>>\n>>> - Rebase on top of buffer API rework\n>>> - Expose stream formats\n>>> - Rework camera data config\n>>> ---\n>>> src/libcamera/pipeline/meson.build | 1 +\n>>> src/libcamera/pipeline/simple/meson.build | 3 +\n>>> src/libcamera/pipeline/simple/simple.cpp | 710 ++++++++++++++++++++++\n>>> 3 files changed, 714 insertions(+)\n>>> create mode 100644 src/libcamera/pipeline/simple/meson.build\n>>> create mode 100644 src/libcamera/pipeline/simple/simple.cpp\n>>>\n>>> diff --git a/src/libcamera/pipeline/meson.build b/src/libcamera/pipeline/meson.build\n>>> index 0d466225a72e..606ba31a0319 100644\n>>> --- a/src/libcamera/pipeline/meson.build\n>>> +++ b/src/libcamera/pipeline/meson.build\n>>> @@ -5,3 +5,4 @@ libcamera_sources += files([\n>>> \n>>> subdir('ipu3')\n>>> subdir('rkisp1')\n>>> +subdir('simple')\n>>> diff --git a/src/libcamera/pipeline/simple/meson.build b/src/libcamera/pipeline/simple/meson.build\n>>> new file mode 100644\n>>> index 000000000000..4945a3e173cf\n>>> --- /dev/null\n>>> +++ b/src/libcamera/pipeline/simple/meson.build\n>>> @@ -0,0 +1,3 @@\n>>> +libcamera_sources += files([\n>>> + 'simple.cpp',\n>>> +])\n>>> diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp\n>>> new file mode 100644\n>>> index 000000000000..545a99fe31ca\n>>> --- /dev/null\n>>> +++ b/src/libcamera/pipeline/simple/simple.cpp\n\n\n\n>>> +\tif (video_->caps().isMultiplanar()) {\n>>> +\t\tLOG(SimplePipeline, Error)\n>>> +\t\t\t<< \"V4L2 multiplanar devices are not supported\";\n>>> +\t\treturn false;\n>>\n>> Same here. For the same reasons.\n> \n> Do you mean that your multiplanar device works with this series ?\n\nqcom-camss uses multiplanar API for single-planar formats\n(just like the current RPi's unicam driver iirc).\n\nAnd on db410c with ov5640 sensor this works with this series\n(if I am not trying to fix the enumeration of pipeline configurations).\n\nThough (as it worked as is) I didn't check how exactly this happened.\nMaybe I'll get better understanding of the details now that I am trying to figure out\nwhy BA81 isn't currently working for me (see my earlier email today).\n\n\nThanks,\nAndrey\n\n>>> +\t}\n>>> +\n>>> +\tvideo_->bufferReady.connect(this, &SimplePipelineHandler::bufferReady);\n>>> +\n>>> +\t/*\n>>> +\t * Create one camera data instance for each sensor and gather all\n>>> +\t * entities in all pipelines.\n>>> +\t */\n>>> +\tstd::vector> pipelines;\n>>> +\tstd::set entities;\n>>> +\n>>> +\tpipelines.reserve(sensors.size());\n>>> +\n>>> +\tfor (MediaEntity *sensor : sensors) {\n>>> +\t\tstd::unique_ptr data =\n>>> +\t\t\tstd::make_unique(this, sensor,\n>>> +\t\t\t\t\t\t\t videos[0]);\n>>> +\t\tif (!data->isValid()) {\n>>> +\t\t\tLOG(SimplePipeline, Error)\n>>> +\t\t\t\t<< \"No valid pipeline for sensor '\"\n>>> +\t\t\t\t<< sensor->name() << \"', skipping\";\n>>> +\t\t\tcontinue;\n>>> +\t\t}\n>>> +\n>>> +\t\tfor (SimpleCameraData::Entity &entity : data->entities_)\n>>> +\t\t\tentities.insert(entity.entity);\n>>> +\n>>> +\t\tpipelines.push_back(std::move(data));\n>>> +\t}\n>>> +\n>>> +\tif (entities.empty())\n>>> +\t\treturn false;\n>>> +\n>>> +\t/* Create and open V4L2Subdev instances for all the entities. */\n>>> +\tfor (MediaEntity *entity : entities) {\n>>> +\t\tauto elem = subdevs_.emplace(std::piecewise_construct,\n>>> +\t\t\t\t\t std::forward_as_tuple(entity),\n>>> +\t\t\t\t\t std::forward_as_tuple(entity));\n>>> +\t\tV4L2Subdevice *subdev = &elem.first->second;\n>>> +\t\tint ret = subdev->open();\n>>> +\t\tif (ret < 0) {\n>>> +\t\t\tLOG(SimplePipeline, Error)\n>>> +\t\t\t\t<< \"Failed to open \" << subdev->deviceNode()\n>>> +\t\t\t\t<< \": \" << strerror(-ret);\n>>> +\t\t\treturn false;\n>>> +\t\t}\n>>> +\t}\n>>> +\n>>> +\t/* Initialize each pipeline and register a corresponding camera. */\n>>> +\tfor (std::unique_ptr &data : pipelines) {\n>>> +\t\tint ret = data->init();\n>>> +\t\tif (ret < 0)\n>>> +\t\t\tcontinue;\n>>> +\n>>> +\t\tstd::shared_ptr camera =\n>>> +\t\t\tCamera::create(this, data->sensor_->entity()->name(),\n>>> +\t\t\t\t data->streams());\n>>> +\t\tregisterCamera(std::move(camera), std::move(data));\n>>> +\t}\n>>> +\n>>> +\treturn true;\n>>> +}\n>>> +\n>>> +V4L2Subdevice *SimplePipelineHandler::subdev(const MediaEntity *entity)\n>>> +{\n>>> +\tauto iter = subdevs_.find(entity);\n>>> +\tif (iter == subdevs_.end())\n>>> +\t\treturn nullptr;\n>>> +\n>>> +\treturn &iter->second;\n>>> +}\n>>> +\n>>> +/* -----------------------------------------------------------------------------\n>>> + * Buffer Handling\n>>> + */\n>>> +\n>>> +void SimplePipelineHandler::bufferReady(FrameBuffer *buffer)\n>>> +{\n>>> +\tASSERT(activeCamera_);\n>>> +\tRequest *request = buffer->request();\n>>> +\tcompleteBuffer(activeCamera_, request, buffer);\n>>> +\tcompleteRequest(activeCamera_, request);\n>>> +}\n>>> +\n>>> +REGISTER_PIPELINE_HANDLER(SimplePipelineHandler);\n>>> +\n>>> +} /* namespace libcamera */\n>","headers":{"Return-Path":"","Received":["from mail-lf1-x143.google.com (mail-lf1-x143.google.com\n\t[IPv6:2a00:1450:4864:20::143])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 6752A6279B\n\tfor ;\n\tWed, 8 Apr 2020 19:52:03 +0200 (CEST)","by mail-lf1-x143.google.com with SMTP id s13so5815072lfb.9\n\tfor ;\n\tWed, 08 Apr 2020 10:52:03 -0700 (PDT)","from [192.168.118.216] (37-144-159-139.broadband.corbina.ru.\n\t[37.144.159.139]) by smtp.gmail.com with ESMTPSA id\n\tm21sm797051ljj.56.2020.04.08.10.52.01\n\t(version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);\n\tWed, 08 Apr 2020 10:52:02 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key; \n\tunprotected) header.d=linaro.org\n\theader.i=@linaro.org header.b=\"H7kPvu8r\"; \n\tdkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=subject:to:cc:references:from:message-id:date:user-agent\n\t:mime-version:in-reply-to:content-language:content-transfer-encoding; \n\tbh=kdTL/0RaJf+Jy9ZflGLl/UpmaSukDGsck+ddckecYmQ=;\n\tb=H7kPvu8rKfg2YsVXkLpaQUH6Ede2/sNR8ayBd8OxjnY+fx9UuoZGjNGJV0bBh0Sk3w\n\tbIrkdGc4g3/vmIZEqV57vt9b+rFoROW+GVBmwmu+R/ZEvAObiEtG7x9uPA6alXY22lax\n\tHjztEciCjmEdSR9DK9oPtaTodtAPbBRut8RnRpkLJtt4i5FpZfmAYKXHTXonshT4aJ4/\n\tZTFqV449IMJdo4Hby++q1ezAo4rA+13N8wN4GkmqTKIWRXWv7p66QLaDUqfaxoxbEk7F\n\tkGiq3Yf3i52Ky2x6rII11xHyKeUgb46IL/wh+W8BLOXTdSMXjwBmkILZd4FSg3dwRgOD\n\t+BxQ==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:subject:to:cc:references:from:message-id:date\n\t:user-agent:mime-version:in-reply-to:content-language\n\t:content-transfer-encoding;\n\tbh=kdTL/0RaJf+Jy9ZflGLl/UpmaSukDGsck+ddckecYmQ=;\n\tb=XIiX0Yx5pTnlP7duy7eEbd2qcn1Ut9qglnWvC1W4/qHY4xebkGBj/Tb6dmMnCLVPlE\n\txSqZjcsoYnm5oJNPcrsOv47j1S8IB57AjfUWJE0m5XHC3oJT5pNd9S3oa381yxpswgED\n\tIUkEse7huxc/IDicSEmtpE18mYnmJkUOQUniBWFqryzL++G3PY5Kv91PldqSh9Z3vzgm\n\t3hncdnMm7Jik3L9q0SdZT37vPhhHXMlqVBaEN1B6XEgcLCoukIEkCd5kfOLdINdtde8P\n\tLj8BfJ9G+RMTwiuaoDl3/dXNZK4Yi5hRrC/de2ukIqYSw8ET02LFLH7AkQup2/VWUzow\n\tfK1Q==","X-Gm-Message-State":"AGi0PuYijQ4KUQT7epKg+w7r0tc3lhWfjqafOxUC6iJ32TfAzhIX5DeO\n\tlZv/zbD2tV+T6onO9EsfaFypQw==","X-Google-Smtp-Source":"APiQypI1bdRhmozBuxrSVAnw0w5/nZg1Wlr6ZOou0VTl5aN20B/07SXKL+Gh9lN8rAyf21hNug/xaA==","X-Received":"by 2002:a19:7507:: with SMTP id y7mr815080lfe.121.1586368322699; \n\tWed, 08 Apr 2020 10:52:02 -0700 (PDT)","To":"Laurent Pinchart ","Cc":"libcamera-devel@lists.libcamera.org, Martijn Braam , \n\tBenjamin GAIGNARD ","References":"<20200320014839.14683-1-laurent.pinchart@ideasonboard.com>\n\t<20200320014839.14683-10-laurent.pinchart@ideasonboard.com>\n\t<053e8bb2-1d50-1aa5-b09a-fdb6718e3a41@linaro.org>\n\t<20200404000825.GE9690@pendragon.ideasonboard.com>","From":"Andrey Konovalov ","Message-ID":"<22ea7c4d-814c-2d59-e747-9c2698a53fdb@linaro.org>","Date":"Wed, 8 Apr 2020 20:52:00 +0300","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101\n\tThunderbird/68.4.1","MIME-Version":"1.0","In-Reply-To":"<20200404000825.GE9690@pendragon.ideasonboard.com>","Content-Type":"text/plain; charset=utf-8; format=flowed","Content-Language":"en-US","Content-Transfer-Encoding":"7bit","Subject":"Re: [libcamera-devel] [PATCH v3 09/11] libcamera: pipeline: Add a\n\tsimple pipeline handler","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","X-List-Received-Date":"Wed, 08 Apr 2020 17:52:03 -0000"}},{"id":4482,"web_url":"https://patchwork.libcamera.org/comment/4482/","msgid":"<6c615d90-e383-c936-783d-36628bc18794@linaro.org>","date":"2020-04-21T20:42:11","subject":"Re: [libcamera-devel] [PATCH v3 09/11] libcamera: pipeline: Add a\n\tsimple pipeline handler","submitter":{"id":25,"url":"https://patchwork.libcamera.org/api/people/25/","name":"Andrey Konovalov","email":"andrey.konovalov@linaro.org"},"content":"Hi Laurent,\n\nOn 08.04.2020 20:40, Andrey Konovalov wrote:\n> Hi Laurent,\n> \n> On 04.04.2020 03:12, Laurent Pinchart wrote:\n>> Hi Andrey,\n>>\n>> On Fri, Apr 03, 2020 at 04:13:35PM +0300, Andrey Konovalov wrote:\n>>> On 31.03.2020 14:45, Kieran Bingham wrote:\n>>>> On 20/03/2020 01:48, Laurent Pinchart wrote:\n>>>>> From: Martijn Braam \n>>>>>\n>>>>> This new pipeline handler aims at supporting any simple device without\n>>>>> requiring any device-specific code. Simple devices are currently defined\n>>>>> as a graph made of one or multiple camera sensors and a single video\n>>>>> node, with each sensor connected to the video node through a linear\n>>>>> pipeline.\n>>>>>\n>>>>> The simple pipeline handler will automatically parse the media graph,\n>>>>> enumerate sensors, build supported stream configurations, and configure\n>>>>> the pipeline, without any device-specific knowledge. It doesn't support\n>>>>> configuration of any processing in the pipeline at the moment, but may\n>>>>> be extended to support simple processing such as format conversion or\n>>>>> scaling in the future.\n>>>>>\n>>>>> The only device-specific information in the pipeline handler is the list\n>>>>> of supported drivers, required for device matching. We may be able to\n>>>>> remove this in the future by matching with the simple pipeline handler\n>>>>> as a last resort option, after all other pipeline handlers have been\n>>>>> tried.\n>>>>>\n>>>>> Signed-off-by: Martijn Braam \n>>>>> Signed-off-by: Laurent Pinchart \n>>>>\n>>>> Reviewed-by: Kieran Bingham \n>>>>\n>>>> Some comments and discussion points below, but I think we should work\n>>>> towards trying to get this integrated so it can actually be used (which\n>>>> is where we'll find the nuances and corner cases as people test it)\n>>>>\n>>>>> ---\n>>>>> Changes since v2:\n>>>>>\n>>>>> - Log an error when setupFormats() fail\n>>>>> - Propagate getFormat() and setFormat() errors to the caller of\n>>>>>     setupFormats()\n>>>>> - Reorder variable declarations in validate()\n>>>>> - Add \\todo comment related to the selection of the default format\n>>>>> - Use log Error instead of Info if pipeline isn't valid\n>>>>> - Rebase on top of V4L2PixelFormat\n>>>>>\n>>>>> Changes since v1:\n>>>>>\n>>>>> - Rebase on top of buffer API rework\n>>>>> - Expose stream formats\n>>>>> - Rework camera data config\n>>>>> ---\n>>>>>    src/libcamera/pipeline/meson.build        |   1 +\n>>>>>    src/libcamera/pipeline/simple/meson.build |   3 +\n>>>>>    src/libcamera/pipeline/simple/simple.cpp  | 710 ++++++++++++++++++++++\n>>>>>    3 files changed, 714 insertions(+)\n>>>>>    create mode 100644 src/libcamera/pipeline/simple/meson.build\n>>>>>    create mode 100644 src/libcamera/pipeline/simple/simple.cpp\n>>>>>\n>>>>> diff --git a/src/libcamera/pipeline/meson.build b/src/libcamera/pipeline/meson.build\n>>>>> index 0d466225a72e..606ba31a0319 100644\n>>>>> --- a/src/libcamera/pipeline/meson.build\n>>>>> +++ b/src/libcamera/pipeline/meson.build\n>>>>> @@ -5,3 +5,4 @@ libcamera_sources += files([\n>>>>>    subdir('ipu3')\n>>>>>    subdir('rkisp1')\n>>>>> +subdir('simple')\n>>>>> diff --git a/src/libcamera/pipeline/simple/meson.build b/src/libcamera/pipeline/simple/meson.build\n>>>>> new file mode 100644\n>>>>> index 000000000000..4945a3e173cf\n>>>>> --- /dev/null\n>>>>> +++ b/src/libcamera/pipeline/simple/meson.build\n>>>>> @@ -0,0 +1,3 @@\n>>>>> +libcamera_sources += files([\n>>>>> +    'simple.cpp',\n>>>>> +])\n>>>>> diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp\n>>>>> new file mode 100644\n>>>\n>>> \n>>>\n>>>>> +int SimpleCameraData::init()\n>>>>> +{\n>>>>> +    SimplePipelineHandler *pipe = static_cast(pipe_);\n>>>>> +    V4L2VideoDevice *video = pipe->video();\n>>>>> +    int ret;\n>>>>> +\n>>>>> +    /*\n>>>>> +     * Enumerate the possible pipeline configurations. For each media bus\n>>>>> +     * format supported by the sensor, propagate the formats through the\n>>>>> +     * pipeline, and enumerate the corresponding possible V4L2 pixel\n>>>>> +     * formats on the video node.\n>>>>> +     */\n>>>>> +    for (unsigned int code : sensor_->mbusCodes()) {\n>>>>> +        V4L2SubdeviceFormat format{ code, sensor_->resolution() };\n>>>>> +\n>>>>> +        /*\n>>>>> +         * Setup links first as some subdev drivers take active links\n>>>>> +         * into account to propaget TRY formats. So is life :-(\n>>>>\n>>>> /propaget/propagate/\n>>>>\n>>>> /So is life/Such is life/\n>>>>\n>>>>\n>>>>> +         */\n>>>>> +        ret = setupLinks();\n>>>>> +        if (ret < 0)\n>>>>> +            return ret;\n>>>>> +\n>>>>> +        ret = setupFormats(&format, V4L2Subdevice::TryFormat);\n>>>>> +        if (ret < 0) {\n>>>>> +            LOG(SimplePipeline, Error)\n>>>>> +                << \"Failed to setup pipeline for media bus code \"\n>>>>> +                << utils::hex(code, 4);\n>>>>\n>>>> Oh dear, seems we need mbusCode.toString() too! hehe\n>>>>\n>>>>> +            return ret;\n>>>>\n>>>> Andrey states that this return prevents iterating potentially successful\n>>>> mbus-codes, and should instead 'continue'...\n>>>\n>>> I've taken a deeper look at the ov5640 + db410c logs I've received, and the above\n>>> return *could* be a potential problem, but definitely not in this (ov5640 + db410c)\n>>> case.\n>>>\n>>> We only saw SimpleCameraData::setupFormats() returning error due to failing\n>>> sensor_->setFormat(format) call.\n>>> This happened if before the cam is started (and the stream cfg isn't set to a\n>>> particular width*height) the sensor is configured for the frame rate greater than 15 fps.\n>>> In particular, in the ov5640's driver probe function the initial mode is set to\n>>> \"YUV422 UYVY VGA@30fps\".\n>>> ov5640_enum_frame_size() doesn't consider the current fps at all, so the maximum\n>>> frame size reported is 2592x1944. But it only works at 15fps max, and an attempt\n>>> to set the sensor format to a one with the maximum possible frame size of 2592x1944\n>>> results in -EINVAL returned by SUBDEV_S_FMT.\n>>> (So in our ov5640 + db410c case replacing the \"return\" with \"continue\" does let the pipeline\n>>> handler to try all the other media bus codes if the first one fails, but they all fail the\n>>> same way as the first one does.)\n>>>\n>>> If this issue with the frame rate is worked around in some way, the enumeration\n>>> of the possible pipeline configurations completes kind of OK: a good configuration\n>>> is selected in the end. But during the enumeration all the configurations, the\n>>> broken ones included are considered as the working ones:\n>>>\n>>> The links are set up this way (\"entity name\":pad_number]):\n>>>\n>>> \"ov5640 4-0076\":0 -> \"msm_csiphy0\":0,\"msm_csiphy0\":1 -> \"msm_csid0\":0,\"msm_csid0\":1 ->\n>>>     -> \"msm_ispif0\":0,\"msm_ispif0\":1 -> \"msm_vfe0_rdi0\":0\n>>>\n>>>\n>>> SimpleCameraData::setupFormats() creates a broken pipeline when it tries the sensor format\n>>> not supported by the rest of the entities. This results in e.g. the following:\n>>>\n>>> \"ov5640 4-0076\":0[MEDIA_BUS_FMT_RGB565_2X8_BE] -> [MEDIA_BUS_FMT_UYVY8_2X8]\"msm_csiphy0\":0,\n>>>      \"msm_csiphy0\":1[MEDIA_BUS_FMT_UYVY8_2X8] -> [MEDIA_BUS_FMT_UYVY8_2X8]\"msm_csid0\":0, etc\n>>>\n>>> I.e. an attempt to set fmt on \"msm_csiphy0\":0 to MEDIA_BUS_FMT_RGB565_2X8_BE results in the\n>>> format set to MEDIA_BUS_FMT_UYVY8_2X8 and no error is returned here (and down the pipeline\n>>> as MEDIA_BUS_FMT_UYVY8_2X8 is set OK for all the other pads). So we get the first link misconfigured,\n>>> but the result is considered as a good pipeline as no error was returned while the format\n>>> from the sensor output was propagated from the sensor output down to \"msm_vfe0_rdi0\":0.\n>>> And the broken configuration is stored in the formats_ map (in the end, it works in our\n>>> ov5640+db410c case as the broken configurations are rewritten with the valid ones later).\n>>>\n>>> Who/what is to ensure that for each of the links, the source and the sink pads of the same link\n>>> are configured to the same format?\n>>\n>> The kernel is responsible for validating that at stream on time, but I\n>> think we should also verify it in setupFormats(). Would you like to send\n>> a patch on top of this series ?\n\nI've just posted the two patches to address that.\n\nThanks,\nAndrey\n\n> I've got something which gets to the point when 'cam' starts streaming, but no buffer comes from the\n> video device.\n> \n> That is now SimpleCameraData::init() enumerates the possible pipeline configurations correctly (skips\n> the broken ones). But with the patch the StreamConfiguration now uses 'BA81' format which doesn't\n> work (though looks valid for the pipeline) instead of 'YUYV' before which worked OK. (The format is\n> selected in SimplePipelineHandler::CreateConfiguration() by choosing the first entry in the formats\n> map.)\n> \n> I am looking into this now. But \"debugging by emails\" isn't the fastest process unfortunately (I don't\n> have ov5640 for db410c locally).\n> \n> \n> Thanks,\n> Andrey\n> \n>>>>> +        }\n>>>>> +\n>>>>> +        std::map> videoFormats =\n>>>>> +            video->formats(format.mbus_code);\n>>>>> +\n>>>>> +        LOG(SimplePipeline, Debug)\n>>>>> +            << \"Adding configuration for \" << format.size.toString()\n>>>>> +            << \" in pixel formats [ \"\n>>>>> +            << utils::join(videoFormats, \", \",\n>>>>> +                       [](const auto &f) {\n>>>>> +                           return f.first.toString();\n>>>>> +                       })\n>>>>\n>>>> Aha, I kinda like the utils::join!\n>>>>\n>>>>> +            << \" ]\";\n>>>>> +\n>>>>> +        /*\n>>>>> +         * Store the configuration in the formats_ map, mapping\n>>>>> +         * PixelFormat to configuration. Any previously stored value is\n>>>>\n>>>> /to configuration/to the current configuration/ ?\n>>>>\n>>>>> +         * overwritten, as the pipeline handler currently doesn't care\n>>>>> +         * about how a particular PixelFormat is achieved.\n>>>>\n>>>> As long as there's an output, I agree. If devices have specific\n>>>> constraints such as better performance or such with a particular\n>>>> mbus-code, then they need to create a platform specific pipeline handler\n>>>> to deal with the nuances.\n>>>>\n>>>> Maybe there might be tuning hints or something later, but certainly not now.\n>>>>\n>>>>\n>>>>> +         */\n>>>>> +        for (const auto &videoFormat : videoFormats) {\n>>>>> +            PixelFormat pixelFormat = video->toPixelFormat(videoFormat.first);\n>>>>> +            if (!pixelFormat)\n>>>>> +                continue;\n>>>>> +\n>>>>> +            Configuration config;\n>>>>> +            config.code = code;\n>>>>> +            config.pixelFormat = pixelFormat;\n>>>>> +            config.size = format.size;\n>>>>> +\n>>>>> +            formats_[pixelFormat] = config;\n>>>>> +        }\n>>>>> +    }\n>>>>> +\n>>>>> +    if (formats_.empty()) {\n>>>>> +        LOG(SimplePipeline, Error) << \"No valid configuration found\";\n>>>>> +        return -EINVAL;\n>>>>> +    }\n>>>>> +\n>>>>> +    return 0;\n>>>>> +}\n>>\n>> [snip]\n>>","headers":{"Return-Path":"","Received":["from mail-lj1-x241.google.com (mail-lj1-x241.google.com\n\t[IPv6:2a00:1450:4864:20::241])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 3808260406\n\tfor ;\n\tTue, 21 Apr 2020 22:42:14 +0200 (CEST)","by mail-lj1-x241.google.com with SMTP id w20so9164374ljj.0\n\tfor ;\n\tTue, 21 Apr 2020 13:42:14 -0700 (PDT)","from [192.168.118.216] (37-144-159-139.broadband.corbina.ru.\n\t[37.144.159.139]) by smtp.gmail.com with ESMTPSA id\n\tw19sm2958141ljo.54.2020.04.21.13.42.12\n\t(version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);\n\tTue, 21 Apr 2020 13:42:12 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key; \n\tunprotected) header.d=linaro.org\n\theader.i=@linaro.org header.b=\"n20d+0V1\"; \n\tdkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=subject:from:to:cc:references:message-id:date:user-agent\n\t:mime-version:in-reply-to:content-language:content-transfer-encoding; \n\tbh=UA8ZRpoH3y90NxI8kducZFIYL0sE+JkNI95iFmPSiEU=;\n\tb=n20d+0V1fO37eZgGzw0S08TQXMUbTgllh6ft5AE5XP17ZGyCMjEeq5NJwnpQWyUFsU\n\tz6RlE2OblbnyWSamoKksYrMfGErgtbQmG+LIEdBIk9zCLcGSGwDBiDLvzk3l1nuiNbxT\n\t212HHtIHnq8fkx6b+UXW4mslBQke66j5BK53Z6sVofHH0wCMrYjBQbIzrZJ7NATN9A8U\n\teonvXS07jflqXIvtMKEReHk+AdnKUjelogNOZ4gOQpnGH65zYuIXJ/fa26xzjibq28m5\n\t+U6KvDG6LLaEQwMmoYF2MftbE5p9eYBTYitL1PJt7Wt9YGjzc2WPng+U+qT5PlP/HvN6\n\tmvyA==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:subject:from:to:cc:references:message-id:date\n\t:user-agent:mime-version:in-reply-to:content-language\n\t:content-transfer-encoding;\n\tbh=UA8ZRpoH3y90NxI8kducZFIYL0sE+JkNI95iFmPSiEU=;\n\tb=FzgtXkj4p5nGprqXzuk8EHQI1sic15O7ofsWJLEcc2r0UU8IG47Mogu0oWSTeizO4v\n\tFU04g9l/RfoHXdHjcvbLBhBN2CffLGkLvjoidEk6t9h5y2EfeMX3FfNYHLplSdK9O60v\n\tLgGX/Ipa9eiO76/AvhQ2txAhGTSDLtkcypRb8eapvofFekfAbZUM+DHpjHX8XqGSj9EQ\n\twkuM9G4UD+wJmCbdS8AO5IV4I3jPc5RESslltQVwYKyMR0FrpX6bQ2NsP29fd3jk7+NK\n\t6c1ExnT2A8UOeNUEqtcYhbCxuwFUOyth/ul6paOzf50FHPaI9aiuKbuyvPMjbZ3H08Bj\n\taTyA==","X-Gm-Message-State":"AGi0PuZUJSyqvsWop9YuIltvy7xBYKeauchJpZOxAWcqAqOJB5yqOdLm\n\t6nKDg32Tzzb3aLP9Gpdtn5sd/A==","X-Google-Smtp-Source":"APiQypIqus2kHvIFeaUnxpBSA7b7FBE/gQJ3z6f3E6mjfc+tuXXwUabCHr+UHoULXi7+nYw3BLCXdg==","X-Received":"by 2002:a2e:9c09:: with SMTP id s9mr595241lji.169.1587501733488; \n\tTue, 21 Apr 2020 13:42:13 -0700 (PDT)","From":"Andrey Konovalov ","To":"Laurent Pinchart ","Cc":"kieran.bingham@ideasonboard.com, libcamera-devel@lists.libcamera.org,\n\tMartijn Braam ,\n\tBenjamin GAIGNARD ","References":"<20200320014839.14683-1-laurent.pinchart@ideasonboard.com>\n\t<20200320014839.14683-10-laurent.pinchart@ideasonboard.com>\n\t<43484c6b-44ac-4bd7-88e4-b46070840315@ideasonboard.com>\n\t<0519333c-3d17-325a-704b-664e475f556c@linaro.org>\n\t<20200404001202.GF9690@pendragon.ideasonboard.com>\n\t<13e80003-3f82-9c62-50fe-1505a753c7d9@linaro.org>","Message-ID":"<6c615d90-e383-c936-783d-36628bc18794@linaro.org>","Date":"Tue, 21 Apr 2020 23:42:11 +0300","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101\n\tThunderbird/68.7.0","MIME-Version":"1.0","In-Reply-To":"<13e80003-3f82-9c62-50fe-1505a753c7d9@linaro.org>","Content-Type":"text/plain; charset=utf-8; format=flowed","Content-Language":"en-US","Content-Transfer-Encoding":"8bit","Subject":"Re: [libcamera-devel] [PATCH v3 09/11] libcamera: pipeline: Add a\n\tsimple pipeline handler","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","X-List-Received-Date":"Tue, 21 Apr 2020 20:42:14 -0000"}}]