{"id":3744,"url":"https://patchwork.libcamera.org/api/patches/3744/?format=json","web_url":"https://patchwork.libcamera.org/patch/3744/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20200510115810.21938-6-laurent.pinchart@ideasonboard.com>","date":"2020-05-10T11:58:09","name":"[libcamera-devel,v5,5/6] libcamera: pipeline: simple: Integrate converter support","commit_ref":null,"pull_url":null,"state":"accepted","archived":false,"hash":"e42a9328df1778f4543fbe24f2b704b59761e05f","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/?format=json","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/3744/mbox/","series":[{"id":890,"url":"https://patchwork.libcamera.org/api/series/890/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=890","date":"2020-05-10T11:58:04","name":"Simple pipeline handler","version":5,"mbox":"https://patchwork.libcamera.org/series/890/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/3744/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/3744/checks/","tags":{},"headers":{"Return-Path":"","Received":["from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id B9D8D60BEC\n\tfor ;\n\tSun, 10 May 2020 13:58:22 +0200 (CEST)","from pendragon.bb.dnainternet.fi (81-175-216-236.bb.dnainternet.fi\n\t[81.175.216.236])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 08C4C73E;\n\tSun, 10 May 2020 13:58:21 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ideasonboard.com\n\theader.i=@ideasonboard.com\n\theader.b=\"EidLgGt9\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1589111902;\n\tbh=+wZqfUbabNXDp4NXGLaBy/11cFyxT8yJfj+mt8/YUjc=;\n\th=From:To:Cc:Subject:Date:In-Reply-To:References:From;\n\tb=EidLgGt9Ne05EG53K2TBqlPCqflNwJmso0WnSoO1eQ5piZpkOCSCj1n8Wly+VGMt8\n\tWg95qXRjSPWo2XqX4X1kql/GxVUVjpnpaB2aDJJA8PV0XSVF6WG7qin4e9FJ/91M8F\n\t6PXJGBq/kf0oZJntthRs4umdRtLUYgLIuDljSFVE=","From":"Laurent Pinchart ","To":"libcamera-devel@lists.libcamera.org","Cc":"Martijn Braam ","Date":"Sun, 10 May 2020 14:58:09 +0300","Message-Id":"<20200510115810.21938-6-laurent.pinchart@ideasonboard.com>","X-Mailer":"git-send-email 2.26.2","In-Reply-To":"<20200510115810.21938-1-laurent.pinchart@ideasonboard.com>","References":"<20200510115810.21938-1-laurent.pinchart@ideasonboard.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=UTF-8","Content-Transfer-Encoding":"8bit","Subject":"[libcamera-devel] [PATCH v5 5/6] libcamera: pipeline: simple:\n\tIntegrate converter support","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","X-List-Received-Date":"Sun, 10 May 2020 11:58:24 -0000"},"content":"Add support for an optional format converter, supported by the\nSimpleConverter class. If a converter is available for the pipeline, it\nwill be used to expose additional pixel formats.\n\nSigned-off-by: Laurent Pinchart \nReviewed-by: Niklas Söderlund \n---\nChanges since v2:\n\n- Rebase on top of V4L2PixelFormat\n- Don't crash if the device has no converter\n---\n src/libcamera/pipeline/simple/simple.cpp | 196 ++++++++++++++++++++---\n 1 file changed, 178 insertions(+), 18 deletions(-)","diff":"diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp\nindex 95417500b6b5..d5a3bf61252b 100644\n--- a/src/libcamera/pipeline/simple/simple.cpp\n+++ b/src/libcamera/pipeline/simple/simple.cpp\n@@ -11,6 +11,7 @@\n #include \n #include \n #include \n+#include \n #include \n #include \n #include \n@@ -31,17 +32,24 @@\n #include \"v4l2_subdevice.h\"\n #include \"v4l2_videodevice.h\"\n \n+#include \"converter.h\"\n+\n namespace libcamera {\n \n LOG_DEFINE_CATEGORY(SimplePipeline)\n \n class SimplePipelineHandler;\n \n+struct SimplePipelineInfo {\n+\tconst char *driver;\n+\tconst char *converter;\n+};\n+\n namespace {\n \n-static const char * const drivers[] = {\n-\t\"imx7-csi\",\n-\t\"sun6i-csi\",\n+static const SimplePipelineInfo supportedDevices[] = {\n+\t{ \"imx7-csi\", \"pxp\" },\n+\t{ \"sun6i-csi\", nullptr },\n };\n \n } /* namespace */\n@@ -88,6 +96,8 @@ public:\n \n \tconst V4L2SubdeviceFormat &sensorFormat() { return sensorFormat_; }\n \n+\tbool needConversion() const { return needConversion_; }\n+\n private:\n \t/*\n \t * The SimpleCameraData instance is guaranteed to be valid as long as\n@@ -98,6 +108,7 @@ private:\n \tconst SimpleCameraData *data_;\n \n \tV4L2SubdeviceFormat sensorFormat_;\n+\tbool needConversion_;\n };\n \n class SimplePipelineHandler : public PipelineHandler\n@@ -120,6 +131,7 @@ public:\n \n \tV4L2VideoDevice *video() { return video_; }\n \tV4L2Subdevice *subdev(const MediaEntity *entity);\n+\tSimpleConverter *converter() { return converter_; }\n \n protected:\n \tint queueRequestDevice(Camera *camera, Request *request) override;\n@@ -136,11 +148,17 @@ private:\n \tint createCamera(MediaEntity *sensor);\n \n \tvoid bufferReady(FrameBuffer *buffer);\n+\tvoid converterDone(FrameBuffer *input, FrameBuffer *output);\n \n \tMediaDevice *media_;\n \tV4L2VideoDevice *video_;\n \tstd::map subdevs_;\n \n+\tSimpleConverter *converter_;\n+\tbool useConverter_;\n+\tstd::vector> converterBuffers_;\n+\tstd::queue converterQueue_;\n+\n \tCamera *activeCamera_;\n };\n \n@@ -222,6 +240,7 @@ int SimpleCameraData::init()\n {\n \tSimplePipelineHandler *pipe = static_cast(pipe_);\n \tV4L2VideoDevice *video = pipe->video();\n+\tSimpleConverter *converter = pipe->converter();\n \tint ret;\n \n \t/*\n@@ -278,7 +297,13 @@ int SimpleCameraData::init()\n \t\t\tconfig.pixelFormat = pixelFormat;\n \t\t\tconfig.size = format.size;\n \n-\t\t\tformats_[pixelFormat] = config;\n+\t\t\tif (!converter) {\n+\t\t\t\tformats_[pixelFormat] = config;\n+\t\t\t\tcontinue;\n+\t\t\t}\n+\n+\t\t\tfor (PixelFormat format : converter->formats(pixelFormat))\n+\t\t\t\tformats_[format] = config;\n \t\t}\n \t}\n \n@@ -417,6 +442,8 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate()\n \t\tstatus = Adjusted;\n \t}\n \n+\tneedConversion_ = cfg.pixelFormat != pipeConfig.pixelFormat;\n+\n \tcfg.bufferCount = 3;\n \n \treturn status;\n@@ -427,13 +454,14 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate()\n */\n \n SimplePipelineHandler::SimplePipelineHandler(CameraManager *manager)\n-\t: PipelineHandler(manager), video_(nullptr)\n+\t: PipelineHandler(manager), video_(nullptr), converter_(nullptr)\n {\n }\n \n SimplePipelineHandler::~SimplePipelineHandler()\n {\n \tdelete video_;\n+\tdelete converter_;\n }\n \n CameraConfiguration *SimplePipelineHandler::generateConfiguration(Camera *camera,\n@@ -499,22 +527,37 @@ int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c)\n \t\treturn ret;\n \n \t/* Configure the video node. */\n-\tV4L2PixelFormat videoFormat = video_->toV4L2PixelFormat(cfg.pixelFormat);\n+\tV4L2PixelFormat videoFormat = video_->toV4L2PixelFormat(pipeConfig.pixelFormat);\n \n-\tV4L2DeviceFormat outputFormat = {};\n-\toutputFormat.fourcc = videoFormat;\n-\toutputFormat.size = cfg.size;\n+\tV4L2DeviceFormat captureFormat = {};\n+\tcaptureFormat.fourcc = videoFormat;\n+\tcaptureFormat.size = cfg.size;\n \n-\tret = video_->setFormat(&outputFormat);\n+\tret = video_->setFormat(&captureFormat);\n \tif (ret)\n \t\treturn ret;\n \n-\tif (outputFormat.size != cfg.size || outputFormat.fourcc != videoFormat) {\n+\tif (captureFormat.fourcc != videoFormat || captureFormat.size != cfg.size) {\n \t\tLOG(SimplePipeline, Error)\n \t\t\t<< \"Unable to configure capture in \" << cfg.toString();\n \t\treturn -EINVAL;\n \t}\n \n+\t/* Configure the converter if required. */\n+\tuseConverter_ = config->needConversion();\n+\n+\tif (useConverter_) {\n+\t\tint ret = converter_->configure(pipeConfig.pixelFormat,\n+\t\t\t\t\t\tcfg.pixelFormat, cfg.size);\n+\t\tif (ret < 0) {\n+\t\t\tLOG(SimplePipeline, Error)\n+\t\t\t\t<< \"Unable to configure converter\";\n+\t\t\treturn ret;\n+\t\t}\n+\n+\t\tLOG(SimplePipeline, Debug) << \"Using format converter\";\n+\t}\n+\n \tcfg.setStream(&data->stream_);\n \n \treturn 0;\n@@ -525,24 +568,47 @@ int SimplePipelineHandler::exportFrameBuffers(Camera *camera, Stream *stream,\n {\n \tunsigned int count = stream->configuration().bufferCount;\n \n-\treturn video_->exportBuffers(count, buffers);\n+\t/*\n+\t * Export buffers on the converter or capture video node, depending on\n+\t * whether the converter is used or not.\n+\t */\n+\tif (useConverter_)\n+\t\treturn converter_->exportBuffers(count, buffers);\n+\telse\n+\t\treturn video_->exportBuffers(count, buffers);\n }\n \n int SimplePipelineHandler::start(Camera *camera)\n {\n \tSimpleCameraData *data = cameraData(camera);\n \tunsigned int count = data->stream_.configuration().bufferCount;\n+\tint ret;\n \n-\tint ret = video_->importBuffers(count);\n+\tif (useConverter_)\n+\t\tret = video_->allocateBuffers(count, &converterBuffers_);\n+\telse\n+\t\tret = video_->importBuffers(count);\n \tif (ret < 0)\n \t\treturn ret;\n \n \tret = video_->streamOn();\n \tif (ret < 0) {\n-\t\tvideo_->releaseBuffers();\n+\t\tstop(camera);\n \t\treturn ret;\n \t}\n \n+\tif (useConverter_) {\n+\t\tret = converter_->start(count);\n+\t\tif (ret < 0) {\n+\t\t\tstop(camera);\n+\t\t\treturn ret;\n+\t\t}\n+\n+\t\t/* Queue all internal buffers for capture. */\n+\t\tfor (std::unique_ptr &buffer : converterBuffers_)\n+\t\t\tvideo_->queueBuffer(buffer.get());\n+\t}\n+\n \tactiveCamera_ = camera;\n \n \treturn 0;\n@@ -550,8 +616,13 @@ int SimplePipelineHandler::start(Camera *camera)\n \n void SimplePipelineHandler::stop(Camera *camera)\n {\n+\tif (useConverter_)\n+\t\tconverter_->stop();\n+\n \tvideo_->streamOff();\n \tvideo_->releaseBuffers();\n+\n+\tconverterBuffers_.clear();\n \tactiveCamera_ = nullptr;\n }\n \n@@ -567,6 +638,15 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request)\n \t\treturn -ENOENT;\n \t}\n \n+\t/*\n+\t * If conversion is needed, push the buffer to the converter queue, it\n+\t * will be handed to the converter in the capture completion handler.\n+\t */\n+\tif (useConverter_) {\n+\t\tconverterQueue_.push(buffer);\n+\t\treturn 0;\n+\t}\n+\n \treturn video_->queueBuffer(buffer);\n }\n \n@@ -576,11 +656,20 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request)\n \n bool SimplePipelineHandler::match(DeviceEnumerator *enumerator)\n {\n-\tfor (const char *driver : drivers) {\n-\t\tDeviceMatch dm(driver);\n+\tMediaDevice *converter = nullptr;\n+\n+\tfor (const SimplePipelineInfo &info : supportedDevices) {\n+\t\tDeviceMatch dm(info.driver);\n \t\tmedia_ = acquireMediaDevice(enumerator, dm);\n-\t\tif (media_)\n+\t\tif (!media_)\n+\t\t\tcontinue;\n+\n+\t\tif (!info.converter)\n \t\t\tbreak;\n+\n+\t\tDeviceMatch converterMatch(info.converter);\n+\t\tconverter = acquireMediaDevice(enumerator, converterMatch);\n+\t\tbreak;\n \t}\n \n \tif (!media_)\n@@ -635,6 +724,19 @@ bool SimplePipelineHandler::match(DeviceEnumerator *enumerator)\n \n \tvideo_->bufferReady.connect(this, &SimplePipelineHandler::bufferReady);\n \n+\t/* Open the converter, if any. */\n+\tif (converter) {\n+\t\tconverter_ = new SimpleConverter(converter);\n+\t\tif (converter_->open() < 0) {\n+\t\t\tLOG(SimplePipeline, Warning)\n+\t\t\t\t<< \"Failed to open converter, disabling format conversion\";\n+\t\t\tdelete converter_;\n+\t\t\tconverter_ = nullptr;\n+\t\t}\n+\n+\t\tconverter_->bufferReady.connect(this, &SimplePipelineHandler::converterDone);\n+\t}\n+\n \t/*\n \t * Create one camera data instance for each sensor and gather all\n \t * entities in all pipelines.\n@@ -709,12 +811,70 @@ V4L2Subdevice *SimplePipelineHandler::subdev(const MediaEntity *entity)\n \n void SimplePipelineHandler::bufferReady(FrameBuffer *buffer)\n {\n-\tASSERT(activeCamera_);\n+\t/*\n+\t * If an error occurred during capture, or if the buffer was cancelled,\n+\t * complete the request, even if the converter is in use as there's no\n+\t * point converting an erroneous buffer.\n+\t */\n+\tif (buffer->metadata().status != FrameMetadata::FrameSuccess) {\n+\t\tif (useConverter_) {\n+\t\t\t/* Requeue the buffer for capture. */\n+\t\t\tvideo_->queueBuffer(buffer);\n+\n+\t\t\t/*\n+\t\t\t * Get the next user-facing buffer to complete the\n+\t\t\t * request.\n+\t\t\t */\n+\t\t\tif (converterQueue_.empty())\n+\t\t\t\treturn;\n+\n+\t\t\tbuffer = converterQueue_.front();\n+\t\t\tconverterQueue_.pop();\n+\t\t}\n+\n+\t\tRequest *request = buffer->request();\n+\t\tcompleteBuffer(activeCamera_, request, buffer);\n+\t\tcompleteRequest(activeCamera_, request);\n+\t\treturn;\n+\t}\n+\n+\t/*\n+\t * Queue the captured and the request buffer to the converter if format\n+\t * conversion is needed. If there's no queued request, just requeue the\n+\t * captured buffer for capture.\n+\t */\n+\tif (useConverter_) {\n+\t\tif (converterQueue_.empty()) {\n+\t\t\tvideo_->queueBuffer(buffer);\n+\t\t\treturn;\n+\t\t}\n+\n+\t\tFrameBuffer *output = converterQueue_.front();\n+\t\tconverterQueue_.pop();\n+\n+\t\tconverter_->queueBuffers(buffer, output);\n+\t\treturn;\n+\t}\n+\n+\t/* Otherwise simply complete the request. */\n \tRequest *request = buffer->request();\n \tcompleteBuffer(activeCamera_, request, buffer);\n \tcompleteRequest(activeCamera_, request);\n }\n \n+void SimplePipelineHandler::converterDone(FrameBuffer *input,\n+\t\t\t\t\t FrameBuffer *output)\n+{\n+\t/* Complete the request. */\n+\tASSERT(activeCamera_);\n+\tRequest *request = output->request();\n+\tcompleteBuffer(activeCamera_, request, output);\n+\tcompleteRequest(activeCamera_, request);\n+\n+\t/* Queue the input buffer back for capture. */\n+\tvideo_->queueBuffer(input);\n+}\n+\n REGISTER_PIPELINE_HANDLER(SimplePipelineHandler);\n \n } /* namespace libcamera */\n","prefixes":["libcamera-devel","v5","5/6"]}