From patchwork Mon Aug 10 13:01:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 9277 X-Patchwork-Delegate: kieran.bingham@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 17F5ABD87C for ; Mon, 10 Aug 2020 13:01:21 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 94A3360F72; Mon, 10 Aug 2020 15:01:20 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="P0/XFj4S"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 779076038C for ; Mon, 10 Aug 2020 15:01:19 +0200 (CEST) Received: from Q.local (cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 1B921F9; Mon, 10 Aug 2020 15:01:14 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1597064474; bh=wYWeTIid+4V8JqSnee2isbIqvoQhayeEDa8xdVj1CpQ=; h=From:To:Cc:Subject:Date:From; b=P0/XFj4SJncM3vtXGf/P+LW46XSPTaryqQyhqAO07HCatz+npIWuzdygFUGjM8stB VHZMgPq/Y+bhhILIOPwb7b+gYgTzk3jAVds0Q2Vmmdq93TduaZ2/03vwp8IFWIu9pV +zeDNyznwPd/s3zrLhLuNk+AlrrNLs9byRFo3gLU= From: Kieran Bingham To: libcamera devel Date: Mon, 10 Aug 2020 14:01:10 +0100 Message-Id: <20200810130110.429191-1-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH] Documentation: Vivid Pipeline X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Provide the patches for the Vivid pipeline within the Documentation tree. These patches are not destined for direct integration, but serve as a reference for implementing a new pipeline handler. Keep the patches within the tree to allow tracking updates, and making them accessible to developers. Signed-off-by: Kieran Bingham --- .../0000-cover-letter.patch | 56 ++++++ ...ne-Introduce-skeleton-Vivid-Pipeline.patch | 139 +++++++++++++++ ...ibcamera-pipeline-vivid-Match-device.patch | 54 ++++++ ...amera-pipeline-vivid-Create-a-Camera.patch | 110 ++++++++++++ ...ne-vivid-Generate-and-validate-Strea.patch | 160 ++++++++++++++++++ ...-pipeline-vivid-Configure-the-device.patch | 71 ++++++++ ...ne-vivid-Buffer-handling-and-stream-.patch | 102 +++++++++++ ...camera-pipeline-vivid-Queue-requests.patch | 45 +++++ ...peline-vivid-Initialise-key-controls.patch | 70 ++++++++ ...amera-pipeline-vivid-Handle-controls.patch | 133 +++++++++++++++ 10 files changed, 940 insertions(+) create mode 100644 Documentation/vivid-pipeline-handler/0000-cover-letter.patch create mode 100644 Documentation/vivid-pipeline-handler/0001-libcamera-pipeline-Introduce-skeleton-Vivid-Pipeline.patch create mode 100644 Documentation/vivid-pipeline-handler/0002-libcamera-pipeline-vivid-Match-device.patch create mode 100644 Documentation/vivid-pipeline-handler/0003-libcamera-pipeline-vivid-Create-a-Camera.patch create mode 100644 Documentation/vivid-pipeline-handler/0004-libcamera-pipeline-vivid-Generate-and-validate-Strea.patch create mode 100644 Documentation/vivid-pipeline-handler/0005-libcamera-pipeline-vivid-Configure-the-device.patch create mode 100644 Documentation/vivid-pipeline-handler/0006-libcamera-pipeline-vivid-Buffer-handling-and-stream-.patch create mode 100644 Documentation/vivid-pipeline-handler/0007-libcamera-pipeline-vivid-Queue-requests.patch create mode 100644 Documentation/vivid-pipeline-handler/0008-libcamera-pipeline-vivid-Initialise-key-controls.patch create mode 100644 Documentation/vivid-pipeline-handler/0009-libcamera-pipeline-vivid-Handle-controls.patch diff --git a/Documentation/vivid-pipeline-handler/0000-cover-letter.patch b/Documentation/vivid-pipeline-handler/0000-cover-letter.patch new file mode 100644 index 000000000000..25e61d4313ff --- /dev/null +++ b/Documentation/vivid-pipeline-handler/0000-cover-letter.patch @@ -0,0 +1,56 @@ +From 03bbac3265554b73e6ddcdebaf95090062e3669f Mon Sep 17 00:00:00 2001 +From: Kieran Bingham +Date: Mon, 10 Aug 2020 13:28:23 +0100 +Subject: [PATCH 0/9] libcamera: pipeline: Add a VIVID Pipeline Handler + +Introduce a new pipeline handler to support the virtual video test +driver provided by V4L2. + +These patches are intended to serve as a code-guide for writing a new +pipeline handler, and can be referenced along side the Pipeline Handler +Writers Guide documentation. + +To test these patches, apply locally to your libcamera sources by +executing the following: + + git am Documentation/vivid-pipeline-handler/* + +Then test by ensuring you have the vivid module loaded: + + sudo modprobe vivid + +To enable the new pipeline handler the build must be re-configured. A +limitation in the meson build system means that this requires a full +reconfiguration. The easiest solution of which is to remove any existing +build and commence with a fresh build tree. + +Configure, build and test: + + rm -r build + meson build + cd build + ninja + + + +Kieran Bingham (9): + libcamera: pipeline: Introduce skeleton Vivid Pipeline + libcamera: pipeline: vivid: Match device + libcamera: pipeline: vivid: Create a Camera + libcamera: pipeline: vivid: Generate and validate StreamConfigurations + libcamera: pipeline: vivid: Configure the device + libcamera: pipeline: vivid: Buffer handling and stream control + libcamera: pipeline: vivid: Queue requests + libcamera: pipeline: vivid: Initialise key controls + libcamera: pipeline: vivid: Handle controls + + meson_options.txt | 2 +- + src/libcamera/pipeline/vivid/meson.build | 5 + + src/libcamera/pipeline/vivid/vivid.cpp | 382 +++++++++++++++++++++++ + 3 files changed, 388 insertions(+), 1 deletion(-) + create mode 100644 src/libcamera/pipeline/vivid/meson.build + create mode 100644 src/libcamera/pipeline/vivid/vivid.cpp + +-- +2.25.1 + diff --git a/Documentation/vivid-pipeline-handler/0001-libcamera-pipeline-Introduce-skeleton-Vivid-Pipeline.patch b/Documentation/vivid-pipeline-handler/0001-libcamera-pipeline-Introduce-skeleton-Vivid-Pipeline.patch new file mode 100644 index 000000000000..ca1dc48e4fb9 --- /dev/null +++ b/Documentation/vivid-pipeline-handler/0001-libcamera-pipeline-Introduce-skeleton-Vivid-Pipeline.patch @@ -0,0 +1,139 @@ +From 2480348e61bfcbc8a113a03e6090308f9b904acb Mon Sep 17 00:00:00 2001 +From: Kieran Bingham +Date: Fri, 10 Jul 2020 15:13:05 +0100 +Subject: [PATCH 1/9] libcamera: pipeline: Introduce skeleton Vivid Pipeline + +Provide all of the skeleton stubs to succesfully compile +and register a new Pipeline Handler for the Vivid test device. + +Meson must be reconfigured to ensure that this pipeline handler is +included in the selected pipelines configuration, and after building, we +can test that the PipelineHandler is successfully registered by listing +the cameras on the system with LIBCAMERA_LOG_LEVELS enabled: + +""" +LIBCAMERA_LOG_LEVELS=Pipeline:0 ./build-vivid/src/cam/cam -l +[230:30:03.624102821] [2867886] DEBUG Pipeline pipeline_handler.cpp:680 Registered pipeline handler "PipelineHandlerVivid" +Available cameras: +""" + +Signed-off-by: Kieran Bingham +--- + meson_options.txt | 2 +- + src/libcamera/pipeline/vivid/meson.build | 5 ++ + src/libcamera/pipeline/vivid/vivid.cpp | 78 ++++++++++++++++++++++++ + 3 files changed, 84 insertions(+), 1 deletion(-) + create mode 100644 src/libcamera/pipeline/vivid/meson.build + create mode 100644 src/libcamera/pipeline/vivid/vivid.cpp + +diff --git a/meson_options.txt b/meson_options.txt +index e9e815fde366..ea38f1553240 100644 +--- a/meson_options.txt ++++ b/meson_options.txt +@@ -16,7 +16,7 @@ option('gstreamer', + + option('pipelines', + type : 'array', +- choices : ['ipu3', 'raspberrypi', 'rkisp1', 'simple', 'uvcvideo', 'vimc'], ++ choices : ['ipu3', 'raspberrypi', 'rkisp1', 'simple', 'uvcvideo', 'vimc', 'vivid'], + description : 'Select which pipeline handlers to include') + + option('qcam', +diff --git a/src/libcamera/pipeline/vivid/meson.build b/src/libcamera/pipeline/vivid/meson.build +new file mode 100644 +index 000000000000..086bb825387c +--- /dev/null ++++ b/src/libcamera/pipeline/vivid/meson.build +@@ -0,0 +1,5 @@ ++# SPDX-License-Identifier: CC0-1.0 ++ ++libcamera_sources += files([ ++ 'vivid.cpp', ++]) +diff --git a/src/libcamera/pipeline/vivid/vivid.cpp b/src/libcamera/pipeline/vivid/vivid.cpp +new file mode 100644 +index 000000000000..4418f616fe84 +--- /dev/null ++++ b/src/libcamera/pipeline/vivid/vivid.cpp +@@ -0,0 +1,78 @@ ++/* SPDX-License-Identifier: LGPL-2.1-or-later */ ++/* ++ * Copyright (C) 2020, Google Inc. ++ * ++ * vivid.cpp - Pipeline handler for the vivid capture device ++ */ ++ ++#include "libcamera/internal/log.h" ++#include "libcamera/internal/pipeline_handler.h" ++ ++namespace libcamera { ++ ++LOG_DEFINE_CATEGORY(VIVID) ++ ++class PipelineHandlerVivid : public PipelineHandler ++{ ++public: ++ PipelineHandlerVivid(CameraManager *manager); ++ ++ CameraConfiguration *generateConfiguration(Camera *camera, ++ const StreamRoles &roles) override; ++ int configure(Camera *camera, CameraConfiguration *config) override; ++ ++ int exportFrameBuffers(Camera *camera, Stream *stream, ++ std::vector> *buffers) override; ++ ++ int start(Camera *camera) override; ++ void stop(Camera *camera) override; ++ ++ int queueRequestDevice(Camera *camera, Request *request) override; ++ ++ bool match(DeviceEnumerator *enumerator) override; ++}; ++ ++PipelineHandlerVivid::PipelineHandlerVivid(CameraManager *manager) ++ : PipelineHandler(manager) ++{ ++} ++ ++CameraConfiguration *PipelineHandlerVivid::generateConfiguration(Camera *camera, ++ const StreamRoles &roles) ++{ ++ return nullptr; ++} ++ ++int PipelineHandlerVivid::configure(Camera *camera, CameraConfiguration *config) ++{ ++ return -1; ++} ++ ++int PipelineHandlerVivid::exportFrameBuffers(Camera *camera, Stream *stream, ++ std::vector> *buffers) ++{ ++ return -1; ++} ++ ++int PipelineHandlerVivid::start(Camera *camera) ++{ ++ return -1; ++} ++ ++void PipelineHandlerVivid::stop(Camera *camera) ++{ ++} ++ ++int PipelineHandlerVivid::queueRequestDevice(Camera *camera, Request *request) ++{ ++ return -1; ++} ++ ++bool PipelineHandlerVivid::match(DeviceEnumerator *enumerator) ++{ ++ return false; ++} ++ ++REGISTER_PIPELINE_HANDLER(PipelineHandlerVivid); ++ ++} /* namespace libcamera */ +-- +2.25.1 + diff --git a/Documentation/vivid-pipeline-handler/0002-libcamera-pipeline-vivid-Match-device.patch b/Documentation/vivid-pipeline-handler/0002-libcamera-pipeline-vivid-Match-device.patch new file mode 100644 index 000000000000..a48eeeac1fe9 --- /dev/null +++ b/Documentation/vivid-pipeline-handler/0002-libcamera-pipeline-vivid-Match-device.patch @@ -0,0 +1,54 @@ +From d676cff10779a5a8f286cf0553f52c5f747436f0 Mon Sep 17 00:00:00 2001 +From: Kieran Bingham +Date: Fri, 10 Jul 2020 15:37:55 +0100 +Subject: [PATCH 2/9] libcamera: pipeline: vivid: Match device + +Verify that we can match on our expected device(s). + +Use a temporary debug print to check that the pipeline finds +our device: + +""" +LIBCAMERA_LOG_LEVELS=Pipeline,VIVID:0 ./src/cam/cam -l + +[230:51:10.670503423] [2872877] DEBUG VIVID vivid.cpp:81 Obtained Vivid Device +""" + +Signed-off-by: Kieran Bingham +--- + src/libcamera/pipeline/vivid/vivid.cpp | 12 +++++++++++- + 1 file changed, 11 insertions(+), 1 deletion(-) + +diff --git a/src/libcamera/pipeline/vivid/vivid.cpp b/src/libcamera/pipeline/vivid/vivid.cpp +index 4418f616fe84..9811f6ef5095 100644 +--- a/src/libcamera/pipeline/vivid/vivid.cpp ++++ b/src/libcamera/pipeline/vivid/vivid.cpp +@@ -5,6 +5,7 @@ + * vivid.cpp - Pipeline handler for the vivid capture device + */ + ++#include "libcamera/internal/device_enumerator.h" + #include "libcamera/internal/log.h" + #include "libcamera/internal/pipeline_handler.h" + +@@ -70,7 +71,16 @@ int PipelineHandlerVivid::queueRequestDevice(Camera *camera, Request *request) + + bool PipelineHandlerVivid::match(DeviceEnumerator *enumerator) + { +- return false; ++ DeviceMatch dm("vivid"); ++ dm.add("vivid-000-vid-cap"); ++ ++ MediaDevice *media = acquireMediaDevice(enumerator, dm); ++ if (!media) ++ return false; ++ ++ LOG(VIVID, Debug) << "Obtained Vivid Device"; ++ ++ return false; // Prevent infinite loops for now + } + + REGISTER_PIPELINE_HANDLER(PipelineHandlerVivid); +-- +2.25.1 + diff --git a/Documentation/vivid-pipeline-handler/0003-libcamera-pipeline-vivid-Create-a-Camera.patch b/Documentation/vivid-pipeline-handler/0003-libcamera-pipeline-vivid-Create-a-Camera.patch new file mode 100644 index 000000000000..1d8552752047 --- /dev/null +++ b/Documentation/vivid-pipeline-handler/0003-libcamera-pipeline-vivid-Create-a-Camera.patch @@ -0,0 +1,110 @@ +From f80f723a010501c6d159ae9601c2787d5d02906f Mon Sep 17 00:00:00 2001 +From: Kieran Bingham +Date: Fri, 10 Jul 2020 16:28:36 +0100 +Subject: [PATCH 3/9] libcamera: pipeline: vivid: Create a Camera + +Create a VividCameraData inheriting from the CameraData to handle camera +specific data, and use it to create and register the camera with the +CameraManager. + +This can now be tested to see that the camera becomes available to +applications: + +""" +LIBCAMERA_LOG_LEVELS=Pipeline,VIVID:0 ./src/cam/cam -l +[231:44:49.325333712] [2880028] INFO IPAManager ipa_manager.cpp:136 libcamera is not installed. Adding '/home/libcamera/build-vivid/src/ipa' to the IPA search path +[231:44:49.325428449] [2880028] WARN IPAManager ipa_manager.cpp:147 No IPA found in '/usr/local/lib/x86_64-linux-gnu/libcamera' +[231:44:49.325446253] [2880028] INFO Camera camera_manager.cpp:283 libcamera v0.0.11+713-d175334d-dirty +Available cameras: +1: vivid +""" + +Signed-off-by: Kieran Bingham +--- + src/libcamera/pipeline/vivid/vivid.cpp | 54 +++++++++++++++++++++++++- + 1 file changed, 52 insertions(+), 2 deletions(-) + +diff --git a/src/libcamera/pipeline/vivid/vivid.cpp b/src/libcamera/pipeline/vivid/vivid.cpp +index 9811f6ef5095..a8922ce70ed4 100644 +--- a/src/libcamera/pipeline/vivid/vivid.cpp ++++ b/src/libcamera/pipeline/vivid/vivid.cpp +@@ -5,14 +5,46 @@ + * vivid.cpp - Pipeline handler for the vivid capture device + */ + ++#include ++ + #include "libcamera/internal/device_enumerator.h" + #include "libcamera/internal/log.h" ++#include "libcamera/internal/media_device.h" + #include "libcamera/internal/pipeline_handler.h" ++#include "libcamera/internal/v4l2_videodevice.h" + + namespace libcamera { + + LOG_DEFINE_CATEGORY(VIVID) + ++class VividCameraData : public CameraData ++{ ++public: ++ VividCameraData(PipelineHandler *pipe, MediaDevice *media) ++ : CameraData(pipe), media_(media), video_(nullptr) ++ { ++ } ++ ++ ~VividCameraData() ++ { ++ delete video_; ++ } ++ ++ int init(); ++ ++ MediaDevice *media_; ++ V4L2VideoDevice *video_; ++ Stream stream_; ++}; ++ ++class VividCameraConfiguration : public CameraConfiguration ++{ ++public: ++ VividCameraConfiguration(); ++ ++ Status validate() override; ++}; ++ + class PipelineHandlerVivid : public PipelineHandler + { + public: +@@ -78,9 +110,27 @@ bool PipelineHandlerVivid::match(DeviceEnumerator *enumerator) + if (!media) + return false; + +- LOG(VIVID, Debug) << "Obtained Vivid Device"; ++ std::unique_ptr data = std::make_unique(this, media); ++ ++ /* Locate and open the capture video node. */ ++ if (data->init()) ++ return false; ++ ++ /* Create and register the camera. */ ++ std::set streams{ &data->stream_ }; ++ std::shared_ptr camera = Camera::create(this, data->video_->deviceName(), streams); ++ registerCamera(std::move(camera), std::move(data)); ++ ++ return true; ++} ++ ++int VividCameraData::init() ++{ ++ video_ = new V4L2VideoDevice(media_->getEntityByName("vivid-000-vid-cap")); ++ if (video_->open()) ++ return -ENODEV; + +- return false; // Prevent infinite loops for now ++ return 0; + } + + REGISTER_PIPELINE_HANDLER(PipelineHandlerVivid); +-- +2.25.1 + diff --git a/Documentation/vivid-pipeline-handler/0004-libcamera-pipeline-vivid-Generate-and-validate-Strea.patch b/Documentation/vivid-pipeline-handler/0004-libcamera-pipeline-vivid-Generate-and-validate-Strea.patch new file mode 100644 index 000000000000..43a203cffa7b --- /dev/null +++ b/Documentation/vivid-pipeline-handler/0004-libcamera-pipeline-vivid-Generate-and-validate-Strea.patch @@ -0,0 +1,160 @@ +From 928ab8ae5728677c21bd31309b782d0856441aa2 Mon Sep 17 00:00:00 2001 +From: Kieran Bingham +Date: Fri, 10 Jul 2020 16:44:01 +0100 +Subject: [PATCH 4/9] libcamera: pipeline: vivid: Generate and validate + StreamConfigurations + +Implement the support for Generating and Validating the streams the +Camera can provide. + +Vivid is a simple case with only a single stream. + +Test the configurations can be generated and reported with cam -I: + +""" +LIBCAMERA_LOG_LEVELS=Pipeline,VIVID:0 ./src/cam/cam -c 1 -I +[232:02:09.633067174] [2882911] INFO IPAManager ipa_manager.cpp:136 libcamera is not installed. Adding '/home//libcamera/build-vivid/src/ipa' to the IPA search path +[232:02:09.633332451] [2882911] WARN IPAManager ipa_manager.cpp:147 No IPA found in '/usr/local/lib/x86_64-linux-gnu/libcamera' +[232:02:09.633373414] [2882911] INFO Camera camera_manager.cpp:283 libcamera v0.0.11+714-d1ebd889-dirty +Using camera vivid +0: 1280x720-BGR888 + * Pixelformat: NV21 (320x180)-(3840x2160)/(+0,+0) + - 320x180 + - 640x360 + - 640x480 + - 1280x720 + - 1920x1080 + - 3840x2160 + * Pixelformat: NV12 (320x180)-(3840x2160)/(+0,+0) + - 320x180 + - 640x360 + - 640x480 + - 1280x720 + - 1920x1080 + - 3840x2160 + * Pixelformat: BGRA8888 (320x180)-(3840x2160)/(+0,+0) + - 320x180 + - 640x360 + - 640x480 + - 1280x720 + - 1920x1080 + - 3840x2160 + * Pixelformat: RGBA8888 (320x180)-(3840x2160)/(+0,+0) + - 320x180 + - 640x360 + - 640x480 + - 1280x720 + - 1920x1080 + - 3840x2160 + +""" + +Signed-off-by: Kieran Bingham +--- + src/libcamera/pipeline/vivid/vivid.cpp | 74 +++++++++++++++++++++++++- + 1 file changed, 73 insertions(+), 1 deletion(-) + +diff --git a/src/libcamera/pipeline/vivid/vivid.cpp b/src/libcamera/pipeline/vivid/vivid.cpp +index a8922ce70ed4..9e95bae8bc30 100644 +--- a/src/libcamera/pipeline/vivid/vivid.cpp ++++ b/src/libcamera/pipeline/vivid/vivid.cpp +@@ -6,6 +6,7 @@ + */ + + #include ++#include + + #include "libcamera/internal/device_enumerator.h" + #include "libcamera/internal/log.h" +@@ -63,8 +64,50 @@ public: + int queueRequestDevice(Camera *camera, Request *request) override; + + bool match(DeviceEnumerator *enumerator) override; ++ ++private: ++ int processControls(VividCameraData *data, Request *request); ++ ++ VividCameraData *cameraData(const Camera *camera) ++ { ++ return static_cast( ++ PipelineHandler::cameraData(camera)); ++ } + }; + ++VividCameraConfiguration::VividCameraConfiguration() ++ : CameraConfiguration() ++{ ++} ++ ++CameraConfiguration::Status VividCameraConfiguration::validate() ++{ ++ Status status = Valid; ++ ++ if (config_.empty()) ++ return Invalid; ++ ++ /* Cap the number of entries to the available streams. */ ++ if (config_.size() > 1) { ++ config_.resize(1); ++ status = Adjusted; ++ } ++ ++ StreamConfiguration &cfg = config_[0]; ++ ++ /* Adjust the pixel format. */ ++ const std::vector formats = cfg.formats().pixelformats(); ++ if (std::find(formats.begin(), formats.end(), cfg.pixelFormat) == formats.end()) { ++ cfg.pixelFormat = cfg.formats().pixelformats()[0]; ++ LOG(VIVID, Debug) << "Adjusting format to " << cfg.pixelFormat.toString(); ++ status = Adjusted; ++ } ++ ++ cfg.bufferCount = 4; ++ ++ return status; ++} ++ + PipelineHandlerVivid::PipelineHandlerVivid(CameraManager *manager) + : PipelineHandler(manager) + { +@@ -73,7 +116,36 @@ PipelineHandlerVivid::PipelineHandlerVivid(CameraManager *manager) + CameraConfiguration *PipelineHandlerVivid::generateConfiguration(Camera *camera, + const StreamRoles &roles) + { +- return nullptr; ++ CameraConfiguration *config = new VividCameraConfiguration(); ++ VividCameraData *data = cameraData(camera); ++ ++ if (roles.empty()) ++ return config; ++ ++ std::map> v4l2Formats = ++ data->video_->formats(); ++ std::map> deviceFormats; ++ std::transform(v4l2Formats.begin(), v4l2Formats.end(), ++ std::inserter(deviceFormats, deviceFormats.begin()), ++ [&](const decltype(v4l2Formats)::value_type &format) { ++ return decltype(deviceFormats)::value_type{ ++ format.first.toPixelFormat(), ++ format.second ++ }; ++ }); ++ ++ StreamFormats formats(deviceFormats); ++ StreamConfiguration cfg(formats); ++ ++ cfg.pixelFormat = formats::BGR888; ++ cfg.size = { 1280, 720 }; ++ cfg.bufferCount = 4; ++ ++ config->addConfiguration(cfg); ++ ++ config->validate(); ++ ++ return config; + } + + int PipelineHandlerVivid::configure(Camera *camera, CameraConfiguration *config) +-- +2.25.1 + diff --git a/Documentation/vivid-pipeline-handler/0005-libcamera-pipeline-vivid-Configure-the-device.patch b/Documentation/vivid-pipeline-handler/0005-libcamera-pipeline-vivid-Configure-the-device.patch new file mode 100644 index 000000000000..c750ce4e80d4 --- /dev/null +++ b/Documentation/vivid-pipeline-handler/0005-libcamera-pipeline-vivid-Configure-the-device.patch @@ -0,0 +1,71 @@ +From 08df6b40552e67c3607aa1d6802f47beec76e151 Mon Sep 17 00:00:00 2001 +From: Kieran Bingham +Date: Mon, 13 Jul 2020 09:51:51 +0100 +Subject: [PATCH 5/9] libcamera: pipeline: vivid: Configure the device + +When the configurations have been generated and validated, they can be +applied to a device. + +Vivid supports only a single stream, so it directly obtains the first +StreamConfiguration from the CameraConfiguration. + +The VIVID catpure device is a V4L2Video device, so we generate a +V4L2DeviceFormat to apply directly to the capture device node. + +Note that we explicitly convert the libcamera Format stored in +cfg.pixelFormat to a V4L2PixelFormat using the helpers provided by the +V4L2VideoDevice to ensure that any multiplanar formats are handled +correctly and accordingly. + +Following the call to set the format using the Kernel API, if the format +has been adjusted in any way by the kernel driver, then we have failed +to correctly handle the validation stages, and thus the configure +operation is idendified has having failed. + +Finally stream specific data can be directly stored and set as +reflecting the state of the stream. + +[NOTE: the cfg.setStream() call here associates the stream to the +StreamConfiguration however that should quite likely be done as part of +the validation process. TBD] + +Signed-off-by: Kieran Bingham +--- + src/libcamera/pipeline/vivid/vivid.cpp | 21 ++++++++++++++++++++- + 1 file changed, 20 insertions(+), 1 deletion(-) + +diff --git a/src/libcamera/pipeline/vivid/vivid.cpp b/src/libcamera/pipeline/vivid/vivid.cpp +index 9e95bae8bc30..dbc19424e75a 100644 +--- a/src/libcamera/pipeline/vivid/vivid.cpp ++++ b/src/libcamera/pipeline/vivid/vivid.cpp +@@ -150,7 +150,26 @@ CameraConfiguration *PipelineHandlerVivid::generateConfiguration(Camera *camera, + + int PipelineHandlerVivid::configure(Camera *camera, CameraConfiguration *config) + { +- return -1; ++ VividCameraData *data = cameraData(camera); ++ StreamConfiguration &cfg = config->at(0); ++ int ret; ++ ++ V4L2DeviceFormat format = {}; ++ format.fourcc = data->video_->toV4L2PixelFormat(cfg.pixelFormat); ++ format.size = cfg.size; ++ ++ ret = data->video_->setFormat(&format); ++ if (ret) ++ return ret; ++ ++ if (format.size != cfg.size || ++ format.fourcc != data->video_->toV4L2PixelFormat(cfg.pixelFormat)) ++ return -EINVAL; ++ ++ cfg.setStream(&data->stream_); ++ cfg.stride = format.planes[0].bpl; ++ ++ return 0; + } + + int PipelineHandlerVivid::exportFrameBuffers(Camera *camera, Stream *stream, +-- +2.25.1 + diff --git a/Documentation/vivid-pipeline-handler/0006-libcamera-pipeline-vivid-Buffer-handling-and-stream-.patch b/Documentation/vivid-pipeline-handler/0006-libcamera-pipeline-vivid-Buffer-handling-and-stream-.patch new file mode 100644 index 000000000000..b04ef3e10473 --- /dev/null +++ b/Documentation/vivid-pipeline-handler/0006-libcamera-pipeline-vivid-Buffer-handling-and-stream-.patch @@ -0,0 +1,102 @@ +From 8b6d4200c60513bdb9c435c94d512efb4f59441c Mon Sep 17 00:00:00 2001 +From: Kieran Bingham +Date: Mon, 13 Jul 2020 10:46:26 +0100 +Subject: [PATCH 6/9] libcamera: pipeline: vivid: Buffer handling and stream + control + +We can now add buffer management, and connect up our bufferReady signal +to a callback. + +Note that we provide the ability to export buffers from our capture +device (data->video_) using the exportBuffers() functionality from the +V4L2VideoDevice which allows a FrameBufferAllocater to obtain buffers +from this device. + +When buffers are obtained through the exportFrameBuffers API, they are +orphaned and left unassociated with the device, and must be reimported +at start() time anyway. This allows the same interface to be used +whether internal buffers, or external buffers are used for the stream. + +When a buffer completes, we call the buffer completion handler on the +pipeline handler, and because we have only a single stream, we can also +immediately complete the request. + +Signed-off-by: Kieran Bingham +--- + src/libcamera/pipeline/vivid/vivid.cpp | 35 ++++++++++++++++++++++++-- + 1 file changed, 33 insertions(+), 2 deletions(-) + +diff --git a/src/libcamera/pipeline/vivid/vivid.cpp b/src/libcamera/pipeline/vivid/vivid.cpp +index dbc19424e75a..1a945a744055 100644 +--- a/src/libcamera/pipeline/vivid/vivid.cpp ++++ b/src/libcamera/pipeline/vivid/vivid.cpp +@@ -32,6 +32,7 @@ public: + } + + int init(); ++ void bufferReady(FrameBuffer *buffer); + + MediaDevice *media_; + V4L2VideoDevice *video_; +@@ -175,16 +176,36 @@ int PipelineHandlerVivid::configure(Camera *camera, CameraConfiguration *config) + int PipelineHandlerVivid::exportFrameBuffers(Camera *camera, Stream *stream, + std::vector> *buffers) + { +- return -1; ++ VividCameraData *data = cameraData(camera); ++ unsigned int count = stream->configuration().bufferCount; ++ ++ return data->video_->exportBuffers(count, buffers); + } + + int PipelineHandlerVivid::start(Camera *camera) + { +- return -1; ++ VividCameraData *data = cameraData(camera); ++ unsigned int count = data->stream_.configuration().bufferCount; ++ ++ int ret = data->video_->importBuffers(count); ++ if (ret < 0) ++ return ret; ++ ++ ret = data->video_->streamOn(); ++ if (ret < 0) { ++ data->ipa_->stop(); ++ data->video_->releaseBuffers(); ++ return ret; ++ } ++ ++ return 0; + } + + void PipelineHandlerVivid::stop(Camera *camera) + { ++ VividCameraData *data = cameraData(camera); ++ data->video_->streamOff(); ++ data->video_->releaseBuffers(); + } + + int PipelineHandlerVivid::queueRequestDevice(Camera *camera, Request *request) +@@ -221,9 +242,19 @@ int VividCameraData::init() + if (video_->open()) + return -ENODEV; + ++ video_->bufferReady.connect(this, &VividCameraData::bufferReady); ++ + return 0; + } + ++void VividCameraData::bufferReady(FrameBuffer *buffer) ++{ ++ Request *request = buffer->request(); ++ ++ pipe_->completeBuffer(camera_, request, buffer); ++ pipe_->completeRequest(camera_, request); ++} ++ + REGISTER_PIPELINE_HANDLER(PipelineHandlerVivid); + + } /* namespace libcamera */ +-- +2.25.1 + diff --git a/Documentation/vivid-pipeline-handler/0007-libcamera-pipeline-vivid-Queue-requests.patch b/Documentation/vivid-pipeline-handler/0007-libcamera-pipeline-vivid-Queue-requests.patch new file mode 100644 index 000000000000..a896e6d12e37 --- /dev/null +++ b/Documentation/vivid-pipeline-handler/0007-libcamera-pipeline-vivid-Queue-requests.patch @@ -0,0 +1,45 @@ +From 22e1f2c2736cd860a63086695f1dded0b2b2aa0e Mon Sep 17 00:00:00 2001 +From: Kieran Bingham +Date: Mon, 13 Jul 2020 11:35:11 +0100 +Subject: [PATCH 7/9] libcamera: pipeline: vivid: Queue requests + +When a request is given to a pipeline handler, it must parse the request +and identify what actions the pipeline handler should take to enact on hardware. + +In the case of the VIVID pipeline handler, we identify the buffer from the only +supported stream, and queue it to the video capture device. + +Signed-off-by: Kieran Bingham +--- + src/libcamera/pipeline/vivid/vivid.cpp | 15 ++++++++++++++- + 1 file changed, 14 insertions(+), 1 deletion(-) + +diff --git a/src/libcamera/pipeline/vivid/vivid.cpp b/src/libcamera/pipeline/vivid/vivid.cpp +index 1a945a744055..4362e73f49a5 100644 +--- a/src/libcamera/pipeline/vivid/vivid.cpp ++++ b/src/libcamera/pipeline/vivid/vivid.cpp +@@ -210,7 +210,20 @@ void PipelineHandlerVivid::stop(Camera *camera) + + int PipelineHandlerVivid::queueRequestDevice(Camera *camera, Request *request) + { +- return -1; ++ VividCameraData *data = cameraData(camera); ++ FrameBuffer *buffer = request->findBuffer(&data->stream_); ++ if (!buffer) { ++ LOG(VIVID, Error) ++ << "Attempt to queue request with invalid stream"; ++ ++ return -ENOENT; ++ } ++ ++ int ret = data->video_->queueBuffer(buffer); ++ if (ret < 0) ++ return ret; ++ ++ return 0; + } + + bool PipelineHandlerVivid::match(DeviceEnumerator *enumerator) +-- +2.25.1 + diff --git a/Documentation/vivid-pipeline-handler/0008-libcamera-pipeline-vivid-Initialise-key-controls.patch b/Documentation/vivid-pipeline-handler/0008-libcamera-pipeline-vivid-Initialise-key-controls.patch new file mode 100644 index 000000000000..8e0fca45fa04 --- /dev/null +++ b/Documentation/vivid-pipeline-handler/0008-libcamera-pipeline-vivid-Initialise-key-controls.patch @@ -0,0 +1,70 @@ +From 703b1d24259934781c9f9ff80774e5845876c47f Mon Sep 17 00:00:00 2001 +From: Kieran Bingham +Date: Mon, 13 Jul 2020 12:48:37 +0100 +Subject: [PATCH 8/9] libcamera: pipeline: vivid: Initialise key controls + +The VIVID pipeline handler retains state globally of it's controls. +Ensure that when we configure this specific pipeline we set initial +parameters on the device that suit our (specific) needs. + +This introduces how controls can be set directly on a device, however +under normal circumstances controls should usually be set from libcamera +controls as part of a request. These are VIVID specific only. + +Signed-off-by: Kieran Bingham +--- + src/libcamera/pipeline/vivid/vivid.cpp | 31 ++++++++++++++++++++++++++ + 1 file changed, 31 insertions(+) + +diff --git a/src/libcamera/pipeline/vivid/vivid.cpp b/src/libcamera/pipeline/vivid/vivid.cpp +index 4362e73f49a5..1744d78f2f28 100644 +--- a/src/libcamera/pipeline/vivid/vivid.cpp ++++ b/src/libcamera/pipeline/vivid/vivid.cpp +@@ -14,6 +14,18 @@ + #include "libcamera/internal/pipeline_handler.h" + #include "libcamera/internal/v4l2_videodevice.h" + ++#define VIVID_CID_VIVID_BASE (0x00f00000 | 0xf000) ++#define VIVID_CID_VIVID_CLASS (0x00f00000 | 1) ++#define VIVID_CID_TEST_PATTERN (VIVID_CID_VIVID_BASE + 0) ++#define VIVID_CID_OSD_TEXT_MODE (VIVID_CID_VIVID_BASE + 1) ++#define VIVID_CID_HOR_MOVEMENT (VIVID_CID_VIVID_BASE + 2) ++#define VIVID_CID_VERT_MOVEMENT (VIVID_CID_VIVID_BASE + 3) ++#define VIVID_CID_SHOW_BORDER (VIVID_CID_VIVID_BASE + 4) ++#define VIVID_CID_SHOW_SQUARE (VIVID_CID_VIVID_BASE + 5) ++#define VIVID_CID_INSERT_SAV (VIVID_CID_VIVID_BASE + 6) ++#define VIVID_CID_INSERT_EAV (VIVID_CID_VIVID_BASE + 7) ++#define VIVID_CID_VBI_CAP_INTERLACED (VIVID_CID_VIVID_BASE + 8) ++ + namespace libcamera { + + LOG_DEFINE_CATEGORY(VIVID) +@@ -167,6 +179,25 @@ int PipelineHandlerVivid::configure(Camera *camera, CameraConfiguration *config) + format.fourcc != data->video_->toV4L2PixelFormat(cfg.pixelFormat)) + return -EINVAL; + ++ /* Set initial controls specific to VIVID */ ++ ControlList controls(data->video_->controls()); ++ controls.set(VIVID_CID_TEST_PATTERN, 0); /* Vertical Colour Bars */ ++ controls.set(VIVID_CID_OSD_TEXT_MODE, 0); /* Display all OSD */ ++ ++ /* Ensure clear colours configured. */ ++ controls.set(V4L2_CID_BRIGHTNESS, 128); ++ controls.set(V4L2_CID_CONTRAST, 128); ++ controls.set(V4L2_CID_SATURATION, 128); ++ ++ /* Enable movement to visualise buffer updates. */ ++ controls.set(VIVID_CID_HOR_MOVEMENT, 5); ++ ++ ret = data->video_->setControls(&controls); ++ if (ret) { ++ LOG(VIVID, Error) << "Failed to set controls: " << ret; ++ return ret < 0 ? ret : -EINVAL; ++ } ++ + cfg.setStream(&data->stream_); + cfg.stride = format.planes[0].bpl; + +-- +2.25.1 + diff --git a/Documentation/vivid-pipeline-handler/0009-libcamera-pipeline-vivid-Handle-controls.patch b/Documentation/vivid-pipeline-handler/0009-libcamera-pipeline-vivid-Handle-controls.patch new file mode 100644 index 000000000000..ea88ef44c102 --- /dev/null +++ b/Documentation/vivid-pipeline-handler/0009-libcamera-pipeline-vivid-Handle-controls.patch @@ -0,0 +1,133 @@ +From 03bbac3265554b73e6ddcdebaf95090062e3669f Mon Sep 17 00:00:00 2001 +From: Kieran Bingham +Date: Mon, 13 Jul 2020 12:56:11 +0100 +Subject: [PATCH 9/9] libcamera: pipeline: vivid: Handle controls + +When constructing the camera, we parse the available controls on the +video capture device, and map supported controls to libcamera controls, +and initialise the defaults. + +The controls are handled during queueRequestDevice for each request and +applied to the device through the capture node. + +Signed-off-by: Kieran Bingham +--- + src/libcamera/pipeline/vivid/vivid.cpp | 80 +++++++++++++++++++++++++- + 1 file changed, 79 insertions(+), 1 deletion(-) + +diff --git a/src/libcamera/pipeline/vivid/vivid.cpp b/src/libcamera/pipeline/vivid/vivid.cpp +index 1744d78f2f28..b8b8e3ae0287 100644 +--- a/src/libcamera/pipeline/vivid/vivid.cpp ++++ b/src/libcamera/pipeline/vivid/vivid.cpp +@@ -5,7 +5,11 @@ + * vivid.cpp - Pipeline handler for the vivid capture device + */ + ++#include ++ + #include ++#include ++#include + #include + + #include "libcamera/internal/device_enumerator.h" +@@ -239,6 +243,46 @@ void PipelineHandlerVivid::stop(Camera *camera) + data->video_->releaseBuffers(); + } + ++int PipelineHandlerVivid::processControls(VividCameraData *data, Request *request) ++{ ++ ControlList controls(data->video_->controls()); ++ ++ for (auto it : request->controls()) { ++ unsigned int id = it.first; ++ unsigned int offset; ++ uint32_t cid; ++ ++ if (id == controls::Brightness) { ++ cid = V4L2_CID_BRIGHTNESS; ++ offset = 128; ++ } else if (id == controls::Contrast) { ++ cid = V4L2_CID_CONTRAST; ++ offset = 0; ++ } else if (id == controls::Saturation) { ++ cid = V4L2_CID_SATURATION; ++ offset = 0; ++ } else { ++ continue; ++ } ++ ++ int32_t value = lroundf(it.second.get() * 128 + offset); ++ controls.set(cid, utils::clamp(value, 0, 255)); ++ } ++ ++ for (const auto &ctrl : controls) ++ LOG(VIVID, Debug) ++ << "Setting control " << utils::hex(ctrl.first) ++ << " to " << ctrl.second.toString(); ++ ++ int ret = data->video_->setControls(&controls); ++ if (ret) { ++ LOG(VIVID, Error) << "Failed to set controls: " << ret; ++ return ret < 0 ? ret : -EINVAL; ++ } ++ ++ return ret; ++} ++ + int PipelineHandlerVivid::queueRequestDevice(Camera *camera, Request *request) + { + VividCameraData *data = cameraData(camera); +@@ -250,7 +294,11 @@ int PipelineHandlerVivid::queueRequestDevice(Camera *camera, Request *request) + return -ENOENT; + } + +- int ret = data->video_->queueBuffer(buffer); ++ int ret = processControls(data, request); ++ if (ret < 0) ++ return ret; ++ ++ ret = data->video_->queueBuffer(buffer); + if (ret < 0) + return ret; + +@@ -288,6 +336,36 @@ int VividCameraData::init() + + video_->bufferReady.connect(this, &VividCameraData::bufferReady); + ++ /* Initialise the supported controls. */ ++ const ControlInfoMap &controls = video_->controls(); ++ ControlInfoMap::Map ctrls; ++ ++ for (const auto &ctrl : controls) { ++ const ControlId *id; ++ ControlInfo info; ++ ++ switch (ctrl.first->id()) { ++ case V4L2_CID_BRIGHTNESS: ++ id = &controls::Brightness; ++ info = ControlInfo{ { -1.0f }, { 1.0f }, { 0.0f } }; ++ break; ++ case V4L2_CID_CONTRAST: ++ id = &controls::Contrast; ++ info = ControlInfo{ { 0.0f }, { 2.0f }, { 1.0f } }; ++ break; ++ case V4L2_CID_SATURATION: ++ id = &controls::Saturation; ++ info = ControlInfo{ { 0.0f }, { 2.0f }, { 1.0f } }; ++ break; ++ default: ++ continue; ++ } ++ ++ ctrls.emplace(id, info); ++ } ++ ++ controlInfo_ = std::move(ctrls); ++ + return 0; + } + +-- +2.25.1 +