From patchwork Thu Aug 8 00:04:01 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 1745 Return-Path: Received: from vsp-unauthed02.binero.net (vsp-unauthed02.binero.net [195.74.38.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3C07260E31 for ; Thu, 8 Aug 2019 02:04:10 +0200 (CEST) X-Halon-ID: 068f0e8b-b970-11e9-bdc3-005056917a89 Authorized-sender: niklas@soderlund.pp.se Received: from bismarck.berto.se (unknown [145.14.112.32]) by bin-vsp-out-01.atm.binero.net (Halon) with ESMTPA id 068f0e8b-b970-11e9-bdc3-005056917a89; Thu, 08 Aug 2019 02:04:01 +0200 (CEST) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org Date: Thu, 8 Aug 2019 02:04:01 +0200 Message-Id: <20190808000404.10841-2-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20190808000404.10841-1-niklas.soderlund@ragnatech.se> References: <20190808000404.10841-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 1/4] libcamera: pipeline: vimc: Initialize device pointers X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Aug 2019 00:04:10 -0000 As the device pointers are deleted in the pipelines destructor it's not a good idea to have them uninitialized. Signed-off-by: Niklas Söderlund Reviewed-by: Laurent Pinchart --- src/libcamera/pipeline/vimc.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/vimc.cpp b/src/libcamera/pipeline/vimc.cpp index 61b68a32ea508af8..3d6808222a8a2c5d 100644 --- a/src/libcamera/pipeline/vimc.cpp +++ b/src/libcamera/pipeline/vimc.cpp @@ -35,7 +35,7 @@ class VimcCameraData : public CameraData { public: VimcCameraData(PipelineHandler *pipe) - : CameraData(pipe) + : CameraData(pipe), video_(nullptr), sensor_(nullptr) { } From patchwork Thu Aug 8 00:04:02 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 1746 Return-Path: Received: from bin-mail-out-06.binero.net (bin-mail-out-06.binero.net [195.74.38.229]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BFF8961623 for ; Thu, 8 Aug 2019 02:04:10 +0200 (CEST) X-Halon-ID: 06e79393-b970-11e9-bdc3-005056917a89 Authorized-sender: niklas@soderlund.pp.se Received: from bismarck.berto.se (unknown [145.14.112.32]) by bin-vsp-out-01.atm.binero.net (Halon) with ESMTPA id 06e79393-b970-11e9-bdc3-005056917a89; Thu, 08 Aug 2019 02:04:01 +0200 (CEST) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org Date: Thu, 8 Aug 2019 02:04:02 +0200 Message-Id: <20190808000404.10841-3-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20190808000404.10841-1-niklas.soderlund@ragnatech.se> References: <20190808000404.10841-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 2/4] libcamera: pipeline: vimc: Switch to using the RGB/YUV Capture video node X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Aug 2019 00:04:11 -0000 Linux commit 85ab1aa1fac17bcd ("media: vimc: deb: fix default sink bayer format") which is part of v5.2 changes the default media bus format for the debayer subdevices. This leads to a -EPIPE error when trying to use the raw capture video device nodes. Fix this by moving the vimc pipeline to use the RGB/YUV Capture capture video node. As a consequence of this change the scaler in the vimc pipeline is used and a hard coded upscale of 3 is present in the video pipeline. This limits the sizes exposed and accepted by libcamera to multiples of 3. The raw capture video node still needs to be handled by the pipeline as its format needs to be updated to allow the pipeline format validation to pass. Signed-off-by: Niklas Söderlund Reviewed-by: Laurent Pinchart --- src/libcamera/pipeline/vimc.cpp | 112 ++++++++++++++++++++++++++++---- test/camera/buffer_import.cpp | 8 +-- 2 files changed, 104 insertions(+), 16 deletions(-) diff --git a/src/libcamera/pipeline/vimc.cpp b/src/libcamera/pipeline/vimc.cpp index 3d6808222a8a2c5d..55975cae066d71cd 100644 --- a/src/libcamera/pipeline/vimc.cpp +++ b/src/libcamera/pipeline/vimc.cpp @@ -10,6 +10,8 @@ #include #include +#include + #include #include #include @@ -25,6 +27,7 @@ #include "pipeline_handler.h" #include "utils.h" #include "v4l2_controls.h" +#include "v4l2_subdevice.h" #include "v4l2_videodevice.h" namespace libcamera { @@ -35,21 +38,28 @@ class VimcCameraData : public CameraData { public: VimcCameraData(PipelineHandler *pipe) - : CameraData(pipe), video_(nullptr), sensor_(nullptr) + : CameraData(pipe), sensor_(nullptr), debayer_(nullptr), + scaler_(nullptr), video_(nullptr), raw_(nullptr) { } ~VimcCameraData() { delete sensor_; + delete debayer_; + delete scaler_; delete video_; + delete raw_; } int init(MediaDevice *media); void bufferReady(Buffer *buffer); - V4L2VideoDevice *video_; CameraSensor *sensor_; + V4L2Subdevice *debayer_; + V4L2Subdevice *scaler_; + V4L2VideoDevice *video_; + V4L2VideoDevice *raw_; Stream stream_; }; @@ -80,6 +90,8 @@ public: int queueRequest(Camera *camera, Request *request) override; + int initLinks(MediaDevice *media); + bool match(DeviceEnumerator *enumerator) override; private: @@ -131,8 +143,11 @@ CameraConfiguration::Status VimcCameraConfiguration::validate() /* Clamp the size based on the device limits. */ const Size size = cfg.size; - cfg.size.width = std::max(16U, std::min(4096U, cfg.size.width)); - cfg.size.height = std::max(16U, std::min(2160U, cfg.size.height)); + /* Dimensions need to be a multiple of 3 for the scaler to work. */ + cfg.size.width = std::max(18U, std::min(4096U, cfg.size.width)); + cfg.size.height = std::max(18U, std::min(2160U, cfg.size.height)); + cfg.size.width -= cfg.size.width % 3; + cfg.size.height -= cfg.size.height % 3; if (cfg.size != size) { LOG(VIMC, Debug) @@ -160,7 +175,7 @@ CameraConfiguration *PipelineHandlerVimc::generateConfiguration(Camera *camera, StreamConfiguration cfg{}; cfg.pixelFormat = V4L2_PIX_FMT_RGB24; - cfg.size = { 640, 480 }; + cfg.size = { 1920, 1080 }; cfg.bufferCount = 4; config->addConfiguration(cfg); @@ -176,6 +191,33 @@ int PipelineHandlerVimc::configure(Camera *camera, CameraConfiguration *config) StreamConfiguration &cfg = config->at(0); int ret; + /* The scaler hardcodes a x3 scale-up ratio. */ + V4L2SubdeviceFormat subformat = {}; + subformat.mbus_code = MEDIA_BUS_FMT_SGRBG8_1X8; + subformat.size = { cfg.size.width / 3, cfg.size.height / 3 }; + + ret = data->sensor_->setFormat(&subformat); + if (ret) + return ret; + + ret = data->debayer_->setFormat(0, &subformat); + if (ret) + return ret; + + subformat.mbus_code = MEDIA_BUS_FMT_RGB888_1X24; + ret = data->debayer_->setFormat(1, &subformat); + if (ret) + return ret; + + ret = data->scaler_->setFormat(0, &subformat); + if (ret) + return ret; + + subformat.size = cfg.size; + ret = data->scaler_->setFormat(1, &subformat); + if (ret) + return ret; + V4L2DeviceFormat format = {}; format.fourcc = cfg.pixelFormat; format.size = cfg.size; @@ -188,6 +230,17 @@ int PipelineHandlerVimc::configure(Camera *camera, CameraConfiguration *config) format.fourcc != cfg.pixelFormat) return -EINVAL; + /* + * Format has to be set on the raw capture video node, otherwise the + * vimc driver will fail pipeline validation. + */ + format.fourcc = V4L2_PIX_FMT_SGRBG8; + format.size = { cfg.size.width / 3, cfg.size.height / 3 }; + + ret = data->raw_->setFormat(&format); + if (ret) + return ret; + cfg.setStream(&data->stream_); return 0; @@ -292,6 +345,26 @@ int PipelineHandlerVimc::queueRequest(Camera *camera, Request *request) return 0; } +int PipelineHandlerVimc::initLinks(MediaDevice *media) +{ + MediaLink *link; + int ret; + + ret = media->disableLinks(); + if (ret < 0) + return ret; + + link = media->link("Debayer B", 1, "Scaler", 0); + if (!link) + return -ENODEV; + + ret = link->setEnabled(true); + if (ret < 0) + return ret; + + return 0; +} + bool PipelineHandlerVimc::match(DeviceEnumerator *enumerator) { DeviceMatch dm("vimc"); @@ -318,6 +391,9 @@ bool PipelineHandlerVimc::match(DeviceEnumerator *enumerator) std::unique_ptr data = utils::make_unique(this); + if (initLinks(media)) + return false; + /* Locate and open the capture video node. */ if (data->init(media)) return false; @@ -335,18 +411,30 @@ int VimcCameraData::init(MediaDevice *media) { int ret; - /* Create and open the video device and the camera sensor. */ - video_ = new V4L2VideoDevice(media->getEntityByName("Raw Capture 1")); - if (video_->open()) - return -ENODEV; - - video_->bufferReady.connect(this, &VimcCameraData::bufferReady); - + /* Create and open the camera sensor, debayer, scaler and video device. */ sensor_ = new CameraSensor(media->getEntityByName("Sensor B")); ret = sensor_->init(); if (ret) return ret; + debayer_ = new V4L2Subdevice(media->getEntityByName("Debayer B")); + if (debayer_->open()) + return -ENODEV; + + scaler_ = new V4L2Subdevice(media->getEntityByName("Scaler")); + if (scaler_->open()) + return -ENODEV; + + video_ = new V4L2VideoDevice(media->getEntityByName("RGB/YUV Capture")); + if (video_->open()) + return -ENODEV; + + video_->bufferReady.connect(this, &VimcCameraData::bufferReady); + + raw_ = new V4L2VideoDevice(media->getEntityByName("Raw Capture 1")); + if (raw_->open()) + return -ENODEV; + /* Initialise the supported controls. */ const V4L2ControlInfoMap &controls = sensor_->controls(); for (const auto &ctrl : controls) { diff --git a/test/camera/buffer_import.cpp b/test/camera/buffer_import.cpp index 400d02b350c1aa8f..620ee66dee0570bc 100644 --- a/test/camera/buffer_import.cpp +++ b/test/camera/buffer_import.cpp @@ -72,12 +72,12 @@ public: return ret; } - format_.size.width = 640; - format_.size.height = 480; + format_.size.width = 1920; + format_.size.height = 1080; format_.fourcc = V4L2_PIX_FMT_RGB24; format_.planesCount = 1; - format_.planes[0].size = 640 * 480 * 3; - format_.planes[0].bpl = 640 * 3; + format_.planes[0].size = 1920 * 1080 * 3; + format_.planes[0].bpl = 1920 * 3; if (video_->setFormat(&format_)) { cleanup(); From patchwork Thu Aug 8 00:04:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 1747 Return-Path: Received: from vsp-unauthed02.binero.net (vsp-unauthed02.binero.net [195.74.38.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 47EF66161A for ; Thu, 8 Aug 2019 02:04:11 +0200 (CEST) X-Halon-ID: 0751ce3d-b970-11e9-bdc3-005056917a89 Authorized-sender: niklas@soderlund.pp.se Received: from bismarck.berto.se (unknown [145.14.112.32]) by bin-vsp-out-01.atm.binero.net (Halon) with ESMTPA id 0751ce3d-b970-11e9-bdc3-005056917a89; Thu, 08 Aug 2019 02:04:02 +0200 (CEST) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org Date: Thu, 8 Aug 2019 02:04:03 +0200 Message-Id: <20190808000404.10841-4-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20190808000404.10841-1-niklas.soderlund@ragnatech.se> References: <20190808000404.10841-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 3/4] tests: v4l2_videodevice: Set media bus and pixel formats for vimc X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Aug 2019 00:04:12 -0000 Most of the video device tests are based on vimc and Linux commit 85ab1aa1fac17bcd ("media: vimc: deb: fix default sink bayer format") changes the default media bus format for the debayer subdevices. This leads to a -EPIPE error when trying to use the raw capture video device nodes. Fix this by explicitly setting media bus and pixel formats to known good values which works before and after the upstream change. Signed-off-by: Niklas Söderlund Reviewed-by: Laurent Pinchart --- .../v4l2_videodevice_test.cpp | 26 +++++++++++++++++++ test/v4l2_videodevice/v4l2_videodevice_test.h | 7 ++++- 2 files changed, 32 insertions(+), 1 deletion(-) diff --git a/test/v4l2_videodevice/v4l2_videodevice_test.cpp b/test/v4l2_videodevice/v4l2_videodevice_test.cpp index b26d06ad45197c8f..a0d269fef7f43895 100644 --- a/test/v4l2_videodevice/v4l2_videodevice_test.cpp +++ b/test/v4l2_videodevice/v4l2_videodevice_test.cpp @@ -8,6 +8,8 @@ #include #include +#include + #include "v4l2_videodevice_test.h" #include "device_enumerator.h" @@ -69,6 +71,28 @@ int V4L2VideoDeviceTest::init() if (capture_->getFormat(&format)) return TestFail; + if (driver_ == "vimc") { + sensor_ = new CameraSensor(media_->getEntityByName("Sensor A")); + if (sensor_->init()) + return TestFail; + + debayer_ = new V4L2Subdevice(media_->getEntityByName("Debayer A")); + if (debayer_->open()) + return TestFail; + + format.fourcc = V4L2_PIX_FMT_SBGGR8; + + V4L2SubdeviceFormat subformat = {}; + subformat.mbus_code = MEDIA_BUS_FMT_SBGGR8_1X8; + subformat.size = format.size; + + if (sensor_->setFormat(&subformat)) + return TestFail; + + if (debayer_->setFormat(0, &subformat)) + return TestFail; + } + format.size.width = 640; format.size.height = 480; if (capture_->setFormat(&format)) @@ -83,5 +107,7 @@ void V4L2VideoDeviceTest::cleanup() capture_->releaseBuffers(); capture_->close(); + delete debayer_; + delete sensor_; delete capture_; }; diff --git a/test/v4l2_videodevice/v4l2_videodevice_test.h b/test/v4l2_videodevice/v4l2_videodevice_test.h index 3321b5a4f98fdb1d..34dd231c6d9d108d 100644 --- a/test/v4l2_videodevice/v4l2_videodevice_test.h +++ b/test/v4l2_videodevice/v4l2_videodevice_test.h @@ -13,8 +13,10 @@ #include "test.h" +#include "camera_sensor.h" #include "device_enumerator.h" #include "media_device.h" +#include "v4l2_subdevice.h" #include "v4l2_videodevice.h" using namespace libcamera; @@ -23,7 +25,8 @@ class V4L2VideoDeviceTest : public Test { public: V4L2VideoDeviceTest(const char *driver, const char *entity) - : driver_(driver), entity_(entity), capture_(nullptr) + : driver_(driver), entity_(entity), sensor_(nullptr), + debayer_(nullptr), capture_(nullptr) { } @@ -35,6 +38,8 @@ protected: std::string entity_; std::unique_ptr enumerator_; std::shared_ptr media_; + CameraSensor *sensor_; + V4L2Subdevice *debayer_; V4L2VideoDevice *capture_; BufferPool pool_; }; From patchwork Thu Aug 8 00:04:04 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 1748 Return-Path: Received: from bin-mail-out-06.binero.net (bin-mail-out-06.binero.net [195.74.38.229]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 792E561623 for ; Thu, 8 Aug 2019 02:04:11 +0200 (CEST) X-Halon-ID: 07b99d47-b970-11e9-bdc3-005056917a89 Authorized-sender: niklas@soderlund.pp.se Received: from bismarck.berto.se (unknown [145.14.112.32]) by bin-vsp-out-01.atm.binero.net (Halon) with ESMTPA id 07b99d47-b970-11e9-bdc3-005056917a89; Thu, 08 Aug 2019 02:04:03 +0200 (CEST) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org Date: Thu, 8 Aug 2019 02:04:04 +0200 Message-Id: <20190808000404.10841-5-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20190808000404.10841-1-niklas.soderlund@ragnatech.se> References: <20190808000404.10841-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 4/4] tests: camera-sensor: Test using invalid media bus format X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Aug 2019 00:04:12 -0000 Linux commit b6c61a6c37317efd ("media: vimc: propagate pixel format in the stream") changes the sensor in the vimc media graph to accept all media bus format currently described in Linux. This prevents the camera-sensor test case to verify that a supported media bus format is selected from a list of defined formats, fix this by using an invalid media bus format in the test case. Signed-off-by: Niklas Söderlund Reviewed-by: Laurent Pinchart --- test/camera-sensor.cpp | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/test/camera-sensor.cpp b/test/camera-sensor.cpp index 092cbdd8111ce70d..9fe59cc98d79ccab 100644 --- a/test/camera-sensor.cpp +++ b/test/camera-sensor.cpp @@ -83,7 +83,8 @@ protected: return TestFail; } - V4L2SubdeviceFormat format = sensor_->getFormat({ MEDIA_BUS_FMT_UYVY8_2X8, + /* Use an invalid format and make sure it's not selected. */ + V4L2SubdeviceFormat format = sensor_->getFormat({ 0xdeadbeef, MEDIA_BUS_FMT_SBGGR10_1X10, MEDIA_BUS_FMT_BGR888_1X24 }, Size(1024, 768));