{"id":19620,"url":"https://patchwork.libcamera.org/api/patches/19620/?format=json","web_url":"https://patchwork.libcamera.org/patch/19620/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20240301212121.9072-30-laurent.pinchart@ideasonboard.com>","date":"2024-03-01T21:21:18","name":"[PATCH/RFC,29/32] pipeline: raspberrypi: vc4: Configure format on Unicam subdev","commit_ref":null,"pull_url":null,"state":"rfc","archived":false,"hash":"e8880ee5c2e55233d8d37384c27dc22185151889","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/?format=json","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/19620/mbox/","series":[{"id":4197,"url":"https://patchwork.libcamera.org/api/series/4197/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=4197","date":"2024-03-01T21:20:49","name":"libcamera: Support the upstream Unicam driver","version":1,"mbox":"https://patchwork.libcamera.org/series/4197/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/19620/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/19620/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 5BB99C3260\n\tfor ;\n\tFri, 1 Mar 2024 21:22:04 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 18D6362C84;\n\tFri, 1 Mar 2024 22:22:04 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 3400662C94\n\tfor ;\n\tFri, 1 Mar 2024 22:22:02 +0100 (CET)","from pendragon.ideasonboard.com (89-27-53-110.bb.dnainternet.fi\n\t[89.27.53.110])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id B16EE3DF7;\n\tFri, 1 Mar 2024 22:21:47 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"ByigtAHf\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1709328107;\n\tbh=LGJCkU8WiFH6nxUqWNKrukB57KBiBbJ9SDEhPjBxC3I=;\n\th=From:To:Cc:Subject:Date:In-Reply-To:References:From;\n\tb=ByigtAHfZn7Sf6mKKFxJfplcF38unKFtJu2CEmTtwlc5DvvykLjohcj4ewZDkdFXS\n\txTtsjj4z+/ghTSz37RzADKnD1xaQlQ3CO6ugaToOTPGBmiPuGHmbnv5RNykOHWjKku\n\twn4jfV+UB3M/SG4scyMA/7u0LbyfiMasNF/laXhg=","From":"Laurent Pinchart ","To":"libcamera-devel@lists.libcamera.org","Subject":"[PATCH/RFC 29/32] pipeline: raspberrypi: vc4: Configure format on\n\tUnicam subdev","Date":"Fri, 1 Mar 2024 23:21:18 +0200","Message-ID":"<20240301212121.9072-30-laurent.pinchart@ideasonboard.com>","X-Mailer":"git-send-email 2.43.0","In-Reply-To":"<20240301212121.9072-1-laurent.pinchart@ideasonboard.com>","References":"<20240301212121.9072-1-laurent.pinchart@ideasonboard.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Sakari Ailus ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"The mainline Unicam driver creates a V4L2 subdevice, which needs to be\nconfigured. Create and open a corresponding V4L2Subdevice instance and\nconfigure the format on its sink pad in the platformConfigure()\nfunction.\n\nPresence of the Unicam subdev is required, to avoid extra complexity.\nThis drops support for the driver from the Raspberry Pi downstream\nkernel. Users are expected to update their kernel to use the mainline\nUnicam driver.\n\nSigned-off-by: Laurent Pinchart \n---\n src/libcamera/pipeline/rpi/vc4/vc4.cpp | 78 ++++++++++++++++++++++----\n 1 file changed, 68 insertions(+), 10 deletions(-)","diff":"diff --git a/src/libcamera/pipeline/rpi/vc4/vc4.cpp b/src/libcamera/pipeline/rpi/vc4/vc4.cpp\nindex da925ba5f793..81e0bc1b8f7a 100644\n--- a/src/libcamera/pipeline/rpi/vc4/vc4.cpp\n+++ b/src/libcamera/pipeline/rpi/vc4/vc4.cpp\n@@ -5,6 +5,8 @@\n * vc4.cpp - Pipeline handler for VC4-based Raspberry Pi devices\n */\n \n+#include \n+\n #include \n #include \n #include \n@@ -12,6 +14,7 @@\n #include \n \n #include \"libcamera/internal/device_enumerator.h\"\n+#include \"libcamera/internal/v4l2_subdevice.h\"\n \n #include \"../common/pipeline_base.h\"\n #include \"../common/rpi_stream.h\"\n@@ -32,6 +35,10 @@ namespace {\n enum class Unicam : unsigned int { Image, Embedded };\n enum class Isp : unsigned int { Input, Output0, Output1, Stats };\n \n+static constexpr unsigned int kUnicamSinkPad = 0;\n+static constexpr unsigned int kUnicamSourceImagePad = 1;\n+static constexpr unsigned int kUnicamSourceMetadataPad = 2;\n+\n } /* namespace */\n \n class Vc4CameraData final : public RPi::CameraData\n@@ -82,6 +89,8 @@ public:\n \tvoid setIspControls(const ControlList &controls);\n \tvoid setCameraTimeout(uint32_t maxFrameLengthMs);\n \n+\tstd::unique_ptr unicamSubdev_;\n+\n \t/* Array of Unicam and ISP device streams and associated buffers/streams. */\n \tRPi::Device unicam_;\n \tRPi::Device isp_;\n@@ -310,6 +319,7 @@ int PipelineHandlerVc4::platformRegister(std::unique_ptr &camer\n \tif (!data->dmaHeap_.isValid())\n \t\treturn -ENOMEM;\n \n+\tMediaEntity *unicamSubdev = unicam->getEntityByName(\"unicam\");\n \tMediaEntity *unicamImage = unicam->getEntityByName(\"unicam-image\");\n \tMediaEntity *unicamEmbedded = unicam->getEntityByName(\"unicam-embedded\");\n \tMediaEntity *ispOutput0 = isp->getEntityByName(\"bcm2835-isp0-output0\");\n@@ -321,7 +331,13 @@ int PipelineHandlerVc4::platformRegister(std::unique_ptr &camer\n \t !ispOutput0 || !ispCapture1 || !ispCapture2 || !ispCapture3)\n \t\treturn -ENOENT;\n \n-\t/* Create the unicam video streams. */\n+\tif (!unicamSubdev) {\n+\t\tLOG(RPI, Error) << \"Downstream Unicam driver not supported, please update your kernel!\";\n+\t\treturn -EINVAL;\n+\t}\n+\n+\t/* Create the unicam subdev and video streams. */\n+\tdata->unicamSubdev_ = std::make_unique(unicamSubdev);\n \tdata->unicam_[Unicam::Image] = RPi::Stream(\"Unicam Image\", unicamImage);\n \tdata->unicam_[Unicam::Embedded] = RPi::Stream(\"Unicam Embedded\", unicamEmbedded);\n \n@@ -350,6 +366,10 @@ int PipelineHandlerVc4::platformRegister(std::unique_ptr &camer\n \t * The below grouping is just for convenience so that we can easily\n \t * iterate over all streams in one go.\n \t */\n+\tint ret = data->unicamSubdev_->open();\n+\tif (ret < 0)\n+\t\treturn ret;\n+\n \tdata->streams_.push_back(&data->unicam_[Unicam::Image]);\n \tif (data->sensorMetadata_)\n \t\tdata->streams_.push_back(&data->unicam_[Unicam::Embedded]);\n@@ -358,16 +378,11 @@ int PipelineHandlerVc4::platformRegister(std::unique_ptr &camer\n \t\tdata->streams_.push_back(&stream);\n \n \tfor (auto stream : data->streams_) {\n-\t\tint ret = stream->dev()->open();\n+\t\tret = stream->dev()->open();\n \t\tif (ret)\n \t\t\treturn ret;\n \t}\n \n-\tif (!data->unicam_[Unicam::Image].dev()->caps().hasMediaController()) {\n-\t\tLOG(RPI, Error) << \"Unicam driver does not use the MediaController, please update your kernel!\";\n-\t\treturn -EINVAL;\n-\t}\n-\n \t/* Write up all the IPA connections. */\n \tdata->ipa_->processStatsComplete.connect(data, &Vc4CameraData::processStatsComplete);\n \tdata->ipa_->prepareIspComplete.connect(data, &Vc4CameraData::prepareIspComplete);\n@@ -530,7 +545,50 @@ int Vc4CameraData::platformPipelineConfigure(const std::unique_ptr &\n int Vc4CameraData::platformConfigure(const RPi::RPiCameraConfiguration *rpiConfig)\n {\n \t/*\n-\t * 1. Configure the Unicam video devices.\n+\t * 1. Configure the Unicam subdev.\n+\t *\n+\t * Start by setting up routes, and then set the formats on the sink pad\n+\t * streams. They will be automatically propagated to the source pads by\n+\t * the kernel.\n+\t */\n+\n+\tconst V4L2Subdevice::Stream imageStream{\n+\t\tkUnicamSinkPad,\n+\t\tsensor_->imageStream().stream\n+\t};\n+\tconst V4L2Subdevice::Stream embeddedDataStream{\n+\t\tkUnicamSinkPad,\n+\t\tsensor_->embeddedDataStream().value_or(V4L2Subdevice::Stream{}).stream\n+\t};\n+\n+\tV4L2Subdevice::Routing routing;\n+\n+\trouting.emplace_back(imageStream, V4L2Subdevice::Stream{ kUnicamSourceImagePad, 0 },\n+\t\t\t V4L2_SUBDEV_ROUTE_FL_ACTIVE);\n+\n+\tif (sensorMetadata_)\n+\t\trouting.emplace_back(embeddedDataStream,\n+\t\t\t\t V4L2Subdevice::Stream{ kUnicamSourceMetadataPad, 0 },\n+\t\t\t\t V4L2_SUBDEV_ROUTE_FL_ACTIVE);\n+\n+\tint ret = unicamSubdev_->setRouting(&routing);\n+\tif (ret)\n+\t\treturn ret;\n+\n+\tV4L2SubdeviceFormat subdevFormat = rpiConfig->sensorFormat_;\n+\tret = unicamSubdev_->setFormat(imageStream, &subdevFormat);\n+\tif (ret)\n+\t\treturn ret;\n+\n+\tif (sensorMetadata_) {\n+\t\tsubdevFormat = sensor_->embeddedDataFormat();\n+\t\tret = unicamSubdev_->setFormat(embeddedDataStream, &subdevFormat);\n+\t\tif (ret)\n+\t\t\treturn ret;\n+\t}\n+\n+\t/*\n+\t * 2. Configure the Unicam video devices.\n \t */\n \n \tV4L2VideoDevice *unicam = unicam_[Unicam::Image].dev();\n@@ -553,7 +611,7 @@ int Vc4CameraData::platformConfigure(const RPi::RPiCameraConfiguration *rpiConfi\n \t\t\t\t\t\t\t\t BayerFormat::Packing::CSI2);\n \t}\n \n-\tint ret = unicam->setFormat(&unicamFormat);\n+\tret = unicam->setFormat(&unicamFormat);\n \tif (ret)\n \t\treturn ret;\n \n@@ -581,7 +639,7 @@ int Vc4CameraData::platformConfigure(const RPi::RPiCameraConfiguration *rpiConfi\n \t}\n \n \t/*\n-\t * 2. Configure the ISP.\n+\t * 3. Configure the ISP.\n \t */\n \n \tret = isp_[Isp::Input].dev()->setFormat(&unicamFormat);\n","prefixes":["PATCH/RFC","29/32"]}