{"id":26591,"url":"https://patchwork.libcamera.org/api/patches/26591/?format=json","web_url":"https://patchwork.libcamera.org/patch/26591/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20260501105137.439519-4-maxbretschneider@protonmail.com>","date":"2026-05-01T10:52:13","name":"[RFC,v1,3/3] libcamera: pipeline: virtual: Support StreamRole::Raw and Bayer formats","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"89c6ec4093eaebf115abbab9cd3e15bca8efa8a0","submitter":{"id":266,"url":"https://patchwork.libcamera.org/api/people/266/?format=json","name":"Max Bretschneider","email":"maxbretschneider@protonmail.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/26591/mbox/","series":[{"id":5892,"url":"https://patchwork.libcamera.org/api/series/5892/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=5892","date":"2026-05-01T10:52:02","name":"libcamera: pipeline: virtual: Add raw Bayer frame support","version":1,"mbox":"https://patchwork.libcamera.org/series/5892/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/26591/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/26591/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id B81D2BDCB5\n\tfor ;\n\tFri, 1 May 2026 10:52:20 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 661766301A;\n\tFri, 1 May 2026 12:52:20 +0200 (CEST)","from mail-43166.protonmail.ch (mail-43166.protonmail.ch\n\t[185.70.43.166])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id A725B62FE8\n\tfor ;\n\tFri, 1 May 2026 12:52:18 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key;\n\tunprotected) header.d=protonmail.com header.i=@protonmail.com\n\theader.b=\"UqC0zwWV\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com;\n\ts=protonmail3; t=1777632738; x=1777891938;\n\tbh=ZJPOY3/OwUQG4BZcbsfUxqqrPOe0aqRGxct2M8cNYEw=;\n\th=Date:To:From:Cc:Subject:Message-ID:In-Reply-To:References:\n\tFeedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID:\n\tMessage-ID:BIMI-Selector;\n\tb=UqC0zwWVupwyTxL9Ct9aqFQiPp9rbdS3kLIbv1Z5rDdyEtt4+nI7XQ8etg5fVKmFt\n\tWzwyhhDd8p4EMxjI36nZnlEHL7bZNZpIF+V1swdJo7Orw9msRY/ObI7flYG1sfRoFe\n\tlcBHwWCs2qu7fT1wAPmtgVAtzvB1PYOqVO/CRojGREiZh7zn4czaEEo0Xabz/yGHFc\n\t7koxeDzJ/mIwJ2zp1F8IlePEXGt9YfMQ+HD9OQER5xlABkdnfxinKawJluc93pnbU2\n\tZF5WZzXU1FecM8Mqqop6cSiI0HTk7rEN+tcYzkA2Zim3vrv7an2gG7bLnkhaGlOmVR\n\tcVkY1IuJRknNQ==","Date":"Fri, 01 May 2026 10:52:13 +0000","To":"libcamera-devel@lists.libcamera.org","From":"Max Bretschneider ","Cc":"Max Bretschneider ","Subject":"[RFC PATCH v1 3/3] libcamera: pipeline: virtual: Support\n\tStreamRole::Raw and Bayer formats","Message-ID":"<20260501105137.439519-4-maxbretschneider@protonmail.com>","In-Reply-To":"<20260501105137.439519-1-maxbretschneider@protonmail.com>","References":"<20260501105137.439519-1-maxbretschneider@protonmail.com>","Feedback-ID":"122687743:user:proton","X-Pm-Message-ID":"90c2765669d8b9305bc89b33b05cb288f6a206ee","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Transfer-Encoding":"quoted-printable","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Extend generateConfiguration() to return a Bayer StreamConfiguration\nwhen the StreamRole::Raw is requested on a camera configured with\nraw_frames.\n\nExtend validate() to allow Bayer formats, also set the correct stride\nand frameSize for single plane Bayer buffers. Sensor properties required\nfor SoftISP integration are also declared in match().\n\nSigned-off-by: Max Bretschneider \n---\n src/libcamera/pipeline/virtual/virtual.cpp | 123 ++++++++++++++++++---\n src/libcamera/pipeline/virtual/virtual.h | 3 +-\n 2 files changed, 109 insertions(+), 17 deletions(-)\n\n--\n2.43.0","diff":"diff --git a/src/libcamera/pipeline/virtual/virtual.cpp b/src/libcamera/pipeline/virtual/virtual.cpp\nindex 81d2ddda..beec2f5a 100644\n--- a/src/libcamera/pipeline/virtual/virtual.cpp\n+++ b/src/libcamera/pipeline/virtual/virtual.cpp\n@@ -19,6 +19,7 @@\n #include \n #include \n #include \n+#include \n #include \n\n #include \n@@ -37,7 +38,6 @@\n #include \"libcamera/internal/framebuffer.h\"\n #include \"libcamera/internal/pipeline_handler.h\"\n #include \"libcamera/internal/request.h\"\n-#include \"libcamera/internal/value_node.h\"\n\n #include \"pipeline/virtual/config_parser.h\"\n\n@@ -202,21 +202,28 @@ CameraConfiguration::Status VirtualCameraConfiguration::validate()\n \t\t\tadjusted = true;\n \t\t}\n\n-\t\tif (cfg.pixelFormat != formats::NV12) {\n-\t\t\tcfg.pixelFormat = formats::NV12;\n-\t\t\tstatus = Adjusted;\n-\t\t\tadjusted = true;\n-\t\t}\n+\t\tconst PixelFormatInfo &fmtInfo = PixelFormatInfo::info(cfg.pixelFormat);\n+\t\tconst bool rawStream = fmtInfo.colourEncoding == PixelFormatInfo::ColourEncodingRAW;\n\n-\t\tif (cfg.colorSpace != ColorSpace::Rec709) {\n-\t\t\tcfg.colorSpace = ColorSpace::Rec709;\n-\t\t\tstatus = Adjusted;\n-\t\t\tadjusted = true;\n-\t\t}\n+\t\tif (!rawStream) {\n+\t\t\tif (cfg.pixelFormat != formats::NV12) {\n+\t\t\t\tcfg.pixelFormat = formats::NV12;\n+\t\t\t\tstatus = Adjusted;\n+\t\t\t\tadjusted = true;\n+\t\t\t}\n\n-\t\tif (validateColorSpaces() == Adjusted) {\n-\t\t\tstatus = Adjusted;\n-\t\t\tadjusted = true;\n+\t\t\tif (cfg.colorSpace != ColorSpace::Rec709) {\n+\t\t\t\tcfg.colorSpace = ColorSpace::Rec709;\n+\t\t\t\tstatus = Adjusted;\n+\t\t\t\tadjusted = true;\n+\t\t\t}\n+\n+\t\t\tif (validateColorSpaces() == Adjusted) {\n+\t\t\t\tstatus = Adjusted;\n+\t\t\t\tadjusted = true;\n+\t\t\t}\n+\t\t} else {\n+\t\t\tcfg.colorSpace = ColorSpace::Raw;\n \t\t}\n\n \t\tif (adjusted)\n@@ -267,7 +274,64 @@ PipelineHandlerVirtual::generateConfiguration(Camera *camera,\n \t\tcase StreamRole::Viewfinder:\n \t\t\tbreak;\n\n-\t\tcase StreamRole::Raw:\n+\t\tcase StreamRole::Raw: {\n+\t\t\tif (!std::holds_alternative(data->config_.frame)) {\n+\t\t\t\tLOG(Virtual, Error)\n+\t\t\t\t\t<< \"StreamRole::Raw requested but camera is not configured with raw_frames\";\n+\t\t\t\treturn {};\n+\t\t\t}\n+\n+\t\t\tconst auto &rawFrames = std::get(data->config_.frame);\n+\t\t\tPixelFormat rawFormat;\n+\n+\t\t\t/*\n+\t\t\t * \\todo Possibly replace with a lookup table to\n+\t\t\t * be able to just index by (cfaPatter, bitDepth),\n+\t\t\t * might be cleaner.\n+\t\t\t */\n+\t\t\tauto bayerFormat = [&](PixelFormat f8, PixelFormat f10,\n+\t\t\t\t\t PixelFormat f12, PixelFormat f14,\n+\t\t\t\t\t PixelFormat f16) {\n+\t\t\t\tswitch (rawFrames.bitDepth) {\n+\t\t\t\tcase 8:\n+\t\t\t\t\treturn f8;\n+\t\t\t\tcase 10:\n+\t\t\t\t\treturn f10;\n+\t\t\t\tcase 12:\n+\t\t\t\t\treturn f12;\n+\t\t\t\tcase 14:\n+\t\t\t\t\treturn f14;\n+\t\t\t\tdefault:\n+\t\t\t\t\treturn f16;\n+\t\t\t\t}\n+\t\t\t};\n+\n+\t\t\t/* Map bayer order and bit depth to pixel format */\n+\t\t\tif (rawFrames.cfaPattern == properties::draft::ColorFilterArrangementEnum::RGGB)\n+\t\t\t\trawFormat = bayerFormat(formats::SRGGB8, formats::SRGGB10, formats::SRGGB12, formats::SRGGB14, formats::SRGGB16);\n+\t\t\telse if (rawFrames.cfaPattern == properties::draft::ColorFilterArrangementEnum::BGGR)\n+\t\t\t\trawFormat = bayerFormat(formats::SBGGR8, formats::SBGGR10, formats::SBGGR12, formats::SBGGR14, formats::SBGGR16);\n+\t\t\telse if (rawFrames.cfaPattern == properties::draft::ColorFilterArrangementEnum::GRBG)\n+\t\t\t\trawFormat = bayerFormat(formats::SGRBG8, formats::SGRBG10, formats::SGRBG12, formats::SGRBG14, formats::SGRBG16);\n+\t\t\telse\n+\t\t\t\trawFormat = bayerFormat(formats::SGBRG8, formats::SGBRG10, formats::SGBRG12, formats::SGBRG14, formats::SGBRG16);\n+\n+\t\t\t/*\n+\t\t\t * Use the Bayer format matching the raw frame\n+\t\t\t * configuration.\n+\t\t\t */\n+\t\t\tstd::map> rawStreamFormats;\n+\t\t\trawStreamFormats[rawFormat] = { { data->config_.minResolutionSize, data->config_.maxResolutionSize } };\n+\t\t\tStreamFormats rawFormats(rawStreamFormats);\n+\t\t\tStreamConfiguration rawCfg(rawFormats);\n+\t\t\trawCfg.pixelFormat = rawFormat;\n+\t\t\trawCfg.size = data->config_.maxResolutionSize;\n+\t\t\trawCfg.bufferCount = VirtualCameraConfiguration::kBufferCount;\n+\t\t\trawCfg.colorSpace = ColorSpace::Raw;\n+\t\t\tconfig->addConfiguration(rawCfg);\n+\t\t\tcontinue;\n+\t\t}\n+\n \t\tdefault:\n \t\t\tLOG(Virtual, Error)\n \t\t\t\t<< \"Requested stream role not supported: \" << role;\n@@ -401,6 +465,28 @@ bool PipelineHandlerVirtual::match([[maybe_unused]] DeviceEnumerator *enumerator\n \t\tstd::set streams;\n \t\tfor (auto &streamConfig : data->streamConfigs_)\n \t\t\tstreams.insert(&streamConfig.stream);\n+\n+\t\t/* Add sensor properties and controls required by SoftISP for raw streams */\n+\t\tif (std::holds_alternative(data->config_.frame)) {\n+\t\t\tconst auto &rawFrames = std::get(data->config_.frame);\n+\t\t\tdata->properties_.set(properties::draft::ColorFilterArrangement, static_cast(rawFrames.cfaPattern));\n+\t\t\tdata->properties_.set(properties::PixelArraySize, data->config_.maxResolutionSize);\n+\t\t\tdata->properties_.set(properties::UnitCellSize, Size(1000, 1000));\n+\n+\t\t\t/* Extends existing controlInfo_ with SoftISP required controls */\n+\t\t\tControlInfoMap::Map controls;\n+\t\t\tfor (const auto &[id, info] : data->controlInfo_)\n+\t\t\t\tcontrols[id] = info;\n+\n+\t\t\t/* \\todo Allow configuration via YAML */\n+\t\t\tcontrols[&controls::AnalogueGain] = ControlInfo(1.0f, 16.0f, 1.0f);\n+\t\t\tcontrols[&controls::ExposureTime] = ControlInfo(100, 33333, 10000);\n+\t\t\tcontrols[&controls::AeEnable] = ControlInfo(false, true, true);\n+\t\t\tcontrols[&controls::AwbEnable] = ControlInfo(false, true, true);\n+\n+\t\t\tdata->controlInfo_ = ControlInfoMap(std::move(controls), controls::controls);\n+\t\t}\n+\n \t\tstd::string id = data->config_.id;\n \t\tstd::shared_ptr camera = Camera::create(std::move(data), id, streams);\n\n@@ -434,7 +520,12 @@ bool PipelineHandlerVirtual::initFrameGenerator(Camera *camera)\n \t\t\t [&](ImageFrames &imageFrames) {\n \t\t\t\t for (auto &streamConfig : data->streamConfigs_)\n \t\t\t\t\t streamConfig.frameGenerator = ImageFrameGenerator::create(imageFrames);\n-\t\t\t } },\n+\t\t\t },\n+\t\t\t [&](RawFrames &rawFrames) {\n+\t\t\t\t for (auto &streamConfig : data->streamConfigs_)\n+\t\t\t\t\t streamConfig.frameGenerator = RawFrameGenerator::create(rawFrames);\n+\t\t\t },\n+\t\t },\n \t\t frame);\n\n \tfor (auto &streamConfig : data->streamConfigs_)\ndiff --git a/src/libcamera/pipeline/virtual/virtual.h b/src/libcamera/pipeline/virtual/virtual.h\nindex 215e56fa..48392808 100644\n--- a/src/libcamera/pipeline/virtual/virtual.h\n+++ b/src/libcamera/pipeline/virtual/virtual.h\n@@ -23,11 +23,12 @@\n\n #include \"frame_generator.h\"\n #include \"image_frame_generator.h\"\n+#include \"raw_frame_generator.h\"\n #include \"test_pattern_generator.h\"\n\n namespace libcamera {\n\n-using VirtualFrame = std::variant;\n+using VirtualFrame = std::variant;\n\n class VirtualCameraData : public Camera::Private,\n \t\t\t public Thread,\n","prefixes":["RFC","v1","3/3"]}