{"id":26720,"url":"https://patchwork.libcamera.org/api/covers/26720/?format=json","web_url":"https://patchwork.libcamera.org/cover/26720/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20260511202327.40191-1-maxbretschneider@protonmail.com>","date":"2026-05-11T20:23:37","name":"[RFC,v3,0/2] libcamera: pipeline: virtual: Add raw Bayer frame support","submitter":{"id":266,"url":"https://patchwork.libcamera.org/api/people/266/?format=json","name":"Max Bretschneider","email":"maxbretschneider@protonmail.com"},"mbox":"https://patchwork.libcamera.org/cover/26720/mbox/","series":[{"id":5935,"url":"https://patchwork.libcamera.org/api/series/5935/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=5935","date":"2026-05-11T20:23:37","name":"libcamera: pipeline: virtual: Add raw Bayer frame support","version":3,"mbox":"https://patchwork.libcamera.org/series/5935/mbox/"}],"comments":"https://patchwork.libcamera.org/api/covers/26720/comments/","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 26FEBBDCBD\n\tfor ;\n\tMon, 11 May 2026 20:23:44 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id D56B26302D;\n\tMon, 11 May 2026 22:23:43 +0200 (CEST)","from mail-10631.protonmail.ch (mail-10631.protonmail.ch\n\t[79.135.106.31])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id C298B62E9D\n\tfor ;\n\tMon, 11 May 2026 22:23:42 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key;\n\tunprotected) header.d=protonmail.com header.i=@protonmail.com\n\theader.b=\"KGzf+0n/\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com;\n\ts=protonmail3; t=1778531021; x=1778790221;\n\tbh=r0eOW98/up6XBCuoxFGmY24wsHRHi6LpPyUOfwO4f1E=;\n\th=Date:To:From:Cc:Subject:Message-ID:In-Reply-To:References:\n\tFeedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID:\n\tMessage-ID:BIMI-Selector;\n\tb=KGzf+0n/Vd4zsQ4QwbTypnZ65YMAyRbUZQurP7v+1KCyc2AbDxNDaGSL8omU+3A+d\n\tr1rNlMEdnxd5UUtGnxz2RZ9N859cp6tWWT1rwp8pfcX3O1YGv8w2Ly93glinjGccID\n\tf8dee1Offa4fHLnck/CKaBK4auSG8A19uki4sEejINVM+ld78Uuki9oastQvVPFLjH\n\tr58rl5SYTROtvd2J/kvsty8hlEtVOY/Ff7BSnv9mWJSYvMk+C6ON6IM+vR7xHrb+zo\n\tsVecxbI9RnOy94wtno1rPCFW8WF5uedHCs8qN7HjeVT2gqVcfgtobWCWm4ICy6BU1T\n\t8UEmkWZTap72w==","Date":"Mon, 11 May 2026 20:23:37 +0000","To":"libcamera-devel@lists.libcamera.org","From":"Max Bretschneider ","Cc":"Max Bretschneider ","Subject":"[RFC PATCH v3 0/2] libcamera: pipeline: virtual: Add raw Bayer frame\n\tsupport","Message-ID":"<20260511202327.40191-1-maxbretschneider@protonmail.com>","In-Reply-To":"<20260504091623.3354474-1-maxbretschneider@protonmail.com>","References":"<20260501105137.439519-1-maxbretschneider@protonmail.com>\n\t<20260504091623.3354474-1-maxbretschneider@protonmail.com>","Feedback-ID":"122687743:user:proton","X-Pm-Message-ID":"f6b9f06f68a32cd53c2c3f8270fd7a131a59f585","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Transfer-Encoding":"quoted-printable","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Changes since v2: \n- Use std::get_if instead of holds_alternative + std::get\n- Extracted the file collection into collectFiles() helper\n- Use BayerFormat::toPixelFormat()\n- Drop SoftISP-specific controls and the mention of it\n\nThe virtual pipeline handler currently only produces NV12 output, either\nfrom test patterns or JPEG files. This means that SoftISP, which\nprocesses raw Bayer data, cannot be tested without physical camera\nhardware. A first step to achieving this would be to support raw streams\nin the virtual pipeline handler. This series introduces a RawFrameGenerator\nthat reads binary raw Bayer files from disk and serves them through the virtual\npipeline. The introduced changes are opt-in through the YAML configuration. If\nvirtual cameras are configured with raw_frames, the new code path is\nchosen, paths for the existing test_patterns and frames are unaffected.\n\nThe SoftISP part of this patch's motivation is complemented by David Plowman's\nseries, which introduced the CameraSensorMemory class\n(https://patchwork.libcamera.org/patch/25468/), since SoftISP needs an\nactual CameraSensor for correct construction. The idea is that in the\nfuture it can be used together to serve Raw frames through the\nvirtual pipeline and process them with SoftISP, for example\nfor CI tests or more easily reproducible benchmarks.\n\nTesting: I have used `dd` to create zero filled raw frames to\ntest things such as format negotiation, buffer sizing, general frame\nflow, config parsing (both right/wrong, e.g. rejecting invalid input)\nand bit depth handling. I've written a small bash script to essentially\nautomate this for all n : { 8, 10, 12, 14, 16 } and the four Bayer\nformats in accordance to `src/libcamera/formats.yaml`. I then used numpy to\ntest the generateFrame() functionality by generating a known frame with\ncontent, capturing it through the pipeline to disk with the cam application\nand then reading the output back again to verify that all the pixels match\nthe expected values.\n\nI noticed that the pipeline handler components currently have no\ndedicated test coverage, if I missed them or if there is some preferred\napproach for adding additional testing I'd be open to any suggestions.\nFor now I can also share my scripts if that's wanted.\n\nRan `git clang-format` as well as the `checkstyle.py`. I've run the\nMeson tests both with and without the changes and get the same:\nOk: 51\nExpected Fail: 1\nFail: 1\nUnexpected Pass: 0\nSkipped: 29\nTimeout: 0\n\nOpen questions:\n- The sensor properties set in match() (ColorFilterArrangement,\n PixelArraySize, UnitCellSize) are currently hardcoded, and I've\n left them with a \\todo. Should I move these directly to the yaml as well or proceed differently?\n- Currently every frame is read into framesDatas_ immediately at\n startup. Depending on the number and the size of the raw files, this\n could take up quite a lot of memory. For the use cases I thought of\n (e.g. CI) this would be fine, since the number of files would probably\n be comparatively small there. Still I wanted to bring it up.\n- We weren't sure if the SPDX license header is fine like this. For the\n moment I've just placed my intern company mail, is this fine as is?\n\nDocumentation:\nI've added documentation according to the coding-style.rst,\nif my comments are not sufficient I'd be happy to add more of course.\n\nAI Disclosure:\nI've used DeepWiki to familiarize myself with the libcamera repository\nand to point me at reference implementations and patterns to use as orientation.\n\nThank you all for your time!\n\nMax Bretschneider (2):\n libcamera: pipeline: virtual: Add RawFrameGenerator\n libcamera: pipeline: virtual: Add raw_frames config and Bayer format\n support\n\n .../pipeline/virtual/config_parser.cpp | 156 ++++++++++++++----\n src/libcamera/pipeline/virtual/meson.build | 1 +\n .../pipeline/virtual/raw_frame_generator.cpp | 131 +++++++++++++++\n .../pipeline/virtual/raw_frame_generator.h | 47 ++++++\n src/libcamera/pipeline/virtual/virtual.cpp | 99 +++++++++--\n src/libcamera/pipeline/virtual/virtual.h | 3 +-\n 6 files changed, 389 insertions(+), 48 deletions(-)\n create mode 100644 src/libcamera/pipeline/virtual/raw_frame_generator.cpp\n create mode 100644 src/libcamera/pipeline/virtual/raw_frame_generator.h"}