[{"id":31452,"web_url":"https://patchwork.libcamera.org/comment/31452/","msgid":"<172745476410.532453.10065732689440216657@ping.linuxembedded.co.uk>","date":"2024-09-27T16:32:44","subject":"Re: [PATCH v13 0/7] Add VirtualPipelineHandler","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"Quoting Harvey Yang (2024-09-26 17:29:49)\n> \n> Harvey Yang New message\n> Tue, Sep 10, 12:48 PM\n> to libcamera-devel, me\n> \n> Hi all,\n> \n> This series adds virtual pipeline handler, which doesn't depend on any\n> hardware, like camera sensor or ISP. Currently the configuration\n> supports test patterns and images.\n> \n> It passed the gitlab pipeline:\n> https://gitlab.freedesktop.org/chenghaoyang/libcamera/-/pipelines/1278799\n> \n> I've also checked that it works on qcam.\n> \n> I failed to pass multi_stream_test when trying to enable multiple\n> streams though. Please give me some hints what it tests and what I\n> missed:\n> ```\n> [295:43:43.910237144] [1441841] INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/usr/local/google/home/chenghaoyang/Workspace/libca\n> mera/build/src/ipa' to the IPA search path\n> [295:43:43.914030564] [1441841] INFO Camera camera_manager.cpp:325 libcamera v0.3.1+79-8ca4f033-dirty (2024-08-29T19:18:51UTC)\n> [295:43:43.915493754] [1441844] ERROR DmaBufAllocator dma_buf_allocator.cpp:120 Could not open any dma-buf provider\n> [295:43:43.919118835] [1441841] INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/usr/local/google/home/chenghaoyang/Workspace/libca\n> mera/build/src/ipa' to the IPA search path\n> [295:43:43.922245825] [1441841] INFO Camera camera_manager.cpp:325 libcamera v0.3.1+79-8ca4f033-dirty (2024-08-29T19:18:51UTC)\n> [295:43:43.922820175] [1441846] ERROR DmaBufAllocator dma_buf_allocator.cpp:120 Could not open any dma-buf provider\n> Unable to set the pipeline to the playing state.\n\nI presume this is some failure in the configuration of the virtual test\nenvironment that isn't exposing access to the suitable udmabuf handler?\n\nAnd I wonder if this series is now making the unit tests use the virtual\ncamera instead of vimc ?\n\nIf so - we probably want to have the tests more consistently defined\nmaybe - to either prefer specific cameras - or run on all cameras\navailable?\n\nBut I think fixing this issue probably needs investigation on the\nlibcamera-ci build (https://gitlab.freedesktop.org/camera/libcamera-ci)\n\n--\nKieran\n\n\n> ```\n> \n> Gitlab pipeline failure:\n> https://gitlab.freedesktop.org/chenghaoyang/libcamera/-/pipelines/1260412\n> \n> Updates in v13:\n> - Returned an errno in `FrameGenerete::generateFrame()`\n> \n> Updates in v12:\n> - Added ImageFrameGenerator first, then add Parser.\n> - Removed ScaleMode argument, as only Fill is supported now.\n> - Fixed bug of Stream's configuration usage in configure().\n> \n> Updates in v11:\n> - Allowed a single value in the config file's frame_rates field.\n> \n> Updates in v10:\n> Apply fixes according to Jacopo's and Barnabás' comments.\n> - Split test_pattern and path fields in the yaml format.\n> - Let FrameGenerators control frameCount_.\n> - Fixed match() returning values.\n> \n> Updates in v9: Allocate contiguous memory for planes in the same\n> FrameBuffer.\n> \n> BR,\n> Harvey\n> \n> Harvey Yang (6):\n> libcamera: add DmaBufAllocator::exportBuffers()\n> libcamera: Remove PipelineHandler Fatal check of non-empty\n> MediaDevices\n> libcamera: virtual: Add VirtualPipelineHandler\n> libcamera: virtual: Add ImageFrameGenerator\n> libcamera: virtual: Read config and register cameras based on the\n> config\n> libcamera: software_isp: Refactor SoftwareIsp to use\n> DmaBufAllocator::exportBuffers\n> \n> Konami Shu (1):\n> libcamera: pipeline: Add test pattern for VirtualPipelineHandler\n> \n> .../libcamera/internal/dma_buf_allocator.h | 13 +\n> meson.build | 1 +\n> meson_options.txt | 3 +-\n> src/android/meson.build | 19 -\n> src/libcamera/dma_buf_allocator.cpp | 57 +++\n> src/libcamera/pipeline/virtual/README.md | 48 +++\n> .../pipeline/virtual/data/virtual.yaml | 36 ++\n> .../pipeline/virtual/frame_generator.h | 29 ++\n> .../virtual/image_frame_generator.cpp | 175 ++++++++\n> .../pipeline/virtual/image_frame_generator.h | 51 +++\n> src/libcamera/pipeline/virtual/meson.build | 13 +\n> src/libcamera/pipeline/virtual/parser.cpp | 243 +++++++++++\n> src/libcamera/pipeline/virtual/parser.h | 39 ++\n> .../virtual/test_pattern_generator.cpp | 137 +++++++\n> .../pipeline/virtual/test_pattern_generator.h | 53 +++\n> src/libcamera/pipeline/virtual/virtual.cpp | 382 ++++++++++++++++++\n> src/libcamera/pipeline/virtual/virtual.h | 61 +++\n> src/libcamera/pipeline_handler.cpp | 11 +-\n> src/libcamera/software_isp/software_isp.cpp | 20 +-\n> src/meson.build | 19 +\n> 20 files changed, 1368 insertions(+), 42 deletions(-)\n> create mode 100644 src/libcamera/pipeline/virtual/README.md\n> create mode 100644 src/libcamera/pipeline/virtual/data/virtual.yaml\n> create mode 100644 src/libcamera/pipeline/virtual/frame_generator.h\n> create mode 100644 src/libcamera/pipeline/virtual/image_frame_generator.cpp\n> create mode 100644 src/libcamera/pipeline/virtual/image_frame_generator.h\n> create mode 100644 src/libcamera/pipeline/virtual/meson.build\n> create mode 100644 src/libcamera/pipeline/virtual/parser.cpp\n> create mode 100644 src/libcamera/pipeline/virtual/parser.h\n> create mode 100644 src/libcamera/pipeline/virtual/test_pattern_generator.cpp\n> create mode 100644 src/libcamera/pipeline/virtual/test_pattern_generator.h\n> create mode 100644 src/libcamera/pipeline/virtual/virtual.cpp\n> create mode 100644 src/libcamera/pipeline/virtual/virtual.h\n> \n> -- \n> 2.46.1.824.gd892dcdcdd-goog\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id C3AF0C0F1B\n\tfor ;\n\tFri, 27 Sep 2024 16:32:50 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id A22046350F;\n\tFri, 27 Sep 2024 18:32:49 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 9657A634F2\n\tfor ;\n\tFri, 27 Sep 2024 18:32:47 +0200 (CEST)","from pendragon.ideasonboard.com\n\t(cpc89244-aztw30-2-0-cust6594.18-1.cable.virginm.net [86.31.185.195])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 2F1D9163;\n\tFri, 27 Sep 2024 18:31:18 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"ZePk0xrl\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1727454678;\n\tbh=NWdBL7oUE7k1dA/Mc2lIN3YKIm0EpiSfLM653d/FEqE=;\n\th=In-Reply-To:References:Subject:From:Cc:To:Date:From;\n\tb=ZePk0xrlSKuXi6XX8W2EI78ub2AfrK3U9ro3dVjb5ArB1E1cgohATvqfWwz9pCv7Q\n\t2Brv6ek+lPpO/m8tupYQW1YTXMWMAAjFCZqyQ6ewa5L8u49u04nZAf7KciJyFxrRuS\n\t1o185rb3L40IIan/jn8knmkD2HVh/tSrJyRRhY+A=","Content-Type":"text/plain; charset=\"utf-8\"","MIME-Version":"1.0","Content-Transfer-Encoding":"quoted-printable","In-Reply-To":"<20240926163141.1593026-1-chenghaoyang@google.com>","References":"<20240926163141.1593026-1-chenghaoyang@google.com>","Subject":"Re: [PATCH v13 0/7] Add VirtualPipelineHandler","From":"Kieran Bingham ","Cc":"Harvey Yang ","To":"Harvey Yang ,\n\tlibcamera-devel@lists.libcamera.org","Date":"Fri, 27 Sep 2024 17:32:44 +0100","Message-ID":"<172745476410.532453.10065732689440216657@ping.linuxembedded.co.uk>","User-Agent":"alot/0.10","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":31455,"web_url":"https://patchwork.libcamera.org/comment/31455/","msgid":"","date":"2024-09-27T17:55:19","subject":"Re: [PATCH v13 0/7] Add VirtualPipelineHandler","submitter":{"id":117,"url":"https://patchwork.libcamera.org/api/people/117/","name":"Cheng-Hao Yang","email":"chenghaoyang@chromium.org"},"content":"Hi Kieran,\n\nOn Sat, Sep 28, 2024 at 12:32 AM Kieran Bingham\n wrote:\n>\n> Quoting Harvey Yang (2024-09-26 17:29:49)\n> >\n> > Harvey Yang New message\n> > Tue, Sep 10, 12:48 PM\n> > to libcamera-devel, me\n> >\n> > Hi all,\n> >\n> > This series adds virtual pipeline handler, which doesn't depend on any\n> > hardware, like camera sensor or ISP. Currently the configuration\n> > supports test patterns and images.\n> >\n> > It passed the gitlab pipeline:\n> > https://gitlab.freedesktop.org/chenghaoyang/libcamera/-/pipelines/1278799\n> >\n> > I've also checked that it works on qcam.\n> >\n> > I failed to pass multi_stream_test when trying to enable multiple\n> > streams though. Please give me some hints what it tests and what I\n> > missed:\n> > ```\n> > [295:43:43.910237144] [1441841] INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/usr/local/google/home/chenghaoyang/Workspace/libca\n> > mera/build/src/ipa' to the IPA search path\n> > [295:43:43.914030564] [1441841] INFO Camera camera_manager.cpp:325 libcamera v0.3.1+79-8ca4f033-dirty (2024-08-29T19:18:51UTC)\n> > [295:43:43.915493754] [1441844] ERROR DmaBufAllocator dma_buf_allocator.cpp:120 Could not open any dma-buf provider\n> > [295:43:43.919118835] [1441841] INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/usr/local/google/home/chenghaoyang/Workspace/libca\n> > mera/build/src/ipa' to the IPA search path\n> > [295:43:43.922245825] [1441841] INFO Camera camera_manager.cpp:325 libcamera v0.3.1+79-8ca4f033-dirty (2024-08-29T19:18:51UTC)\n> > [295:43:43.922820175] [1441846] ERROR DmaBufAllocator dma_buf_allocator.cpp:120 Could not open any dma-buf provider\n> > Unable to set the pipeline to the playing state.\n>\n> I presume this is some failure in the configuration of the virtual test\n> environment that isn't exposing access to the suitable udmabuf handler?\n\nActually, I don't think so. I added logs in exportFrameBuffer(), and the\ntest doesn't produce that log, which indicates the test fails before using\nDmaBufAllocator. Also, the test still fails with memfd workaround:\nhttps://gitlab.freedesktop.org/chenghaoyang/libcamera/-/pipelines/1279655\n\n>\n> And I wonder if this series is now making the unit tests use the virtual\n> camera instead of vimc ?\n\nIn my own trials, it seems that multi_stream_test constructs both\nvirtual and vimc pipeline handlers. I'm not exactly sure what it tests\nthough.\n\n>\n> If so - we probably want to have the tests more consistently defined\n> maybe - to either prefer specific cameras - or run on all cameras\n> available?\n>\n> But I think fixing this issue probably needs investigation on the\n> libcamera-ci build (https://gitlab.freedesktop.org/camera/libcamera-ci)\n>\n> --\n> Kieran\n>\n>\n> > ```\n> >\n> > Gitlab pipeline failure:\n> > https://gitlab.freedesktop.org/chenghaoyang/libcamera/-/pipelines/1260412\n> >\n> > Updates in v13:\n> > - Returned an errno in `FrameGenerete::generateFrame()`\n> >\n> > Updates in v12:\n> > - Added ImageFrameGenerator first, then add Parser.\n> > - Removed ScaleMode argument, as only Fill is supported now.\n> > - Fixed bug of Stream's configuration usage in configure().\n> >\n> > Updates in v11:\n> > - Allowed a single value in the config file's frame_rates field.\n> >\n> > Updates in v10:\n> > Apply fixes according to Jacopo's and Barnabás' comments.\n> > - Split test_pattern and path fields in the yaml format.\n> > - Let FrameGenerators control frameCount_.\n> > - Fixed match() returning values.\n> >\n> > Updates in v9: Allocate contiguous memory for planes in the same\n> > FrameBuffer.\n> >\n> > BR,\n> > Harvey\n> >\n> > Harvey Yang (6):\n> > libcamera: add DmaBufAllocator::exportBuffers()\n> > libcamera: Remove PipelineHandler Fatal check of non-empty\n> > MediaDevices\n> > libcamera: virtual: Add VirtualPipelineHandler\n> > libcamera: virtual: Add ImageFrameGenerator\n> > libcamera: virtual: Read config and register cameras based on the\n> > config\n> > libcamera: software_isp: Refactor SoftwareIsp to use\n> > DmaBufAllocator::exportBuffers\n> >\n> > Konami Shu (1):\n> > libcamera: pipeline: Add test pattern for VirtualPipelineHandler\n> >\n> > .../libcamera/internal/dma_buf_allocator.h | 13 +\n> > meson.build | 1 +\n> > meson_options.txt | 3 +-\n> > src/android/meson.build | 19 -\n> > src/libcamera/dma_buf_allocator.cpp | 57 +++\n> > src/libcamera/pipeline/virtual/README.md | 48 +++\n> > .../pipeline/virtual/data/virtual.yaml | 36 ++\n> > .../pipeline/virtual/frame_generator.h | 29 ++\n> > .../virtual/image_frame_generator.cpp | 175 ++++++++\n> > .../pipeline/virtual/image_frame_generator.h | 51 +++\n> > src/libcamera/pipeline/virtual/meson.build | 13 +\n> > src/libcamera/pipeline/virtual/parser.cpp | 243 +++++++++++\n> > src/libcamera/pipeline/virtual/parser.h | 39 ++\n> > .../virtual/test_pattern_generator.cpp | 137 +++++++\n> > .../pipeline/virtual/test_pattern_generator.h | 53 +++\n> > src/libcamera/pipeline/virtual/virtual.cpp | 382 ++++++++++++++++++\n> > src/libcamera/pipeline/virtual/virtual.h | 61 +++\n> > src/libcamera/pipeline_handler.cpp | 11 +-\n> > src/libcamera/software_isp/software_isp.cpp | 20 +-\n> > src/meson.build | 19 +\n> > 20 files changed, 1368 insertions(+), 42 deletions(-)\n> > create mode 100644 src/libcamera/pipeline/virtual/README.md\n> > create mode 100644 src/libcamera/pipeline/virtual/data/virtual.yaml\n> > create mode 100644 src/libcamera/pipeline/virtual/frame_generator.h\n> > create mode 100644 src/libcamera/pipeline/virtual/image_frame_generator.cpp\n> > create mode 100644 src/libcamera/pipeline/virtual/image_frame_generator.h\n> > create mode 100644 src/libcamera/pipeline/virtual/meson.build\n> > create mode 100644 src/libcamera/pipeline/virtual/parser.cpp\n> > create mode 100644 src/libcamera/pipeline/virtual/parser.h\n> > create mode 100644 src/libcamera/pipeline/virtual/test_pattern_generator.cpp\n> > create mode 100644 src/libcamera/pipeline/virtual/test_pattern_generator.h\n> > create mode 100644 src/libcamera/pipeline/virtual/virtual.cpp\n> > create mode 100644 src/libcamera/pipeline/virtual/virtual.h\n> >\n> > --\n> > 2.46.1.824.gd892dcdcdd-goog\n> >","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id B516CC0F1B\n\tfor ;\n\tFri, 27 Sep 2024 17:55:34 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 1EAFD6350F;\n\tFri, 27 Sep 2024 19:55:33 +0200 (CEST)","from mail-lj1-x235.google.com (mail-lj1-x235.google.com\n\t[IPv6:2a00:1450:4864:20::235])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 5E6D9634F2\n\tfor ;\n\tFri, 27 Sep 2024 19:55:31 +0200 (CEST)","by mail-lj1-x235.google.com with SMTP id\n\t38308e7fff4ca-2fabe5c8c26so4637031fa.2\n\tfor ;\n\tFri, 27 Sep 2024 10:55:31 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=chromium.org header.i=@chromium.org\n\theader.b=\"iDc3ighk\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=chromium.org; s=google; t=1727459730; x=1728064530;\n\tdarn=lists.libcamera.org; \n\th=content-transfer-encoding:cc:to:subject:message-id:date:from\n\t:in-reply-to:references:mime-version:from:to:cc:subject:date\n\t:message-id:reply-to;\n\tbh=h/1XZSm6QnEb90qWSyThJPHeUbPgnab3rJXuSXsoncA=;\n\tb=iDc3ighkG4Er4s3SR8PbDckygjxhrbl4SuazaJNPbmOue41U0RvPjSfsQjUtEizgDT\n\tVRTZ5tfE59tf6UDbTNhLnEg7At/Ypyw0zdtIONRaeAO1WU1uLW9LFnpcsJhs7NOsZjca\n\tWcascOeFS+B4RPdLnGFTCmQ4/4ItQczRlD5fo=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20230601; t=1727459730; x=1728064530;\n\th=content-transfer-encoding:cc:to:subject:message-id:date:from\n\t:in-reply-to:references:mime-version:x-gm-message-state:from:to:cc\n\t:subject:date:message-id:reply-to;\n\tbh=h/1XZSm6QnEb90qWSyThJPHeUbPgnab3rJXuSXsoncA=;\n\tb=BBYpUA32uFcP324wvRfN9bWzf7unR7pX8y33b4Wa0IWCUdej8TsgVNon9L2id77W2m\n\tLOvxPLaw4+qhWd8Xocmaobk2gGERoW3SEdso/lwoLCijP+5j4KdwxSRQRvimsG4xU6p0\n\trEY0c5IdrONDXxFfcuIz/oS2Zl56deLOIiktjFgDPm4nvcyiIDEuVbN7bCcFdjE9azhC\n\tVF27sLvn43RTkFGVTXQAEzrBj7d+dcfipo2iphCuapCdL+v4Z4Y/eM6+ryzlL1KkA2mv\n\tEMv0cliiA1q0Evvh3PmJx2LnXvMH9cwMAP9Rv4ADZQyhxKgSBVmJ7jPAwTQAa9H8sQ1z\n\teoqw==","X-Gm-Message-State":"AOJu0YzPYYIvrUc773qyFyEBKvl7FNadkuP6HeObWW3Y3Me9WlPKTbM+\n\tNkGHjJJ4aOme50XUk1UxpAUPpwAKdFJDhXuE+XDW6Su18/w/TR6jATz2QduCLEfdPrYPOUMDRXX\n\t5GfyvAaYzuveMjhdOjkb4So2RMCUhKY5Tpyc5636NOqtedqnHaQ==","X-Google-Smtp-Source":"AGHT+IE0kiB6lJyY9bdbTZPNNrLLX076IPPeixmJmcWwod2V21T+x3sR44GA8/Uux8+9ZVC0O1Aru2djRxCXiIs+XsQ=","X-Received":"by 2002:a2e:a987:0:b0:2ef:2344:deec with SMTP id\n\t38308e7fff4ca-2f9d41d6b63mr23313651fa.45.1727459730440;\n\tFri, 27 Sep 2024 10:55:30 -0700 (PDT)","MIME-Version":"1.0","References":"<20240926163141.1593026-1-chenghaoyang@google.com>\n\t<172745476410.532453.10065732689440216657@ping.linuxembedded.co.uk>","In-Reply-To":"<172745476410.532453.10065732689440216657@ping.linuxembedded.co.uk>","From":"Cheng-Hao Yang ","Date":"Sat, 28 Sep 2024 01:55:19 +0800","Message-ID":"","Subject":"Re: [PATCH v13 0/7] Add VirtualPipelineHandler","To":"Kieran Bingham ","Cc":"libcamera-devel@lists.libcamera.org,\n\tHarvey Yang ","Content-Type":"text/plain; charset=\"UTF-8\"","Content-Transfer-Encoding":"quoted-printable","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]