{"id":19683,"url":"https://patchwork.libcamera.org/api/1.1/covers/19683/?format=json","web_url":"https://patchwork.libcamera.org/cover/19683/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20240312105851.243780-1-alexander.gordeev@opensynergy.com>","date":"2024-03-12T10:58:50","name":"[RFC,v9,0/1] Virtio video device specification","submitter":{"id":167,"url":"https://patchwork.libcamera.org/api/1.1/people/167/?format=json","name":"Alexander Gordeev","email":"Alexander.Gordeev@opensynergy.com"},"mbox":"https://patchwork.libcamera.org/cover/19683/mbox/","series":[{"id":4216,"url":"https://patchwork.libcamera.org/api/1.1/series/4216/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=4216","date":"2024-03-12T10:58:50","name":"Virtio video device specification","version":9,"mbox":"https://patchwork.libcamera.org/series/4216/mbox/"}],"comments":"https://patchwork.libcamera.org/api/covers/19683/comments/","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 7321BBD1F1\n\tfor ;\n\tTue, 12 Mar 2024 11:29:02 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 1783962C80;\n\tTue, 12 Mar 2024 12:29:02 +0100 (CET)","from repost01.tmes.trendmicro.eu (repost01.tmes.trendmicro.eu\n\t[18.185.115.27])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 7AA606286C\n\tfor ;\n\tTue, 12 Mar 2024 11:59:51 +0100 (CET)","from 104.47.11.169_.trendmicro.com (unknown [172.21.162.72])\n\tby repost01.tmes.trendmicro.eu (Postfix) with SMTP id 1D5FE10000E3F; \n\tTue, 12 Mar 2024 10:59:51 +0000 (UTC)","from DEU01-FR2-obe.outbound.protection.outlook.com (unknown\n\t[104.47.11.169])\n\tby repre01.tmes.trendmicro.eu (Trend Micro Email Security) with\n\tESMTPS id 177DD10002BAC; Tue, 12 Mar 2024 10:59:48 +0000 (UTC)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=opensynergy.com header.i=@opensynergy.com\n\theader.b=\"gdZEmqKd\"; dkim-atps=neutral","X-TM-MAIL-RECEIVED-TIME":"1710241188.095000","X-TM-MAIL-UUID":"da981b40-d838-41ac-96e1-632cc07132b4","ARC-Seal":"i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;\n\tb=d2uP+sGQB5MBhxrx/zulgxPwtiEWXaCD1QDMO8xRUqJxgKeFpCkz6555c4Fl1TXre/I9SbouUpDP2jhlL2gXEtWVz/kPhz0tB1XDhkhCIUIk+hnxgPcu26eaGgEm4GIoHXatOwGPR+pDwKuEi9Qs627EY+kjC7uOIF0QvZNMZ5P8tgs6EKuLbmfuYwu7sK7isQSFvqfDOR/EpuS73DozZ2TGEOOsl93GP2stKcPNd0nT+I77W0lX9ppgDHLvopEz9isZ1osryvKyUzIW8YywGfqOtJlHU/qnp597LyxbLnehUZgaSkKB5eETInmEofBYBri+/KHXet4fO0WTS91AgQ==","ARC-Message-Signature":"i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;\n\ts=arcselector9901;\n\th=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;\n\tbh=lFzgbUIdZz6uNcbyByJ82Rc8jbhZER0oBBcJ17XRJuw=;\n\tb=TjZ1ZP/Y3YNoxDfjEginXzPVTy7V4PvLcWbSS+wfvuh6cQV57ONcBsL7JAvywIHslNboOsady/HZvadxR6QeFUx/yGKnAfJhmLrYAfR5Js+pgnBJdoeKRldsgJ/zr7D+QlN6cZtuivnbIV627AA5Fd2oKjkoZhbO3xIq3ldHPSCcQsm82V4ejLopPKFRztlq1vytgeqFWHNJ/31quj8TvfQffsrAycpkFbhkuGxxXAagANSEYO2WMobabSH/CXJWC6Ie3O/ZikjFWEppXbxcIP+g/spKJ6KqG1XD7iJCSb8/nRQX6Lvy0n9TebG1UMTZG4/mFjX8W+UmvuUFlNg+CA==","ARC-Authentication-Results":"i=1; mx.microsoft.com 1; spf=pass (sender ip is\n\t217.66.60.4) smtp.rcpttodomain=chromium.org\n\tsmtp.mailfrom=opensynergy.com; \n\tdmarc=pass (p=reject sp=reject pct=100) action=none\n\theader.from=opensynergy.com; dkim=none (message not signed);\n\tarc=none (0)","X-MS-Exchange-Authentication-Results":"spf=pass (sender IP is 217.66.60.4)\n\tsmtp.mailfrom=opensynergy.com; dkim=none (message not signed)\n\theader.d=none;dmarc=pass action=none header.from=opensynergy.com;","Received-SPF":"Pass (protection.outlook.com: domain of opensynergy.com\n\tdesignates 217.66.60.4 as permitted sender)\n\treceiver=protection.outlook.com; \n\tclient-ip=217.66.60.4; helo=SR-MAIL-03.open-synergy.com; pr=C","From":"Alexander Gordeev ","To":"virtio-comment@lists.oasis-open.org","Subject":"[RFC PATCH v9 0/1] Virtio video device specification","Date":"Tue, 12 Mar 2024 11:58:50 +0100","Message-Id":"<20240312105851.243780-1-alexander.gordeev@opensynergy.com>","X-Mailer":"git-send-email 2.40.1","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-EOPAttributedMessage":"0","X-MS-PublicTrafficType":"Email","X-MS-TrafficTypeDiagnostic":"AM3PEPF0000A798:EE_|BEYP281MB3812:EE_","Content-Type":"text/plain","X-MS-Office365-Filtering-Correlation-Id":"da3edb60-2f49-4793-d02c-08dc42838775","X-MS-Exchange-SenderADCheck":"1","X-MS-Exchange-AntiSpam-Relay":"0","X-Microsoft-Antispam":"BCL:0;","X-Microsoft-Antispam-Message-Info":"P4OC/rYxyOvmbLbDuYY0SLjN86qoQrCRbxU5baTDFzR2SUGdTA6LevWih1/4A4ntkBmUrhYBiDdNKXcI24kgWH70foQW0FDdx2z3ltJpWXBCJoTVRfMZCWfNnc19SX5XVzItwCc2/Ithmh/3XoFZpSKbngfJep9wBEkM/U7HmTVB4fBOfeQjVB+ubyGTpdQTXSZQYvO5nrvLs24zZGy9+pYsM07uzI9Et2kcCQv5KzsyVRVff0pmugh9MG+I062P8xSdaAJAR3mbLXOPS8dKoxE0cSD+7wb7S65LZ+8NcDre7aeY6enUCV+6u5iA2ODpUvAKHB5SFBysbXLQidgKiCGWMMQT8X+cAki5XMvswroHbC/rt86DC7c4g+0yl5tpXsjZu12TNqA+hTCqM0J+WeDbClSKdFwzGQYWnotAZ/93kCfk10E7EetoQFNKADiP00dzD05dwHAwei3Mo5YpWIIkTbSgJfeSj9fzElS7b7nMo1PrQ0/YBOq40Zr8Spw2PLzi2fE2gj2CEaDN3arcMYjqfmemhCwjOOijFtlCKvpT6E7PP9TH7MhlUvHaLd92ae3PVYOPdeXXpE0dl1XJIWxOFtTelr5b5QQm7zBSe/OsWikshbDMhpLF28I2wjmgQu+A7CRgaK07GU++1mwYTqv7S3T/JZVDYJcPdxIRwrOeXStDcAKfwPJl3ALvskUBcbX5wmBaJkxwT+toHqPP/g==","X-Forefront-Antispam-Report":"CIP:217.66.60.4; CTRY:DE; LANG:en; SCL:1; SRV:; \n\tIPV:NLI; SFV:NSPM; H:SR-MAIL-03.open-synergy.com;\n\tPTR:InfoDomainNonexistent; CAT:NONE;\n\tSFS:(13230031)(1800799015)(7416005)(376005)(82310400014)(36860700004);\n\tDIR:OUT; SFP:1102; ","X-OriginatorOrg":"opensynergy.com","X-MS-Exchange-CrossTenant-OriginalArrivalTime":"12 Mar 2024 10:59:45.9936\n\t(UTC)","X-MS-Exchange-CrossTenant-Network-Message-Id":"da3edb60-2f49-4793-d02c-08dc42838775","X-MS-Exchange-CrossTenant-Id":"800fae25-9b1b-4edc-993d-c939c4e84a64","X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp":"TenantId=800fae25-9b1b-4edc-993d-c939c4e84a64;\n\tIp=[217.66.60.4]; \n\tHelo=[SR-MAIL-03.open-synergy.com]","X-MS-Exchange-CrossTenant-AuthSource":"AM3PEPF0000A798.eurprd04.prod.outlook.com","X-MS-Exchange-CrossTenant-AuthAs":"Anonymous","X-MS-Exchange-CrossTenant-FromEntityHeader":"HybridOnPrem","X-MS-Exchange-Transport-CrossTenantHeadersStamped":"BEYP281MB3812","X-TM-AS-ERS":"104.47.11.169-0.0.0.0","X-TMASE-Version":"StarCloud-1.3-9.1.1015-28246.006","X-TMASE-Result":"10--17.288200-4.000000","X-TMASE-MatchedRID":"KlGK0EfpKrtKqT77NMMyNxZu3Ax5Ngq859KGlOljPvuQEjOZZEyhAras\n\tXTFkoEYAhYNEGrSdk3jcBYoN9IgCH+7cQoOH30eiquZVB+P8J0YAf7GEWnsTRGSVHUW5n9Lh9NF\n\tKrIIXgXyHl7GO9ptPH77GcxyzvMofQlotKdXOGlGK7q7iyFn58ee6+kao1dvdkvRBElnsHbOIV9\n\tdyzzz/jqyFqPcqKs/nN5KkjJKvdUstHkTd8If/QVhkmCjWYvWlusBvtbk1teAugFBQy+58BQZIL\n\tdGsDavCr3RRr3ptPn3D85EiRKYySmBU2FL1ofut7lU0NYyj5VxWEpmiB2wI29THFoOVTUKPbafs\n\tkoGxs3me8Tj8YASJOQjHjNQtMA4kG7nDH5J3xCYBmIwrrOlq1WGsEiRO8BnIpZd6jUkGtmC6dXa\n\tAYUzGlrIoaeeAvJLFt4fpjyEpFNq1MsxF+lLocdRBUD49QCgpDyrqR96pNEL9fsjZBWN6mnPeH8\n\t07NzYLrKVhusxXDCsphiMDWApKO/dniAo1PYHZEPSTHBgcfv6i3c5/PPV1uPt7J0kZt8YlGQCt0\n\tp3VQOxlVdfr0ykIxos72H+RnuZCeQczFQh2XEv4MTnOvlqF3MU/m8gE5vZPrf+sQ5qSfyxI+l65\n\tvTxt4rqaKIHX1SvitvjNRakP96uZd6OkvSXQQ3x2eIlQG1sMfC4IwOLvyuf/oGTNGh63fyJUNkN\n\trt5r93+6k97GsyDiUTGVAhB5EbQ==","X-TMASE-XGENCLOUD":"5a69dfce-6f07-4538-9004-834dc4c6a3ff-0-0-200-0","X-TM-Deliver-Signature":"65153075E6A4CF5C93B0C4250A12FB16","X-TM-Addin-Auth":"bbBll7Y/N6b5JwSwH6sKQIBikFBffGlM0VQTRgyWuPKi2E/qwgojkJArpar\n\t4YLaK+w1gjeTnePafIiW+gixHdSnrGLegdx7PKhXUUd/HFV01zQu1v7wTGyhe3Yp4nEat3jU7BR\n\tFxZwuXQslSKk+FvOGSHEFvQPFxcVUlswVcOUbA7Jy85qVdbHCIwUQ4VIn9RoJkT4hbhn4lMmBuG\n\tIKMVVqCUR2548mE9mI6qNJHNgu7O4yFmlvjDsKpnMAEsXGq8pXwhDfUbpRaAKyplWfUVZXaN5p4\n\tEVzxezUlgIKfH8n/8S6EVx7+uxla6/pyOaYn.PLsjzK4E/+GHK/8oqAgOD21ucU7/29It0mFTFF\n\tXHi0flf4lLgMD5VLOhunr9nj7j6fZMaSZxgrmQ/U1ak3xyUJCD8+nhxNjtBgYOPGYIAA5o7qqP0\n\tzAlFM5tT4pelP9uaQ/QNN5q6l2ghx1r59yMJnbooC94vcGvisxBz8QDMxDyI2fD86+yQ+bTIANU\n\tmc77W0cppOnjY0p6EiaELgm/XjS6g5ibZpwzHsAxSixsw4AQIItsSErCDI2NKia98iGr4aTGiWk\n\tlDKHh/TPl5EPsOL/3NHXRYFSbFjYDVpnLRGZUr7n9840BI81CLmgPDZE+TB4bC5RVyXBwbTw1vz\n\tERQA==","X-TM-Addin-ProductCode":"EMS","DKIM-Signature":"v=1; a=rsa-sha256; c=simple/simple; d=opensynergy.com;\n\ts=TM-DKIM-20210503141657; t=1710241191;\n\tbh=vxxP2+iaKewLGClE3Fzeah/djsRIX/5doINiMRwSrgI=; l=4690;\n\th=From:To:Date;\n\tb=gdZEmqKdprTyhWZxCrZeQrFiOq8paJ66xAdZQrkAZ90Ji/dpgDHR7aCN2c7QhJpxu\n\tdGp4vVZMQ7Mg5A3ltKCZMsRgsNYsdjIFp+cBqBG6qtgDgWnQAxuh4Tt/4gk21vIODx\n\tEbx115D64SXkW0o3ydmgNZuCWnEOYV+E3qv4jQhCaM/54CIB43POetieZkeVSw2aXJ\n\tOjYF5mljx1ZUb44jqKT/lR8VzfPzNRlN9EMrbeNCmvRzdTNvX8jibIv5f0gnrvboTa\n\tXyxTsB89n9hbsy3P0d9nLzlgaBT6ttMdle1fZFZddcNwWdkMVBkjDXgike9pRGz47+\n\twjY4Uw91Jmlyg==","X-Mailman-Approved-At":"Tue, 12 Mar 2024 12:29:00 +0100","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Albert Esteve , hmazur@google.com,\n\t\"Michael S . Tsirkin\" ,\n\tDaniel Almeida ,\n\tMarcin Wojtas ,\n\tAndrii Cherniavskyi ,\n\tlibcamera-devel@lists.libcamera.org, bgrzesik@google.com,\n\tAndrew Gazizov ,\n\tEnric Balletbo i Serra ,\n\tGustavo Padovan ,\n\tKeiichi Watanabe , zyta@google.com,\n\tlinux-media@vger.kernel.org,\n\tAlexander Gordeev ,\n\talex.bennee@linaro.org, \n\tmikrawczyk@google.com, Matti.Moell@opensynergy.com,\n\tAlexandre Courbot ,\n\tCornelia Huck , bag@semihalf.com, srosek@google.com,\n\tNicolas Dufresne ,\n\tEnrico Granata ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Hi,\n\nThis is the 9th version of virtio-video device patch. Hopefully it is\nthe last RFC version. There are some major changes and now I think it\nis good enough conceptually, but may need a few minor changes, when the\ndriver gets updated. Sorry, it took way longer, than I expected.\nThere are a lot of details to care about and also a lot of distractions.\nPlease review.\n\nChangelog v8 -> v9:\n1. Chenged representation of device capabilities/parameters to\nType-Length-Value (TLV).\nTLV representation is already used in the kernel, see [1][2]. It is\nmentioned in the virtio sound device specification. It is also\ncompatible with struct v4l2_ext_control, that is used to set V4L2\ncontrols in batches. Here I'm trying to extend the approach onto many\nother things: device capabilities, formats, buffers/resources, crop\nsettings. This way it is possible to get all the device capabilities in\none call, then set all the parameters/controls in one call. For us this\nis important due to significant hypercall delays. I hope this might be\napplicable in the V4L2 core as well in the future, so I'm very\ninterested in comments from linux-media community.\n\n2. Changed back the representation of capabilities to pre v7 model.\nVideo decoders and encoders are basically converters between\nmultidimensional shapes of all supported encoded formats and all\nsupported raw formats. This was roughly the way the capabilities were\nrepresented in v1-v6 (excluding the controls), then in v7 and v8 I\ntried to do it closer to the V4L2 discovery process. Now I think this\nwas a mistake, the previously used method may look more complex, but it\nfits better. This approach is similar to the way the capabilities are\ndescribed in gstreamer (they don't seem to have explicit links between\nthe sets though).\n\n3. Replaced the internal device's per stream resource queues with\nvirtqueues.\nThe idea briefly: we had to avoid blocking the descriptors for a long\ntime, so I implemented delayed responses in v8. Then there is no\nnatural limit for the number of elements in the resource queues\nanymore. What to do if the queue becomes full? So there are a bunch of\nthings to define in the spec. I decided to replace these queues with\nvirtqueues, because the virtqueues are already defined well enough.\nThere are of course some upsides and downsides. I think upsides are\noverwhelming and I'd be happy to discuss this if anyone is interested.\n\n4. Renamed commandq to mainq since commands can be queued to per stream\nqueues as well.\n\n5. Renamed STREAM_CREATE -> STREAM_OPEN, STREAM_DESTROY -> STREAM_CLOSE.\nNow the per stream virtqueues are opened rather than created.\n\n6. Enabled RESET on the output queue too.\nI didn't know how to handle a VIDIOC_STREAMOFF on the capture queue\ncoming from guest user-space otherwise.\n\n7. Merged ATTACH_BACKING command into SET_PARAMS.\nI felt, that it really belongs to the parameters alongside the\nnum_resources, it allows the driver to see if the resources are detached\nafter a SET_PARAMS or not, also might be helpful during debugging. Still\nthere is an issue: SG lists can be quite large. If it is large and the\nresults are returned on the eventq, then this significantly increases\nthe eventq's descriptor sizes. Avoided this by adding a requirement for\nthe device to cap the TLV length to 8.\n\n8. Added DRM format modifiers.\n9. Added FWHT codec.\n10. Addressed review comments.\n11. The spec size went down from 24 pages to 19 pages.\n\nFull PDF: https://drive.google.com/file/d/1gj-kOBp1uvCB5IzrtOTjf-zunP3f-EFq/view?usp=sharing\nPDF with the video section only: https://drive.google.com/file/d/1fH8XaSYkwkb2beB9thS7FOvFysUM22Q5/view?usp=sharing\nPlease note, that external links are broken in the short version.\n\n[1] https://docs.kernel.org/userspace-api/netlink/intro.html?highlight=tlv\n[2] https://docs.kernel.org/sound/kernel-api/writing-an-alsa-driver.html?highlight=tlv\n[3] https://lore.kernel.org/linux-media/20230922100303.GF19112@pendragon.ideasonboard.com/\n\nAlexander Gordeev (1):\n virtio-video: Add virtio video device specification\n\n conformance.tex | 4 +\n content.tex | 1 +\n device-types/video/description.tex | 1607 +++++++++++++++++++++\n device-types/video/device-conformance.tex | 22 +\n device-types/video/driver-conformance.tex | 20 +\n introduction.tex | 21 +\n 6 files changed, 1675 insertions(+)\n create mode 100644 device-types/video/description.tex\n create mode 100644 device-types/video/device-conformance.tex\n create mode 100644 device-types/video/driver-conformance.tex"}