From patchwork Wed Aug 4 12:43:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 13195 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id ED252C3235 for ; Wed, 4 Aug 2021 12:43:32 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 331B46882A; Wed, 4 Aug 2021 14:43:32 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="pp5v4thU"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 034AE6026E for ; Wed, 4 Aug 2021 14:43:30 +0200 (CEST) Received: from pendragon.lan (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 8139724F for ; Wed, 4 Aug 2021 14:43:29 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1628081009; bh=L9yLxay/WS0zp5KfqpJQXy1h95X5lnQTkgnxIs6dbmA=; h=From:To:Subject:Date:From; b=pp5v4thUX3J5GJPPL247ig8qdgw5l1pVcTLbk2hnIEl+XYkmGtHk0rTk230UCR3Jn EiPL4CR7UcQnDbC4h/t6pJOyGduT8AF5RyNu2R0KHWBtMpaENSRa9UOVGyYmZ8Bs01 TLRTno6sMorL6q9LER+iBCr8pHM+53fRpf2VKgUM= From: Laurent Pinchart To: libcamera-devel@lists.libcamera.org Date: Wed, 4 Aug 2021 15:43:06 +0300 Message-Id: <20210804124314.8044-1-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.31.1 MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 0/8] libcamera: Add DRM/KMS viewfinder display to cam X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Hello, This patch series extends the cam application to implement live viewfinder display using a DRM/KMS device. Supported display features are currently fairly limited (scaling and overlays are not supported for instance), but should nonetheless be a good first step. Patches 1/8 extends to EventLoop class to support file descriptor events, needed by KMS. Patches 2/8 to 4/8 then refactor the cam application and the BufferWriter class to introduce an abstract FrameSink class. Patch 5/8 adds a set of classes to wrap the DRM/KMS API (using libdrm). Patches 6/8 and 7/8 add a DRM/KMS frame sink implementation, and patch 8/8 wires it up in the cam application. I've kept patch 7/8 separate from 6/8 to outline in the history that it's one of multiple options, and make it easier to switch to a different implementation later if needed. Enabling the display is a costly operation, so doing it at start() time would be best. However, we would need a frame buffer to be displayed at that point, and none is available yet from the camera. Patch 7/8 thus delays enabling the display to the first frame. The operation isn't too costly from a userspace point of view as the atomic request is committed asynchronously (although it still requires two ioctl calls, one to create the mode blob - but this could be moved to start() - and one to commit the request). Another option would be to allocate an extra dumb buffer and fill it with static data for the first frame. I've tested this with the simple pipeline handler on an R-Car H3 Salvator-XS board, running cam -c 'platform/vimc.0 Sensor B' \ -s pixelformat=BGR888,width=1440,height=900 -C -D HDMI-A-1 with vimc as a source. As the R-Car DU requires DMA contiguous memory by default (IOMMU support is enabled), a patch is required to use the vb2 dma-contig allocator in vimc. It can be found at [1]. The size of the stream has to match the size of the display, as scaling isn't supported yet. As vimc requires the output size to be a multiple of 3, this restricts the possible resolutions (we should *really* fix this in the vimc driver). Compared to v2, review comments have been taken into account. Please see changelogs in individual patches for details. Compared to v1, the frame sink implementation is now based on requests instead of buffers, which should address Niklas' concerns. [1] https://lore.kernel.org/linux-media/20210730001939.30769-1-laurent.pinchart+renesas@ideasonboard.com/T/#u Laurent Pinchart (8): cam: event_loop: Add support for file descriptor events cam: Add FrameSink base class cam: Turn BufferWriter into a FrameSink cam: Rename BufferWriter to FileSink cam: Add DRM helper classes cam: Add KMS sink class cam: kms_sink: Enable display on first frame cam: Add support for viewfinder through DRM/KMS src/cam/buffer_writer.h | 31 - src/cam/camera_session.cpp | 94 ++- src/cam/camera_session.h | 6 +- src/cam/drm.cpp | 663 +++++++++++++++++++ src/cam/drm.h | 331 +++++++++ src/cam/event_loop.cpp | 44 ++ src/cam/event_loop.h | 20 + src/cam/{buffer_writer.cpp => file_sink.cpp} | 49 +- src/cam/file_sink.h | 38 ++ src/cam/frame_sink.cpp | 67 ++ src/cam/frame_sink.h | 34 + src/cam/kms_sink.cpp | 297 +++++++++ src/cam/kms_sink.h | 74 +++ src/cam/main.cpp | 6 + src/cam/main.h | 1 + src/cam/meson.build | 17 +- 16 files changed, 1718 insertions(+), 54 deletions(-) delete mode 100644 src/cam/buffer_writer.h create mode 100644 src/cam/drm.cpp create mode 100644 src/cam/drm.h rename src/cam/{buffer_writer.cpp => file_sink.cpp} (67%) create mode 100644 src/cam/file_sink.h create mode 100644 src/cam/frame_sink.cpp create mode 100644 src/cam/frame_sink.h create mode 100644 src/cam/kms_sink.cpp create mode 100644 src/cam/kms_sink.h