From patchwork Mon Aug 1 00:05:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 16877 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BB95CBE173 for ; Mon, 1 Aug 2022 00:05:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 13396603F2; Mon, 1 Aug 2022 02:05:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1659312351; bh=eoTMFFzec7AYczKR6X/cb+4Bh2MBLQalHxaTzfMe4bc=; h=To:Date:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=3tF9x4R81LVHT7z+aIjlY+zNASy33nJD7gRch92+N4JaI1J1NhqxyDbLreeykbkvf FlVGTqzGSrlXAf4jroXr20eB0HzGnUnIJm8O445pEnpABXQHDdQF9Wvg2duStGAKZJ ofVmzLsOkBMKKGbRbR+PsArJ1a+K+30fH+kPLyYa3zT1/spwFWyCuyMN2yDGDW0tCo rmD9F4l27czPpyl2Q+Vny9OS2kZuvP1euOHdTL86sRM7cf97jK7RaI0inGBEvBMsdw gqIVsRXRWhHdzpOW/RsfUSmyp8J+BIb++wdUGnalSPC6A3WPNBlQhzCyXuNwUBhlZZ k9LLFK9r92+TA== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E3F04603F2 for ; Mon, 1 Aug 2022 02:05:48 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="emW5yIrQ"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 43E1030B for ; Mon, 1 Aug 2022 02:05:48 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1659312348; bh=eoTMFFzec7AYczKR6X/cb+4Bh2MBLQalHxaTzfMe4bc=; h=From:To:Subject:Date:From; b=emW5yIrQtyMT/B7EZhjflUETDkpbCXs2t6XBQPxy9GTCgkE4CjuJ39sfWQe584puD 0KYNFqNJAxDCmxRZ7ZvDy18WP9aPJ9oIudEhi3/G9Vmfd5qOrFa6MKXwgV6qZ0mkJf 7MNBN5wMxgQ3qCxXzO7/4YE5JQhf6QvBewLY2Cd0= To: libcamera-devel@lists.libcamera.org Date: Mon, 1 Aug 2022 03:05:30 +0300 Message-Id: <20220801000543.3501-1-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 00/13] libcamera: pipeline: simple: Support the NXP i.MX8 ISI X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Hello, This patch series adds support for the NXP i.MX8 ISI to the simple pipeline handler. The ISI (which stands for Image Sensor Interface) is an IP core found in multiple SoCs from the NXP i.MX8 family. It combines a crossbar switch with processing pipelines that each include a scaler, a colorspace converter and a DMA engine to capture frames from multiple inputs to memory. As data flows linearly from the source to the DMA engine, the ISI is a good candidate for the simple pipeline handler (while the ISI can be used with Bayer sensors by bypassing the processing pipeline to capture raw data, most use cases will use RGB or YUV sources). The ISI driver has been posted to the linux-media mailing list, currently as a v2 ([1]). It depends on the V4L2 routing API to control the crossbar switch, and as the API is still under development ([2]), this blocks merging the driver for the time being. However, the routing API is now at v12, and the development effort isn't slowing down, we may consider merging support for the API in libcamera before it hits upstream. I would appreciate feedback on this. The series starts with three patches that update the Linux kernel header, with 01/13 moving to headers from kernel v5.19, and patches 02/13 and 03/13 adding new pixel formats that have been merged in subsystem trees and will be present in v5.20. Patch 04/13 then adds support for those formats to libcamera. If we decide that the routing API shouldn't be merged in libcamera yet, these first four patches are still candidates for upstreaming without delay. Patch 05/13 then adds the V4L2 routing API to the kernel headers, and patches 06/13 to 08/13 extend the V4L2Subdevice class to support the API. Patches 09/13 to 12/13 extend the simple pipeline handler to take routes into account during pipeline discovery and configuration. Finally, patch 13/13 enables support for the ISI in the simple pipeline handler. [1] https://lore.kernel.org/linux-media/20220712000251.13607-1-laurent.pinchart@ideasonboard.com/ [2] https://lore.kernel.org/linux-media/20220301161156.1119557-1-tomi.valkeinen@ideasonboard.com/ Jacopo Mondi (4): libcamera: v4l2_subdevice: Change V4L2Subdevice::Whence libcamera: v4l2_subdevice: Collect subdev capabilities libcamera: v4l2_subdevice: Add support for the V4L2 subdev routing API libcamera: pipeline: simple: Reset routing table of subdevs Laurent Pinchart (7): include: linux: Update kernel headers to version v5.19 include: drm_fourcc: Add AVUY and XVUY 4:4:4 packet formats include: linux: Add V4L2 YUVA32 and YUVX32 pixel formats libcamera: formats: Add AVUY8888 and XVUY8888 formats include: linux: Add V4L2 subdev internal routing API libcamera: pipeline: simple: Setup links in the context of sink entities libcamera: pipeline: simple: Add support for NXP ISI Phi-Bang Nguyen (2): libcamera: pipeline: simple: Walk pipeline using subdev internal routing libcamera: pipeline: simple: Don't disable links carrying other streams include/libcamera/internal/v4l2_subdevice.h | 28 +- include/linux/README | 2 +- include/linux/dma-buf.h | 4 +- include/linux/drm_fourcc.h | 88 +++++- include/linux/intel-ipu3.h | 35 ++- include/linux/v4l2-controls.h | 301 +++++++++++++++++++- include/linux/v4l2-subdev.h | 88 +++++- include/linux/videodev2.h | 15 +- src/libcamera/formats.cpp | 26 ++ src/libcamera/formats.yaml | 4 + src/libcamera/pipeline/simple/simple.cpp | 173 ++++++++++- src/libcamera/v4l2_pixelformat.cpp | 4 + src/libcamera/v4l2_subdevice.cpp | 161 ++++++++++- 13 files changed, 877 insertions(+), 52 deletions(-) base-commit: 69ae75b0cc211f82665b3e92fb3de64a9852b403 Tested-by: Jacopo Mondi