From patchwork Mon Oct 24 00:03:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17663 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A9537BDB16 for ; Mon, 24 Oct 2022 00:04:24 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B55A762ED7; Mon, 24 Oct 2022 02:04:23 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1666569863; bh=syxSYGPu4oGXI77zNPgI3E9nNA/ccBIRdNQUr3uMZ1M=; h=To:Date:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=Oq8pyGwAkpvn6AD93F5HTovVFVJRTewXjtFqxE/T+OisUQi0mEvSL5B/Tg1i7IJJL 4qz1Utqcy67f5AOFmfZ1gSlf4sz1jOZhqSJHczw59OlxvtNOKElRx+6FoAW5X8aw0f tZIPB9CSiYg4nTAeFGOYG+ZHOk+PTcQrUXpmXSw9Ia+hkSG7jq/JyesvmY+hom02Sj Z2onKyINJPiJJcdapQxlxGpfBWGTHpkJhACyFDDuzGTeBPa+E4eK+FiLWV2izgGL8H 4A9SpeDFU24rIwi9q9van176trE6Hmye1DnMjVnjySXMzI6EuKPlhS50QsPjT9EY53 lsLNqfteZbJsg== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 383D462EA6 for ; Mon, 24 Oct 2022 02:04:22 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="oTVKNWYw"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 48B14471; Mon, 24 Oct 2022 02:04:21 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1666569861; bh=syxSYGPu4oGXI77zNPgI3E9nNA/ccBIRdNQUr3uMZ1M=; h=From:To:Cc:Subject:Date:From; b=oTVKNWYwHABFtFCldv0agEj6dFCH6nSnKGxKGivfhx5l5/1aMEs08kmmrKTng8abl UgMwU920Y18PXvWGlqeR/Jcxhbj4U+N9iVk1KyqJdUeacB2JKy4bBCXFfCEEohCwK/ tCJsT2A4QhDSu2SozMOXcNDybXIP3eUsmwgmBNo0= To: libcamera-devel@lists.libcamera.org Date: Mon, 24 Oct 2022 03:03:43 +0300 Message-Id: <20221024000356.29521-1-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.37.4 MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 00/13] Add Bayer format support for RkISP1 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Hello, This patch series implements support for raw Bayer capture in the RkISP1 pipeline handler and IPA module. I have taken over Florian's v2, addressed my review comments, and added further fixes and improvements. The RkISP1 can capture raw frames by bypassing the ISP, feeding the input data directly to the resizer (which has to be bypassed as well, with a 1:1 scaling ratio). In this mode, the ISP doesn't consume parameters or produce statistics, which prevents most algorithms from running. The notable exception is the AGC algorithm which still needs to run in order to set manual exposure time and analogue gain (automatic mode is not available). The series contains a few drive-by fixes and cleanups. Patch 01/13 is one of those, it reorders the algorithm operations in the .h and .cpp files. If it gets accepted, I will likely merge it sooner than later and separately from the rest of the series as it has a high risk of causing conflicts. Patch 02/13 is another small cleanup in the RkISP1 AGC. Patch 03/13 starts adding support for raw capture by addressing the IPA side first, allowing algorithms to report if they support raw capture, and skipping the ones that don't at runtime when capturing raw frames. The next patch, 04/13, adds support for manual gain and exposure time in the AGC algorithm. I have picked it up from the list (posted as "[PATCH v3] ipa: rkisp1: Add support for manual gain and exposure"), in order to implement raw support in AGC on top in patch 05/13. With patches 06/13 to 08/13 we get back on the libcamera side, with more drive-by improvements that turn StreamRole into a scoped enum and return a unique pointer from PipelineHandler::generateConfiguration(). These could also be merged independently from the rest of the series. Patches 09/13 to 11/13 then prepare the RkISP1 pipeline handler. Patch 09/13 is a small refactoring to prepare for the larger changes, patch 10/13 an improvement previously posted to the list (as "[RFC PATCH v2] pipeline: rkisp1: Query the driver for formats") that I found useful, and patch 11/13 a bug fix. Finally, patches 12/13 and 13/13 implement raw capture support in the pipeline handler. I have split this into runtime support (12/13) and configuration support (13/13) to ease review. Patch 13/13 could possibly be further split into three patches for configure(), validate() and generateConfiguration(), but that's more complicated and I think that, while relatively big, 13/13 is still reviewable. The result has been tested on an NXP i.MX8MP with an IMX219 camera sensor. I haven't noticed any regression with non-raw capture, and have successfully captured raw frames in different resolutions by specifying either a raw role or a raw pixel format on the cam command line. Florian Sylvestre (2): pipeline: rkisp1: Support raw Bayer capture at runtime pipeline: rkisp1: Support raw Bayer capture configuration Laurent Pinchart (9): ipa: Sort algorithm operations based on calling order ipa: rkisp1: Move shutter speed and analogue gain limits from agc to sensor ipa: rkisp1: Support raw capture in IPA operations ipa: rkisp1: agc: Support raw capture libcamera: stream: Add operator<<() to print StreamRole as a string libcamera: stream: Turn StreamRole into scoped enumeration libcamera: pipeline_handler: Return unique_ptr from generateConfiguration() pipeline: rkisp1: Pass info pointer to tryCompleteRequest() pipeline: rkisp1: Fix stream size validation Paul Elder (2): ipa: rkisp1: Add support for manual gain and exposure pipeline: rkisp1: Query the driver for formats include/libcamera/internal/pipeline_handler.h | 2 +- include/libcamera/ipa/rkisp1.mojom | 2 +- include/libcamera/stream.h | 5 +- src/apps/lc-compliance/capture_test.cpp | 17 +- src/gstreamer/gstlibcamerapad.cpp | 20 +- src/ipa/ipu3/algorithms/af.cpp | 32 +- src/ipa/ipu3/algorithms/awb.cpp | 166 +++++----- src/ipa/libipa/algorithm.cpp | 34 +- src/ipa/libipa/algorithm.h | 14 +- src/ipa/rkisp1/algorithms/agc.cpp | 184 +++++++---- src/ipa/rkisp1/algorithms/agc.h | 6 + src/ipa/rkisp1/algorithms/algorithm.h | 12 +- src/ipa/rkisp1/algorithms/awb.cpp | 94 +++--- src/ipa/rkisp1/algorithms/awb.h | 6 +- src/ipa/rkisp1/ipa_context.cpp | 33 +- src/ipa/rkisp1/ipa_context.h | 40 ++- src/ipa/rkisp1/rkisp1.cpp | 69 +++- src/libcamera/camera.cpp | 8 +- src/libcamera/pipeline/ipu3/ipu3.cpp | 10 +- .../pipeline/raspberrypi/raspberrypi.cpp | 12 +- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 309 ++++++++++++------ src/libcamera/pipeline/rkisp1/rkisp1_path.cpp | 254 ++++++++++++-- src/libcamera/pipeline/rkisp1/rkisp1_path.h | 15 +- src/libcamera/pipeline/simple/simple.cpp | 12 +- src/libcamera/pipeline/uvcvideo/uvcvideo.cpp | 8 +- src/libcamera/pipeline/vimc/vimc.cpp | 8 +- src/libcamera/pipeline_handler.cpp | 3 +- src/libcamera/stream.cpp | 19 ++ 28 files changed, 957 insertions(+), 437 deletions(-) base-commit: f9c859343936db876f13d550ff9adca96a9ef528