From patchwork Thu Nov 24 02:51:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17851 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0C6A2BDE6B for ; Thu, 24 Nov 2022 02:51:53 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6721F6331B; Thu, 24 Nov 2022 03:51:52 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1669258312; bh=7j+J0hN5+fPe5zoNQarr5dJXIpzVDMvZrG75KMruB7s=; h=To:Date:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=bj15OiRbXGFBw2XEysJMV8yk8eSA90u/pjrZJv9H5k7ILpYbgDgkzGu/WjKRItFuo nV+ZHPdkDpCFdVz+YVzeUiERcjAG/DU9UryHBymdTF/9JvPybaNuQBBxCd5Y4jHNv4 9zry/6ohWsAGo++zMcM+Ue7cfGWzwHR7w3KReCyTZGcOn3kDM5Dl13KuJIwiOMSLk4 VbrOdcKhlip9IiaohmjstNGCwbnXJ26AUladUHo8pwEmqiPZVhfY0CgJOVvNVO4Fj6 BY37iYH2asy7jX4ZAXTvO6SdELyWQeFiMFprO3fvDHz1HwfCyevGR0okcc1MN5xSj7 u5jN9rzRtj84w== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 94F41603CE for ; Thu, 24 Nov 2022 03:51:50 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="HDz3je+0"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id F1698496 for ; Thu, 24 Nov 2022 03:51:49 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1669258310; bh=7j+J0hN5+fPe5zoNQarr5dJXIpzVDMvZrG75KMruB7s=; h=From:To:Subject:Date:From; b=HDz3je+0NlvBTEjZrZHyn7O+hhWdUMhEF4rdpMWMwb2fQCoBDchxZtWIMAA8w2tO1 mpZX3A6T15FeieON5RiJ1NP4umBSaHD7LocOt/B4fHD9c4lcOlfTgREolKM9ntNA1Q SSlkbU/PsO6V+mBznZBW5HM3U9/v72V/qYRDAy8Q= To: libcamera-devel@lists.libcamera.org Date: Thu, 24 Nov 2022 04:51:24 +0200 Message-Id: <20221124025133.17875-1-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.37.4 MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 0/9] Add Bayer format support for RkISP1 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Hello, This patch series implements support for raw Bayer capture in the RkISP1 pipeline handler and IPA module. I have taken over Florian's v2, posted v3, and addressed review comments in this v4, with further bug fixes. The RkISP1 can capture raw frames by bypassing the ISP, feeding the input data directly to the resizer (which has to be bypassed as well, with a 1:1 scaling ratio). In this mode, the ISP doesn't consume parameters or produce statistics, which prevents most algorithms from running. The notable exception is the AGC algorithm which still needs to run in order to set manual exposure time and analogue gain (automatic mode is not available). The series starts with a few drive-by fixes and cleanups. Patches 1/9 and 2/9 turn StreamRole into a scoped enum. These could be merged independently from the rest of the series if a v5 is needed for the remaining patches. Patch 3/9 starts adding support for raw capture by addressing the IPA side first, allowing algorithms to report if they support raw capture, and skipping the ones that don't at runtime when capturing raw frames. The next patch, 4/9, adds support for manual gain and exposure time in the AGC algorithm. I have picked it up from the list (posted as "[PATCH v3] ipa: rkisp1: Add support for manual gain and exposure"), in order to implement raw support in AGC on top in patch 5/9. Patches 6/9 and 7/9 then prepare the RkISP1 pipeline handler. Patch 6/9 is an improvement previously posted to the list (as "[RFC PATCH v2] pipeline: rkisp1: Query the driver for formats") that I found useful, and patch 7/9 a bug fix. Finally, patches 8/9 and 3/9 implement raw capture support in the pipeline handler. I have split this into runtime support (8/9) and configuration support (9/9) to ease review. Patch 9/9 could possibly be further split into three patches for configure(), validate() and generateConfiguration(), but that's more complicated and I think that, while relatively big, 13/13 is still reviewable. The result has been tested on an NXP i.MX8MP with an IMX219 camera sensor. I haven't noticed any regression with non-raw capture, and have successfully captured raw frames in different resolutions by specifying either a raw role or a raw pixel format on the cam command line. I have also successfully tested manual gain and exposure with YUV formats using a capture script. Florian Sylvestre (2): pipeline: rkisp1: Support raw Bayer capture at runtime pipeline: rkisp1: Support raw Bayer capture configuration Laurent Pinchart (5): libcamera: stream: Add operator<<() to print StreamRole as a string libcamera: stream: Turn StreamRole into scoped enumeration ipa: rkisp1: Support raw capture in IPA operations ipa: rkisp1: agc: Support raw capture pipeline: rkisp1: Fix stream size validation Paul Elder (2): ipa: rkisp1: Add support for manual gain and exposure pipeline: rkisp1: Query the driver for formats include/libcamera/stream.h | 5 +- src/android/camera_capabilities.cpp | 2 +- src/apps/lc-compliance/capture_test.cpp | 17 +- src/gstreamer/gstlibcamerapad.cpp | 20 +- src/ipa/rkisp1/algorithms/agc.cpp | 100 +++++-- src/ipa/rkisp1/algorithms/agc.h | 6 + src/ipa/rkisp1/algorithms/algorithm.h | 12 +- src/ipa/rkisp1/ipa_context.cpp | 5 + src/ipa/rkisp1/ipa_context.h | 15 +- src/ipa/rkisp1/rkisp1.cpp | 65 ++++- src/libcamera/pipeline/imx8-isi/imx8-isi.cpp | 8 +- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 275 ++++++++++++------ src/libcamera/pipeline/rkisp1/rkisp1_path.cpp | 255 ++++++++++++++-- src/libcamera/pipeline/rkisp1/rkisp1_path.h | 15 +- src/libcamera/stream.cpp | 19 ++ 15 files changed, 665 insertions(+), 154 deletions(-)