From patchwork Sat Jun 4 18:59:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 16155 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 36D42BD161 for ; Sat, 4 Jun 2022 18:59:59 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 68C0765635; Sat, 4 Jun 2022 20:59:58 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1654369198; bh=p+F5vwtkejrROtrhFEyAnfO0fdxWXx7uOpN1TnBbRqM=; h=To:Date:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=WJPiQRm/B9tjAuCluQj56/M/ArJFDqWjGvHM6fOEaiMq1WMnGbU6XNDnZV8SViPrv qbSxOXEMFbncZWjloYBn3v/oMEsSneCe8wjyllr+2DK9NijmhlOu+rb71pNR4Tmijl K/jVKO+6/SCE8nf8GL6r+dHRcqaqVbQKiHUNAiBCcCiRjcZUAx6pa9tyLsVP8RxAm1 ktJ2qFvfIdn1aDnPPB2BWuSIupb+rpxxQ6el0lISE5MxCLcIGbe+dvcZ94Y2eHxKdQ xpD20QsBlseujLG3IYPkIQIFGAt3zeRQnB55//uHpfFszEEVh5R9s/0oK2yFBxwIEM 9LjnFjfZ1wFXQ== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CD93A633A6 for ; Sat, 4 Jun 2022 20:59:56 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="uGZgUeSi"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (85-76-79-203-nat.elisa-mobile.fi [85.76.79.203]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 818D96D4; Sat, 4 Jun 2022 20:59:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1654369196; bh=p+F5vwtkejrROtrhFEyAnfO0fdxWXx7uOpN1TnBbRqM=; h=From:To:Cc:Subject:Date:From; b=uGZgUeSirgNG4zzD52rMAuTiTw36ltE3pgotZSy7e5Oc2aI90U9XuaoHqTNwKiuos FIipFSRgnZeZzODNKqTZrsqSqqFdN8j/ex6HjWmKZ74smFcKU1fWWNgvk22LzGKsqd LB9Sbs22HW1EQIWsoWjscN6+5Ymv2F+SHwYiNumM= To: libcamera-devel@lists.libcamera.org Date: Sat, 4 Jun 2022 21:59:25 +0300 Message-Id: <20220604185939.29163-1-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH v2 00/14] Replace boost JSON parser with libyaml in Raspberry Pi IPA X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Hello, As mentioned in the subject, this patch series replaces the boost JSON parser with the YamlParser implementation based on libyaml for the Raspberry Pi IPA module. The series starts with a few cleanups and extensions in YamlParser, with patch 01/14 turning the enum Type into an enum class, patch 02/14 extending the size() function to support dictionaries in addition to lists, and patch 03/14 moving the parser from using FILE to File for a better RAII-style management of files. Patch 04/14 continues with a small cleanup in the YamlParser unit tests. Patches 05/14 and 06/14 introduces the first important functional change in the series: they preserve the order of items in YAML dictionaries. This is needed by the Raspberry Pi IPA, as it instantiates algorithms in the same order as they are declared in the tuning file. Patch 05/14 extends the YamlParser unit test to ensure order is preserved (which fails), and patch 06/14 preserves the order (which fixes the test). Patch 07/14 contains the second big change: a new iterator API for YamlObject. This allows iterating over the elements of a list of dictionaly object. Unlike the boost property_tree implementation that iterates over a pair of std::string key and ptree value, the YamlObject iterators differ for lists and dictionaries. This provides a more familiar API for lists, at the expense of requiring adapter objects (see the asDict() and asList() functions). I'm interested in feedback on this approach, hence the RFC. Patch 08/14 then extends the unit tests to cover the iterator API, and patch 09/14 uses the API in the Android camera HAL. Patch 10/14 completes that part of the series by removing the then unused memberNames() function. The series continue with two extensions to the YamlParser. In patch 11/14 the get() function receives a fix to properly check the value range for 32-bit integers, and in patch 12/14 new specializations of the function are added for 16-bit integers, needed by the Raspberry Pi IPA. The last part of the series converts the Raspberry Pi IPA from boost to YamlParser. Small changes are needed in patch 13/14 to convert tabs to spaces in the tuning JSON files, as they confuse the YAML parser. Patch 14/14 then converts the IPA module, and drops the dependency on boost. When compiled in release mode with clang-13, the Raspberry Pi IPA module .text section shrank from 1317086 to 963614 bytes. I haven't been able to test the series on a Raspberry Pi yet (second reason for the RFC) as I don't have access to my board at the moment. Laurent Pinchart (14): libcamera: yaml_object: Turn Type into an enum class libcamera: yaml_parser: Extend YamlObject::size() to dictionaries libcamera: yaml_parser: Switch from FILE to File test: yaml-parser: Use write() instead of fwrite() test: yaml-parser: Test dictionary items ordering libcamera: yaml_parser: Preserve order of items in dictionary libcamera: yaml_parser: Add iterator API test: yaml_parser: Extend tests to cover the iterator API android: Use the YamlObject iterator API libcamera: yaml_parser: Remove memberNames() function libcamera: yaml_parser: Fix range checks for 32-bit integers libcamera: yaml_parser: Add get() specializations for 16-bit integers ipa: raspberrypi: Replace tabs with spaces in tuning data files ipa: raspberrypi: Use YamlParser to replace dependency on boost README.rst | 6 - include/libcamera/internal/yaml_parser.h | 142 +++++++++- src/android/camera_hal_config.cpp | 22 +- src/ipa/raspberrypi/controller/algorithm.cpp | 2 +- src/ipa/raspberrypi/controller/algorithm.hpp | 6 +- src/ipa/raspberrypi/controller/controller.cpp | 27 +- src/ipa/raspberrypi/controller/pwl.cpp | 12 +- src/ipa/raspberrypi/controller/pwl.hpp | 5 +- src/ipa/raspberrypi/controller/rpi/agc.cpp | 94 +++---- src/ipa/raspberrypi/controller/rpi/agc.hpp | 10 +- src/ipa/raspberrypi/controller/rpi/alsc.cpp | 94 +++---- src/ipa/raspberrypi/controller/rpi/alsc.hpp | 2 +- src/ipa/raspberrypi/controller/rpi/awb.cpp | 89 +++--- src/ipa/raspberrypi/controller/rpi/awb.hpp | 8 +- .../controller/rpi/black_level.cpp | 12 +- .../controller/rpi/black_level.hpp | 2 +- src/ipa/raspberrypi/controller/rpi/ccm.cpp | 28 +- src/ipa/raspberrypi/controller/rpi/ccm.hpp | 4 +- .../raspberrypi/controller/rpi/contrast.cpp | 18 +- .../raspberrypi/controller/rpi/contrast.hpp | 2 +- src/ipa/raspberrypi/controller/rpi/dpc.cpp | 4 +- src/ipa/raspberrypi/controller/rpi/dpc.hpp | 2 +- src/ipa/raspberrypi/controller/rpi/geq.cpp | 10 +- src/ipa/raspberrypi/controller/rpi/geq.hpp | 2 +- src/ipa/raspberrypi/controller/rpi/lux.cpp | 12 +- src/ipa/raspberrypi/controller/rpi/lux.hpp | 2 +- src/ipa/raspberrypi/controller/rpi/noise.cpp | 6 +- src/ipa/raspberrypi/controller/rpi/noise.hpp | 2 +- src/ipa/raspberrypi/controller/rpi/sdn.cpp | 6 +- src/ipa/raspberrypi/controller/rpi/sdn.hpp | 2 +- .../raspberrypi/controller/rpi/sharpen.cpp | 8 +- .../raspberrypi/controller/rpi/sharpen.hpp | 2 +- src/ipa/raspberrypi/data/imx219.json | 8 +- src/ipa/raspberrypi/data/imx219_noir.json | 10 +- src/ipa/raspberrypi/data/imx290.json | 18 +- src/ipa/raspberrypi/data/imx477.json | 8 +- src/ipa/raspberrypi/data/imx477_noir.json | 10 +- src/ipa/raspberrypi/data/ov5647.json | 10 +- src/ipa/raspberrypi/data/ov5647_noir.json | 12 +- src/ipa/raspberrypi/data/se327m12.json | 6 +- src/ipa/raspberrypi/meson.build | 1 - src/ipa/raspberrypi/raspberrypi.cpp | 1 + src/libcamera/yaml_parser.cpp | 263 +++++++++++++----- test/yaml-parser.cpp | 112 +++++--- 44 files changed, 691 insertions(+), 411 deletions(-) base-commit: 7ef83e02284715db95e3f4cd61744290baff2840