[{"id":30645,"web_url":"https://patchwork.libcamera.org/comment/30645/","msgid":"<1c24fdf8-f01d-4e46-881d-442c75db3024@ideasonboard.com>","date":"2024-08-07T08:30:22","subject":"Re: [PATCH v5 16/18] Documentation: Split public/private\n\tdocumentation","submitter":{"id":156,"url":"https://patchwork.libcamera.org/api/people/156/","name":"Dan Scally","email":"dan.scally@ideasonboard.com"},"content":"Hi Laurent - thanks for the patch\n\nOn 05/08/2024 15:36, Laurent Pinchart wrote:\n> From: Daniel Scally \n>\n> The API reference pages generated by Doxygen are comprehensive, but\n> therefore quite overwhelming for application developers who will\n> likely never need to use the majority of the library's objects. To\n> reduce the complexity of the documentation, split it into two runs of\n> doxygen.\n>\n> The first run of doxygen is for the public API. We pass a specific list\n> of source files to parse, which is built from the arrays of public\n> headers and sources in meson build files. This ensures that we only\n> generate the documentation for code from those files.\n>\n> A custom Python script is needed to add the list of input files to\n> Doxyfile, as several of the objects included in the header and source\n> array are custom_tgt objects, which can't be handled as strings to\n> populate a variable in the configuration data.\n>\n> The headers defining the Extensible and Object classes (class.h and\n> object.h respectively), as well as the corresponding source files, are\n> excluded from the public API documentation despite being referenced in\n> the meson public headers and sources arrays. This is due to the fact\n> that public API classes inherit from Extensible and Object, making the\n> Extensible and Object classes part of the public ABI. Those two base\n> classes are however implementation details and must not be accessed\n> directly by application code.\n>\n> The second run of doxygen is for the internal API. This contains\n> documentation for all of the library's objects as it currently does.\n> This set will now be output into build/Documentation/internal-api-html.\n>\n> Signed-off-by: Daniel Scally \n> Co-developed-by: Laurent Pinchart \n> Signed-off-by: Laurent Pinchart \n> ---\n\nThe python script to generate the list of input files is definitely better than scattered foreach \nloops collecting a list in a hacky way - good improvement.\n\n> .../{Doxyfile.in => Doxyfile-common.in} | 24 -------\n> Documentation/Doxyfile-internal.in | 30 ++++++++\n> Documentation/Doxyfile-public.in | 20 ++++++\n> Documentation/gen-doxyfile.py | 46 ++++++++++++\n> Documentation/meson.build | 70 +++++++++++++++++--\n> 5 files changed, 160 insertions(+), 30 deletions(-)\n> rename Documentation/{Doxyfile.in => Doxyfile-common.in} (62%)\n> create mode 100644 Documentation/Doxyfile-internal.in\n> create mode 100644 Documentation/Doxyfile-public.in\n> create mode 100755 Documentation/gen-doxyfile.py\n>\n> diff --git a/Documentation/Doxyfile.in b/Documentation/Doxyfile-common.in\n> similarity index 62%\n> rename from Documentation/Doxyfile.in\n> rename to Documentation/Doxyfile-common.in\n> index 6e5a3830ec38..a70aee430e77 100644\n> --- a/Documentation/Doxyfile.in\n> +++ b/Documentation/Doxyfile-common.in\n> @@ -17,20 +17,11 @@ EXTENSION_MAPPING = h=C++\n> \n> TOC_INCLUDE_HEADINGS = 0\n> \n> -INTERNAL_DOCS = YES\n> CASE_SENSE_NAMES = YES\n> \n> QUIET = YES\n> WARN_AS_ERROR = @WARN_AS_ERROR@\n> \n> -INPUT = \"@TOP_SRCDIR@/Documentation\" \\\n> - \"@TOP_SRCDIR@/include/libcamera\" \\\n> - \"@TOP_SRCDIR@/src/ipa/ipu3\" \\\n> - \"@TOP_SRCDIR@/src/ipa/libipa\" \\\n> - \"@TOP_SRCDIR@/src/libcamera\" \\\n> - \"@TOP_BUILDDIR@/include/libcamera\" \\\n> - \"@TOP_BUILDDIR@/src/libcamera\"\n> -\n> FILE_PATTERNS = *.c \\\n> *.cpp \\\n> *.dox \\\n> @@ -38,19 +29,6 @@ FILE_PATTERNS = *.c \\\n> \n> RECURSIVE = YES\n> \n> -EXCLUDE = @TOP_SRCDIR@/include/libcamera/base/span.h \\\n> - @TOP_SRCDIR@/include/libcamera/internal/device_enumerator_sysfs.h \\\n> - @TOP_SRCDIR@/include/libcamera/internal/device_enumerator_udev.h \\\n> - @TOP_SRCDIR@/include/libcamera/internal/ipc_pipe_unixsocket.h \\\n> - @TOP_SRCDIR@/src/libcamera/device_enumerator_sysfs.cpp \\\n> - @TOP_SRCDIR@/src/libcamera/device_enumerator_udev.cpp \\\n> - @TOP_SRCDIR@/src/libcamera/ipc_pipe_unixsocket.cpp \\\n> - @TOP_SRCDIR@/src/libcamera/pipeline/ \\\n> - @TOP_SRCDIR@/src/libcamera/tracepoints.cpp \\\n> - @TOP_BUILDDIR@/include/libcamera/internal/tracepoints.h \\\n> - @TOP_BUILDDIR@/include/libcamera/ipa/soft_ipa_interface.h \\\n> - @TOP_BUILDDIR@/src/libcamera/proxy/\n> -\n> EXCLUDE_PATTERNS = @TOP_BUILDDIR@/include/libcamera/ipa/*_serializer.h \\\n> @TOP_BUILDDIR@/include/libcamera/ipa/*_proxy.h \\\n> @TOP_BUILDDIR@/include/libcamera/ipa/ipu3_*.h \\\n> @@ -73,8 +51,6 @@ EXCLUDE_SYMBOLS = libcamera::BoundMethodArgs \\\n> \n> EXCLUDE_SYMLINKS = YES\n> \n> -HTML_OUTPUT = api-html\n> -\n> GENERATE_LATEX = NO\n> \n> MACRO_EXPANSION = YES\n> diff --git a/Documentation/Doxyfile-internal.in b/Documentation/Doxyfile-internal.in\n> new file mode 100644\n> index 000000000000..5e0310091416\n> --- /dev/null\n> +++ b/Documentation/Doxyfile-internal.in\n> @@ -0,0 +1,30 @@\n> +# SPDX-License-Identifier: CC-BY-SA-4.0\n> +\n> +@INCLUDE_PATH = @TOP_BUILDDIR@/Documentation\n> +@INCLUDE = Doxyfile-common\n> +\n> +HIDE_UNDOC_CLASSES = NO\n> +HIDE_UNDOC_MEMBERS = NO\n> +HTML_OUTPUT = internal-api-html\n> +INTERNAL_DOCS = YES\n> +\n> +INPUT = \"@TOP_SRCDIR@/Documentation\" \\\n> + \"@TOP_SRCDIR@/include/libcamera\" \\\n> + \"@TOP_SRCDIR@/src/ipa/ipu3\" \\\n> + \"@TOP_SRCDIR@/src/ipa/libipa\" \\\n> + \"@TOP_SRCDIR@/src/libcamera\" \\\n> + \"@TOP_BUILDDIR@/include/libcamera\" \\\n> + \"@TOP_BUILDDIR@/src/libcamera\"\n> +\n> +EXCLUDE = @TOP_SRCDIR@/include/libcamera/base/span.h \\\n> + @TOP_SRCDIR@/include/libcamera/internal/device_enumerator_sysfs.h \\\n> + @TOP_SRCDIR@/include/libcamera/internal/device_enumerator_udev.h \\\n> + @TOP_SRCDIR@/include/libcamera/internal/ipc_pipe_unixsocket.h \\\n> + @TOP_SRCDIR@/src/libcamera/device_enumerator_sysfs.cpp \\\n> + @TOP_SRCDIR@/src/libcamera/device_enumerator_udev.cpp \\\n> + @TOP_SRCDIR@/src/libcamera/ipc_pipe_unixsocket.cpp \\\n> + @TOP_SRCDIR@/src/libcamera/pipeline/ \\\n> + @TOP_SRCDIR@/src/libcamera/tracepoints.cpp \\\n> + @TOP_BUILDDIR@/include/libcamera/internal/tracepoints.h \\\n> + @TOP_BUILDDIR@/include/libcamera/ipa/soft_ipa_interface.h \\\n> + @TOP_BUILDDIR@/src/libcamera/proxy/\n> diff --git a/Documentation/Doxyfile-public.in b/Documentation/Doxyfile-public.in\n> new file mode 100644\n> index 000000000000..36bb57584a07\n> --- /dev/null\n> +++ b/Documentation/Doxyfile-public.in\n> @@ -0,0 +1,20 @@\n> +# SPDX-License-Identifier: CC-BY-SA-4.0\n> +\n> +@INCLUDE_PATH = @TOP_BUILDDIR@/Documentation\n> +@INCLUDE = Doxyfile-common\n> +\n> +HIDE_UNDOC_CLASSES = YES\n> +HIDE_UNDOC_MEMBERS = YES\n> +HTML_OUTPUT = api-html\n> +INTERNAL_DOCS = NO\n> +\n> +INPUT = \"@TOP_SRCDIR@/Documentation\" \\\n> + ${inputs}\n> +\n> +EXCLUDE = @TOP_SRCDIR@/include/libcamera/base/class.h \\\n> + @TOP_SRCDIR@/include/libcamera/base/object.h \\\n> + @TOP_SRCDIR@/include/libcamera/base/span.h \\\n> + @TOP_SRCDIR@/src/libcamera/base/class.cpp \\\n> + @TOP_SRCDIR@/src/libcamera/base/object.cpp\n> +\n> +PREDEFINED += __DOXYGEN_PUBLIC__\n> diff --git a/Documentation/gen-doxyfile.py b/Documentation/gen-doxyfile.py\n> new file mode 100755\n> index 000000000000..c2a4184dd195\n> --- /dev/null\n> +++ b/Documentation/gen-doxyfile.py\n> @@ -0,0 +1,46 @@\n> +#!/usr/bin/env python3\n> +# SPDX-License-Identifier: GPL-2.0-or-later\n> +# Copyright (C) 2024, Google Inc.\nDoes this mean it's drawn from elsewhere? Or is it a copy-paste error?\n> +#\n> +# Author: Laurent Pinchart \n> +#\n> +# Generate Doxyfile from a template\n> +\n> +import argparse\n> +import os\n> +import string\n> +import sys\n> +\n> +\n> +def fill_template(template, data):\n> +\n> + template = open(template, 'rb').read()\n> + template = template.decode('utf-8')\n> + template = string.Template(template)\nI'd add a line break here.\n> + return template.substitute(data)\n> +\n> +\n> +def main(argv):\n> +\n> + # Parse command line arguments\nThe comment can probably be skipped.\n> + parser = argparse.ArgumentParser()\n> + parser.add_argument('-o', dest='output', metavar='file',\n> + type=argparse.FileType('w', encoding='utf-8'),\n> + default=sys.stdout,\n> + help='Output file name (default: standard output)')\n> + parser.add_argument('template', metavar='doxyfile.tmpl', type=str,\n> + help='Doxyfile template')\n> + parser.add_argument('inputs', type=str, nargs='*',\n> + help='Input files')\n> +\n> + args = parser.parse_args(argv[1:])\n> +\n> + inputs = [f'\"{os.path.realpath(input)}\"' for input in args.inputs]\n> + data = fill_template(args.template, {'inputs': (' \\\\\\n' + ' ' * 25).join(inputs)})\n> + args.output.write(data)\n> +\n> + return 0\n> +\n> +\n> +if __name__ == '__main__':\n> + sys.exit(main(sys.argv))\n> diff --git a/Documentation/meson.build b/Documentation/meson.build\n> index 1d84ed815b50..bf28233a2ce8 100644\n> --- a/Documentation/meson.build\n> +++ b/Documentation/meson.build\n> @@ -24,9 +24,9 @@ if doxygen.found() and dot.found()\n> \n> cdata.set('PREDEFINED', ' \\\\\\n\\t\\t\\t '.join(doxygen_predefined))\n> \n> - doxyfile = configure_file(input : 'Doxyfile.in',\n> - output : 'Doxyfile',\n> - configuration : cdata)\n> + doxyfile_common = configure_file(input : 'Doxyfile-common.in',\n> + output : 'Doxyfile-common',\n> + configuration : cdata)\n> \n> doxygen_public_input = [\n> libcamera_base_public_headers,\n> @@ -50,17 +50,75 @@ if doxygen.found() and dot.found()\n> doxygen_internal_input += [ipu3_ipa_sources]\n> endif\n> \n> - custom_target('doxygen',\n> + # We run doxygen twice - the first run excludes internal API objects as it\n> + # is intended to document the public API only. A second run covers all of\n> + # the library's objects for libcamera developers. Common configuration is\n> + # set in an initially generated Doxyfile, which is then included by the two\n> + # final Doxyfiles. We collected a list of the public sources in\n> + # doxygen_public_sources to pass to the public run and include it in cdata\n> + # here - although both Doxyfile and Doxyfile-internal will also receive\n> + # the same parameter they simply ignore it.\nThis comment needs correcting now as it references the older method.\n> +\n> + # This is the \"public\" run of doxygen generating an abridged version of the\n> + # API's documentation.\n> +\n> + doxyfile_tmpl = configure_file(input : 'Doxyfile-public.in',\n> + output : 'Doxyfile-public.tmpl',\n> + configuration : cdata)\n> +\n> + doxyfile = custom_target('doxyfile-public',\n> + input : [\n> + doxygen_public_input,\n> + ],\n> + output : 'Doxyfile-public',\n> + command : [\n> + 'gen-doxyfile.py',\n> + '-o', '@OUTPUT@',\n> + doxyfile_tmpl,\n> + '@INPUT@',\n> + ])\n> +\n> + custom_target('doxygen-public',\n> input : [\n> doxyfile,\n> - doxygen_public_input,\n> - doxygen_internal_input,\n> + doxyfile_common,\n> ],\n> output : 'api-html',\n> command : [doxygen, doxyfile],\n> install : true,\n> install_dir : doc_install_dir,\n> install_tag : 'doc')\n> +\n> + # This is the internal documentation, which hard-codes a list of directories\n> + # to parse in its doxyfile.\n> +\n> + doxyfile_tmpl = configure_file(input : 'Doxyfile-internal.in',\n> + output : 'Doxyfile-internal.tmpl',\n> + configuration : cdata)\n> +\n> + doxyfile = custom_target('doxyfile-internal',\n> + input : [\n> + doxygen_public_input,\n> + doxygen_internal_input,\n> + ],\n> + output : 'Doxyfile-internal',\n> + command : [\n> + 'gen-doxyfile.py',\n> + '-o', '@OUTPUT@',\n> + doxyfile_tmpl,\n> + '@INPUT@',\n> + ])\nI think we can skip the script for the internal run -there's no templated values to be replaced and \nthe generated Doxyfile-internal is identical to the Doxyfile-internal.tmpl.\n> +\n> + custom_target('doxygen-internal',\n> + input : [\n> + doxyfile,\n> + doxyfile_common,\n> + ],\n> + output : 'internal-api-html',\n> + command : [doxygen, doxyfile],\n> + install : true,\n> + install_dir : doc_install_dir,\n> + install_tag : 'doc-internal')\n> endif\n> \n> #","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 714FABE173\n\tfor ;\n\tWed, 7 Aug 2024 08:30:28 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 74E8263393;\n\tWed, 7 Aug 2024 10:30:27 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 432DE61946\n\tfor ;\n\tWed, 7 Aug 2024 10:30:26 +0200 (CEST)","from [192.168.0.43]\n\t(cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id F3E926AF;\n\tWed, 7 Aug 2024 10:29:32 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"F32o7unA\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1723019373;\n\tbh=kPENKf0KDQk/f61cTELQ/qYA9RDfkhRWKKNtv7XwR78=;\n\th=Date:Subject:To:References:From:In-Reply-To:From;\n\tb=F32o7unAeOICezVGXl/ncP158HDpTvFc0OI5lMX1sptIqV897cnXJHV4b1zl6ehxb\n\th08x/lMSK2NB7Y+UM3lLQ4guTkyLK5SlXEAUqv481HpyLZ7qUcCMarMJE9PJRTspsv\n\t+3u2Lf6G57pEDqJfNKza51ySXnWXBMhqJOuMV/tw=","Message-ID":"<1c24fdf8-f01d-4e46-881d-442c75db3024@ideasonboard.com>","Date":"Wed, 7 Aug 2024 09:30:22 +0100","MIME-Version":"1.0","User-Agent":"Mozilla Thunderbird","Subject":"Re: [PATCH v5 16/18] Documentation: Split public/private\n\tdocumentation","To":"Laurent Pinchart ,\n\tlibcamera-devel@lists.libcamera.org","References":"<20240805143654.20870-1-laurent.pinchart@ideasonboard.com>\n\t<20240805143654.20870-17-laurent.pinchart@ideasonboard.com>","Content-Language":"en-US","From":"Dan Scally ","Autocrypt":"addr=dan.scally@ideasonboard.com; keydata=\n\txsFNBGLydlEBEADa5O2s0AbUguprfvXOQun/0a8y2Vk6BqkQALgeD6KnXSWwaoCULp18etYW\n\tB31bfgrdphXQ5kUQibB0ADK8DERB4wrzrUb5CMxLBFE7mQty+v5NsP0OFNK9XTaAOcmD+Ove\n\teIjYvqurAaro91jrRVrS1gBRxIFqyPgNvwwL+alMZhn3/2jU2uvBmuRrgnc/e9cHKiuT3Dtq\n\tMHGPKL2m+plk+7tjMoQFfexoQ1JKugHAjxAhJfrkXh6uS6rc01bYCyo7ybzg53m1HLFJdNGX\n\tsUKR+dQpBs3SY4s66tc1sREJqdYyTsSZf80HjIeJjU/hRunRo4NjRIJwhvnK1GyjOvvuCKVU\n\tRWpY8dNjNu5OeAfdrlvFJOxIE9M8JuYCQTMULqd1NuzbpFMjc9524U3Cngs589T7qUMPb1H1\n\tNTA81LmtJ6Y+IV5/kiTUANflpzBwhu18Ok7kGyCq2a2jsOcVmk8gZNs04gyjuj8JziYwwLbf\n\tvzABwpFVcS8aR+nHIZV1HtOzyw8CsL8OySc3K9y+Y0NRpziMRvutrppzgyMb9V+N31mK9Mxl\n\t1YkgaTl4ciNWpdfUe0yxH03OCuHi3922qhPLF4XX5LN+NaVw5Xz2o3eeWklXdouxwV7QlN33\n\tu4+u2FWzKxDqO6WLQGjxPE0mVB4Gh5Pa1Vb0ct9Ctg0qElvtGQARAQABzShEYW4gU2NhbGx5\n\tIDxkYW4uc2NhbGx5QGlkZWFzb25ib2FyZC5jb20+wsGNBBMBCAA3FiEEsdtt8OWP7+8SNfQe\n\tkiQuh/L+GMQFAmLydlIFCQWjmoACGwMECwkIBwUVCAkKCwUWAgMBAAAKCRCSJC6H8v4YxDI2\n\tEAC2Gz0iyaXJkPInyshrREEWbo0CA6v5KKf3I/HlMPqkZ48bmGoYm4mEQGFWZJAT3K4ir8bg\n\tcEfs9V54gpbrZvdwS4abXbUK4WjKwEs8HK3XJv1WXUN2bsz5oEJWZUImh9gD3naiLLI9QMMm\n\tw/aZkT+NbN5/2KvChRWhdcha7+2Te4foOY66nIM+pw2FZM6zIkInLLUik2zXOhaZtqdeJZQi\n\tHSPU9xu7TRYN4cvdZAnSpG7gQqmLm5/uGZN1/sB3kHTustQtSXKMaIcD/DMNI3JN/t+RJVS7\n\tc0Jh/ThzTmhHyhxx3DRnDIy7kwMI4CFvmhkVC2uNs9kWsj1DuX5kt8513mvfw2OcX9UnNKmZ\n\tnhNCuF6DxVrL8wjOPuIpiEj3V+K7DFF1Cxw1/yrLs8dYdYh8T8vCY2CHBMsqpESROnTazboh\n\tAiQ2xMN1cyXtX11Qwqm5U3sykpLbx2BcmUUUEAKNsM//Zn81QXKG8vOx0ZdMfnzsCaCzt8f6\n\t9dcDBBI3tJ0BI9ByiocqUoL6759LM8qm18x3FYlxvuOs4wSGPfRVaA4yh0pgI+ModVC2Pu3y\n\tejE/IxeatGqJHh6Y+iJzskdi27uFkRixl7YJZvPJAbEn7kzSi98u/5ReEA8Qhc8KO/B7wprj\n\txjNMZNYd0Eth8+WkixHYj752NT5qshKJXcyUU87BTQRi8nZSARAAx0BJayh1Fhwbf4zoY56x\n\txHEpT6DwdTAYAetd3yiKClLVJadYxOpuqyWa1bdfQWPb+h4MeXbWw/53PBgn7gI2EA7ebIRC\n\tPJJhAIkeym7hHZoxqDQTGDJjxFEL11qF+U3rhWiL2Zt0Pl+zFq0eWYYVNiXjsIS4FI2+4m16\n\ttPbDWZFJnSZ828VGtRDQdhXfx3zyVX21lVx1bX4/OZvIET7sVUufkE4hrbqrrufre7wsjD1t\n\t8MQKSapVrr1RltpzPpScdoxknOSBRwOvpp57pJJe5A0L7+WxJ+vQoQXj0j+5tmIWOAV1qBQp\n\thyoyUk9JpPfntk2EKnZHWaApFp5TcL6c5LhUvV7F6XwOjGPuGlZQCWXee9dr7zym8iR3irWT\n\t+49bIh5PMlqSLXJDYbuyFQHFxoiNdVvvf7etvGfqFYVMPVjipqfEQ38ST2nkzx+KBICz7uwj\n\tJwLBdTXzGFKHQNckGMl7F5QdO/35An/QcxBnHVMXqaSd12tkJmoRVWduwuuoFfkTY5mUV3uX\n\txGj3iVCK4V+ezOYA7c2YolfRCNMTza6vcK/P4tDjjsyBBZrCCzhBvd4VVsnnlZhVaIxoky4K\n\taL+AP+zcQrUZmXmgZjXOLryGnsaeoVrIFyrU6ly90s1y3KLoPsDaTBMtnOdwxPmo1xisH8oL\n\ta/VRgpFBfojLPxMAEQEAAcLBfAQYAQgAJhYhBLHbbfDlj+/vEjX0HpIkLofy/hjEBQJi8nZT\n\tBQkFo5qAAhsMAAoJEJIkLofy/hjEXPcQAMIPNqiWiz/HKu9W4QIf1OMUpKn3YkVIj3p3gvfM\n\tRes4fGX94Ji599uLNrPoxKyaytC4R6BTxVriTJjWK8mbo9jZIRM4vkwkZZ2bu98EweSucxbp\n\tvjESsvMXGgxniqV/RQ/3T7LABYRoIUutARYq58p5HwSP0frF0fdFHYdTa2g7MYZl1ur2JzOC\n\tFHRpGadlNzKDE3fEdoMobxHB3Lm6FDml5GyBAA8+dQYVI0oDwJ3gpZPZ0J5Vx9RbqXe8RDuR\n\tdu90hvCJkq7/tzSQ0GeD3BwXb9/R/A4dVXhaDd91Q1qQXidI+2jwhx8iqiYxbT+DoAUkQRQy\n\txBtoCM1CxH7u45URUgD//fxYr3D4B1SlonA6vdaEdHZOGwECnDpTxecENMbz/Bx7qfrmd901\n\tD+N9SjIwrbVhhSyUXYnSUb8F+9g2RDY42Sk7GcYxIeON4VzKqWM7hpkXZ47pkK0YodO+dRKM\n\tyMcoUWrTK0Uz6UzUGKoJVbxmSW/EJLEGoI5p3NWxWtScEVv8mO49gqQdrRIOheZycDmHnItt\n\t9Qjv00uFhEwv2YfiyGk6iGF2W40s2pH2t6oeuGgmiZ7g6d0MEK8Ql/4zPItvr1c1rpwpXUC1\n\tu1kQWgtnNjFHX3KiYdqjcZeRBiry1X0zY+4Y24wUU0KsEewJwjhmCKAsju1RpdlPg2kC","In-Reply-To":"<20240805143654.20870-17-laurent.pinchart@ideasonboard.com>","Content-Type":"text/plain; charset=UTF-8; format=flowed","Content-Transfer-Encoding":"7bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":30670,"web_url":"https://patchwork.libcamera.org/comment/30670/","msgid":"<172303113182.1687952.3559570839316346035@ping.linuxembedded.co.uk>","date":"2024-08-07T11:45:31","subject":"Re: [PATCH v5 16/18] Documentation: Split public/private\n\tdocumentation","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"Quoting Dan Scally (2024-08-07 09:30:22)\n> Hi Laurent - thanks for the patch\n> \n> On 05/08/2024 15:36, Laurent Pinchart wrote:\n> > From: Daniel Scally \n> >\n> > The API reference pages generated by Doxygen are comprehensive, but\n> > therefore quite overwhelming for application developers who will\n> > likely never need to use the majority of the library's objects. To\n> > reduce the complexity of the documentation, split it into two runs of\n> > doxygen.\n> >\n> > The first run of doxygen is for the public API. We pass a specific list\n> > of source files to parse, which is built from the arrays of public\n> > headers and sources in meson build files. This ensures that we only\n> > generate the documentation for code from those files.\n> >\n> > A custom Python script is needed to add the list of input files to\n> > Doxyfile, as several of the objects included in the header and source\n> > array are custom_tgt objects, which can't be handled as strings to\n> > populate a variable in the configuration data.\n> >\n> > The headers defining the Extensible and Object classes (class.h and\n> > object.h respectively), as well as the corresponding source files, are\n> > excluded from the public API documentation despite being referenced in\n> > the meson public headers and sources arrays. This is due to the fact\n> > that public API classes inherit from Extensible and Object, making the\n> > Extensible and Object classes part of the public ABI. Those two base\n> > classes are however implementation details and must not be accessed\n> > directly by application code.\n> >\n> > The second run of doxygen is for the internal API. This contains\n> > documentation for all of the library's objects as it currently does.\n> > This set will now be output into build/Documentation/internal-api-html.\n\nSounds good.\n\n> >\n> > Signed-off-by: Daniel Scally \n> > Co-developed-by: Laurent Pinchart \n> > Signed-off-by: Laurent Pinchart \n> > ---\n> \n> The python script to generate the list of input files is definitely better than scattered foreach \n> loops collecting a list in a hacky way - good improvement.\n> \n> > .../{Doxyfile.in => Doxyfile-common.in} | 24 -------\n> > Documentation/Doxyfile-internal.in | 30 ++++++++\n> > Documentation/Doxyfile-public.in | 20 ++++++\n> > Documentation/gen-doxyfile.py | 46 ++++++++++++\n> > Documentation/meson.build | 70 +++++++++++++++++--\n> > 5 files changed, 160 insertions(+), 30 deletions(-)\n> > rename Documentation/{Doxyfile.in => Doxyfile-common.in} (62%)\n> > create mode 100644 Documentation/Doxyfile-internal.in\n> > create mode 100644 Documentation/Doxyfile-public.in\n> > create mode 100755 Documentation/gen-doxyfile.py\n> >\n> > diff --git a/Documentation/Doxyfile.in b/Documentation/Doxyfile-common.in\n> > similarity index 62%\n> > rename from Documentation/Doxyfile.in\n> > rename to Documentation/Doxyfile-common.in\n> > index 6e5a3830ec38..a70aee430e77 100644\n> > --- a/Documentation/Doxyfile.in\n> > +++ b/Documentation/Doxyfile-common.in\n> > @@ -17,20 +17,11 @@ EXTENSION_MAPPING = h=C++\n> > \n> > TOC_INCLUDE_HEADINGS = 0\n> > \n> > -INTERNAL_DOCS = YES\n> > CASE_SENSE_NAMES = YES\n> > \n> > QUIET = YES\n> > WARN_AS_ERROR = @WARN_AS_ERROR@\n> > \n> > -INPUT = \"@TOP_SRCDIR@/Documentation\" \\\n> > - \"@TOP_SRCDIR@/include/libcamera\" \\\n> > - \"@TOP_SRCDIR@/src/ipa/ipu3\" \\\n> > - \"@TOP_SRCDIR@/src/ipa/libipa\" \\\n> > - \"@TOP_SRCDIR@/src/libcamera\" \\\n> > - \"@TOP_BUILDDIR@/include/libcamera\" \\\n> > - \"@TOP_BUILDDIR@/src/libcamera\"\n> > -\n> > FILE_PATTERNS = *.c \\\n> > *.cpp \\\n> > *.dox \\\n> > @@ -38,19 +29,6 @@ FILE_PATTERNS = *.c \\\n> > \n> > RECURSIVE = YES\n> > \n> > -EXCLUDE = @TOP_SRCDIR@/include/libcamera/base/span.h \\\n> > - @TOP_SRCDIR@/include/libcamera/internal/device_enumerator_sysfs.h \\\n> > - @TOP_SRCDIR@/include/libcamera/internal/device_enumerator_udev.h \\\n> > - @TOP_SRCDIR@/include/libcamera/internal/ipc_pipe_unixsocket.h \\\n> > - @TOP_SRCDIR@/src/libcamera/device_enumerator_sysfs.cpp \\\n> > - @TOP_SRCDIR@/src/libcamera/device_enumerator_udev.cpp \\\n> > - @TOP_SRCDIR@/src/libcamera/ipc_pipe_unixsocket.cpp \\\n> > - @TOP_SRCDIR@/src/libcamera/pipeline/ \\\n> > - @TOP_SRCDIR@/src/libcamera/tracepoints.cpp \\\n> > - @TOP_BUILDDIR@/include/libcamera/internal/tracepoints.h \\\n> > - @TOP_BUILDDIR@/include/libcamera/ipa/soft_ipa_interface.h \\\n> > - @TOP_BUILDDIR@/src/libcamera/proxy/\n> > -\n> > EXCLUDE_PATTERNS = @TOP_BUILDDIR@/include/libcamera/ipa/*_serializer.h \\\n> > @TOP_BUILDDIR@/include/libcamera/ipa/*_proxy.h \\\n> > @TOP_BUILDDIR@/include/libcamera/ipa/ipu3_*.h \\\n> > @@ -73,8 +51,6 @@ EXCLUDE_SYMBOLS = libcamera::BoundMethodArgs \\\n> > \n> > EXCLUDE_SYMLINKS = YES\n> > \n> > -HTML_OUTPUT = api-html\n> > -\n> > GENERATE_LATEX = NO\n> > \n> > MACRO_EXPANSION = YES\n> > diff --git a/Documentation/Doxyfile-internal.in b/Documentation/Doxyfile-internal.in\n> > new file mode 100644\n> > index 000000000000..5e0310091416\n> > --- /dev/null\n> > +++ b/Documentation/Doxyfile-internal.in\n> > @@ -0,0 +1,30 @@\n> > +# SPDX-License-Identifier: CC-BY-SA-4.0\n> > +\n> > +@INCLUDE_PATH = @TOP_BUILDDIR@/Documentation\n> > +@INCLUDE = Doxyfile-common\n> > +\n> > +HIDE_UNDOC_CLASSES = NO\n> > +HIDE_UNDOC_MEMBERS = NO\n> > +HTML_OUTPUT = internal-api-html\n> > +INTERNAL_DOCS = YES\n> > +\n> > +INPUT = \"@TOP_SRCDIR@/Documentation\" \\\n> > + \"@TOP_SRCDIR@/include/libcamera\" \\\n> > + \"@TOP_SRCDIR@/src/ipa/ipu3\" \\\n> > + \"@TOP_SRCDIR@/src/ipa/libipa\" \\\n> > + \"@TOP_SRCDIR@/src/libcamera\" \\\n> > + \"@TOP_BUILDDIR@/include/libcamera\" \\\n> > + \"@TOP_BUILDDIR@/src/libcamera\"\n> > +\n> > +EXCLUDE = @TOP_SRCDIR@/include/libcamera/base/span.h \\\n> > + @TOP_SRCDIR@/include/libcamera/internal/device_enumerator_sysfs.h \\\n> > + @TOP_SRCDIR@/include/libcamera/internal/device_enumerator_udev.h \\\n> > + @TOP_SRCDIR@/include/libcamera/internal/ipc_pipe_unixsocket.h \\\n> > + @TOP_SRCDIR@/src/libcamera/device_enumerator_sysfs.cpp \\\n> > + @TOP_SRCDIR@/src/libcamera/device_enumerator_udev.cpp \\\n> > + @TOP_SRCDIR@/src/libcamera/ipc_pipe_unixsocket.cpp \\\n> > + @TOP_SRCDIR@/src/libcamera/pipeline/ \\\n> > + @TOP_SRCDIR@/src/libcamera/tracepoints.cpp \\\n> > + @TOP_BUILDDIR@/include/libcamera/internal/tracepoints.h \\\n> > + @TOP_BUILDDIR@/include/libcamera/ipa/soft_ipa_interface.h \\\n> > + @TOP_BUILDDIR@/src/libcamera/proxy/\n> > diff --git a/Documentation/Doxyfile-public.in b/Documentation/Doxyfile-public.in\n> > new file mode 100644\n> > index 000000000000..36bb57584a07\n> > --- /dev/null\n> > +++ b/Documentation/Doxyfile-public.in\n> > @@ -0,0 +1,20 @@\n> > +# SPDX-License-Identifier: CC-BY-SA-4.0\n> > +\n> > +@INCLUDE_PATH = @TOP_BUILDDIR@/Documentation\n> > +@INCLUDE = Doxyfile-common\n> > +\n> > +HIDE_UNDOC_CLASSES = YES\n> > +HIDE_UNDOC_MEMBERS = YES\n> > +HTML_OUTPUT = api-html\n> > +INTERNAL_DOCS = NO\n> > +\n> > +INPUT = \"@TOP_SRCDIR@/Documentation\" \\\n> > + ${inputs}\n> > +\n> > +EXCLUDE = @TOP_SRCDIR@/include/libcamera/base/class.h \\\n> > + @TOP_SRCDIR@/include/libcamera/base/object.h \\\n> > + @TOP_SRCDIR@/include/libcamera/base/span.h \\\n> > + @TOP_SRCDIR@/src/libcamera/base/class.cpp \\\n> > + @TOP_SRCDIR@/src/libcamera/base/object.cpp\n> > +\n> > +PREDEFINED += __DOXYGEN_PUBLIC__\n> > diff --git a/Documentation/gen-doxyfile.py b/Documentation/gen-doxyfile.py\n> > new file mode 100755\n> > index 000000000000..c2a4184dd195\n> > --- /dev/null\n> > +++ b/Documentation/gen-doxyfile.py\n> > @@ -0,0 +1,46 @@\n> > +#!/usr/bin/env python3\n> > +# SPDX-License-Identifier: GPL-2.0-or-later\n> > +# Copyright (C) 2024, Google Inc.\n> Does this mean it's drawn from elsewhere? Or is it a copy-paste error?\n> > +#\n> > +# Author: Laurent Pinchart \n> > +#\n> > +# Generate Doxyfile from a template\n> > +\n> > +import argparse\n> > +import os\n> > +import string\n> > +import sys\n> > +\n> > +\n> > +def fill_template(template, data):\n> > +\n> > + template = open(template, 'rb').read()\n> > + template = template.decode('utf-8')\n> > + template = string.Template(template)\n> I'd add a line break here.\n> > + return template.substitute(data)\n> > +\n> > +\n> > +def main(argv):\n> > +\n> > + # Parse command line arguments\n> The comment can probably be skipped.\n> > + parser = argparse.ArgumentParser()\n> > + parser.add_argument('-o', dest='output', metavar='file',\n> > + type=argparse.FileType('w', encoding='utf-8'),\n> > + default=sys.stdout,\n> > + help='Output file name (default: standard output)')\n> > + parser.add_argument('template', metavar='doxyfile.tmpl', type=str,\n> > + help='Doxyfile template')\n> > + parser.add_argument('inputs', type=str, nargs='*',\n> > + help='Input files')\n> > +\n> > + args = parser.parse_args(argv[1:])\n> > +\n> > + inputs = [f'\"{os.path.realpath(input)}\"' for input in args.inputs]\n> > + data = fill_template(args.template, {'inputs': (' \\\\\\n' + ' ' * 25).join(inputs)})\n> > + args.output.write(data)\n> > +\n> > + return 0\n> > +\n> > +\n> > +if __name__ == '__main__':\n> > + sys.exit(main(sys.argv))\n> > diff --git a/Documentation/meson.build b/Documentation/meson.build\n> > index 1d84ed815b50..bf28233a2ce8 100644\n> > --- a/Documentation/meson.build\n> > +++ b/Documentation/meson.build\n> > @@ -24,9 +24,9 @@ if doxygen.found() and dot.found()\n> > \n> > cdata.set('PREDEFINED', ' \\\\\\n\\t\\t\\t '.join(doxygen_predefined))\n> > \n> > - doxyfile = configure_file(input : 'Doxyfile.in',\n> > - output : 'Doxyfile',\n> > - configuration : cdata)\n> > + doxyfile_common = configure_file(input : 'Doxyfile-common.in',\n> > + output : 'Doxyfile-common',\n> > + configuration : cdata)\n> > \n> > doxygen_public_input = [\n> > libcamera_base_public_headers,\n> > @@ -50,17 +50,75 @@ if doxygen.found() and dot.found()\n> > doxygen_internal_input += [ipu3_ipa_sources]\n> > endif\n> > \n> > - custom_target('doxygen',\n> > + # We run doxygen twice - the first run excludes internal API objects as it\n> > + # is intended to document the public API only. A second run covers all of\n> > + # the library's objects for libcamera developers. Common configuration is\n> > + # set in an initially generated Doxyfile, which is then included by the two\n> > + # final Doxyfiles. We collected a list of the public sources in\n> > + # doxygen_public_sources to pass to the public run and include it in cdata\n> > + # here - although both Doxyfile and Doxyfile-internal will also receive\n> > + # the same parameter they simply ignore it.\n> This comment needs correcting now as it references the older method.\n> > +\n> > + # This is the \"public\" run of doxygen generating an abridged version of the\n> > + # API's documentation.\n> > +\n> > + doxyfile_tmpl = configure_file(input : 'Doxyfile-public.in',\n> > + output : 'Doxyfile-public.tmpl',\n> > + configuration : cdata)\n> > +\n> > + doxyfile = custom_target('doxyfile-public',\n> > + input : [\n> > + doxygen_public_input,\n> > + ],\n> > + output : 'Doxyfile-public',\n> > + command : [\n> > + 'gen-doxyfile.py',\n> > + '-o', '@OUTPUT@',\n> > + doxyfile_tmpl,\n> > + '@INPUT@',\n> > + ])\n> > +\n> > + custom_target('doxygen-public',\n> > input : [\n> > doxyfile,\n> > - doxygen_public_input,\n> > - doxygen_internal_input,\n> > + doxyfile_common,\n\nAt least in my mail client - the indentation looks odd here and might\nwant checking.\n\n> > ],\n> > output : 'api-html',\n> > command : [doxygen, doxyfile],\n> > install : true,\n> > install_dir : doc_install_dir,\n> > install_tag : 'doc')\n> > +\n> > + # This is the internal documentation, which hard-codes a list of directories\n> > + # to parse in its doxyfile.\n> > +\n> > + doxyfile_tmpl = configure_file(input : 'Doxyfile-internal.in',\n> > + output : 'Doxyfile-internal.tmpl',\n> > + configuration : cdata)\n> > +\n> > + doxyfile = custom_target('doxyfile-internal',\n> > + input : [\n> > + doxygen_public_input,\n> > + doxygen_internal_input,\n> > + ],\n> > + output : 'Doxyfile-internal',\n> > + command : [\n> > + 'gen-doxyfile.py',\n> > + '-o', '@OUTPUT@',\n> > + doxyfile_tmpl,\n> > + '@INPUT@',\n> > + ])\n> I think we can skip the script for the internal run -there's no templated values to be replaced and \n> the generated Doxyfile-internal is identical to the Doxyfile-internal.tmpl.\n\nI wonder if that's \"future proofing\" but I don't mind either way.\n\nI have nothing much to add except a potential whitespace comment on top\nof Dan's comments, so with those handled:\n\nReviewed-by: Kieran Bingham \n\n> > +\n> > + custom_target('doxygen-internal',\n> > + input : [\n> > + doxyfile,\n> > + doxyfile_common,\n> > + ],\n> > + output : 'internal-api-html',\n> > + command : [doxygen, doxyfile],\n> > + install : true,\n> > + install_dir : doc_install_dir,\n> > + install_tag : 'doc-internal')\n> > endif\n> > \n> > #","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id E229CC323E\n\tfor ;\n\tWed, 7 Aug 2024 11:45:37 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id B29616338D;\n\tWed, 7 Aug 2024 13:45:36 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 36BD96337E\n\tfor ;\n\tWed, 7 Aug 2024 13:45:35 +0200 (CEST)","from pendragon.ideasonboard.com\n\t(cpc89244-aztw30-2-0-cust6594.18-1.cable.virginm.net [86.31.185.195])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 50FF16AF;\n\tWed, 7 Aug 2024 13:44:42 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"EgeWO3bX\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1723031082;\n\tbh=6DDPFgAVCUm9Am6pwpw/Y2jMh7tGlOYUuYLjssmfTEI=;\n\th=In-Reply-To:References:Subject:From:To:Date:From;\n\tb=EgeWO3bX6FWYvb3ztdirVb4xx5IfSyjHq1gnRJ2wcPXgy491Jejh0dD8Y7pRGqsH5\n\tweTBcHXQuWsNUgdYY/III+gVDtOV6h4HZxnZMM0ik8KT8GEuzX7TOZ2uLaSttiqXRh\n\tSPCQ/VvZUrQuSGUXAErMOsSCwQRRaSs4t079M31s=","Content-Type":"text/plain; charset=\"utf-8\"","MIME-Version":"1.0","Content-Transfer-Encoding":"quoted-printable","In-Reply-To":"<1c24fdf8-f01d-4e46-881d-442c75db3024@ideasonboard.com>","References":"<20240805143654.20870-1-laurent.pinchart@ideasonboard.com>\n\t<20240805143654.20870-17-laurent.pinchart@ideasonboard.com>\n\t<1c24fdf8-f01d-4e46-881d-442c75db3024@ideasonboard.com>","Subject":"Re: [PATCH v5 16/18] Documentation: Split public/private\n\tdocumentation","From":"Kieran Bingham ","To":"Dan Scally ,\n\tLaurent Pinchart ,\n\tlibcamera-devel@lists.libcamera.org","Date":"Wed, 07 Aug 2024 12:45:31 +0100","Message-ID":"<172303113182.1687952.3559570839316346035@ping.linuxembedded.co.uk>","User-Agent":"alot/0.10","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":30681,"web_url":"https://patchwork.libcamera.org/comment/30681/","msgid":"<20240807133113.GH15173@pendragon.ideasonboard.com>","date":"2024-08-07T13:31:13","subject":"Re: [PATCH v5 16/18] Documentation: Split public/private\n\tdocumentation","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"On Wed, Aug 07, 2024 at 12:45:31PM +0100, Kieran Bingham wrote:\n> Quoting Dan Scally (2024-08-07 09:30:22)\n> > On 05/08/2024 15:36, Laurent Pinchart wrote:\n> > > From: Daniel Scally \n> > >\n> > > The API reference pages generated by Doxygen are comprehensive, but\n> > > therefore quite overwhelming for application developers who will\n> > > likely never need to use the majority of the library's objects. To\n> > > reduce the complexity of the documentation, split it into two runs of\n> > > doxygen.\n> > >\n> > > The first run of doxygen is for the public API. We pass a specific list\n> > > of source files to parse, which is built from the arrays of public\n> > > headers and sources in meson build files. This ensures that we only\n> > > generate the documentation for code from those files.\n> > >\n> > > A custom Python script is needed to add the list of input files to\n> > > Doxyfile, as several of the objects included in the header and source\n> > > array are custom_tgt objects, which can't be handled as strings to\n> > > populate a variable in the configuration data.\n> > >\n> > > The headers defining the Extensible and Object classes (class.h and\n> > > object.h respectively), as well as the corresponding source files, are\n> > > excluded from the public API documentation despite being referenced in\n> > > the meson public headers and sources arrays. This is due to the fact\n> > > that public API classes inherit from Extensible and Object, making the\n> > > Extensible and Object classes part of the public ABI. Those two base\n> > > classes are however implementation details and must not be accessed\n> > > directly by application code.\n> > >\n> > > The second run of doxygen is for the internal API. This contains\n> > > documentation for all of the library's objects as it currently does.\n> > > This set will now be output into build/Documentation/internal-api-html.\n> \n> Sounds good.\n> \n> > >\n> > > Signed-off-by: Daniel Scally \n> > > Co-developed-by: Laurent Pinchart \n> > > Signed-off-by: Laurent Pinchart \n> > > ---\n> > \n> > The python script to generate the list of input files is definitely better than scattered foreach \n> > loops collecting a list in a hacky way - good improvement.\n> > \n> > > .../{Doxyfile.in => Doxyfile-common.in} | 24 -------\n> > > Documentation/Doxyfile-internal.in | 30 ++++++++\n> > > Documentation/Doxyfile-public.in | 20 ++++++\n> > > Documentation/gen-doxyfile.py | 46 ++++++++++++\n> > > Documentation/meson.build | 70 +++++++++++++++++--\n> > > 5 files changed, 160 insertions(+), 30 deletions(-)\n> > > rename Documentation/{Doxyfile.in => Doxyfile-common.in} (62%)\n> > > create mode 100644 Documentation/Doxyfile-internal.in\n> > > create mode 100644 Documentation/Doxyfile-public.in\n> > > create mode 100755 Documentation/gen-doxyfile.py\n> > >\n> > > diff --git a/Documentation/Doxyfile.in b/Documentation/Doxyfile-common.in\n> > > similarity index 62%\n> > > rename from Documentation/Doxyfile.in\n> > > rename to Documentation/Doxyfile-common.in\n> > > index 6e5a3830ec38..a70aee430e77 100644\n> > > --- a/Documentation/Doxyfile.in\n> > > +++ b/Documentation/Doxyfile-common.in\n> > > @@ -17,20 +17,11 @@ EXTENSION_MAPPING = h=C++\n> > > \n> > > TOC_INCLUDE_HEADINGS = 0\n> > > \n> > > -INTERNAL_DOCS = YES\n> > > CASE_SENSE_NAMES = YES\n> > > \n> > > QUIET = YES\n> > > WARN_AS_ERROR = @WARN_AS_ERROR@\n> > > \n> > > -INPUT = \"@TOP_SRCDIR@/Documentation\" \\\n> > > - \"@TOP_SRCDIR@/include/libcamera\" \\\n> > > - \"@TOP_SRCDIR@/src/ipa/ipu3\" \\\n> > > - \"@TOP_SRCDIR@/src/ipa/libipa\" \\\n> > > - \"@TOP_SRCDIR@/src/libcamera\" \\\n> > > - \"@TOP_BUILDDIR@/include/libcamera\" \\\n> > > - \"@TOP_BUILDDIR@/src/libcamera\"\n> > > -\n> > > FILE_PATTERNS = *.c \\\n> > > *.cpp \\\n> > > *.dox \\\n> > > @@ -38,19 +29,6 @@ FILE_PATTERNS = *.c \\\n> > > \n> > > RECURSIVE = YES\n> > > \n> > > -EXCLUDE = @TOP_SRCDIR@/include/libcamera/base/span.h \\\n> > > - @TOP_SRCDIR@/include/libcamera/internal/device_enumerator_sysfs.h \\\n> > > - @TOP_SRCDIR@/include/libcamera/internal/device_enumerator_udev.h \\\n> > > - @TOP_SRCDIR@/include/libcamera/internal/ipc_pipe_unixsocket.h \\\n> > > - @TOP_SRCDIR@/src/libcamera/device_enumerator_sysfs.cpp \\\n> > > - @TOP_SRCDIR@/src/libcamera/device_enumerator_udev.cpp \\\n> > > - @TOP_SRCDIR@/src/libcamera/ipc_pipe_unixsocket.cpp \\\n> > > - @TOP_SRCDIR@/src/libcamera/pipeline/ \\\n> > > - @TOP_SRCDIR@/src/libcamera/tracepoints.cpp \\\n> > > - @TOP_BUILDDIR@/include/libcamera/internal/tracepoints.h \\\n> > > - @TOP_BUILDDIR@/include/libcamera/ipa/soft_ipa_interface.h \\\n> > > - @TOP_BUILDDIR@/src/libcamera/proxy/\n> > > -\n> > > EXCLUDE_PATTERNS = @TOP_BUILDDIR@/include/libcamera/ipa/*_serializer.h \\\n> > > @TOP_BUILDDIR@/include/libcamera/ipa/*_proxy.h \\\n> > > @TOP_BUILDDIR@/include/libcamera/ipa/ipu3_*.h \\\n> > > @@ -73,8 +51,6 @@ EXCLUDE_SYMBOLS = libcamera::BoundMethodArgs \\\n> > > \n> > > EXCLUDE_SYMLINKS = YES\n> > > \n> > > -HTML_OUTPUT = api-html\n> > > -\n> > > GENERATE_LATEX = NO\n> > > \n> > > MACRO_EXPANSION = YES\n> > > diff --git a/Documentation/Doxyfile-internal.in b/Documentation/Doxyfile-internal.in\n> > > new file mode 100644\n> > > index 000000000000..5e0310091416\n> > > --- /dev/null\n> > > +++ b/Documentation/Doxyfile-internal.in\n> > > @@ -0,0 +1,30 @@\n> > > +# SPDX-License-Identifier: CC-BY-SA-4.0\n> > > +\n> > > +@INCLUDE_PATH = @TOP_BUILDDIR@/Documentation\n> > > +@INCLUDE = Doxyfile-common\n> > > +\n> > > +HIDE_UNDOC_CLASSES = NO\n> > > +HIDE_UNDOC_MEMBERS = NO\n> > > +HTML_OUTPUT = internal-api-html\n> > > +INTERNAL_DOCS = YES\n> > > +\n> > > +INPUT = \"@TOP_SRCDIR@/Documentation\" \\\n> > > + \"@TOP_SRCDIR@/include/libcamera\" \\\n> > > + \"@TOP_SRCDIR@/src/ipa/ipu3\" \\\n> > > + \"@TOP_SRCDIR@/src/ipa/libipa\" \\\n> > > + \"@TOP_SRCDIR@/src/libcamera\" \\\n> > > + \"@TOP_BUILDDIR@/include/libcamera\" \\\n> > > + \"@TOP_BUILDDIR@/src/libcamera\"\n> > > +\n> > > +EXCLUDE = @TOP_SRCDIR@/include/libcamera/base/span.h \\\n> > > + @TOP_SRCDIR@/include/libcamera/internal/device_enumerator_sysfs.h \\\n> > > + @TOP_SRCDIR@/include/libcamera/internal/device_enumerator_udev.h \\\n> > > + @TOP_SRCDIR@/include/libcamera/internal/ipc_pipe_unixsocket.h \\\n> > > + @TOP_SRCDIR@/src/libcamera/device_enumerator_sysfs.cpp \\\n> > > + @TOP_SRCDIR@/src/libcamera/device_enumerator_udev.cpp \\\n> > > + @TOP_SRCDIR@/src/libcamera/ipc_pipe_unixsocket.cpp \\\n> > > + @TOP_SRCDIR@/src/libcamera/pipeline/ \\\n> > > + @TOP_SRCDIR@/src/libcamera/tracepoints.cpp \\\n> > > + @TOP_BUILDDIR@/include/libcamera/internal/tracepoints.h \\\n> > > + @TOP_BUILDDIR@/include/libcamera/ipa/soft_ipa_interface.h \\\n> > > + @TOP_BUILDDIR@/src/libcamera/proxy/\n> > > diff --git a/Documentation/Doxyfile-public.in b/Documentation/Doxyfile-public.in\n> > > new file mode 100644\n> > > index 000000000000..36bb57584a07\n> > > --- /dev/null\n> > > +++ b/Documentation/Doxyfile-public.in\n> > > @@ -0,0 +1,20 @@\n> > > +# SPDX-License-Identifier: CC-BY-SA-4.0\n> > > +\n> > > +@INCLUDE_PATH = @TOP_BUILDDIR@/Documentation\n> > > +@INCLUDE = Doxyfile-common\n> > > +\n> > > +HIDE_UNDOC_CLASSES = YES\n> > > +HIDE_UNDOC_MEMBERS = YES\n> > > +HTML_OUTPUT = api-html\n> > > +INTERNAL_DOCS = NO\n> > > +\n> > > +INPUT = \"@TOP_SRCDIR@/Documentation\" \\\n> > > + ${inputs}\n> > > +\n> > > +EXCLUDE = @TOP_SRCDIR@/include/libcamera/base/class.h \\\n> > > + @TOP_SRCDIR@/include/libcamera/base/object.h \\\n> > > + @TOP_SRCDIR@/include/libcamera/base/span.h \\\n> > > + @TOP_SRCDIR@/src/libcamera/base/class.cpp \\\n> > > + @TOP_SRCDIR@/src/libcamera/base/object.cpp\n> > > +\n> > > +PREDEFINED += __DOXYGEN_PUBLIC__\n> > > diff --git a/Documentation/gen-doxyfile.py b/Documentation/gen-doxyfile.py\n> > > new file mode 100755\n> > > index 000000000000..c2a4184dd195\n> > > --- /dev/null\n> > > +++ b/Documentation/gen-doxyfile.py\n> > > @@ -0,0 +1,46 @@\n> > > +#!/usr/bin/env python3\n> > > +# SPDX-License-Identifier: GPL-2.0-or-later\n> > > +# Copyright (C) 2024, Google Inc.\n> >\n> > Does this mean it's drawn from elsewhere? Or is it a copy-paste error?\n> >\n> > > +#\n> > > +# Author: Laurent Pinchart \n> > > +#\n> > > +# Generate Doxyfile from a template\n> > > +\n> > > +import argparse\n> > > +import os\n> > > +import string\n> > > +import sys\n> > > +\n> > > +\n> > > +def fill_template(template, data):\n> > > +\n> > > + template = open(template, 'rb').read()\n> > > + template = template.decode('utf-8')\n> > > + template = string.Template(template)\n> >\n> > I'd add a line break here.\n\nDone.\n\n> > > + return template.substitute(data)\n> > > +\n> > > +\n> > > +def main(argv):\n> > > +\n> > > + # Parse command line arguments\n> >\n> > The comment can probably be skipped.\n\nI'm fine with that.\n\n\n> > > + parser = argparse.ArgumentParser()\n> > > + parser.add_argument('-o', dest='output', metavar='file',\n> > > + type=argparse.FileType('w', encoding='utf-8'),\n> > > + default=sys.stdout,\n> > > + help='Output file name (default: standard output)')\n> > > + parser.add_argument('template', metavar='doxyfile.tmpl', type=str,\n> > > + help='Doxyfile template')\n> > > + parser.add_argument('inputs', type=str, nargs='*',\n> > > + help='Input files')\n> > > +\n> > > + args = parser.parse_args(argv[1:])\n> > > +\n> > > + inputs = [f'\"{os.path.realpath(input)}\"' for input in args.inputs]\n> > > + data = fill_template(args.template, {'inputs': (' \\\\\\n' + ' ' * 25).join(inputs)})\n> > > + args.output.write(data)\n> > > +\n> > > + return 0\n> > > +\n> > > +\n> > > +if __name__ == '__main__':\n> > > + sys.exit(main(sys.argv))\n> > > diff --git a/Documentation/meson.build b/Documentation/meson.build\n> > > index 1d84ed815b50..bf28233a2ce8 100644\n> > > --- a/Documentation/meson.build\n> > > +++ b/Documentation/meson.build\n> > > @@ -24,9 +24,9 @@ if doxygen.found() and dot.found()\n> > > \n> > > cdata.set('PREDEFINED', ' \\\\\\n\\t\\t\\t '.join(doxygen_predefined))\n> > > \n> > > - doxyfile = configure_file(input : 'Doxyfile.in',\n> > > - output : 'Doxyfile',\n> > > - configuration : cdata)\n> > > + doxyfile_common = configure_file(input : 'Doxyfile-common.in',\n> > > + output : 'Doxyfile-common',\n> > > + configuration : cdata)\n> > > \n> > > doxygen_public_input = [\n> > > libcamera_base_public_headers,\n> > > @@ -50,17 +50,75 @@ if doxygen.found() and dot.found()\n> > > doxygen_internal_input += [ipu3_ipa_sources]\n> > > endif\n> > > \n> > > - custom_target('doxygen',\n> > > + # We run doxygen twice - the first run excludes internal API objects as it\n> > > + # is intended to document the public API only. A second run covers all of\n> > > + # the library's objects for libcamera developers. Common configuration is\n> > > + # set in an initially generated Doxyfile, which is then included by the two\n> > > + # final Doxyfiles. We collected a list of the public sources in\n> > > + # doxygen_public_sources to pass to the public run and include it in cdata\n> > > + # here - although both Doxyfile and Doxyfile-internal will also receive\n> > > + # the same parameter they simply ignore it.\n> >\n> > This comment needs correcting now as it references the older method.\n\nI'll fix it.\n\n> > > +\n> > > + # This is the \"public\" run of doxygen generating an abridged version of the\n> > > + # API's documentation.\n> > > +\n> > > + doxyfile_tmpl = configure_file(input : 'Doxyfile-public.in',\n> > > + output : 'Doxyfile-public.tmpl',\n> > > + configuration : cdata)\n> > > +\n> > > + doxyfile = custom_target('doxyfile-public',\n> > > + input : [\n> > > + doxygen_public_input,\n> > > + ],\n> > > + output : 'Doxyfile-public',\n> > > + command : [\n> > > + 'gen-doxyfile.py',\n> > > + '-o', '@OUTPUT@',\n> > > + doxyfile_tmpl,\n> > > + '@INPUT@',\n> > > + ])\n> > > +\n> > > + custom_target('doxygen-public',\n> > > input : [\n> > > doxyfile,\n> > > - doxygen_public_input,\n> > > - doxygen_internal_input,\n> > > + doxyfile_common,\n> \n> At least in my mail client - the indentation looks odd here and might\n> want checking.\n\nIt looks fine in the source code as far as I can tell. Can you fix your\nmail client ? :-)\n\n> > > ],\n> > > output : 'api-html',\n> > > command : [doxygen, doxyfile],\n> > > install : true,\n> > > install_dir : doc_install_dir,\n> > > install_tag : 'doc')\n> > > +\n> > > + # This is the internal documentation, which hard-codes a list of directories\n> > > + # to parse in its doxyfile.\n> > > +\n> > > + doxyfile_tmpl = configure_file(input : 'Doxyfile-internal.in',\n> > > + output : 'Doxyfile-internal.tmpl',\n> > > + configuration : cdata)\n> > > +\n> > > + doxyfile = custom_target('doxyfile-internal',\n> > > + input : [\n> > > + doxygen_public_input,\n> > > + doxygen_internal_input,\n> > > + ],\n> > > + output : 'Doxyfile-internal',\n> > > + command : [\n> > > + 'gen-doxyfile.py',\n> > > + '-o', '@OUTPUT@',\n> > > + doxyfile_tmpl,\n> > > + '@INPUT@',\n> > > + ])\n> >\n> > I think we can skip the script for the internal run -there's no\n> > templated values to be replaced and the generated Doxyfile-internal\n> > is identical to the Doxyfile-internal.tmpl.\n> \n> I wonder if that's \"future proofing\" but I don't mind either way.\n\nThe goal was to keep the code flow identical, and I considered\nfuture-proofing too. I don't mind dropping it though.\n\n> I have nothing much to add except a potential whitespace comment on top\n> of Dan's comments, so with those handled:\n> \n> Reviewed-by: Kieran Bingham \n> \n> > > +\n> > > + custom_target('doxygen-internal',\n> > > + input : [\n> > > + doxyfile,\n> > > + doxyfile_common,\n> > > + ],\n> > > + output : 'internal-api-html',\n> > > + command : [doxygen, doxyfile],\n> > > + install : true,\n> > > + install_dir : doc_install_dir,\n> > > + install_tag : 'doc-internal')\n> > > endif\n> > > \n> > > #","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 2A124BE173\n\tfor ;\n\tWed, 7 Aug 2024 13:31:39 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 53CB66337F;\n\tWed, 7 Aug 2024 15:31:38 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 673786337E\n\tfor ;\n\tWed, 7 Aug 2024 15:31:36 +0200 (CEST)","from pendragon.ideasonboard.com (81-175-209-231.bb.dnainternet.fi\n\t[81.175.209.231])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 6F5806AF;\n\tWed, 7 Aug 2024 15:30:43 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"TK0s0/KE\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1723037443;\n\tbh=KkSm7JKop/vnNUgqzs5l+xVH6pdHN+augwj+4A5GmPY=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=TK0s0/KEJWv/k4RyX0TuXnKab9ltK26BCzaI54dhxPpNQ1+nJoM7uf9DpIbLOmziL\n\toSyHLBMdegMFGKFVS5xzPQjvquyErGdUMlwkap6Mg21MfWQOFxi9otGuczBQK7ScPI\n\tJRgLRQDq+pVq5ZKNII8J+omChFsQ2zr2tcnw8FsY=","Date":"Wed, 7 Aug 2024 16:31:13 +0300","From":"Laurent Pinchart ","To":"Kieran Bingham ","Cc":"Dan Scally ,\n\tlibcamera-devel@lists.libcamera.org","Subject":"Re: [PATCH v5 16/18] Documentation: Split public/private\n\tdocumentation","Message-ID":"<20240807133113.GH15173@pendragon.ideasonboard.com>","References":"<20240805143654.20870-1-laurent.pinchart@ideasonboard.com>\n\t<20240805143654.20870-17-laurent.pinchart@ideasonboard.com>\n\t<1c24fdf8-f01d-4e46-881d-442c75db3024@ideasonboard.com>\n\t<172303113182.1687952.3559570839316346035@ping.linuxembedded.co.uk>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<172303113182.1687952.3559570839316346035@ping.linuxembedded.co.uk>","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":30682,"web_url":"https://patchwork.libcamera.org/comment/30682/","msgid":"","date":"2024-08-07T13:45:03","subject":"Re: [PATCH v5 16/18] Documentation: Split public/private\n\tdocumentation","submitter":{"id":156,"url":"https://patchwork.libcamera.org/api/people/156/","name":"Dan Scally","email":"dan.scally@ideasonboard.com"},"content":"Hi Laurent\n\nOn 07/08/2024 14:31, Laurent Pinchart wrote:\n> On Wed, Aug 07, 2024 at 12:45:31PM +0100, Kieran Bingham wrote:\n>> Quoting Dan Scally (2024-08-07 09:30:22)\n>>> On 05/08/2024 15:36, Laurent Pinchart wrote:\n>>>> From: Daniel Scally \n>>>>\n>>>> The API reference pages generated by Doxygen are comprehensive, but\n>>>> therefore quite overwhelming for application developers who will\n>>>> likely never need to use the majority of the library's objects. To\n>>>> reduce the complexity of the documentation, split it into two runs of\n>>>> doxygen.\n>>>>\n>>>> The first run of doxygen is for the public API. We pass a specific list\n>>>> of source files to parse, which is built from the arrays of public\n>>>> headers and sources in meson build files. This ensures that we only\n>>>> generate the documentation for code from those files.\n>>>>\n>>>> A custom Python script is needed to add the list of input files to\n>>>> Doxyfile, as several of the objects included in the header and source\n>>>> array are custom_tgt objects, which can't be handled as strings to\n>>>> populate a variable in the configuration data.\n>>>>\n>>>> The headers defining the Extensible and Object classes (class.h and\n>>>> object.h respectively), as well as the corresponding source files, are\n>>>> excluded from the public API documentation despite being referenced in\n>>>> the meson public headers and sources arrays. This is due to the fact\n>>>> that public API classes inherit from Extensible and Object, making the\n>>>> Extensible and Object classes part of the public ABI. Those two base\n>>>> classes are however implementation details and must not be accessed\n>>>> directly by application code.\n>>>>\n>>>> The second run of doxygen is for the internal API. This contains\n>>>> documentation for all of the library's objects as it currently does.\n>>>> This set will now be output into build/Documentation/internal-api-html.\n>> Sounds good.\n>>\n>>>> Signed-off-by: Daniel Scally \n>>>> Co-developed-by: Laurent Pinchart \n>>>> Signed-off-by: Laurent Pinchart \n>>>> ---\n>>> The python script to generate the list of input files is definitely better than scattered foreach\n>>> loops collecting a list in a hacky way - good improvement.\n>>>\n>>>> .../{Doxyfile.in => Doxyfile-common.in} | 24 -------\n>>>> Documentation/Doxyfile-internal.in | 30 ++++++++\n>>>> Documentation/Doxyfile-public.in | 20 ++++++\n>>>> Documentation/gen-doxyfile.py | 46 ++++++++++++\n>>>> Documentation/meson.build | 70 +++++++++++++++++--\n>>>> 5 files changed, 160 insertions(+), 30 deletions(-)\n>>>> rename Documentation/{Doxyfile.in => Doxyfile-common.in} (62%)\n>>>> create mode 100644 Documentation/Doxyfile-internal.in\n>>>> create mode 100644 Documentation/Doxyfile-public.in\n>>>> create mode 100755 Documentation/gen-doxyfile.py\n>>>>\n>>>> diff --git a/Documentation/Doxyfile.in b/Documentation/Doxyfile-common.in\n>>>> similarity index 62%\n>>>> rename from Documentation/Doxyfile.in\n>>>> rename to Documentation/Doxyfile-common.in\n>>>> index 6e5a3830ec38..a70aee430e77 100644\n>>>> --- a/Documentation/Doxyfile.in\n>>>> +++ b/Documentation/Doxyfile-common.in\n>>>> @@ -17,20 +17,11 @@ EXTENSION_MAPPING = h=C++\n>>>> \n>>>> TOC_INCLUDE_HEADINGS = 0\n>>>> \n>>>> -INTERNAL_DOCS = YES\n>>>> CASE_SENSE_NAMES = YES\n>>>> \n>>>> QUIET = YES\n>>>> WARN_AS_ERROR = @WARN_AS_ERROR@\n>>>> \n>>>> -INPUT = \"@TOP_SRCDIR@/Documentation\" \\\n>>>> - \"@TOP_SRCDIR@/include/libcamera\" \\\n>>>> - \"@TOP_SRCDIR@/src/ipa/ipu3\" \\\n>>>> - \"@TOP_SRCDIR@/src/ipa/libipa\" \\\n>>>> - \"@TOP_SRCDIR@/src/libcamera\" \\\n>>>> - \"@TOP_BUILDDIR@/include/libcamera\" \\\n>>>> - \"@TOP_BUILDDIR@/src/libcamera\"\n>>>> -\n>>>> FILE_PATTERNS = *.c \\\n>>>> *.cpp \\\n>>>> *.dox \\\n>>>> @@ -38,19 +29,6 @@ FILE_PATTERNS = *.c \\\n>>>> \n>>>> RECURSIVE = YES\n>>>> \n>>>> -EXCLUDE = @TOP_SRCDIR@/include/libcamera/base/span.h \\\n>>>> - @TOP_SRCDIR@/include/libcamera/internal/device_enumerator_sysfs.h \\\n>>>> - @TOP_SRCDIR@/include/libcamera/internal/device_enumerator_udev.h \\\n>>>> - @TOP_SRCDIR@/include/libcamera/internal/ipc_pipe_unixsocket.h \\\n>>>> - @TOP_SRCDIR@/src/libcamera/device_enumerator_sysfs.cpp \\\n>>>> - @TOP_SRCDIR@/src/libcamera/device_enumerator_udev.cpp \\\n>>>> - @TOP_SRCDIR@/src/libcamera/ipc_pipe_unixsocket.cpp \\\n>>>> - @TOP_SRCDIR@/src/libcamera/pipeline/ \\\n>>>> - @TOP_SRCDIR@/src/libcamera/tracepoints.cpp \\\n>>>> - @TOP_BUILDDIR@/include/libcamera/internal/tracepoints.h \\\n>>>> - @TOP_BUILDDIR@/include/libcamera/ipa/soft_ipa_interface.h \\\n>>>> - @TOP_BUILDDIR@/src/libcamera/proxy/\n>>>> -\n>>>> EXCLUDE_PATTERNS = @TOP_BUILDDIR@/include/libcamera/ipa/*_serializer.h \\\n>>>> @TOP_BUILDDIR@/include/libcamera/ipa/*_proxy.h \\\n>>>> @TOP_BUILDDIR@/include/libcamera/ipa/ipu3_*.h \\\n>>>> @@ -73,8 +51,6 @@ EXCLUDE_SYMBOLS = libcamera::BoundMethodArgs \\\n>>>> \n>>>> EXCLUDE_SYMLINKS = YES\n>>>> \n>>>> -HTML_OUTPUT = api-html\n>>>> -\n>>>> GENERATE_LATEX = NO\n>>>> \n>>>> MACRO_EXPANSION = YES\n>>>> diff --git a/Documentation/Doxyfile-internal.in b/Documentation/Doxyfile-internal.in\n>>>> new file mode 100644\n>>>> index 000000000000..5e0310091416\n>>>> --- /dev/null\n>>>> +++ b/Documentation/Doxyfile-internal.in\n>>>> @@ -0,0 +1,30 @@\n>>>> +# SPDX-License-Identifier: CC-BY-SA-4.0\n>>>> +\n>>>> +@INCLUDE_PATH = @TOP_BUILDDIR@/Documentation\n>>>> +@INCLUDE = Doxyfile-common\n>>>> +\n>>>> +HIDE_UNDOC_CLASSES = NO\n>>>> +HIDE_UNDOC_MEMBERS = NO\n>>>> +HTML_OUTPUT = internal-api-html\n>>>> +INTERNAL_DOCS = YES\n>>>> +\n>>>> +INPUT = \"@TOP_SRCDIR@/Documentation\" \\\n>>>> + \"@TOP_SRCDIR@/include/libcamera\" \\\n>>>> + \"@TOP_SRCDIR@/src/ipa/ipu3\" \\\n>>>> + \"@TOP_SRCDIR@/src/ipa/libipa\" \\\n>>>> + \"@TOP_SRCDIR@/src/libcamera\" \\\n>>>> + \"@TOP_BUILDDIR@/include/libcamera\" \\\n>>>> + \"@TOP_BUILDDIR@/src/libcamera\"\n>>>> +\n>>>> +EXCLUDE = @TOP_SRCDIR@/include/libcamera/base/span.h \\\n>>>> + @TOP_SRCDIR@/include/libcamera/internal/device_enumerator_sysfs.h \\\n>>>> + @TOP_SRCDIR@/include/libcamera/internal/device_enumerator_udev.h \\\n>>>> + @TOP_SRCDIR@/include/libcamera/internal/ipc_pipe_unixsocket.h \\\n>>>> + @TOP_SRCDIR@/src/libcamera/device_enumerator_sysfs.cpp \\\n>>>> + @TOP_SRCDIR@/src/libcamera/device_enumerator_udev.cpp \\\n>>>> + @TOP_SRCDIR@/src/libcamera/ipc_pipe_unixsocket.cpp \\\n>>>> + @TOP_SRCDIR@/src/libcamera/pipeline/ \\\n>>>> + @TOP_SRCDIR@/src/libcamera/tracepoints.cpp \\\n>>>> + @TOP_BUILDDIR@/include/libcamera/internal/tracepoints.h \\\n>>>> + @TOP_BUILDDIR@/include/libcamera/ipa/soft_ipa_interface.h \\\n>>>> + @TOP_BUILDDIR@/src/libcamera/proxy/\n>>>> diff --git a/Documentation/Doxyfile-public.in b/Documentation/Doxyfile-public.in\n>>>> new file mode 100644\n>>>> index 000000000000..36bb57584a07\n>>>> --- /dev/null\n>>>> +++ b/Documentation/Doxyfile-public.in\n>>>> @@ -0,0 +1,20 @@\n>>>> +# SPDX-License-Identifier: CC-BY-SA-4.0\n>>>> +\n>>>> +@INCLUDE_PATH = @TOP_BUILDDIR@/Documentation\n>>>> +@INCLUDE = Doxyfile-common\n>>>> +\n>>>> +HIDE_UNDOC_CLASSES = YES\n>>>> +HIDE_UNDOC_MEMBERS = YES\n>>>> +HTML_OUTPUT = api-html\n>>>> +INTERNAL_DOCS = NO\n>>>> +\n>>>> +INPUT = \"@TOP_SRCDIR@/Documentation\" \\\n>>>> + ${inputs}\n>>>> +\n>>>> +EXCLUDE = @TOP_SRCDIR@/include/libcamera/base/class.h \\\n>>>> + @TOP_SRCDIR@/include/libcamera/base/object.h \\\n>>>> + @TOP_SRCDIR@/include/libcamera/base/span.h \\\n>>>> + @TOP_SRCDIR@/src/libcamera/base/class.cpp \\\n>>>> + @TOP_SRCDIR@/src/libcamera/base/object.cpp\n>>>> +\n>>>> +PREDEFINED += __DOXYGEN_PUBLIC__\n>>>> diff --git a/Documentation/gen-doxyfile.py b/Documentation/gen-doxyfile.py\n>>>> new file mode 100755\n>>>> index 000000000000..c2a4184dd195\n>>>> --- /dev/null\n>>>> +++ b/Documentation/gen-doxyfile.py\n>>>> @@ -0,0 +1,46 @@\n>>>> +#!/usr/bin/env python3\n>>>> +# SPDX-License-Identifier: GPL-2.0-or-later\n>>>> +# Copyright (C) 2024, Google Inc.\n>>> Does this mean it's drawn from elsewhere? Or is it a copy-paste error?\n>>>\n>>>> +#\n>>>> +# Author: Laurent Pinchart \n>>>> +#\n>>>> +# Generate Doxyfile from a template\n>>>> +\n>>>> +import argparse\n>>>> +import os\n>>>> +import string\n>>>> +import sys\n>>>> +\n>>>> +\n>>>> +def fill_template(template, data):\n>>>> +\n>>>> + template = open(template, 'rb').read()\n>>>> + template = template.decode('utf-8')\n>>>> + template = string.Template(template)\n>>> I'd add a line break here.\n> Done.\n>\n>>>> + return template.substitute(data)\n>>>> +\n>>>> +\n>>>> +def main(argv):\n>>>> +\n>>>> + # Parse command line arguments\n>>> The comment can probably be skipped.\n> I'm fine with that.\n>\n>\n>>>> + parser = argparse.ArgumentParser()\n>>>> + parser.add_argument('-o', dest='output', metavar='file',\n>>>> + type=argparse.FileType('w', encoding='utf-8'),\n>>>> + default=sys.stdout,\n>>>> + help='Output file name (default: standard output)')\n>>>> + parser.add_argument('template', metavar='doxyfile.tmpl', type=str,\n>>>> + help='Doxyfile template')\n>>>> + parser.add_argument('inputs', type=str, nargs='*',\n>>>> + help='Input files')\n>>>> +\n>>>> + args = parser.parse_args(argv[1:])\n>>>> +\n>>>> + inputs = [f'\"{os.path.realpath(input)}\"' for input in args.inputs]\n>>>> + data = fill_template(args.template, {'inputs': (' \\\\\\n' + ' ' * 25).join(inputs)})\n>>>> + args.output.write(data)\n>>>> +\n>>>> + return 0\n>>>> +\n>>>> +\n>>>> +if __name__ == '__main__':\n>>>> + sys.exit(main(sys.argv))\n>>>> diff --git a/Documentation/meson.build b/Documentation/meson.build\n>>>> index 1d84ed815b50..bf28233a2ce8 100644\n>>>> --- a/Documentation/meson.build\n>>>> +++ b/Documentation/meson.build\n>>>> @@ -24,9 +24,9 @@ if doxygen.found() and dot.found()\n>>>> \n>>>> cdata.set('PREDEFINED', ' \\\\\\n\\t\\t\\t '.join(doxygen_predefined))\n>>>> \n>>>> - doxyfile = configure_file(input : 'Doxyfile.in',\n>>>> - output : 'Doxyfile',\n>>>> - configuration : cdata)\n>>>> + doxyfile_common = configure_file(input : 'Doxyfile-common.in',\n>>>> + output : 'Doxyfile-common',\n>>>> + configuration : cdata)\n>>>> \n>>>> doxygen_public_input = [\n>>>> libcamera_base_public_headers,\n>>>> @@ -50,17 +50,75 @@ if doxygen.found() and dot.found()\n>>>> doxygen_internal_input += [ipu3_ipa_sources]\n>>>> endif\n>>>> \n>>>> - custom_target('doxygen',\n>>>> + # We run doxygen twice - the first run excludes internal API objects as it\n>>>> + # is intended to document the public API only. A second run covers all of\n>>>> + # the library's objects for libcamera developers. Common configuration is\n>>>> + # set in an initially generated Doxyfile, which is then included by the two\n>>>> + # final Doxyfiles. We collected a list of the public sources in\n>>>> + # doxygen_public_sources to pass to the public run and include it in cdata\n>>>> + # here - although both Doxyfile and Doxyfile-internal will also receive\n>>>> + # the same parameter they simply ignore it.\n>>> This comment needs correcting now as it references the older method.\n> I'll fix it.\n>\n>>>> +\n>>>> + # This is the \"public\" run of doxygen generating an abridged version of the\n>>>> + # API's documentation.\n>>>> +\n>>>> + doxyfile_tmpl = configure_file(input : 'Doxyfile-public.in',\n>>>> + output : 'Doxyfile-public.tmpl',\n>>>> + configuration : cdata)\n>>>> +\n>>>> + doxyfile = custom_target('doxyfile-public',\n>>>> + input : [\n>>>> + doxygen_public_input,\n>>>> + ],\n>>>> + output : 'Doxyfile-public',\n>>>> + command : [\n>>>> + 'gen-doxyfile.py',\n>>>> + '-o', '@OUTPUT@',\n>>>> + doxyfile_tmpl,\n>>>> + '@INPUT@',\n>>>> + ])\n>>>> +\n>>>> + custom_target('doxygen-public',\n>>>> input : [\n>>>> doxyfile,\n>>>> - doxygen_public_input,\n>>>> - doxygen_internal_input,\n>>>> + doxyfile_common,\n>> At least in my mail client - the indentation looks odd here and might\n>> want checking.\n> It looks fine in the source code as far as I can tell. Can you fix your\n> mail client ? :-)\n>\n>>>> ],\n>>>> output : 'api-html',\n>>>> command : [doxygen, doxyfile],\n>>>> install : true,\n>>>> install_dir : doc_install_dir,\n>>>> install_tag : 'doc')\n>>>> +\n>>>> + # This is the internal documentation, which hard-codes a list of directories\n>>>> + # to parse in its doxyfile.\n>>>> +\n>>>> + doxyfile_tmpl = configure_file(input : 'Doxyfile-internal.in',\n>>>> + output : 'Doxyfile-internal.tmpl',\n>>>> + configuration : cdata)\n>>>> +\n>>>> + doxyfile = custom_target('doxyfile-internal',\n>>>> + input : [\n>>>> + doxygen_public_input,\n>>>> + doxygen_internal_input,\n>>>> + ],\n>>>> + output : 'Doxyfile-internal',\n>>>> + command : [\n>>>> + 'gen-doxyfile.py',\n>>>> + '-o', '@OUTPUT@',\n>>>> + doxyfile_tmpl,\n>>>> + '@INPUT@',\n>>>> + ])\n>>> I think we can skip the script for the internal run -there's no\n>>> templated values to be replaced and the generated Doxyfile-internal\n>>> is identical to the Doxyfile-internal.tmpl.\n>> I wonder if that's \"future proofing\" but I don't mind either way.\n> The goal was to keep the code flow identical, and I considered\n> future-proofing too. I don't mind dropping it though.\n\nFair enough; leave it in then (unless you've already dropped it...I also don't mind either way so \nwhatever state it's currently in, is fine).\n\n\nIs an R-b from me appropriate here given I'm down as the author? If so:\n\n\nReviewed-by: Daniel Scally \n\n>> I have nothing much to add except a potential whitespace comment on top\n>> of Dan's comments, so with those handled:\n>>\n>> Reviewed-by: Kieran Bingham \n>>\n>>>> +\n>>>> + custom_target('doxygen-internal',\n>>>> + input : [\n>>>> + doxyfile,\n>>>> + doxyfile_common,\n>>>> + ],\n>>>> + output : 'internal-api-html',\n>>>> + command : [doxygen, doxyfile],\n>>>> + install : true,\n>>>> + install_dir : doc_install_dir,\n>>>> + install_tag : 'doc-internal')\n>>>> endif\n>>>> \n>>>> #","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 2C060C323E\n\tfor ;\n\tWed, 7 Aug 2024 13:45:09 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id EAAF963393;\n\tWed, 7 Aug 2024 15:45:07 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 72AEE6337F\n\tfor ;\n\tWed, 7 Aug 2024 15:45:06 +0200 (CEST)","from [192.168.0.43]\n\t(cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 4D53D6AF;\n\tWed, 7 Aug 2024 15:44:13 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"CLmVp4d3\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1723038253;\n\tbh=A4TNAIAg1eY8OtuAwT8TeeDQWOcvp7u/yE3z7LRoXGg=;\n\th=Date:Subject:To:Cc:References:From:In-Reply-To:From;\n\tb=CLmVp4d3mcXABY39S537FXn8/OHIrgymHx9/VFOMAw5Bex1o6lYAGQySL8RdoBcgA\n\t3K+N+vLe084TWAx22ZPwJLhaAhfGFFYIvE7tXuEyvH9vDkwlMBx8FhkGHrnEWBlq2s\n\tuPq4mrsQtlSRc5LY0Mju5iL6B7p78FCmtm17OqHA=","Message-ID":"","Date":"Wed, 7 Aug 2024 14:45:03 +0100","MIME-Version":"1.0","User-Agent":"Mozilla Thunderbird","Subject":"Re: [PATCH v5 16/18] Documentation: Split public/private\n\tdocumentation","To":"Laurent Pinchart ,\n\tKieran Bingham ","Cc":"libcamera-devel@lists.libcamera.org","References":"<20240805143654.20870-1-laurent.pinchart@ideasonboard.com>\n\t<20240805143654.20870-17-laurent.pinchart@ideasonboard.com>\n\t<1c24fdf8-f01d-4e46-881d-442c75db3024@ideasonboard.com>\n\t<172303113182.1687952.3559570839316346035@ping.linuxembedded.co.uk>\n\t<20240807133113.GH15173@pendragon.ideasonboard.com>","Content-Language":"en-US","From":"Dan Scally ","Autocrypt":"addr=dan.scally@ideasonboard.com; keydata=\n\txsFNBGLydlEBEADa5O2s0AbUguprfvXOQun/0a8y2Vk6BqkQALgeD6KnXSWwaoCULp18etYW\n\tB31bfgrdphXQ5kUQibB0ADK8DERB4wrzrUb5CMxLBFE7mQty+v5NsP0OFNK9XTaAOcmD+Ove\n\teIjYvqurAaro91jrRVrS1gBRxIFqyPgNvwwL+alMZhn3/2jU2uvBmuRrgnc/e9cHKiuT3Dtq\n\tMHGPKL2m+plk+7tjMoQFfexoQ1JKugHAjxAhJfrkXh6uS6rc01bYCyo7ybzg53m1HLFJdNGX\n\tsUKR+dQpBs3SY4s66tc1sREJqdYyTsSZf80HjIeJjU/hRunRo4NjRIJwhvnK1GyjOvvuCKVU\n\tRWpY8dNjNu5OeAfdrlvFJOxIE9M8JuYCQTMULqd1NuzbpFMjc9524U3Cngs589T7qUMPb1H1\n\tNTA81LmtJ6Y+IV5/kiTUANflpzBwhu18Ok7kGyCq2a2jsOcVmk8gZNs04gyjuj8JziYwwLbf\n\tvzABwpFVcS8aR+nHIZV1HtOzyw8CsL8OySc3K9y+Y0NRpziMRvutrppzgyMb9V+N31mK9Mxl\n\t1YkgaTl4ciNWpdfUe0yxH03OCuHi3922qhPLF4XX5LN+NaVw5Xz2o3eeWklXdouxwV7QlN33\n\tu4+u2FWzKxDqO6WLQGjxPE0mVB4Gh5Pa1Vb0ct9Ctg0qElvtGQARAQABzShEYW4gU2NhbGx5\n\tIDxkYW4uc2NhbGx5QGlkZWFzb25ib2FyZC5jb20+wsGNBBMBCAA3FiEEsdtt8OWP7+8SNfQe\n\tkiQuh/L+GMQFAmLydlIFCQWjmoACGwMECwkIBwUVCAkKCwUWAgMBAAAKCRCSJC6H8v4YxDI2\n\tEAC2Gz0iyaXJkPInyshrREEWbo0CA6v5KKf3I/HlMPqkZ48bmGoYm4mEQGFWZJAT3K4ir8bg\n\tcEfs9V54gpbrZvdwS4abXbUK4WjKwEs8HK3XJv1WXUN2bsz5oEJWZUImh9gD3naiLLI9QMMm\n\tw/aZkT+NbN5/2KvChRWhdcha7+2Te4foOY66nIM+pw2FZM6zIkInLLUik2zXOhaZtqdeJZQi\n\tHSPU9xu7TRYN4cvdZAnSpG7gQqmLm5/uGZN1/sB3kHTustQtSXKMaIcD/DMNI3JN/t+RJVS7\n\tc0Jh/ThzTmhHyhxx3DRnDIy7kwMI4CFvmhkVC2uNs9kWsj1DuX5kt8513mvfw2OcX9UnNKmZ\n\tnhNCuF6DxVrL8wjOPuIpiEj3V+K7DFF1Cxw1/yrLs8dYdYh8T8vCY2CHBMsqpESROnTazboh\n\tAiQ2xMN1cyXtX11Qwqm5U3sykpLbx2BcmUUUEAKNsM//Zn81QXKG8vOx0ZdMfnzsCaCzt8f6\n\t9dcDBBI3tJ0BI9ByiocqUoL6759LM8qm18x3FYlxvuOs4wSGPfRVaA4yh0pgI+ModVC2Pu3y\n\tejE/IxeatGqJHh6Y+iJzskdi27uFkRixl7YJZvPJAbEn7kzSi98u/5ReEA8Qhc8KO/B7wprj\n\txjNMZNYd0Eth8+WkixHYj752NT5qshKJXcyUU87BTQRi8nZSARAAx0BJayh1Fhwbf4zoY56x\n\txHEpT6DwdTAYAetd3yiKClLVJadYxOpuqyWa1bdfQWPb+h4MeXbWw/53PBgn7gI2EA7ebIRC\n\tPJJhAIkeym7hHZoxqDQTGDJjxFEL11qF+U3rhWiL2Zt0Pl+zFq0eWYYVNiXjsIS4FI2+4m16\n\ttPbDWZFJnSZ828VGtRDQdhXfx3zyVX21lVx1bX4/OZvIET7sVUufkE4hrbqrrufre7wsjD1t\n\t8MQKSapVrr1RltpzPpScdoxknOSBRwOvpp57pJJe5A0L7+WxJ+vQoQXj0j+5tmIWOAV1qBQp\n\thyoyUk9JpPfntk2EKnZHWaApFp5TcL6c5LhUvV7F6XwOjGPuGlZQCWXee9dr7zym8iR3irWT\n\t+49bIh5PMlqSLXJDYbuyFQHFxoiNdVvvf7etvGfqFYVMPVjipqfEQ38ST2nkzx+KBICz7uwj\n\tJwLBdTXzGFKHQNckGMl7F5QdO/35An/QcxBnHVMXqaSd12tkJmoRVWduwuuoFfkTY5mUV3uX\n\txGj3iVCK4V+ezOYA7c2YolfRCNMTza6vcK/P4tDjjsyBBZrCCzhBvd4VVsnnlZhVaIxoky4K\n\taL+AP+zcQrUZmXmgZjXOLryGnsaeoVrIFyrU6ly90s1y3KLoPsDaTBMtnOdwxPmo1xisH8oL\n\ta/VRgpFBfojLPxMAEQEAAcLBfAQYAQgAJhYhBLHbbfDlj+/vEjX0HpIkLofy/hjEBQJi8nZT\n\tBQkFo5qAAhsMAAoJEJIkLofy/hjEXPcQAMIPNqiWiz/HKu9W4QIf1OMUpKn3YkVIj3p3gvfM\n\tRes4fGX94Ji599uLNrPoxKyaytC4R6BTxVriTJjWK8mbo9jZIRM4vkwkZZ2bu98EweSucxbp\n\tvjESsvMXGgxniqV/RQ/3T7LABYRoIUutARYq58p5HwSP0frF0fdFHYdTa2g7MYZl1ur2JzOC\n\tFHRpGadlNzKDE3fEdoMobxHB3Lm6FDml5GyBAA8+dQYVI0oDwJ3gpZPZ0J5Vx9RbqXe8RDuR\n\tdu90hvCJkq7/tzSQ0GeD3BwXb9/R/A4dVXhaDd91Q1qQXidI+2jwhx8iqiYxbT+DoAUkQRQy\n\txBtoCM1CxH7u45URUgD//fxYr3D4B1SlonA6vdaEdHZOGwECnDpTxecENMbz/Bx7qfrmd901\n\tD+N9SjIwrbVhhSyUXYnSUb8F+9g2RDY42Sk7GcYxIeON4VzKqWM7hpkXZ47pkK0YodO+dRKM\n\tyMcoUWrTK0Uz6UzUGKoJVbxmSW/EJLEGoI5p3NWxWtScEVv8mO49gqQdrRIOheZycDmHnItt\n\t9Qjv00uFhEwv2YfiyGk6iGF2W40s2pH2t6oeuGgmiZ7g6d0MEK8Ql/4zPItvr1c1rpwpXUC1\n\tu1kQWgtnNjFHX3KiYdqjcZeRBiry1X0zY+4Y24wUU0KsEewJwjhmCKAsju1RpdlPg2kC","In-Reply-To":"<20240807133113.GH15173@pendragon.ideasonboard.com>","Content-Type":"text/plain; charset=UTF-8; format=flowed","Content-Transfer-Encoding":"7bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":30684,"web_url":"https://patchwork.libcamera.org/comment/30684/","msgid":"<20240807135119.GA18866@pendragon.ideasonboard.com>","date":"2024-08-07T13:51:19","subject":"Re: [PATCH v5 16/18] Documentation: Split public/private\n\tdocumentation","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"On Wed, Aug 07, 2024 at 02:45:03PM +0100, Daniel Scally wrote:\n> Hi Laurent\n> \n> On 07/08/2024 14:31, Laurent Pinchart wrote:\n> > On Wed, Aug 07, 2024 at 12:45:31PM +0100, Kieran Bingham wrote:\n> >> Quoting Dan Scally (2024-08-07 09:30:22)\n> >>> On 05/08/2024 15:36, Laurent Pinchart wrote:\n> >>>> From: Daniel Scally \n> >>>>\n> >>>> The API reference pages generated by Doxygen are comprehensive, but\n> >>>> therefore quite overwhelming for application developers who will\n> >>>> likely never need to use the majority of the library's objects. To\n> >>>> reduce the complexity of the documentation, split it into two runs of\n> >>>> doxygen.\n> >>>>\n> >>>> The first run of doxygen is for the public API. We pass a specific list\n> >>>> of source files to parse, which is built from the arrays of public\n> >>>> headers and sources in meson build files. This ensures that we only\n> >>>> generate the documentation for code from those files.\n> >>>>\n> >>>> A custom Python script is needed to add the list of input files to\n> >>>> Doxyfile, as several of the objects included in the header and source\n> >>>> array are custom_tgt objects, which can't be handled as strings to\n> >>>> populate a variable in the configuration data.\n> >>>>\n> >>>> The headers defining the Extensible and Object classes (class.h and\n> >>>> object.h respectively), as well as the corresponding source files, are\n> >>>> excluded from the public API documentation despite being referenced in\n> >>>> the meson public headers and sources arrays. This is due to the fact\n> >>>> that public API classes inherit from Extensible and Object, making the\n> >>>> Extensible and Object classes part of the public ABI. Those two base\n> >>>> classes are however implementation details and must not be accessed\n> >>>> directly by application code.\n> >>>>\n> >>>> The second run of doxygen is for the internal API. This contains\n> >>>> documentation for all of the library's objects as it currently does.\n> >>>> This set will now be output into build/Documentation/internal-api-html.\n> >> Sounds good.\n> >>\n> >>>> Signed-off-by: Daniel Scally \n> >>>> Co-developed-by: Laurent Pinchart \n> >>>> Signed-off-by: Laurent Pinchart \n> >>>> ---\n> >>> The python script to generate the list of input files is definitely better than scattered foreach\n> >>> loops collecting a list in a hacky way - good improvement.\n> >>>\n> >>>> .../{Doxyfile.in => Doxyfile-common.in} | 24 -------\n> >>>> Documentation/Doxyfile-internal.in | 30 ++++++++\n> >>>> Documentation/Doxyfile-public.in | 20 ++++++\n> >>>> Documentation/gen-doxyfile.py | 46 ++++++++++++\n> >>>> Documentation/meson.build | 70 +++++++++++++++++--\n> >>>> 5 files changed, 160 insertions(+), 30 deletions(-)\n> >>>> rename Documentation/{Doxyfile.in => Doxyfile-common.in} (62%)\n> >>>> create mode 100644 Documentation/Doxyfile-internal.in\n> >>>> create mode 100644 Documentation/Doxyfile-public.in\n> >>>> create mode 100755 Documentation/gen-doxyfile.py\n> >>>>\n> >>>> diff --git a/Documentation/Doxyfile.in b/Documentation/Doxyfile-common.in\n> >>>> similarity index 62%\n> >>>> rename from Documentation/Doxyfile.in\n> >>>> rename to Documentation/Doxyfile-common.in\n> >>>> index 6e5a3830ec38..a70aee430e77 100644\n> >>>> --- a/Documentation/Doxyfile.in\n> >>>> +++ b/Documentation/Doxyfile-common.in\n> >>>> @@ -17,20 +17,11 @@ EXTENSION_MAPPING = h=C++\n> >>>> \n> >>>> TOC_INCLUDE_HEADINGS = 0\n> >>>> \n> >>>> -INTERNAL_DOCS = YES\n> >>>> CASE_SENSE_NAMES = YES\n> >>>> \n> >>>> QUIET = YES\n> >>>> WARN_AS_ERROR = @WARN_AS_ERROR@\n> >>>> \n> >>>> -INPUT = \"@TOP_SRCDIR@/Documentation\" \\\n> >>>> - \"@TOP_SRCDIR@/include/libcamera\" \\\n> >>>> - \"@TOP_SRCDIR@/src/ipa/ipu3\" \\\n> >>>> - \"@TOP_SRCDIR@/src/ipa/libipa\" \\\n> >>>> - \"@TOP_SRCDIR@/src/libcamera\" \\\n> >>>> - \"@TOP_BUILDDIR@/include/libcamera\" \\\n> >>>> - \"@TOP_BUILDDIR@/src/libcamera\"\n> >>>> -\n> >>>> FILE_PATTERNS = *.c \\\n> >>>> *.cpp \\\n> >>>> *.dox \\\n> >>>> @@ -38,19 +29,6 @@ FILE_PATTERNS = *.c \\\n> >>>> \n> >>>> RECURSIVE = YES\n> >>>> \n> >>>> -EXCLUDE = @TOP_SRCDIR@/include/libcamera/base/span.h \\\n> >>>> - @TOP_SRCDIR@/include/libcamera/internal/device_enumerator_sysfs.h \\\n> >>>> - @TOP_SRCDIR@/include/libcamera/internal/device_enumerator_udev.h \\\n> >>>> - @TOP_SRCDIR@/include/libcamera/internal/ipc_pipe_unixsocket.h \\\n> >>>> - @TOP_SRCDIR@/src/libcamera/device_enumerator_sysfs.cpp \\\n> >>>> - @TOP_SRCDIR@/src/libcamera/device_enumerator_udev.cpp \\\n> >>>> - @TOP_SRCDIR@/src/libcamera/ipc_pipe_unixsocket.cpp \\\n> >>>> - @TOP_SRCDIR@/src/libcamera/pipeline/ \\\n> >>>> - @TOP_SRCDIR@/src/libcamera/tracepoints.cpp \\\n> >>>> - @TOP_BUILDDIR@/include/libcamera/internal/tracepoints.h \\\n> >>>> - @TOP_BUILDDIR@/include/libcamera/ipa/soft_ipa_interface.h \\\n> >>>> - @TOP_BUILDDIR@/src/libcamera/proxy/\n> >>>> -\n> >>>> EXCLUDE_PATTERNS = @TOP_BUILDDIR@/include/libcamera/ipa/*_serializer.h \\\n> >>>> @TOP_BUILDDIR@/include/libcamera/ipa/*_proxy.h \\\n> >>>> @TOP_BUILDDIR@/include/libcamera/ipa/ipu3_*.h \\\n> >>>> @@ -73,8 +51,6 @@ EXCLUDE_SYMBOLS = libcamera::BoundMethodArgs \\\n> >>>> \n> >>>> EXCLUDE_SYMLINKS = YES\n> >>>> \n> >>>> -HTML_OUTPUT = api-html\n> >>>> -\n> >>>> GENERATE_LATEX = NO\n> >>>> \n> >>>> MACRO_EXPANSION = YES\n> >>>> diff --git a/Documentation/Doxyfile-internal.in b/Documentation/Doxyfile-internal.in\n> >>>> new file mode 100644\n> >>>> index 000000000000..5e0310091416\n> >>>> --- /dev/null\n> >>>> +++ b/Documentation/Doxyfile-internal.in\n> >>>> @@ -0,0 +1,30 @@\n> >>>> +# SPDX-License-Identifier: CC-BY-SA-4.0\n> >>>> +\n> >>>> +@INCLUDE_PATH = @TOP_BUILDDIR@/Documentation\n> >>>> +@INCLUDE = Doxyfile-common\n> >>>> +\n> >>>> +HIDE_UNDOC_CLASSES = NO\n> >>>> +HIDE_UNDOC_MEMBERS = NO\n> >>>> +HTML_OUTPUT = internal-api-html\n> >>>> +INTERNAL_DOCS = YES\n> >>>> +\n> >>>> +INPUT = \"@TOP_SRCDIR@/Documentation\" \\\n> >>>> + \"@TOP_SRCDIR@/include/libcamera\" \\\n> >>>> + \"@TOP_SRCDIR@/src/ipa/ipu3\" \\\n> >>>> + \"@TOP_SRCDIR@/src/ipa/libipa\" \\\n> >>>> + \"@TOP_SRCDIR@/src/libcamera\" \\\n> >>>> + \"@TOP_BUILDDIR@/include/libcamera\" \\\n> >>>> + \"@TOP_BUILDDIR@/src/libcamera\"\n> >>>> +\n> >>>> +EXCLUDE = @TOP_SRCDIR@/include/libcamera/base/span.h \\\n> >>>> + @TOP_SRCDIR@/include/libcamera/internal/device_enumerator_sysfs.h \\\n> >>>> + @TOP_SRCDIR@/include/libcamera/internal/device_enumerator_udev.h \\\n> >>>> + @TOP_SRCDIR@/include/libcamera/internal/ipc_pipe_unixsocket.h \\\n> >>>> + @TOP_SRCDIR@/src/libcamera/device_enumerator_sysfs.cpp \\\n> >>>> + @TOP_SRCDIR@/src/libcamera/device_enumerator_udev.cpp \\\n> >>>> + @TOP_SRCDIR@/src/libcamera/ipc_pipe_unixsocket.cpp \\\n> >>>> + @TOP_SRCDIR@/src/libcamera/pipeline/ \\\n> >>>> + @TOP_SRCDIR@/src/libcamera/tracepoints.cpp \\\n> >>>> + @TOP_BUILDDIR@/include/libcamera/internal/tracepoints.h \\\n> >>>> + @TOP_BUILDDIR@/include/libcamera/ipa/soft_ipa_interface.h \\\n> >>>> + @TOP_BUILDDIR@/src/libcamera/proxy/\n> >>>> diff --git a/Documentation/Doxyfile-public.in b/Documentation/Doxyfile-public.in\n> >>>> new file mode 100644\n> >>>> index 000000000000..36bb57584a07\n> >>>> --- /dev/null\n> >>>> +++ b/Documentation/Doxyfile-public.in\n> >>>> @@ -0,0 +1,20 @@\n> >>>> +# SPDX-License-Identifier: CC-BY-SA-4.0\n> >>>> +\n> >>>> +@INCLUDE_PATH = @TOP_BUILDDIR@/Documentation\n> >>>> +@INCLUDE = Doxyfile-common\n> >>>> +\n> >>>> +HIDE_UNDOC_CLASSES = YES\n> >>>> +HIDE_UNDOC_MEMBERS = YES\n> >>>> +HTML_OUTPUT = api-html\n> >>>> +INTERNAL_DOCS = NO\n> >>>> +\n> >>>> +INPUT = \"@TOP_SRCDIR@/Documentation\" \\\n> >>>> + ${inputs}\n> >>>> +\n> >>>> +EXCLUDE = @TOP_SRCDIR@/include/libcamera/base/class.h \\\n> >>>> + @TOP_SRCDIR@/include/libcamera/base/object.h \\\n> >>>> + @TOP_SRCDIR@/include/libcamera/base/span.h \\\n> >>>> + @TOP_SRCDIR@/src/libcamera/base/class.cpp \\\n> >>>> + @TOP_SRCDIR@/src/libcamera/base/object.cpp\n> >>>> +\n> >>>> +PREDEFINED += __DOXYGEN_PUBLIC__\n> >>>> diff --git a/Documentation/gen-doxyfile.py b/Documentation/gen-doxyfile.py\n> >>>> new file mode 100755\n> >>>> index 000000000000..c2a4184dd195\n> >>>> --- /dev/null\n> >>>> +++ b/Documentation/gen-doxyfile.py\n> >>>> @@ -0,0 +1,46 @@\n> >>>> +#!/usr/bin/env python3\n> >>>> +# SPDX-License-Identifier: GPL-2.0-or-later\n> >>>> +# Copyright (C) 2024, Google Inc.\n> >>> Does this mean it's drawn from elsewhere? Or is it a copy-paste error?\n> >>>\n> >>>> +#\n> >>>> +# Author: Laurent Pinchart \n> >>>> +#\n> >>>> +# Generate Doxyfile from a template\n> >>>> +\n> >>>> +import argparse\n> >>>> +import os\n> >>>> +import string\n> >>>> +import sys\n> >>>> +\n> >>>> +\n> >>>> +def fill_template(template, data):\n> >>>> +\n> >>>> + template = open(template, 'rb').read()\n> >>>> + template = template.decode('utf-8')\n> >>>> + template = string.Template(template)\n> >>> I'd add a line break here.\n> > Done.\n> >\n> >>>> + return template.substitute(data)\n> >>>> +\n> >>>> +\n> >>>> +def main(argv):\n> >>>> +\n> >>>> + # Parse command line arguments\n> >>> The comment can probably be skipped.\n> > I'm fine with that.\n> >\n> >\n> >>>> + parser = argparse.ArgumentParser()\n> >>>> + parser.add_argument('-o', dest='output', metavar='file',\n> >>>> + type=argparse.FileType('w', encoding='utf-8'),\n> >>>> + default=sys.stdout,\n> >>>> + help='Output file name (default: standard output)')\n> >>>> + parser.add_argument('template', metavar='doxyfile.tmpl', type=str,\n> >>>> + help='Doxyfile template')\n> >>>> + parser.add_argument('inputs', type=str, nargs='*',\n> >>>> + help='Input files')\n> >>>> +\n> >>>> + args = parser.parse_args(argv[1:])\n> >>>> +\n> >>>> + inputs = [f'\"{os.path.realpath(input)}\"' for input in args.inputs]\n> >>>> + data = fill_template(args.template, {'inputs': (' \\\\\\n' + ' ' * 25).join(inputs)})\n> >>>> + args.output.write(data)\n> >>>> +\n> >>>> + return 0\n> >>>> +\n> >>>> +\n> >>>> +if __name__ == '__main__':\n> >>>> + sys.exit(main(sys.argv))\n> >>>> diff --git a/Documentation/meson.build b/Documentation/meson.build\n> >>>> index 1d84ed815b50..bf28233a2ce8 100644\n> >>>> --- a/Documentation/meson.build\n> >>>> +++ b/Documentation/meson.build\n> >>>> @@ -24,9 +24,9 @@ if doxygen.found() and dot.found()\n> >>>> \n> >>>> cdata.set('PREDEFINED', ' \\\\\\n\\t\\t\\t '.join(doxygen_predefined))\n> >>>> \n> >>>> - doxyfile = configure_file(input : 'Doxyfile.in',\n> >>>> - output : 'Doxyfile',\n> >>>> - configuration : cdata)\n> >>>> + doxyfile_common = configure_file(input : 'Doxyfile-common.in',\n> >>>> + output : 'Doxyfile-common',\n> >>>> + configuration : cdata)\n> >>>> \n> >>>> doxygen_public_input = [\n> >>>> libcamera_base_public_headers,\n> >>>> @@ -50,17 +50,75 @@ if doxygen.found() and dot.found()\n> >>>> doxygen_internal_input += [ipu3_ipa_sources]\n> >>>> endif\n> >>>> \n> >>>> - custom_target('doxygen',\n> >>>> + # We run doxygen twice - the first run excludes internal API objects as it\n> >>>> + # is intended to document the public API only. A second run covers all of\n> >>>> + # the library's objects for libcamera developers. Common configuration is\n> >>>> + # set in an initially generated Doxyfile, which is then included by the two\n> >>>> + # final Doxyfiles. We collected a list of the public sources in\n> >>>> + # doxygen_public_sources to pass to the public run and include it in cdata\n> >>>> + # here - although both Doxyfile and Doxyfile-internal will also receive\n> >>>> + # the same parameter they simply ignore it.\n> >>> This comment needs correcting now as it references the older method.\n> > I'll fix it.\n> >\n> >>>> +\n> >>>> + # This is the \"public\" run of doxygen generating an abridged version of the\n> >>>> + # API's documentation.\n> >>>> +\n> >>>> + doxyfile_tmpl = configure_file(input : 'Doxyfile-public.in',\n> >>>> + output : 'Doxyfile-public.tmpl',\n> >>>> + configuration : cdata)\n> >>>> +\n> >>>> + doxyfile = custom_target('doxyfile-public',\n> >>>> + input : [\n> >>>> + doxygen_public_input,\n> >>>> + ],\n> >>>> + output : 'Doxyfile-public',\n> >>>> + command : [\n> >>>> + 'gen-doxyfile.py',\n> >>>> + '-o', '@OUTPUT@',\n> >>>> + doxyfile_tmpl,\n> >>>> + '@INPUT@',\n> >>>> + ])\n> >>>> +\n> >>>> + custom_target('doxygen-public',\n> >>>> input : [\n> >>>> doxyfile,\n> >>>> - doxygen_public_input,\n> >>>> - doxygen_internal_input,\n> >>>> + doxyfile_common,\n> >> At least in my mail client - the indentation looks odd here and might\n> >> want checking.\n> > It looks fine in the source code as far as I can tell. Can you fix your\n> > mail client ? :-)\n> >\n> >>>> ],\n> >>>> output : 'api-html',\n> >>>> command : [doxygen, doxyfile],\n> >>>> install : true,\n> >>>> install_dir : doc_install_dir,\n> >>>> install_tag : 'doc')\n> >>>> +\n> >>>> + # This is the internal documentation, which hard-codes a list of directories\n> >>>> + # to parse in its doxyfile.\n> >>>> +\n> >>>> + doxyfile_tmpl = configure_file(input : 'Doxyfile-internal.in',\n> >>>> + output : 'Doxyfile-internal.tmpl',\n> >>>> + configuration : cdata)\n> >>>> +\n> >>>> + doxyfile = custom_target('doxyfile-internal',\n> >>>> + input : [\n> >>>> + doxygen_public_input,\n> >>>> + doxygen_internal_input,\n> >>>> + ],\n> >>>> + output : 'Doxyfile-internal',\n> >>>> + command : [\n> >>>> + 'gen-doxyfile.py',\n> >>>> + '-o', '@OUTPUT@',\n> >>>> + doxyfile_tmpl,\n> >>>> + '@INPUT@',\n> >>>> + ])\n> >>> I think we can skip the script for the internal run -there's no\n> >>> templated values to be replaced and the generated Doxyfile-internal\n> >>> is identical to the Doxyfile-internal.tmpl.\n> >> I wonder if that's \"future proofing\" but I don't mind either way.\n> >\n> > The goal was to keep the code flow identical, and I considered\n> > future-proofing too. I don't mind dropping it though.\n> \n> Fair enough; leave it in then (unless you've already dropped it...I also don't mind either way so \n> whatever state it's currently in, is fine).\n\nDone already :-)\n\n> Is an R-b from me appropriate here given I'm down as the author? If so:\n\nI think it is, to record you've reviewed the changes I've made. This is\nwhy I try to sort the trailers in chronological order, and add the\nreviewed-by tags after my SoB line.\n\n> Reviewed-by: Daniel Scally \n> \n> >> I have nothing much to add except a potential whitespace comment on top\n> >> of Dan's comments, so with those handled:\n> >>\n> >> Reviewed-by: Kieran Bingham \n> >>\n> >>>> +\n> >>>> + custom_target('doxygen-internal',\n> >>>> + input : [\n> >>>> + doxyfile,\n> >>>> + doxyfile_common,\n> >>>> + ],\n> >>>> + output : 'internal-api-html',\n> >>>> + command : [doxygen, doxyfile],\n> >>>> + install : true,\n> >>>> + install_dir : doc_install_dir,\n> >>>> + install_tag : 'doc-internal')\n> >>>> endif\n> >>>> \n> >>>> #","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 5A28FBE173\n\tfor ;\n\tWed, 7 Aug 2024 13:51:45 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id A3EE863393;\n\tWed, 7 Aug 2024 15:51:44 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 198156337F\n\tfor ;\n\tWed, 7 Aug 2024 15:51:43 +0200 (CEST)","from pendragon.ideasonboard.com (81-175-209-231.bb.dnainternet.fi\n\t[81.175.209.231])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 1A3C96AF;\n\tWed, 7 Aug 2024 15:50:50 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"rBBXCKEn\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1723038650;\n\tbh=7E76FmD+0JkqaT9C+PAHXBzHR5mHlL/rZqpPsm2ErGk=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=rBBXCKEnmK6qFqym1k/J2e3GOY3M6ZCw5COmGNcYLyTedLEMuAPXZs9qpgJSwTSpq\n\tMD9C9bko1r3toXap3ppW3EPrZwBCRnM0/hPGH/Umb5FtN9srw2TPd9TQRDZCFpBVd2\n\tfWhBhRNanvAwOZvgIMegNWWhfwDWiNIk4fkkEEPE=","Date":"Wed, 7 Aug 2024 16:51:19 +0300","From":"Laurent Pinchart ","To":"Dan Scally ","Cc":"Kieran Bingham ,\n\tlibcamera-devel@lists.libcamera.org","Subject":"Re: [PATCH v5 16/18] Documentation: Split public/private\n\tdocumentation","Message-ID":"<20240807135119.GA18866@pendragon.ideasonboard.com>","References":"<20240805143654.20870-1-laurent.pinchart@ideasonboard.com>\n\t<20240805143654.20870-17-laurent.pinchart@ideasonboard.com>\n\t<1c24fdf8-f01d-4e46-881d-442c75db3024@ideasonboard.com>\n\t<172303113182.1687952.3559570839316346035@ping.linuxembedded.co.uk>\n\t<20240807133113.GH15173@pendragon.ideasonboard.com>\n\t","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]