| Message ID | 20250911230115.25335-7-laurent.pinchart@ideasonboard.com |
|---|---|
| State | Superseded |
| Headers | show |
| Series |
|
| Related | show |
2025. 09. 12. 1:01 keltezéssel, Laurent Pinchart írta: > The libcamera documentation is made of high-level documentation and > guides, written as ReStructuredText and compiled to HTML by Sphinx, and > API reference documentation, written as comments in the code and > compiled to HTML by Doxygen. > > Due to meson limitations that make it difficult to place output files in > subdirectories, the compilation process produces an html/ directory for > the Sphinx documentation, and api-html/ and internal-api-html/ > directories for the Doxygen documentation. When deploying the > documentation to the libcamera.org website, the api-html and > internal-api-html/ directories are moved within html/ to make the > documentation self-contained. > > The Sphinx and Doxygen documentation link to each other. The links are > generated using relative paths, based on the directory hierarchy on the > website. This makes them broken when navigating the documentation in the > build tree or in the directory where libcamera is installed. > > Fix this by standardizing on the directories hierarchy of the website in > the build and install directories: > > - For the build directory, we can't easily build the Doxygen > documentation in a subdirectory of the Sphinx documentation due to > limitations of meson. Keep the existing output directories, and > replace the html/api-html/ and html/internal-api-html/ placeholder > directories with symlinks to the Doxygen output directories. This is > handled by a small custom Sphinx extension. > > - For the install directory, install the Doxygen documentation to > html/api-html/ and html/internal-api-html/. This overwrites the > placeholders. Does it need a special meson option? Installation fails for me: Tried to copy file /tmp/test/usr/share/doc/libcamera-0.5.2/html/internal-api but a directory of that name already exists. Regards, Barnabás Pőcze > > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> > --- > Changes since v1: > > - Split from "[PATCH 4/6] documentation: Include API docs in the sphinx > documentation" > - Avoid copying the doxygen documentation > - Use a sphinx extension to generate symlinks > --- > Documentation/conf.py.in | 26 +++++---- > Documentation/extensions/doxygen-link.py | 73 ++++++++++++++++++++++++ > Documentation/meson.build | 19 ++++-- > 3 files changed, 104 insertions(+), 14 deletions(-) > create mode 100644 Documentation/extensions/doxygen-link.py > > diff --git a/Documentation/conf.py.in b/Documentation/conf.py.in > index 34fa3956f49e..5e0a7cff2984 100644 > --- a/Documentation/conf.py.in > +++ b/Documentation/conf.py.in > @@ -9,14 +9,8 @@ > > # -- Path setup -------------------------------------------------------------- > > -# If extensions (or modules to document with autodoc) are in another directory, > -# add these directories to sys.path here. If the directory is relative to the > -# documentation root, use os.path.abspath to make it absolute, like shown here. > -# > -# import os > -# import sys > -# sys.path.insert(0, os.path.abspath('.')) > - > +import sys > +sys.path.insert(0, "@CURRENT_SRCDIR@/extensions") > > # -- Project information ----------------------------------------------------- > > @@ -39,6 +33,7 @@ author = 'The libcamera documentation authors' > extensions = [ > 'sphinx.ext.graphviz', > 'sphinxcontrib.doxylink', > + 'doxygen-link', > ] > > graphviz_output_format = 'svg' > @@ -75,14 +70,25 @@ pygments_style = None > doxylink = { > 'doxy-pub': ( > '@TOP_BUILDDIR@/Documentation/api-html/tagfile.xml', > - '../api-html/', > + 'api-html/', > ), > 'doxy-int': ( > '@TOP_BUILDDIR@/Documentation/internal-api-html/tagfile.xml', > - '../internal-api-html/', > + 'internal-api-html/', > ), > } > > +doxygen_links = [ > + [ > + '@TOP_BUILDDIR@/Documentation/html/api-html', > + '@TOP_BUILDDIR@/Documentation/api-html', > + ], > + [ > + '@TOP_BUILDDIR@/Documentation/html/internal-api-html', > + '@TOP_BUILDDIR@/Documentation/internal-api-html', > + ], > +] > + > # -- Options for HTML output ------------------------------------------------- > > # The theme to use for HTML and HTML Help pages. See the documentation for > diff --git a/Documentation/extensions/doxygen-link.py b/Documentation/extensions/doxygen-link.py > new file mode 100644 > index 000000000000..8b94a4dd7f0b > --- /dev/null > +++ b/Documentation/extensions/doxygen-link.py > @@ -0,0 +1,73 @@ > +# SPDX-License-Identifier: GPL-2.0-or-later > +# Copyright (C) 2025, Ideas on Board Oy > + > +import contextlib > +import os > +import shutil > +from pathlib import Path > +from sphinx.util import logging > + > +__version__ = "0.0.0" > + > +logger = logging.getLogger(__name__) > + > + > +def on_config_inited(app, config): > + entries = [] > + > + outdir = Path(app.outdir).absolute() > + > + for index, items in enumerate(config.doxygen_links): > + err_msg_prefix = f'Config variable `doxygen_links` contains invalid entry {index} (`{items}`)' > + > + if len(items) != 2: > + raise ValueError(f'{err_msg_prefix}: expected (path, target)') > + > + path = Path(items[0]).absolute() > + target = Path(items[1]).relative_to(path.parent, walk_up=True) > + > + if not path.is_relative_to(outdir): > + raise ValueError(f'{err_msg_prefix}: path `{items[0]}` is outside of output directory `{outdir}`') > + > + entries.append([path, target]) > + > + config.doxygen_links = entries > + > + > +def on_builder_inited(app): > + # Remove the symlinks if they exist, to avoid overwriting the index.html > + # generated by Doxygen with the placeholder index from Sphinx. > + > + for path, target in app.config.doxygen_links: > + if path.is_symlink(): > + logger.info(f'Removing existing symlink {path}') > + os.unlink(path) > + > + > +def on_build_finished(app, exc): > + # Create the symlinks. Remove any existing placeholder directory > + # recursively first. > + > + if exc: > + return > + > + for path, target in app.config.doxygen_links: > + logger.info(f'Creating symlink {path} -> {target}') > + > + if path.is_dir(): > + shutil.rmtree(path) > + > + os.symlink(target, path) > + > + > +def setup(app): > + app.add_config_value('doxygen_links', [], 'env', frozenset({list, tuple})) > + app.connect('config-inited', on_config_inited) > + app.connect('builder-inited', on_builder_inited) > + app.connect('build-finished', on_build_finished) > + > + return { > + "version": __version__, > + "parallel_read_safe": True, > + "parallel_write_safe": True, > + } > diff --git a/Documentation/meson.build b/Documentation/meson.build > index f73407432fff..82d76b257b8c 100644 > --- a/Documentation/meson.build > +++ b/Documentation/meson.build > @@ -12,6 +12,7 @@ dot = find_program('dot', required : get_option('documentation')) > if doxygen.found() and dot.found() > cdata = configuration_data() > cdata.set('VERSION', 'v@0@'.format(libcamera_git_version)) > + cdata.set('CURRENT_SRCDIR', meson.current_source_dir()) > cdata.set('TOP_SRCDIR', meson.project_source_root()) > cdata.set('TOP_BUILDDIR', meson.project_build_root()) > cdata.set('OUTPUT_DIR', meson.current_build_dir()) > @@ -89,7 +90,7 @@ if doxygen.found() and dot.found() > output : 'api-html', > command : [doxygen, doxyfile], > install : true, > - install_dir : doc_install_dir, > + install_dir : doc_install_dir / 'html', > install_tag : 'doc') > > # This is the internal documentation, which hard-codes a list of directories > @@ -109,7 +110,7 @@ if doxygen.found() and dot.found() > output : 'internal-api-html', > command : [doxygen, doxyfile], > install : true, > - install_dir : doc_install_dir, > + install_dir : doc_install_dir / 'html', > install_tag : 'doc-internal') > endif > > @@ -149,7 +150,11 @@ if sphinx.found() > fs = import('fs') > sphinx_conf_dir = fs.parent(sphinx_conf) > > + sphinx_env = environment() > + sphinx_env.set('PYTHONDONTWRITEBYTECODE', '1') > + > docs_sources = [ > + 'api-html/index.rst', > 'camera-sensor-model.rst', > 'code-of-conduct.rst', > 'coding-style.rst', > @@ -164,6 +169,7 @@ if sphinx.found() > 'guides/pipeline-handler.rst', > 'guides/tracing.rst', > 'index.rst', > + 'internal-api-html/index.rst', > 'introduction.rst', > 'lens_driver_requirements.rst', > 'libcamera_architecture.rst', > @@ -183,10 +189,14 @@ if sphinx.found() > input : docs_sources, > output : 'html', > build_by_default : true, > + depend_files : [ > + 'extensions/doxygen-link.py', > + ], > depends : [ > doxygen_public, > doxygen_internal, > ], > + env : sphinx_env, > install : true, > install_dir : doc_install_dir, > install_tag : 'doc') > @@ -195,7 +205,8 @@ if sphinx.found() > command : [sphinx, '-W', '-b', 'linkcheck', > '-c', sphinx_conf_dir, > meson.current_source_dir(), '@OUTPUT@'], > - build_always_stale : true, > input : docs_sources, > - output : 'linkcheck') > + output : 'linkcheck', > + build_always_stale : true, > + env : sphinx_env) > endif
On Fri, Sep 12, 2025 at 02:01:07AM +0300, Laurent Pinchart wrote: > The libcamera documentation is made of high-level documentation and > guides, written as ReStructuredText and compiled to HTML by Sphinx, and > API reference documentation, written as comments in the code and > compiled to HTML by Doxygen. > > Due to meson limitations that make it difficult to place output files in > subdirectories, the compilation process produces an html/ directory for > the Sphinx documentation, and api-html/ and internal-api-html/ > directories for the Doxygen documentation. When deploying the > documentation to the libcamera.org website, the api-html and > internal-api-html/ directories are moved within html/ to make the > documentation self-contained. > > The Sphinx and Doxygen documentation link to each other. The links are > generated using relative paths, based on the directory hierarchy on the > website. This makes them broken when navigating the documentation in the > build tree or in the directory where libcamera is installed. > > Fix this by standardizing on the directories hierarchy of the website in > the build and install directories: > > - For the build directory, we can't easily build the Doxygen > documentation in a subdirectory of the Sphinx documentation due to > limitations of meson. Keep the existing output directories, and > replace the html/api-html/ and html/internal-api-html/ placeholder > directories with symlinks to the Doxygen output directories. This is > handled by a small custom Sphinx extension. > > - For the install directory, install the Doxygen documentation to > html/api-html/ and html/internal-api-html/. This overwrites the > placeholders. > > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> > --- > Changes since v1: > > - Split from "[PATCH 4/6] documentation: Include API docs in the sphinx > documentation" > - Avoid copying the doxygen documentation > - Use a sphinx extension to generate symlinks > --- > Documentation/conf.py.in | 26 +++++---- > Documentation/extensions/doxygen-link.py | 73 ++++++++++++++++++++++++ > Documentation/meson.build | 19 ++++-- > 3 files changed, 104 insertions(+), 14 deletions(-) > create mode 100644 Documentation/extensions/doxygen-link.py > > diff --git a/Documentation/conf.py.in b/Documentation/conf.py.in > index 34fa3956f49e..5e0a7cff2984 100644 > --- a/Documentation/conf.py.in > +++ b/Documentation/conf.py.in > @@ -9,14 +9,8 @@ > > # -- Path setup -------------------------------------------------------------- > > -# If extensions (or modules to document with autodoc) are in another directory, > -# add these directories to sys.path here. If the directory is relative to the > -# documentation root, use os.path.abspath to make it absolute, like shown here. > -# > -# import os > -# import sys > -# sys.path.insert(0, os.path.abspath('.')) > - > +import sys > +sys.path.insert(0, "@CURRENT_SRCDIR@/extensions") > > # -- Project information ----------------------------------------------------- > > @@ -39,6 +33,7 @@ author = 'The libcamera documentation authors' > extensions = [ > 'sphinx.ext.graphviz', > 'sphinxcontrib.doxylink', > + 'doxygen-link', > ] > > graphviz_output_format = 'svg' > @@ -75,14 +70,25 @@ pygments_style = None > doxylink = { > 'doxy-pub': ( > '@TOP_BUILDDIR@/Documentation/api-html/tagfile.xml', > - '../api-html/', > + 'api-html/', > ), > 'doxy-int': ( > '@TOP_BUILDDIR@/Documentation/internal-api-html/tagfile.xml', > - '../internal-api-html/', > + 'internal-api-html/', > ), > } > > +doxygen_links = [ > + [ > + '@TOP_BUILDDIR@/Documentation/html/api-html', > + '@TOP_BUILDDIR@/Documentation/api-html', > + ], > + [ > + '@TOP_BUILDDIR@/Documentation/html/internal-api-html', > + '@TOP_BUILDDIR@/Documentation/internal-api-html', > + ], > +] > + > # -- Options for HTML output ------------------------------------------------- > > # The theme to use for HTML and HTML Help pages. See the documentation for > diff --git a/Documentation/extensions/doxygen-link.py b/Documentation/extensions/doxygen-link.py > new file mode 100644 > index 000000000000..8b94a4dd7f0b > --- /dev/null > +++ b/Documentation/extensions/doxygen-link.py > @@ -0,0 +1,73 @@ > +# SPDX-License-Identifier: GPL-2.0-or-later > +# Copyright (C) 2025, Ideas on Board Oy > + > +import contextlib > +import os > +import shutil > +from pathlib import Path > +from sphinx.util import logging > + > +__version__ = "0.0.0" > + > +logger = logging.getLogger(__name__) > + > + > +def on_config_inited(app, config): > + entries = [] > + > + outdir = Path(app.outdir).absolute() > + > + for index, items in enumerate(config.doxygen_links): > + err_msg_prefix = f'Config variable `doxygen_links` contains invalid entry {index} (`{items}`)' > + > + if len(items) != 2: > + raise ValueError(f'{err_msg_prefix}: expected (path, target)') > + > + path = Path(items[0]).absolute() > + target = Path(items[1]).relative_to(path.parent, walk_up=True) This fails in CI :-( Handler <function on_config_inited at 0x7fa697693e20> for event 'config-inited' threw an exception (exception: PurePath.relative_to() got an unexpected keyword argument 'walk_up') The walk_up argument to relative_to() has been added in Python 3.12. This needs to switch to os.path.relpath(). > + > + if not path.is_relative_to(outdir): > + raise ValueError(f'{err_msg_prefix}: path `{items[0]}` is outside of output directory `{outdir}`') > + > + entries.append([path, target]) > + > + config.doxygen_links = entries > + > + > +def on_builder_inited(app): > + # Remove the symlinks if they exist, to avoid overwriting the index.html > + # generated by Doxygen with the placeholder index from Sphinx. > + > + for path, target in app.config.doxygen_links: > + if path.is_symlink(): > + logger.info(f'Removing existing symlink {path}') > + os.unlink(path) > + > + > +def on_build_finished(app, exc): > + # Create the symlinks. Remove any existing placeholder directory > + # recursively first. > + > + if exc: > + return > + > + for path, target in app.config.doxygen_links: > + logger.info(f'Creating symlink {path} -> {target}') > + > + if path.is_dir(): > + shutil.rmtree(path) > + > + os.symlink(target, path) > + > + > +def setup(app): > + app.add_config_value('doxygen_links', [], 'env', frozenset({list, tuple})) > + app.connect('config-inited', on_config_inited) > + app.connect('builder-inited', on_builder_inited) > + app.connect('build-finished', on_build_finished) > + > + return { > + "version": __version__, > + "parallel_read_safe": True, > + "parallel_write_safe": True, > + } > diff --git a/Documentation/meson.build b/Documentation/meson.build > index f73407432fff..82d76b257b8c 100644 > --- a/Documentation/meson.build > +++ b/Documentation/meson.build > @@ -12,6 +12,7 @@ dot = find_program('dot', required : get_option('documentation')) > if doxygen.found() and dot.found() > cdata = configuration_data() > cdata.set('VERSION', 'v@0@'.format(libcamera_git_version)) > + cdata.set('CURRENT_SRCDIR', meson.current_source_dir()) > cdata.set('TOP_SRCDIR', meson.project_source_root()) > cdata.set('TOP_BUILDDIR', meson.project_build_root()) > cdata.set('OUTPUT_DIR', meson.current_build_dir()) > @@ -89,7 +90,7 @@ if doxygen.found() and dot.found() > output : 'api-html', > command : [doxygen, doxyfile], > install : true, > - install_dir : doc_install_dir, > + install_dir : doc_install_dir / 'html', > install_tag : 'doc') > > # This is the internal documentation, which hard-codes a list of directories > @@ -109,7 +110,7 @@ if doxygen.found() and dot.found() > output : 'internal-api-html', > command : [doxygen, doxyfile], > install : true, > - install_dir : doc_install_dir, > + install_dir : doc_install_dir / 'html', > install_tag : 'doc-internal') > endif > > @@ -149,7 +150,11 @@ if sphinx.found() > fs = import('fs') > sphinx_conf_dir = fs.parent(sphinx_conf) > > + sphinx_env = environment() > + sphinx_env.set('PYTHONDONTWRITEBYTECODE', '1') > + > docs_sources = [ > + 'api-html/index.rst', > 'camera-sensor-model.rst', > 'code-of-conduct.rst', > 'coding-style.rst', > @@ -164,6 +169,7 @@ if sphinx.found() > 'guides/pipeline-handler.rst', > 'guides/tracing.rst', > 'index.rst', > + 'internal-api-html/index.rst', > 'introduction.rst', > 'lens_driver_requirements.rst', > 'libcamera_architecture.rst', > @@ -183,10 +189,14 @@ if sphinx.found() > input : docs_sources, > output : 'html', > build_by_default : true, > + depend_files : [ > + 'extensions/doxygen-link.py', > + ], > depends : [ > doxygen_public, > doxygen_internal, > ], > + env : sphinx_env, > install : true, > install_dir : doc_install_dir, > install_tag : 'doc') > @@ -195,7 +205,8 @@ if sphinx.found() > command : [sphinx, '-W', '-b', 'linkcheck', > '-c', sphinx_conf_dir, > meson.current_source_dir(), '@OUTPUT@'], > - build_always_stale : true, > input : docs_sources, > - output : 'linkcheck') > + output : 'linkcheck', > + build_always_stale : true, > + env : sphinx_env) > endif
On Fri, Sep 12, 2025 at 01:24:59PM +0200, Barnabás Pőcze wrote: > 2025. 09. 12. 1:01 keltezéssel, Laurent Pinchart írta: > > The libcamera documentation is made of high-level documentation and > > guides, written as ReStructuredText and compiled to HTML by Sphinx, and > > API reference documentation, written as comments in the code and > > compiled to HTML by Doxygen. > > > > Due to meson limitations that make it difficult to place output files in > > subdirectories, the compilation process produces an html/ directory for > > the Sphinx documentation, and api-html/ and internal-api-html/ > > directories for the Doxygen documentation. When deploying the > > documentation to the libcamera.org website, the api-html and > > internal-api-html/ directories are moved within html/ to make the > > documentation self-contained. > > > > The Sphinx and Doxygen documentation link to each other. The links are > > generated using relative paths, based on the directory hierarchy on the > > website. This makes them broken when navigating the documentation in the > > build tree or in the directory where libcamera is installed. > > > > Fix this by standardizing on the directories hierarchy of the website in > > the build and install directories: > > > > - For the build directory, we can't easily build the Doxygen > > documentation in a subdirectory of the Sphinx documentation due to > > limitations of meson. Keep the existing output directories, and > > replace the html/api-html/ and html/internal-api-html/ placeholder > > directories with symlinks to the Doxygen output directories. This is > > handled by a small custom Sphinx extension. > > > > - For the install directory, install the Doxygen documentation to > > html/api-html/ and html/internal-api-html/. This overwrites the > > placeholders. > > Does it need a special meson option? Installation fails for me: > > Tried to copy file /tmp/test/usr/share/doc/libcamera-0.5.2/html/internal-api but a directory of that name already exists. That's new to me :-( What meson version are you using ? > > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> > > --- > > Changes since v1: > > > > - Split from "[PATCH 4/6] documentation: Include API docs in the sphinx > > documentation" > > - Avoid copying the doxygen documentation > > - Use a sphinx extension to generate symlinks > > --- > > Documentation/conf.py.in | 26 +++++---- > > Documentation/extensions/doxygen-link.py | 73 ++++++++++++++++++++++++ > > Documentation/meson.build | 19 ++++-- > > 3 files changed, 104 insertions(+), 14 deletions(-) > > create mode 100644 Documentation/extensions/doxygen-link.py > > > > diff --git a/Documentation/conf.py.in b/Documentation/conf.py.in > > index 34fa3956f49e..5e0a7cff2984 100644 > > --- a/Documentation/conf.py.in > > +++ b/Documentation/conf.py.in > > @@ -9,14 +9,8 @@ > > > > # -- Path setup -------------------------------------------------------------- > > > > -# If extensions (or modules to document with autodoc) are in another directory, > > -# add these directories to sys.path here. If the directory is relative to the > > -# documentation root, use os.path.abspath to make it absolute, like shown here. > > -# > > -# import os > > -# import sys > > -# sys.path.insert(0, os.path.abspath('.')) > > - > > +import sys > > +sys.path.insert(0, "@CURRENT_SRCDIR@/extensions") > > > > # -- Project information ----------------------------------------------------- > > > > @@ -39,6 +33,7 @@ author = 'The libcamera documentation authors' > > extensions = [ > > 'sphinx.ext.graphviz', > > 'sphinxcontrib.doxylink', > > + 'doxygen-link', > > ] > > > > graphviz_output_format = 'svg' > > @@ -75,14 +70,25 @@ pygments_style = None > > doxylink = { > > 'doxy-pub': ( > > '@TOP_BUILDDIR@/Documentation/api-html/tagfile.xml', > > - '../api-html/', > > + 'api-html/', > > ), > > 'doxy-int': ( > > '@TOP_BUILDDIR@/Documentation/internal-api-html/tagfile.xml', > > - '../internal-api-html/', > > + 'internal-api-html/', > > ), > > } > > > > +doxygen_links = [ > > + [ > > + '@TOP_BUILDDIR@/Documentation/html/api-html', > > + '@TOP_BUILDDIR@/Documentation/api-html', > > + ], > > + [ > > + '@TOP_BUILDDIR@/Documentation/html/internal-api-html', > > + '@TOP_BUILDDIR@/Documentation/internal-api-html', > > + ], > > +] > > + > > # -- Options for HTML output ------------------------------------------------- > > > > # The theme to use for HTML and HTML Help pages. See the documentation for > > diff --git a/Documentation/extensions/doxygen-link.py b/Documentation/extensions/doxygen-link.py > > new file mode 100644 > > index 000000000000..8b94a4dd7f0b > > --- /dev/null > > +++ b/Documentation/extensions/doxygen-link.py > > @@ -0,0 +1,73 @@ > > +# SPDX-License-Identifier: GPL-2.0-or-later > > +# Copyright (C) 2025, Ideas on Board Oy > > + > > +import contextlib > > +import os > > +import shutil > > +from pathlib import Path > > +from sphinx.util import logging > > + > > +__version__ = "0.0.0" > > + > > +logger = logging.getLogger(__name__) > > + > > + > > +def on_config_inited(app, config): > > + entries = [] > > + > > + outdir = Path(app.outdir).absolute() > > + > > + for index, items in enumerate(config.doxygen_links): > > + err_msg_prefix = f'Config variable `doxygen_links` contains invalid entry {index} (`{items}`)' > > + > > + if len(items) != 2: > > + raise ValueError(f'{err_msg_prefix}: expected (path, target)') > > + > > + path = Path(items[0]).absolute() > > + target = Path(items[1]).relative_to(path.parent, walk_up=True) > > + > > + if not path.is_relative_to(outdir): > > + raise ValueError(f'{err_msg_prefix}: path `{items[0]}` is outside of output directory `{outdir}`') > > + > > + entries.append([path, target]) > > + > > + config.doxygen_links = entries > > + > > + > > +def on_builder_inited(app): > > + # Remove the symlinks if they exist, to avoid overwriting the index.html > > + # generated by Doxygen with the placeholder index from Sphinx. > > + > > + for path, target in app.config.doxygen_links: > > + if path.is_symlink(): > > + logger.info(f'Removing existing symlink {path}') > > + os.unlink(path) > > + > > + > > +def on_build_finished(app, exc): > > + # Create the symlinks. Remove any existing placeholder directory > > + # recursively first. > > + > > + if exc: > > + return > > + > > + for path, target in app.config.doxygen_links: > > + logger.info(f'Creating symlink {path} -> {target}') > > + > > + if path.is_dir(): > > + shutil.rmtree(path) > > + > > + os.symlink(target, path) > > + > > + > > +def setup(app): > > + app.add_config_value('doxygen_links', [], 'env', frozenset({list, tuple})) > > + app.connect('config-inited', on_config_inited) > > + app.connect('builder-inited', on_builder_inited) > > + app.connect('build-finished', on_build_finished) > > + > > + return { > > + "version": __version__, > > + "parallel_read_safe": True, > > + "parallel_write_safe": True, > > + } > > diff --git a/Documentation/meson.build b/Documentation/meson.build > > index f73407432fff..82d76b257b8c 100644 > > --- a/Documentation/meson.build > > +++ b/Documentation/meson.build > > @@ -12,6 +12,7 @@ dot = find_program('dot', required : get_option('documentation')) > > if doxygen.found() and dot.found() > > cdata = configuration_data() > > cdata.set('VERSION', 'v@0@'.format(libcamera_git_version)) > > + cdata.set('CURRENT_SRCDIR', meson.current_source_dir()) > > cdata.set('TOP_SRCDIR', meson.project_source_root()) > > cdata.set('TOP_BUILDDIR', meson.project_build_root()) > > cdata.set('OUTPUT_DIR', meson.current_build_dir()) > > @@ -89,7 +90,7 @@ if doxygen.found() and dot.found() > > output : 'api-html', > > command : [doxygen, doxyfile], > > install : true, > > - install_dir : doc_install_dir, > > + install_dir : doc_install_dir / 'html', > > install_tag : 'doc') > > > > # This is the internal documentation, which hard-codes a list of directories > > @@ -109,7 +110,7 @@ if doxygen.found() and dot.found() > > output : 'internal-api-html', > > command : [doxygen, doxyfile], > > install : true, > > - install_dir : doc_install_dir, > > + install_dir : doc_install_dir / 'html', > > install_tag : 'doc-internal') > > endif > > > > @@ -149,7 +150,11 @@ if sphinx.found() > > fs = import('fs') > > sphinx_conf_dir = fs.parent(sphinx_conf) > > > > + sphinx_env = environment() > > + sphinx_env.set('PYTHONDONTWRITEBYTECODE', '1') > > + > > docs_sources = [ > > + 'api-html/index.rst', > > 'camera-sensor-model.rst', > > 'code-of-conduct.rst', > > 'coding-style.rst', > > @@ -164,6 +169,7 @@ if sphinx.found() > > 'guides/pipeline-handler.rst', > > 'guides/tracing.rst', > > 'index.rst', > > + 'internal-api-html/index.rst', > > 'introduction.rst', > > 'lens_driver_requirements.rst', > > 'libcamera_architecture.rst', > > @@ -183,10 +189,14 @@ if sphinx.found() > > input : docs_sources, > > output : 'html', > > build_by_default : true, > > + depend_files : [ > > + 'extensions/doxygen-link.py', > > + ], > > depends : [ > > doxygen_public, > > doxygen_internal, > > ], > > + env : sphinx_env, > > install : true, > > install_dir : doc_install_dir, > > install_tag : 'doc') > > @@ -195,7 +205,8 @@ if sphinx.found() > > command : [sphinx, '-W', '-b', 'linkcheck', > > '-c', sphinx_conf_dir, > > meson.current_source_dir(), '@OUTPUT@'], > > - build_always_stale : true, > > input : docs_sources, > > - output : 'linkcheck') > > + output : 'linkcheck', > > + build_always_stale : true, > > + env : sphinx_env)
On Fri, Sep 12, 2025 at 08:04:17PM +0300, Laurent Pinchart wrote: > On Fri, Sep 12, 2025 at 01:24:59PM +0200, Barnabás Pőcze wrote: > > 2025. 09. 12. 1:01 keltezéssel, Laurent Pinchart írta: > > > The libcamera documentation is made of high-level documentation and > > > guides, written as ReStructuredText and compiled to HTML by Sphinx, and > > > API reference documentation, written as comments in the code and > > > compiled to HTML by Doxygen. > > > > > > Due to meson limitations that make it difficult to place output files in > > > subdirectories, the compilation process produces an html/ directory for > > > the Sphinx documentation, and api-html/ and internal-api-html/ > > > directories for the Doxygen documentation. When deploying the > > > documentation to the libcamera.org website, the api-html and > > > internal-api-html/ directories are moved within html/ to make the > > > documentation self-contained. > > > > > > The Sphinx and Doxygen documentation link to each other. The links are > > > generated using relative paths, based on the directory hierarchy on the > > > website. This makes them broken when navigating the documentation in the > > > build tree or in the directory where libcamera is installed. > > > > > > Fix this by standardizing on the directories hierarchy of the website in > > > the build and install directories: > > > > > > - For the build directory, we can't easily build the Doxygen > > > documentation in a subdirectory of the Sphinx documentation due to > > > limitations of meson. Keep the existing output directories, and > > > replace the html/api-html/ and html/internal-api-html/ placeholder > > > directories with symlinks to the Doxygen output directories. This is > > > handled by a small custom Sphinx extension. > > > > > > - For the install directory, install the Doxygen documentation to > > > html/api-html/ and html/internal-api-html/. This overwrites the > > > placeholders. > > > > Does it need a special meson option? Installation fails for me: > > > > Tried to copy file /tmp/test/usr/share/doc/libcamera-0.5.2/html/internal-api but a directory of that name already exists. > > That's new to me :-( What meson version are you using ? The behaviour of "meson install" changed with commit 028abfe87c5d3b4dfe8a29472119aa1581beb215 Author: Daan De Meyer <daan.j.demeyer@gmail.com> Date: Fri Apr 11 11:57:12 2025 +0200 minstall: Don't treat symlinks to directories as directories in do_copydir() merged in v1.8.0. *sigh* Oh well. This was the last remaining part of this patch where I thought we may relied a bit on luck, so I suppose it's good we caught it now. I'm tempted to install the doxygen documentation to api-html/ instead of html/api-html/, and keep the symlink. The install tree will be exactly the same as the build tree. Packagers will always have the option to remove the symlink and move api-html/ to html/api-html/, like we'll do when publishing on the website. > > > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> > > > --- > > > Changes since v1: > > > > > > - Split from "[PATCH 4/6] documentation: Include API docs in the sphinx > > > documentation" > > > - Avoid copying the doxygen documentation > > > - Use a sphinx extension to generate symlinks > > > --- > > > Documentation/conf.py.in | 26 +++++---- > > > Documentation/extensions/doxygen-link.py | 73 ++++++++++++++++++++++++ > > > Documentation/meson.build | 19 ++++-- > > > 3 files changed, 104 insertions(+), 14 deletions(-) > > > create mode 100644 Documentation/extensions/doxygen-link.py > > > > > > diff --git a/Documentation/conf.py.in b/Documentation/conf.py.in > > > index 34fa3956f49e..5e0a7cff2984 100644 > > > --- a/Documentation/conf.py.in > > > +++ b/Documentation/conf.py.in > > > @@ -9,14 +9,8 @@ > > > > > > # -- Path setup -------------------------------------------------------------- > > > > > > -# If extensions (or modules to document with autodoc) are in another directory, > > > -# add these directories to sys.path here. If the directory is relative to the > > > -# documentation root, use os.path.abspath to make it absolute, like shown here. > > > -# > > > -# import os > > > -# import sys > > > -# sys.path.insert(0, os.path.abspath('.')) > > > - > > > +import sys > > > +sys.path.insert(0, "@CURRENT_SRCDIR@/extensions") > > > > > > # -- Project information ----------------------------------------------------- > > > > > > @@ -39,6 +33,7 @@ author = 'The libcamera documentation authors' > > > extensions = [ > > > 'sphinx.ext.graphviz', > > > 'sphinxcontrib.doxylink', > > > + 'doxygen-link', > > > ] > > > > > > graphviz_output_format = 'svg' > > > @@ -75,14 +70,25 @@ pygments_style = None > > > doxylink = { > > > 'doxy-pub': ( > > > '@TOP_BUILDDIR@/Documentation/api-html/tagfile.xml', > > > - '../api-html/', > > > + 'api-html/', > > > ), > > > 'doxy-int': ( > > > '@TOP_BUILDDIR@/Documentation/internal-api-html/tagfile.xml', > > > - '../internal-api-html/', > > > + 'internal-api-html/', > > > ), > > > } > > > > > > +doxygen_links = [ > > > + [ > > > + '@TOP_BUILDDIR@/Documentation/html/api-html', > > > + '@TOP_BUILDDIR@/Documentation/api-html', > > > + ], > > > + [ > > > + '@TOP_BUILDDIR@/Documentation/html/internal-api-html', > > > + '@TOP_BUILDDIR@/Documentation/internal-api-html', > > > + ], > > > +] > > > + > > > # -- Options for HTML output ------------------------------------------------- > > > > > > # The theme to use for HTML and HTML Help pages. See the documentation for > > > diff --git a/Documentation/extensions/doxygen-link.py b/Documentation/extensions/doxygen-link.py > > > new file mode 100644 > > > index 000000000000..8b94a4dd7f0b > > > --- /dev/null > > > +++ b/Documentation/extensions/doxygen-link.py > > > @@ -0,0 +1,73 @@ > > > +# SPDX-License-Identifier: GPL-2.0-or-later > > > +# Copyright (C) 2025, Ideas on Board Oy > > > + > > > +import contextlib > > > +import os > > > +import shutil > > > +from pathlib import Path > > > +from sphinx.util import logging > > > + > > > +__version__ = "0.0.0" > > > + > > > +logger = logging.getLogger(__name__) > > > + > > > + > > > +def on_config_inited(app, config): > > > + entries = [] > > > + > > > + outdir = Path(app.outdir).absolute() > > > + > > > + for index, items in enumerate(config.doxygen_links): > > > + err_msg_prefix = f'Config variable `doxygen_links` contains invalid entry {index} (`{items}`)' > > > + > > > + if len(items) != 2: > > > + raise ValueError(f'{err_msg_prefix}: expected (path, target)') > > > + > > > + path = Path(items[0]).absolute() > > > + target = Path(items[1]).relative_to(path.parent, walk_up=True) > > > + > > > + if not path.is_relative_to(outdir): > > > + raise ValueError(f'{err_msg_prefix}: path `{items[0]}` is outside of output directory `{outdir}`') > > > + > > > + entries.append([path, target]) > > > + > > > + config.doxygen_links = entries > > > + > > > + > > > +def on_builder_inited(app): > > > + # Remove the symlinks if they exist, to avoid overwriting the index.html > > > + # generated by Doxygen with the placeholder index from Sphinx. > > > + > > > + for path, target in app.config.doxygen_links: > > > + if path.is_symlink(): > > > + logger.info(f'Removing existing symlink {path}') > > > + os.unlink(path) > > > + > > > + > > > +def on_build_finished(app, exc): > > > + # Create the symlinks. Remove any existing placeholder directory > > > + # recursively first. > > > + > > > + if exc: > > > + return > > > + > > > + for path, target in app.config.doxygen_links: > > > + logger.info(f'Creating symlink {path} -> {target}') > > > + > > > + if path.is_dir(): > > > + shutil.rmtree(path) > > > + > > > + os.symlink(target, path) > > > + > > > + > > > +def setup(app): > > > + app.add_config_value('doxygen_links', [], 'env', frozenset({list, tuple})) > > > + app.connect('config-inited', on_config_inited) > > > + app.connect('builder-inited', on_builder_inited) > > > + app.connect('build-finished', on_build_finished) > > > + > > > + return { > > > + "version": __version__, > > > + "parallel_read_safe": True, > > > + "parallel_write_safe": True, > > > + } > > > diff --git a/Documentation/meson.build b/Documentation/meson.build > > > index f73407432fff..82d76b257b8c 100644 > > > --- a/Documentation/meson.build > > > +++ b/Documentation/meson.build > > > @@ -12,6 +12,7 @@ dot = find_program('dot', required : get_option('documentation')) > > > if doxygen.found() and dot.found() > > > cdata = configuration_data() > > > cdata.set('VERSION', 'v@0@'.format(libcamera_git_version)) > > > + cdata.set('CURRENT_SRCDIR', meson.current_source_dir()) > > > cdata.set('TOP_SRCDIR', meson.project_source_root()) > > > cdata.set('TOP_BUILDDIR', meson.project_build_root()) > > > cdata.set('OUTPUT_DIR', meson.current_build_dir()) > > > @@ -89,7 +90,7 @@ if doxygen.found() and dot.found() > > > output : 'api-html', > > > command : [doxygen, doxyfile], > > > install : true, > > > - install_dir : doc_install_dir, > > > + install_dir : doc_install_dir / 'html', > > > install_tag : 'doc') > > > > > > # This is the internal documentation, which hard-codes a list of directories > > > @@ -109,7 +110,7 @@ if doxygen.found() and dot.found() > > > output : 'internal-api-html', > > > command : [doxygen, doxyfile], > > > install : true, > > > - install_dir : doc_install_dir, > > > + install_dir : doc_install_dir / 'html', > > > install_tag : 'doc-internal') > > > endif > > > > > > @@ -149,7 +150,11 @@ if sphinx.found() > > > fs = import('fs') > > > sphinx_conf_dir = fs.parent(sphinx_conf) > > > > > > + sphinx_env = environment() > > > + sphinx_env.set('PYTHONDONTWRITEBYTECODE', '1') > > > + > > > docs_sources = [ > > > + 'api-html/index.rst', > > > 'camera-sensor-model.rst', > > > 'code-of-conduct.rst', > > > 'coding-style.rst', > > > @@ -164,6 +169,7 @@ if sphinx.found() > > > 'guides/pipeline-handler.rst', > > > 'guides/tracing.rst', > > > 'index.rst', > > > + 'internal-api-html/index.rst', > > > 'introduction.rst', > > > 'lens_driver_requirements.rst', > > > 'libcamera_architecture.rst', > > > @@ -183,10 +189,14 @@ if sphinx.found() > > > input : docs_sources, > > > output : 'html', > > > build_by_default : true, > > > + depend_files : [ > > > + 'extensions/doxygen-link.py', > > > + ], > > > depends : [ > > > doxygen_public, > > > doxygen_internal, > > > ], > > > + env : sphinx_env, > > > install : true, > > > install_dir : doc_install_dir, > > > install_tag : 'doc') > > > @@ -195,7 +205,8 @@ if sphinx.found() > > > command : [sphinx, '-W', '-b', 'linkcheck', > > > '-c', sphinx_conf_dir, > > > meson.current_source_dir(), '@OUTPUT@'], > > > - build_always_stale : true, > > > input : docs_sources, > > > - output : 'linkcheck') > > > + output : 'linkcheck', > > > + build_always_stale : true, > > > + env : sphinx_env)
On Sat, Sep 13, 2025 at 08:45:49PM +0300, Laurent Pinchart wrote: > On Fri, Sep 12, 2025 at 08:04:17PM +0300, Laurent Pinchart wrote: > > On Fri, Sep 12, 2025 at 01:24:59PM +0200, Barnabás Pőcze wrote: > > > 2025. 09. 12. 1:01 keltezéssel, Laurent Pinchart írta: > > > > The libcamera documentation is made of high-level documentation and > > > > guides, written as ReStructuredText and compiled to HTML by Sphinx, and > > > > API reference documentation, written as comments in the code and > > > > compiled to HTML by Doxygen. > > > > > > > > Due to meson limitations that make it difficult to place output files in > > > > subdirectories, the compilation process produces an html/ directory for > > > > the Sphinx documentation, and api-html/ and internal-api-html/ > > > > directories for the Doxygen documentation. When deploying the > > > > documentation to the libcamera.org website, the api-html and > > > > internal-api-html/ directories are moved within html/ to make the > > > > documentation self-contained. > > > > > > > > The Sphinx and Doxygen documentation link to each other. The links are > > > > generated using relative paths, based on the directory hierarchy on the > > > > website. This makes them broken when navigating the documentation in the > > > > build tree or in the directory where libcamera is installed. > > > > > > > > Fix this by standardizing on the directories hierarchy of the website in > > > > the build and install directories: > > > > > > > > - For the build directory, we can't easily build the Doxygen > > > > documentation in a subdirectory of the Sphinx documentation due to > > > > limitations of meson. Keep the existing output directories, and > > > > replace the html/api-html/ and html/internal-api-html/ placeholder > > > > directories with symlinks to the Doxygen output directories. This is > > > > handled by a small custom Sphinx extension. > > > > > > > > - For the install directory, install the Doxygen documentation to > > > > html/api-html/ and html/internal-api-html/. This overwrites the > > > > placeholders. > > > > > > Does it need a special meson option? Installation fails for me: > > > > > > Tried to copy file /tmp/test/usr/share/doc/libcamera-0.5.2/html/internal-api but a directory of that name already exists. > > > > That's new to me :-( What meson version are you using ? > > The behaviour of "meson install" changed with > > commit 028abfe87c5d3b4dfe8a29472119aa1581beb215 > Author: Daan De Meyer <daan.j.demeyer@gmail.com> > Date: Fri Apr 11 11:57:12 2025 +0200 > > minstall: Don't treat symlinks to directories as directories in do_copydir() > > merged in v1.8.0. > > *sigh* > > Oh well. This was the last remaining part of this patch where I thought > we may relied a bit on luck, so I suppose it's good we caught it now. > > I'm tempted to install the doxygen documentation to api-html/ instead of > html/api-html/, and keep the symlink. The install tree will be exactly > the same as the build tree. Packagers will always have the option to > remove the symlink and move api-html/ to html/api-html/, like we'll do > when publishing on the website. That's not going to work. Meson 1.9.0 chokes on installing symlinks to directories, throwing an exception: Traceback (most recent call last): File "/home/laurent/src/tools/meson/mesonbuild/mesonmain.py", line 193, in run return options.run_func(options) ~~~~~~~~~~~~~~~~^^^^^^^^^ File "/home/laurent/src/tools/meson/mesonbuild/minstall.py", line 888, in run installer.do_install(datafilename) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^ File "/home/laurent/src/tools/meson/mesonbuild/minstall.py", line 563, in do_install self.install_targets(d, dm, destdir, fullprefix) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/laurent/src/tools/meson/mesonbuild/minstall.py", line 782, in install_targets self.do_copydir(d, fname, outname, None, install_mode, dm) ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/laurent/src/tools/meson/mesonbuild/minstall.py", line 537, in do_copydir self.do_copyfile(abs_src, abs_dst, follow_symlinks=follow_symlinks) ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/laurent/src/tools/meson/mesonbuild/minstall.py", line 437, in do_copyfile self.copy2(from_file, to_file, follow_symlinks=follow_symlinks) ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/laurent/src/tools/meson/mesonbuild/minstall.py", line 336, in copy2 shutil.copy2(*args, **kwargs) ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.13/shutil.py", line 468, in copy2 copyfile(src, dst, follow_symlinks=follow_symlinks) ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.13/shutil.py", line 260, in copyfile with open(src, 'rb') as fsrc: ~~~~^^^^^^^^^^^ IsADirectoryError: [Errno 21] Is a directory: '/home/laurent/src/iob/oss/libcamera/libcamera/build/x86-gcc-13/Documentation/html/internal-api' That may be a bug in meson, but the fact that it's present in both v1.8.0 and v1.9.0 means that it is probably too dangerous to ignore, even if we can get it fixed. So if we want to work around the problem, we can't have symlinks to directories, which brings us back to square one. Aaargghhhhh. *deep breath* Aaargghhhhh. One option is to keep the current output directories, with the doxygen and sphinx documentation side by side, and override that when publishing to the website. That will require overriding the doxylink configuration option in Documentation/conf.py. We could patch the file manually in the website deployment scripts, or add a meson option. There will then also be the issue of links from doxygen to sphinx, as they currently use hardcoded paths. I'm starting to think we may be better off giving up on this, and keeping sphinx and doxygen documentation side by side on the website as well. > > > > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> > > > > --- > > > > Changes since v1: > > > > > > > > - Split from "[PATCH 4/6] documentation: Include API docs in the sphinx > > > > documentation" > > > > - Avoid copying the doxygen documentation > > > > - Use a sphinx extension to generate symlinks > > > > --- > > > > Documentation/conf.py.in | 26 +++++---- > > > > Documentation/extensions/doxygen-link.py | 73 ++++++++++++++++++++++++ > > > > Documentation/meson.build | 19 ++++-- > > > > 3 files changed, 104 insertions(+), 14 deletions(-) > > > > create mode 100644 Documentation/extensions/doxygen-link.py > > > > > > > > diff --git a/Documentation/conf.py.in b/Documentation/conf.py.in > > > > index 34fa3956f49e..5e0a7cff2984 100644 > > > > --- a/Documentation/conf.py.in > > > > +++ b/Documentation/conf.py.in > > > > @@ -9,14 +9,8 @@ > > > > > > > > # -- Path setup -------------------------------------------------------------- > > > > > > > > -# If extensions (or modules to document with autodoc) are in another directory, > > > > -# add these directories to sys.path here. If the directory is relative to the > > > > -# documentation root, use os.path.abspath to make it absolute, like shown here. > > > > -# > > > > -# import os > > > > -# import sys > > > > -# sys.path.insert(0, os.path.abspath('.')) > > > > - > > > > +import sys > > > > +sys.path.insert(0, "@CURRENT_SRCDIR@/extensions") > > > > > > > > # -- Project information ----------------------------------------------------- > > > > > > > > @@ -39,6 +33,7 @@ author = 'The libcamera documentation authors' > > > > extensions = [ > > > > 'sphinx.ext.graphviz', > > > > 'sphinxcontrib.doxylink', > > > > + 'doxygen-link', > > > > ] > > > > > > > > graphviz_output_format = 'svg' > > > > @@ -75,14 +70,25 @@ pygments_style = None > > > > doxylink = { > > > > 'doxy-pub': ( > > > > '@TOP_BUILDDIR@/Documentation/api-html/tagfile.xml', > > > > - '../api-html/', > > > > + 'api-html/', > > > > ), > > > > 'doxy-int': ( > > > > '@TOP_BUILDDIR@/Documentation/internal-api-html/tagfile.xml', > > > > - '../internal-api-html/', > > > > + 'internal-api-html/', > > > > ), > > > > } > > > > > > > > +doxygen_links = [ > > > > + [ > > > > + '@TOP_BUILDDIR@/Documentation/html/api-html', > > > > + '@TOP_BUILDDIR@/Documentation/api-html', > > > > + ], > > > > + [ > > > > + '@TOP_BUILDDIR@/Documentation/html/internal-api-html', > > > > + '@TOP_BUILDDIR@/Documentation/internal-api-html', > > > > + ], > > > > +] > > > > + > > > > # -- Options for HTML output ------------------------------------------------- > > > > > > > > # The theme to use for HTML and HTML Help pages. See the documentation for > > > > diff --git a/Documentation/extensions/doxygen-link.py b/Documentation/extensions/doxygen-link.py > > > > new file mode 100644 > > > > index 000000000000..8b94a4dd7f0b > > > > --- /dev/null > > > > +++ b/Documentation/extensions/doxygen-link.py > > > > @@ -0,0 +1,73 @@ > > > > +# SPDX-License-Identifier: GPL-2.0-or-later > > > > +# Copyright (C) 2025, Ideas on Board Oy > > > > + > > > > +import contextlib > > > > +import os > > > > +import shutil > > > > +from pathlib import Path > > > > +from sphinx.util import logging > > > > + > > > > +__version__ = "0.0.0" > > > > + > > > > +logger = logging.getLogger(__name__) > > > > + > > > > + > > > > +def on_config_inited(app, config): > > > > + entries = [] > > > > + > > > > + outdir = Path(app.outdir).absolute() > > > > + > > > > + for index, items in enumerate(config.doxygen_links): > > > > + err_msg_prefix = f'Config variable `doxygen_links` contains invalid entry {index} (`{items}`)' > > > > + > > > > + if len(items) != 2: > > > > + raise ValueError(f'{err_msg_prefix}: expected (path, target)') > > > > + > > > > + path = Path(items[0]).absolute() > > > > + target = Path(items[1]).relative_to(path.parent, walk_up=True) > > > > + > > > > + if not path.is_relative_to(outdir): > > > > + raise ValueError(f'{err_msg_prefix}: path `{items[0]}` is outside of output directory `{outdir}`') > > > > + > > > > + entries.append([path, target]) > > > > + > > > > + config.doxygen_links = entries > > > > + > > > > + > > > > +def on_builder_inited(app): > > > > + # Remove the symlinks if they exist, to avoid overwriting the index.html > > > > + # generated by Doxygen with the placeholder index from Sphinx. > > > > + > > > > + for path, target in app.config.doxygen_links: > > > > + if path.is_symlink(): > > > > + logger.info(f'Removing existing symlink {path}') > > > > + os.unlink(path) > > > > + > > > > + > > > > +def on_build_finished(app, exc): > > > > + # Create the symlinks. Remove any existing placeholder directory > > > > + # recursively first. > > > > + > > > > + if exc: > > > > + return > > > > + > > > > + for path, target in app.config.doxygen_links: > > > > + logger.info(f'Creating symlink {path} -> {target}') > > > > + > > > > + if path.is_dir(): > > > > + shutil.rmtree(path) > > > > + > > > > + os.symlink(target, path) > > > > + > > > > + > > > > +def setup(app): > > > > + app.add_config_value('doxygen_links', [], 'env', frozenset({list, tuple})) > > > > + app.connect('config-inited', on_config_inited) > > > > + app.connect('builder-inited', on_builder_inited) > > > > + app.connect('build-finished', on_build_finished) > > > > + > > > > + return { > > > > + "version": __version__, > > > > + "parallel_read_safe": True, > > > > + "parallel_write_safe": True, > > > > + } > > > > diff --git a/Documentation/meson.build b/Documentation/meson.build > > > > index f73407432fff..82d76b257b8c 100644 > > > > --- a/Documentation/meson.build > > > > +++ b/Documentation/meson.build > > > > @@ -12,6 +12,7 @@ dot = find_program('dot', required : get_option('documentation')) > > > > if doxygen.found() and dot.found() > > > > cdata = configuration_data() > > > > cdata.set('VERSION', 'v@0@'.format(libcamera_git_version)) > > > > + cdata.set('CURRENT_SRCDIR', meson.current_source_dir()) > > > > cdata.set('TOP_SRCDIR', meson.project_source_root()) > > > > cdata.set('TOP_BUILDDIR', meson.project_build_root()) > > > > cdata.set('OUTPUT_DIR', meson.current_build_dir()) > > > > @@ -89,7 +90,7 @@ if doxygen.found() and dot.found() > > > > output : 'api-html', > > > > command : [doxygen, doxyfile], > > > > install : true, > > > > - install_dir : doc_install_dir, > > > > + install_dir : doc_install_dir / 'html', > > > > install_tag : 'doc') > > > > > > > > # This is the internal documentation, which hard-codes a list of directories > > > > @@ -109,7 +110,7 @@ if doxygen.found() and dot.found() > > > > output : 'internal-api-html', > > > > command : [doxygen, doxyfile], > > > > install : true, > > > > - install_dir : doc_install_dir, > > > > + install_dir : doc_install_dir / 'html', > > > > install_tag : 'doc-internal') > > > > endif > > > > > > > > @@ -149,7 +150,11 @@ if sphinx.found() > > > > fs = import('fs') > > > > sphinx_conf_dir = fs.parent(sphinx_conf) > > > > > > > > + sphinx_env = environment() > > > > + sphinx_env.set('PYTHONDONTWRITEBYTECODE', '1') > > > > + > > > > docs_sources = [ > > > > + 'api-html/index.rst', > > > > 'camera-sensor-model.rst', > > > > 'code-of-conduct.rst', > > > > 'coding-style.rst', > > > > @@ -164,6 +169,7 @@ if sphinx.found() > > > > 'guides/pipeline-handler.rst', > > > > 'guides/tracing.rst', > > > > 'index.rst', > > > > + 'internal-api-html/index.rst', > > > > 'introduction.rst', > > > > 'lens_driver_requirements.rst', > > > > 'libcamera_architecture.rst', > > > > @@ -183,10 +189,14 @@ if sphinx.found() > > > > input : docs_sources, > > > > output : 'html', > > > > build_by_default : true, > > > > + depend_files : [ > > > > + 'extensions/doxygen-link.py', > > > > + ], > > > > depends : [ > > > > doxygen_public, > > > > doxygen_internal, > > > > ], > > > > + env : sphinx_env, > > > > install : true, > > > > install_dir : doc_install_dir, > > > > install_tag : 'doc') > > > > @@ -195,7 +205,8 @@ if sphinx.found() > > > > command : [sphinx, '-W', '-b', 'linkcheck', > > > > '-c', sphinx_conf_dir, > > > > meson.current_source_dir(), '@OUTPUT@'], > > > > - build_always_stale : true, > > > > input : docs_sources, > > > > - output : 'linkcheck') > > > > + output : 'linkcheck', > > > > + build_always_stale : true, > > > > + env : sphinx_env)
diff --git a/Documentation/conf.py.in b/Documentation/conf.py.in index 34fa3956f49e..5e0a7cff2984 100644 --- a/Documentation/conf.py.in +++ b/Documentation/conf.py.in @@ -9,14 +9,8 @@ # -- Path setup -------------------------------------------------------------- -# If extensions (or modules to document with autodoc) are in another directory, -# add these directories to sys.path here. If the directory is relative to the -# documentation root, use os.path.abspath to make it absolute, like shown here. -# -# import os -# import sys -# sys.path.insert(0, os.path.abspath('.')) - +import sys +sys.path.insert(0, "@CURRENT_SRCDIR@/extensions") # -- Project information ----------------------------------------------------- @@ -39,6 +33,7 @@ author = 'The libcamera documentation authors' extensions = [ 'sphinx.ext.graphviz', 'sphinxcontrib.doxylink', + 'doxygen-link', ] graphviz_output_format = 'svg' @@ -75,14 +70,25 @@ pygments_style = None doxylink = { 'doxy-pub': ( '@TOP_BUILDDIR@/Documentation/api-html/tagfile.xml', - '../api-html/', + 'api-html/', ), 'doxy-int': ( '@TOP_BUILDDIR@/Documentation/internal-api-html/tagfile.xml', - '../internal-api-html/', + 'internal-api-html/', ), } +doxygen_links = [ + [ + '@TOP_BUILDDIR@/Documentation/html/api-html', + '@TOP_BUILDDIR@/Documentation/api-html', + ], + [ + '@TOP_BUILDDIR@/Documentation/html/internal-api-html', + '@TOP_BUILDDIR@/Documentation/internal-api-html', + ], +] + # -- Options for HTML output ------------------------------------------------- # The theme to use for HTML and HTML Help pages. See the documentation for diff --git a/Documentation/extensions/doxygen-link.py b/Documentation/extensions/doxygen-link.py new file mode 100644 index 000000000000..8b94a4dd7f0b --- /dev/null +++ b/Documentation/extensions/doxygen-link.py @@ -0,0 +1,73 @@ +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (C) 2025, Ideas on Board Oy + +import contextlib +import os +import shutil +from pathlib import Path +from sphinx.util import logging + +__version__ = "0.0.0" + +logger = logging.getLogger(__name__) + + +def on_config_inited(app, config): + entries = [] + + outdir = Path(app.outdir).absolute() + + for index, items in enumerate(config.doxygen_links): + err_msg_prefix = f'Config variable `doxygen_links` contains invalid entry {index} (`{items}`)' + + if len(items) != 2: + raise ValueError(f'{err_msg_prefix}: expected (path, target)') + + path = Path(items[0]).absolute() + target = Path(items[1]).relative_to(path.parent, walk_up=True) + + if not path.is_relative_to(outdir): + raise ValueError(f'{err_msg_prefix}: path `{items[0]}` is outside of output directory `{outdir}`') + + entries.append([path, target]) + + config.doxygen_links = entries + + +def on_builder_inited(app): + # Remove the symlinks if they exist, to avoid overwriting the index.html + # generated by Doxygen with the placeholder index from Sphinx. + + for path, target in app.config.doxygen_links: + if path.is_symlink(): + logger.info(f'Removing existing symlink {path}') + os.unlink(path) + + +def on_build_finished(app, exc): + # Create the symlinks. Remove any existing placeholder directory + # recursively first. + + if exc: + return + + for path, target in app.config.doxygen_links: + logger.info(f'Creating symlink {path} -> {target}') + + if path.is_dir(): + shutil.rmtree(path) + + os.symlink(target, path) + + +def setup(app): + app.add_config_value('doxygen_links', [], 'env', frozenset({list, tuple})) + app.connect('config-inited', on_config_inited) + app.connect('builder-inited', on_builder_inited) + app.connect('build-finished', on_build_finished) + + return { + "version": __version__, + "parallel_read_safe": True, + "parallel_write_safe": True, + } diff --git a/Documentation/meson.build b/Documentation/meson.build index f73407432fff..82d76b257b8c 100644 --- a/Documentation/meson.build +++ b/Documentation/meson.build @@ -12,6 +12,7 @@ dot = find_program('dot', required : get_option('documentation')) if doxygen.found() and dot.found() cdata = configuration_data() cdata.set('VERSION', 'v@0@'.format(libcamera_git_version)) + cdata.set('CURRENT_SRCDIR', meson.current_source_dir()) cdata.set('TOP_SRCDIR', meson.project_source_root()) cdata.set('TOP_BUILDDIR', meson.project_build_root()) cdata.set('OUTPUT_DIR', meson.current_build_dir()) @@ -89,7 +90,7 @@ if doxygen.found() and dot.found() output : 'api-html', command : [doxygen, doxyfile], install : true, - install_dir : doc_install_dir, + install_dir : doc_install_dir / 'html', install_tag : 'doc') # This is the internal documentation, which hard-codes a list of directories @@ -109,7 +110,7 @@ if doxygen.found() and dot.found() output : 'internal-api-html', command : [doxygen, doxyfile], install : true, - install_dir : doc_install_dir, + install_dir : doc_install_dir / 'html', install_tag : 'doc-internal') endif @@ -149,7 +150,11 @@ if sphinx.found() fs = import('fs') sphinx_conf_dir = fs.parent(sphinx_conf) + sphinx_env = environment() + sphinx_env.set('PYTHONDONTWRITEBYTECODE', '1') + docs_sources = [ + 'api-html/index.rst', 'camera-sensor-model.rst', 'code-of-conduct.rst', 'coding-style.rst', @@ -164,6 +169,7 @@ if sphinx.found() 'guides/pipeline-handler.rst', 'guides/tracing.rst', 'index.rst', + 'internal-api-html/index.rst', 'introduction.rst', 'lens_driver_requirements.rst', 'libcamera_architecture.rst', @@ -183,10 +189,14 @@ if sphinx.found() input : docs_sources, output : 'html', build_by_default : true, + depend_files : [ + 'extensions/doxygen-link.py', + ], depends : [ doxygen_public, doxygen_internal, ], + env : sphinx_env, install : true, install_dir : doc_install_dir, install_tag : 'doc') @@ -195,7 +205,8 @@ if sphinx.found() command : [sphinx, '-W', '-b', 'linkcheck', '-c', sphinx_conf_dir, meson.current_source_dir(), '@OUTPUT@'], - build_always_stale : true, input : docs_sources, - output : 'linkcheck') + output : 'linkcheck', + build_always_stale : true, + env : sphinx_env) endif
The libcamera documentation is made of high-level documentation and guides, written as ReStructuredText and compiled to HTML by Sphinx, and API reference documentation, written as comments in the code and compiled to HTML by Doxygen. Due to meson limitations that make it difficult to place output files in subdirectories, the compilation process produces an html/ directory for the Sphinx documentation, and api-html/ and internal-api-html/ directories for the Doxygen documentation. When deploying the documentation to the libcamera.org website, the api-html and internal-api-html/ directories are moved within html/ to make the documentation self-contained. The Sphinx and Doxygen documentation link to each other. The links are generated using relative paths, based on the directory hierarchy on the website. This makes them broken when navigating the documentation in the build tree or in the directory where libcamera is installed. Fix this by standardizing on the directories hierarchy of the website in the build and install directories: - For the build directory, we can't easily build the Doxygen documentation in a subdirectory of the Sphinx documentation due to limitations of meson. Keep the existing output directories, and replace the html/api-html/ and html/internal-api-html/ placeholder directories with symlinks to the Doxygen output directories. This is handled by a small custom Sphinx extension. - For the install directory, install the Doxygen documentation to html/api-html/ and html/internal-api-html/. This overwrites the placeholders. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> --- Changes since v1: - Split from "[PATCH 4/6] documentation: Include API docs in the sphinx documentation" - Avoid copying the doxygen documentation - Use a sphinx extension to generate symlinks --- Documentation/conf.py.in | 26 +++++---- Documentation/extensions/doxygen-link.py | 73 ++++++++++++++++++++++++ Documentation/meson.build | 19 ++++-- 3 files changed, 104 insertions(+), 14 deletions(-) create mode 100644 Documentation/extensions/doxygen-link.py