| Message ID | 20250917201742.16406-8-laurent.pinchart@ideasonboard.com |
|---|---|
| State | Accepted |
| Headers | show |
| Series |
|
| Related | show |
Hi Laurent, Thank you for the patch. Quoting Laurent Pinchart (2025-09-17 22:17:38) > The libcamera documentation comprises two parts: pages generated by > Sphinx into the Documentation/html/ directory within the build tree, and > API reference documentation generated by Doxygen into > Documentation/internal-api-html/ and Documentation/api-html/. The two > parts are generated separately, but link to each other. > > From Sphinx to Doxygen, we use the doxylink extension for Sphinx to > generate links to the Doxygen pages corresponding to API elements. The > extension needs to be configured with the paths to the Doxygen > documentation, which are set based on the html/, api-html/ and > internal-api-html/ directories being placed side by side in the same > parent directory. > > Furthermore, we also want to link to the API documentation from the > Sphinx toctree. As toctrees can only link to pages within the Sphinx > documents tree (or to http URLs), we have placeholder .rst documents for > api-html and internal-api-html in the Sphinx documentation tree. Those > generate the Documentation/html/internal-api-html/index.html and > Documentation/html/api-html/index.html placeholder files in the build > tree. We could mention the ../api-html path in these placeholders so that someone reading the docs from the build dir gets a hint where to look. > > The other way around, the API documentation's introduction pagelinks to > Sphinx pages using relative paths. Those paths are hardcoded based on > the api-html/ and internal-api-html/ directories being children of the > html/ directory. > > This results in links being broken in different ways in the build tree, > as well as in the installation directory: the toctree links direct to > the placeholder pages within the html/ directory instead of the Doxygen > documentation in sibling directories, and the Doxygen introduction links > to Sphinx are simply broken. When publishing documentation on the > website we work around those issues by patching several files and moving Do we really patch files? To my understanding we only do the move that is also done by the install step now. Or am I missing something? We could actually do a meson install --destdir= there. > the api-html/ and internal-api-html/ directories to the html/ directory. > > Fixing this is surprisingly difficult. The toctree links can't be > changed to point to a path outside of the Sphinx document tree as this > isn't supported by Sphinx. Using http URLs would link to the > libcamera.org website inside of local documentation, which isn't > acceptable. It may be possible to develop a Sphinx extension to patch > the toctree after it gets parsed, but that would be complex and likely > fragile. Modifying the install path of the Doxygen documentation to > html/api-html/ and html/internal-api-html/ causes issues as the Sphinx > documentation will then overwrite the Doxygen index.html files with the > placeholder indexes. Creating symlinks from html/api-html/ to api-html/ > in the installation directory causes similar problems if 'meson install' > is run twice. Creating the symlinks in the build directory (which was > attempted with a custom Sphinx extension) is also a no-go: starting with > meson v1.8.0, installing symlinks to directories causes an exception due > to a bug in meson. > > The right solution is probably to investigate usage of the doxysphinx > extension. As that's no small amount of work, let's start with a > non-perfect but simple improvement: configure doxylink based on the > api-html/ and internal-api-html/ directories being children of the > Sphinx html/ documentation, and move those two API documentation > directories to html/ during installation with a post-install script. > This fixes links in the installation directory. Links in the build > directory remain broken, but that is not a regression. > > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Phew, that was a ride. Now the initial cp moved into the install step. I like that workaround. Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com> > --- > Documentation/conf.py.in | 4 ++-- > Documentation/install-doxygen.sh | 18 ++++++++++++++++++ > Documentation/meson.build | 3 +++ > 3 files changed, 23 insertions(+), 2 deletions(-) > create mode 100755 Documentation/install-doxygen.sh > > diff --git a/Documentation/conf.py.in b/Documentation/conf.py.in > index 34fa3956f49e..2c75a75799e6 100644 > --- a/Documentation/conf.py.in > +++ b/Documentation/conf.py.in > @@ -75,11 +75,11 @@ pygments_style = None > doxylink = { > 'doxy-pub': ( > '@TOP_BUILDDIR@/Documentation/api-html/tagfile.xml', > - '../api-html/', > + 'api-html/', > ), > 'doxy-int': ( > '@TOP_BUILDDIR@/Documentation/internal-api-html/tagfile.xml', > - '../internal-api-html/', > + 'internal-api-html/', > ), > } > > diff --git a/Documentation/install-doxygen.sh b/Documentation/install-doxygen.sh > new file mode 100755 > index 000000000000..ea5a19dc8fda > --- /dev/null > +++ b/Documentation/install-doxygen.sh > @@ -0,0 +1,18 @@ > +#!/bin/sh > +# SPDX-License-Identifier: GPL-2.0-or-later > +# Copyright (C) 2025, Ideas on Board Oy > +# > +# Author: Laurent Pinchart <laurent.pinchart@ideasonboard.com> > +# > +# Move Doxygen-generated API documentation to correct location > + > +doc_dir="${MESON_INSTALL_DESTDIR_PREFIX}/$1" > +shift > +dirs="$*" > + > +echo "Moving API documentation" > + > +for dir in $dirs ; do > + rm -r "${doc_dir}/html/${dir}" > + mv "${doc_dir}/${dir}" "${doc_dir}/html/" > +done > diff --git a/Documentation/meson.build b/Documentation/meson.build > index 8cf7775902f3..022770968fcf 100644 > --- a/Documentation/meson.build > +++ b/Documentation/meson.build > @@ -193,6 +193,9 @@ if sphinx.found() > install_dir : doc_install_dir, > install_tag : 'doc') > > + meson.add_install_script('install-doxygen.sh', doc_install_dir, > + 'api-html', 'internal-api-html') > + > custom_target('documentation-linkcheck', > command : [sphinx, '-W', '-b', 'linkcheck', > '-c', sphinx_conf_dir, > -- > Regards, > > Laurent Pinchart >
2025. 09. 17. 22:17 keltezéssel, Laurent Pinchart írta: > The libcamera documentation comprises two parts: pages generated by > Sphinx into the Documentation/html/ directory within the build tree, and > API reference documentation generated by Doxygen into > Documentation/internal-api-html/ and Documentation/api-html/. The two > parts are generated separately, but link to each other. > > From Sphinx to Doxygen, we use the doxylink extension for Sphinx to > generate links to the Doxygen pages corresponding to API elements. The > extension needs to be configured with the paths to the Doxygen > documentation, which are set based on the html/, api-html/ and > internal-api-html/ directories being placed side by side in the same > parent directory. > > Furthermore, we also want to link to the API documentation from the > Sphinx toctree. As toctrees can only link to pages within the Sphinx > documents tree (or to http URLs), we have placeholder .rst documents for > api-html and internal-api-html in the Sphinx documentation tree. Those > generate the Documentation/html/internal-api-html/index.html and > Documentation/html/api-html/index.html placeholder files in the build > tree. > > The other way around, the API documentation's introduction pagelinks to > Sphinx pages using relative paths. Those paths are hardcoded based on > the api-html/ and internal-api-html/ directories being children of the > html/ directory. > > This results in links being broken in different ways in the build tree, > as well as in the installation directory: the toctree links direct to > the placeholder pages within the html/ directory instead of the Doxygen > documentation in sibling directories, and the Doxygen introduction links > to Sphinx are simply broken. When publishing documentation on the > website we work around those issues by patching several files and moving > the api-html/ and internal-api-html/ directories to the html/ directory. > > Fixing this is surprisingly difficult. The toctree links can't be > changed to point to a path outside of the Sphinx document tree as this > isn't supported by Sphinx. Using http URLs would link to the > libcamera.org website inside of local documentation, which isn't > acceptable. It may be possible to develop a Sphinx extension to patch > the toctree after it gets parsed, but that would be complex and likely > fragile. Modifying the install path of the Doxygen documentation to > html/api-html/ and html/internal-api-html/ causes issues as the Sphinx > documentation will then overwrite the Doxygen index.html files with the > placeholder indexes. Creating symlinks from html/api-html/ to api-html/ > in the installation directory causes similar problems if 'meson install' > is run twice. Creating the symlinks in the build directory (which was > attempted with a custom Sphinx extension) is also a no-go: starting with > meson v1.8.0, installing symlinks to directories causes an exception due > to a bug in meson. > > The right solution is probably to investigate usage of the doxysphinx > extension. As that's no small amount of work, let's start with a > non-perfect but simple improvement: configure doxylink based on the > api-html/ and internal-api-html/ directories being children of the > Sphinx html/ documentation, and move those two API documentation > directories to html/ during installation with a post-install script. > This fixes links in the installation directory. Links in the build > directory remain broken, but that is not a regression. sphinx -> doxygen links were working in the build directory as far as I can tell. But not anymore after changing the doxylinks configuration here. Which links does the above refer to? Regards, Barnabás Pőcze > > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> > --- > Documentation/conf.py.in | 4 ++-- > Documentation/install-doxygen.sh | 18 ++++++++++++++++++ > Documentation/meson.build | 3 +++ > 3 files changed, 23 insertions(+), 2 deletions(-) > create mode 100755 Documentation/install-doxygen.sh > > diff --git a/Documentation/conf.py.in b/Documentation/conf.py.in > index 34fa3956f49e..2c75a75799e6 100644 > --- a/Documentation/conf.py.in > +++ b/Documentation/conf.py.in > @@ -75,11 +75,11 @@ pygments_style = None > doxylink = { > 'doxy-pub': ( > '@TOP_BUILDDIR@/Documentation/api-html/tagfile.xml', > - '../api-html/', > + 'api-html/', > ), > 'doxy-int': ( > '@TOP_BUILDDIR@/Documentation/internal-api-html/tagfile.xml', > - '../internal-api-html/', > + 'internal-api-html/', > ), > } > > diff --git a/Documentation/install-doxygen.sh b/Documentation/install-doxygen.sh > new file mode 100755 > index 000000000000..ea5a19dc8fda > --- /dev/null > +++ b/Documentation/install-doxygen.sh > @@ -0,0 +1,18 @@ > +#!/bin/sh > +# SPDX-License-Identifier: GPL-2.0-or-later > +# Copyright (C) 2025, Ideas on Board Oy > +# > +# Author: Laurent Pinchart <laurent.pinchart@ideasonboard.com> > +# > +# Move Doxygen-generated API documentation to correct location > + > +doc_dir="${MESON_INSTALL_DESTDIR_PREFIX}/$1" > +shift > +dirs="$*" > + > +echo "Moving API documentation" > + > +for dir in $dirs ; do > + rm -r "${doc_dir}/html/${dir}" > + mv "${doc_dir}/${dir}" "${doc_dir}/html/" > +done > diff --git a/Documentation/meson.build b/Documentation/meson.build > index 8cf7775902f3..022770968fcf 100644 > --- a/Documentation/meson.build > +++ b/Documentation/meson.build > @@ -193,6 +193,9 @@ if sphinx.found() > install_dir : doc_install_dir, > install_tag : 'doc') > > + meson.add_install_script('install-doxygen.sh', doc_install_dir, > + 'api-html', 'internal-api-html') > + > custom_target('documentation-linkcheck', > command : [sphinx, '-W', '-b', 'linkcheck', > '-c', sphinx_conf_dir,
On Thu, Sep 18, 2025 at 09:48:05AM +0200, Barnabás Pőcze wrote: > 2025. 09. 17. 22:17 keltezéssel, Laurent Pinchart írta: > > The libcamera documentation comprises two parts: pages generated by > > Sphinx into the Documentation/html/ directory within the build tree, and > > API reference documentation generated by Doxygen into > > Documentation/internal-api-html/ and Documentation/api-html/. The two > > parts are generated separately, but link to each other. > > > > From Sphinx to Doxygen, we use the doxylink extension for Sphinx to > > generate links to the Doxygen pages corresponding to API elements. The > > extension needs to be configured with the paths to the Doxygen > > documentation, which are set based on the html/, api-html/ and > > internal-api-html/ directories being placed side by side in the same > > parent directory. > > > > Furthermore, we also want to link to the API documentation from the > > Sphinx toctree. As toctrees can only link to pages within the Sphinx > > documents tree (or to http URLs), we have placeholder .rst documents for > > api-html and internal-api-html in the Sphinx documentation tree. Those > > generate the Documentation/html/internal-api-html/index.html and > > Documentation/html/api-html/index.html placeholder files in the build > > tree. > > > > The other way around, the API documentation's introduction pagelinks to > > Sphinx pages using relative paths. Those paths are hardcoded based on > > the api-html/ and internal-api-html/ directories being children of the > > html/ directory. > > > > This results in links being broken in different ways in the build tree, > > as well as in the installation directory: the toctree links direct to > > the placeholder pages within the html/ directory instead of the Doxygen > > documentation in sibling directories, and the Doxygen introduction links > > to Sphinx are simply broken. When publishing documentation on the > > website we work around those issues by patching several files and moving > > the api-html/ and internal-api-html/ directories to the html/ directory. > > > > Fixing this is surprisingly difficult. The toctree links can't be > > changed to point to a path outside of the Sphinx document tree as this > > isn't supported by Sphinx. Using http URLs would link to the > > libcamera.org website inside of local documentation, which isn't > > acceptable. It may be possible to develop a Sphinx extension to patch > > the toctree after it gets parsed, but that would be complex and likely > > fragile. Modifying the install path of the Doxygen documentation to > > html/api-html/ and html/internal-api-html/ causes issues as the Sphinx > > documentation will then overwrite the Doxygen index.html files with the > > placeholder indexes. Creating symlinks from html/api-html/ to api-html/ > > in the installation directory causes similar problems if 'meson install' > > is run twice. Creating the symlinks in the build directory (which was > > attempted with a custom Sphinx extension) is also a no-go: starting with > > meson v1.8.0, installing symlinks to directories causes an exception due > > to a bug in meson. > > > > The right solution is probably to investigate usage of the doxysphinx > > extension. As that's no small amount of work, let's start with a > > non-perfect but simple improvement: configure doxylink based on the > > api-html/ and internal-api-html/ directories being children of the > > Sphinx html/ documentation, and move those two API documentation > > directories to html/ during installation with a post-install script. > > This fixes links in the installation directory. Links in the build > > directory remain broken, but that is not a regression. > > sphinx -> doxygen links were working in the build directory as far as I can tell. > But not anymore after changing the doxylinks configuration here. Which links does > the above refer to? The toctree links are currently broken, and so are the links from doxygen to sphinx. We will have a larger number of broken sphinx -> doxyen links in the build tree with this patch, but we'll fix the installation directory. As the build tree has broken links today already, I think that's an acceptable compromise. I also think the installation directory is more important, as in the build tree people also have access to sources. Application developers in particular are less likely to read documentation from the libcamera build tree, they may not even have the libcamera source tree. > > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> > > --- > > Documentation/conf.py.in | 4 ++-- > > Documentation/install-doxygen.sh | 18 ++++++++++++++++++ > > Documentation/meson.build | 3 +++ > > 3 files changed, 23 insertions(+), 2 deletions(-) > > create mode 100755 Documentation/install-doxygen.sh > > > > diff --git a/Documentation/conf.py.in b/Documentation/conf.py.in > > index 34fa3956f49e..2c75a75799e6 100644 > > --- a/Documentation/conf.py.in > > +++ b/Documentation/conf.py.in > > @@ -75,11 +75,11 @@ pygments_style = None > > doxylink = { > > 'doxy-pub': ( > > '@TOP_BUILDDIR@/Documentation/api-html/tagfile.xml', > > - '../api-html/', > > + 'api-html/', > > ), > > 'doxy-int': ( > > '@TOP_BUILDDIR@/Documentation/internal-api-html/tagfile.xml', > > - '../internal-api-html/', > > + 'internal-api-html/', > > ), > > } > > > > diff --git a/Documentation/install-doxygen.sh b/Documentation/install-doxygen.sh > > new file mode 100755 > > index 000000000000..ea5a19dc8fda > > --- /dev/null > > +++ b/Documentation/install-doxygen.sh > > @@ -0,0 +1,18 @@ > > +#!/bin/sh > > +# SPDX-License-Identifier: GPL-2.0-or-later > > +# Copyright (C) 2025, Ideas on Board Oy > > +# > > +# Author: Laurent Pinchart <laurent.pinchart@ideasonboard.com> > > +# > > +# Move Doxygen-generated API documentation to correct location > > + > > +doc_dir="${MESON_INSTALL_DESTDIR_PREFIX}/$1" > > +shift > > +dirs="$*" > > + > > +echo "Moving API documentation" > > + > > +for dir in $dirs ; do > > + rm -r "${doc_dir}/html/${dir}" > > + mv "${doc_dir}/${dir}" "${doc_dir}/html/" > > +done > > diff --git a/Documentation/meson.build b/Documentation/meson.build > > index 8cf7775902f3..022770968fcf 100644 > > --- a/Documentation/meson.build > > +++ b/Documentation/meson.build > > @@ -193,6 +193,9 @@ if sphinx.found() > > install_dir : doc_install_dir, > > install_tag : 'doc') > > > > + meson.add_install_script('install-doxygen.sh', doc_install_dir, > > + 'api-html', 'internal-api-html') > > + > > custom_target('documentation-linkcheck', > > command : [sphinx, '-W', '-b', 'linkcheck', > > '-c', sphinx_conf_dir,
On Thu, Sep 18, 2025 at 09:23:39AM +0200, Stefan Klug wrote: > Hi Laurent, > > Thank you for the patch. > > Quoting Laurent Pinchart (2025-09-17 22:17:38) > > The libcamera documentation comprises two parts: pages generated by > > Sphinx into the Documentation/html/ directory within the build tree, and > > API reference documentation generated by Doxygen into > > Documentation/internal-api-html/ and Documentation/api-html/. The two > > parts are generated separately, but link to each other. > > > > From Sphinx to Doxygen, we use the doxylink extension for Sphinx to > > generate links to the Doxygen pages corresponding to API elements. The > > extension needs to be configured with the paths to the Doxygen > > documentation, which are set based on the html/, api-html/ and > > internal-api-html/ directories being placed side by side in the same > > parent directory. > > > > Furthermore, we also want to link to the API documentation from the > > Sphinx toctree. As toctrees can only link to pages within the Sphinx > > documents tree (or to http URLs), we have placeholder .rst documents for > > api-html and internal-api-html in the Sphinx documentation tree. Those > > generate the Documentation/html/internal-api-html/index.html and > > Documentation/html/api-html/index.html placeholder files in the build > > tree. > > We could mention the ../api-html path in these placeholders so that > someone reading the docs from the build dir gets a hint where to look. > > > The other way around, the API documentation's introduction pagelinks to > > Sphinx pages using relative paths. Those paths are hardcoded based on > > the api-html/ and internal-api-html/ directories being children of the > > html/ directory. > > > > This results in links being broken in different ways in the build tree, > > as well as in the installation directory: the toctree links direct to > > the placeholder pages within the html/ directory instead of the Doxygen > > documentation in sibling directories, and the Doxygen introduction links > > to Sphinx are simply broken. When publishing documentation on the > > website we work around those issues by patching several files and moving > > Do we really patch files? To my understanding we only do the move that > is also done by the install step now. Or am I missing something? We don't patch, indeed. We override the sphinx options with a custom conf.py. I'll change that in the commit message and write When publishing documentation on the website we work around those issues by overriding conf.py with a custom version and moving the api-html/ and internal-api-html/ directories to the html/ directory. > We could actually do a meson install --destdir= there. > > > the api-html/ and internal-api-html/ directories to the html/ directory. > > > > Fixing this is surprisingly difficult. The toctree links can't be > > changed to point to a path outside of the Sphinx document tree as this > > isn't supported by Sphinx. Using http URLs would link to the > > libcamera.org website inside of local documentation, which isn't > > acceptable. It may be possible to develop a Sphinx extension to patch > > the toctree after it gets parsed, but that would be complex and likely > > fragile. Modifying the install path of the Doxygen documentation to > > html/api-html/ and html/internal-api-html/ causes issues as the Sphinx > > documentation will then overwrite the Doxygen index.html files with the > > placeholder indexes. Creating symlinks from html/api-html/ to api-html/ > > in the installation directory causes similar problems if 'meson install' > > is run twice. Creating the symlinks in the build directory (which was > > attempted with a custom Sphinx extension) is also a no-go: starting with > > meson v1.8.0, installing symlinks to directories causes an exception due > > to a bug in meson. > > > > The right solution is probably to investigate usage of the doxysphinx > > extension. As that's no small amount of work, let's start with a > > non-perfect but simple improvement: configure doxylink based on the > > api-html/ and internal-api-html/ directories being children of the > > Sphinx html/ documentation, and move those two API documentation > > directories to html/ during installation with a post-install script. > > This fixes links in the installation directory. Links in the build > > directory remain broken, but that is not a regression. > > > > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> > > Phew, that was a ride. Now the initial cp moved into the install step. I > like that workaround. > > Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com> > > > --- > > Documentation/conf.py.in | 4 ++-- > > Documentation/install-doxygen.sh | 18 ++++++++++++++++++ > > Documentation/meson.build | 3 +++ > > 3 files changed, 23 insertions(+), 2 deletions(-) > > create mode 100755 Documentation/install-doxygen.sh > > > > diff --git a/Documentation/conf.py.in b/Documentation/conf.py.in > > index 34fa3956f49e..2c75a75799e6 100644 > > --- a/Documentation/conf.py.in > > +++ b/Documentation/conf.py.in > > @@ -75,11 +75,11 @@ pygments_style = None > > doxylink = { > > 'doxy-pub': ( > > '@TOP_BUILDDIR@/Documentation/api-html/tagfile.xml', > > - '../api-html/', > > + 'api-html/', > > ), > > 'doxy-int': ( > > '@TOP_BUILDDIR@/Documentation/internal-api-html/tagfile.xml', > > - '../internal-api-html/', > > + 'internal-api-html/', > > ), > > } > > > > diff --git a/Documentation/install-doxygen.sh b/Documentation/install-doxygen.sh > > new file mode 100755 > > index 000000000000..ea5a19dc8fda > > --- /dev/null > > +++ b/Documentation/install-doxygen.sh > > @@ -0,0 +1,18 @@ > > +#!/bin/sh > > +# SPDX-License-Identifier: GPL-2.0-or-later > > +# Copyright (C) 2025, Ideas on Board Oy > > +# > > +# Author: Laurent Pinchart <laurent.pinchart@ideasonboard.com> > > +# > > +# Move Doxygen-generated API documentation to correct location > > + > > +doc_dir="${MESON_INSTALL_DESTDIR_PREFIX}/$1" > > +shift > > +dirs="$*" > > + > > +echo "Moving API documentation" > > + > > +for dir in $dirs ; do > > + rm -r "${doc_dir}/html/${dir}" > > + mv "${doc_dir}/${dir}" "${doc_dir}/html/" > > +done > > diff --git a/Documentation/meson.build b/Documentation/meson.build > > index 8cf7775902f3..022770968fcf 100644 > > --- a/Documentation/meson.build > > +++ b/Documentation/meson.build > > @@ -193,6 +193,9 @@ if sphinx.found() > > install_dir : doc_install_dir, > > install_tag : 'doc') > > > > + meson.add_install_script('install-doxygen.sh', doc_install_dir, > > + 'api-html', 'internal-api-html') > > + > > custom_target('documentation-linkcheck', > > command : [sphinx, '-W', '-b', 'linkcheck', > > '-c', sphinx_conf_dir,
Quoting Laurent Pinchart (2025-09-18 10:29:56) > On Thu, Sep 18, 2025 at 09:23:39AM +0200, Stefan Klug wrote: > > Hi Laurent, > > > > Thank you for the patch. > > > > Quoting Laurent Pinchart (2025-09-17 22:17:38) > > > The libcamera documentation comprises two parts: pages generated by > > > Sphinx into the Documentation/html/ directory within the build tree, and > > > API reference documentation generated by Doxygen into > > > Documentation/internal-api-html/ and Documentation/api-html/. The two > > > parts are generated separately, but link to each other. > > > > > > From Sphinx to Doxygen, we use the doxylink extension for Sphinx to > > > generate links to the Doxygen pages corresponding to API elements. The > > > extension needs to be configured with the paths to the Doxygen > > > documentation, which are set based on the html/, api-html/ and > > > internal-api-html/ directories being placed side by side in the same > > > parent directory. > > > > > > Furthermore, we also want to link to the API documentation from the > > > Sphinx toctree. As toctrees can only link to pages within the Sphinx > > > documents tree (or to http URLs), we have placeholder .rst documents for > > > api-html and internal-api-html in the Sphinx documentation tree. Those > > > generate the Documentation/html/internal-api-html/index.html and > > > Documentation/html/api-html/index.html placeholder files in the build > > > tree. > > > > We could mention the ../api-html path in these placeholders so that > > someone reading the docs from the build dir gets a hint where to look. > > > > > The other way around, the API documentation's introduction pagelinks to > > > Sphinx pages using relative paths. Those paths are hardcoded based on > > > the api-html/ and internal-api-html/ directories being children of the > > > html/ directory. > > > > > > This results in links being broken in different ways in the build tree, > > > as well as in the installation directory: the toctree links direct to > > > the placeholder pages within the html/ directory instead of the Doxygen > > > documentation in sibling directories, and the Doxygen introduction links > > > to Sphinx are simply broken. When publishing documentation on the > > > website we work around those issues by patching several files and moving > > > > Do we really patch files? To my understanding we only do the move that > > is also done by the install step now. Or am I missing something? > > We don't patch, indeed. We override the sphinx options with a custom > conf.py. I'll change that in the commit message and write > > When publishing documentation on the website we work around those issues > by overriding conf.py with a custom version and moving the api-html/ and > internal-api-html/ directories to the html/ directory. Where does this override happen? I thought we only have a conf.py for the website itself, but docs is now fully self contained (except replicating the install step manually)? Best regards, Stefan > > > We could actually do a meson install --destdir= there. > > > > > the api-html/ and internal-api-html/ directories to the html/ directory. > > > > > > Fixing this is surprisingly difficult. The toctree links can't be > > > changed to point to a path outside of the Sphinx document tree as this > > > isn't supported by Sphinx. Using http URLs would link to the > > > libcamera.org website inside of local documentation, which isn't > > > acceptable. It may be possible to develop a Sphinx extension to patch > > > the toctree after it gets parsed, but that would be complex and likely > > > fragile. Modifying the install path of the Doxygen documentation to > > > html/api-html/ and html/internal-api-html/ causes issues as the Sphinx > > > documentation will then overwrite the Doxygen index.html files with the > > > placeholder indexes. Creating symlinks from html/api-html/ to api-html/ > > > in the installation directory causes similar problems if 'meson install' > > > is run twice. Creating the symlinks in the build directory (which was > > > attempted with a custom Sphinx extension) is also a no-go: starting with > > > meson v1.8.0, installing symlinks to directories causes an exception due > > > to a bug in meson. > > > > > > The right solution is probably to investigate usage of the doxysphinx > > > extension. As that's no small amount of work, let's start with a > > > non-perfect but simple improvement: configure doxylink based on the > > > api-html/ and internal-api-html/ directories being children of the > > > Sphinx html/ documentation, and move those two API documentation > > > directories to html/ during installation with a post-install script. > > > This fixes links in the installation directory. Links in the build > > > directory remain broken, but that is not a regression. > > > > > > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> > > > > Phew, that was a ride. Now the initial cp moved into the install step. I > > like that workaround. > > > > Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com> > > > > > --- > > > Documentation/conf.py.in | 4 ++-- > > > Documentation/install-doxygen.sh | 18 ++++++++++++++++++ > > > Documentation/meson.build | 3 +++ > > > 3 files changed, 23 insertions(+), 2 deletions(-) > > > create mode 100755 Documentation/install-doxygen.sh > > > > > > diff --git a/Documentation/conf.py.in b/Documentation/conf.py.in > > > index 34fa3956f49e..2c75a75799e6 100644 > > > --- a/Documentation/conf.py.in > > > +++ b/Documentation/conf.py.in > > > @@ -75,11 +75,11 @@ pygments_style = None > > > doxylink = { > > > 'doxy-pub': ( > > > '@TOP_BUILDDIR@/Documentation/api-html/tagfile.xml', > > > - '../api-html/', > > > + 'api-html/', > > > ), > > > 'doxy-int': ( > > > '@TOP_BUILDDIR@/Documentation/internal-api-html/tagfile.xml', > > > - '../internal-api-html/', > > > + 'internal-api-html/', > > > ), > > > } > > > > > > diff --git a/Documentation/install-doxygen.sh b/Documentation/install-doxygen.sh > > > new file mode 100755 > > > index 000000000000..ea5a19dc8fda > > > --- /dev/null > > > +++ b/Documentation/install-doxygen.sh > > > @@ -0,0 +1,18 @@ > > > +#!/bin/sh > > > +# SPDX-License-Identifier: GPL-2.0-or-later > > > +# Copyright (C) 2025, Ideas on Board Oy > > > +# > > > +# Author: Laurent Pinchart <laurent.pinchart@ideasonboard.com> > > > +# > > > +# Move Doxygen-generated API documentation to correct location > > > + > > > +doc_dir="${MESON_INSTALL_DESTDIR_PREFIX}/$1" > > > +shift > > > +dirs="$*" > > > + > > > +echo "Moving API documentation" > > > + > > > +for dir in $dirs ; do > > > + rm -r "${doc_dir}/html/${dir}" > > > + mv "${doc_dir}/${dir}" "${doc_dir}/html/" > > > +done > > > diff --git a/Documentation/meson.build b/Documentation/meson.build > > > index 8cf7775902f3..022770968fcf 100644 > > > --- a/Documentation/meson.build > > > +++ b/Documentation/meson.build > > > @@ -193,6 +193,9 @@ if sphinx.found() > > > install_dir : doc_install_dir, > > > install_tag : 'doc') > > > > > > + meson.add_install_script('install-doxygen.sh', doc_install_dir, > > > + 'api-html', 'internal-api-html') > > > + > > > custom_target('documentation-linkcheck', > > > command : [sphinx, '-W', '-b', 'linkcheck', > > > '-c', sphinx_conf_dir, > > -- > Regards, > > Laurent Pinchart
On Thu, Sep 18, 2025 at 01:19:37PM +0200, Stefan Klug wrote: > Quoting Laurent Pinchart (2025-09-18 10:29:56) > > On Thu, Sep 18, 2025 at 09:23:39AM +0200, Stefan Klug wrote: > > > Hi Laurent, > > > > > > Thank you for the patch. > > > > > > Quoting Laurent Pinchart (2025-09-17 22:17:38) > > > > The libcamera documentation comprises two parts: pages generated by > > > > Sphinx into the Documentation/html/ directory within the build tree, and > > > > API reference documentation generated by Doxygen into > > > > Documentation/internal-api-html/ and Documentation/api-html/. The two > > > > parts are generated separately, but link to each other. > > > > > > > > From Sphinx to Doxygen, we use the doxylink extension for Sphinx to > > > > generate links to the Doxygen pages corresponding to API elements. The > > > > extension needs to be configured with the paths to the Doxygen > > > > documentation, which are set based on the html/, api-html/ and > > > > internal-api-html/ directories being placed side by side in the same > > > > parent directory. > > > > > > > > Furthermore, we also want to link to the API documentation from the > > > > Sphinx toctree. As toctrees can only link to pages within the Sphinx > > > > documents tree (or to http URLs), we have placeholder .rst documents for > > > > api-html and internal-api-html in the Sphinx documentation tree. Those > > > > generate the Documentation/html/internal-api-html/index.html and > > > > Documentation/html/api-html/index.html placeholder files in the build > > > > tree. > > > > > > We could mention the ../api-html path in these placeholders so that > > > someone reading the docs from the build dir gets a hint where to look. > > > > > > > The other way around, the API documentation's introduction pagelinks to > > > > Sphinx pages using relative paths. Those paths are hardcoded based on > > > > the api-html/ and internal-api-html/ directories being children of the > > > > html/ directory. > > > > > > > > This results in links being broken in different ways in the build tree, > > > > as well as in the installation directory: the toctree links direct to > > > > the placeholder pages within the html/ directory instead of the Doxygen > > > > documentation in sibling directories, and the Doxygen introduction links > > > > to Sphinx are simply broken. When publishing documentation on the > > > > website we work around those issues by patching several files and moving > > > > > > Do we really patch files? To my understanding we only do the move that > > > is also done by the install step now. Or am I missing something? > > > > We don't patch, indeed. We override the sphinx options with a custom > > conf.py. I'll change that in the commit message and write > > > > When publishing documentation on the website we work around those issues > > by overriding conf.py with a custom version and moving the api-html/ and > > internal-api-html/ directories to the html/ directory. > > Where does this override happen? I thought we only have a conf.py for > the website itself, but docs is now fully self contained (except > replicating the install step manually)? That conf.py sets the doxylink configuration parameters with different paths than the conf.py in libcamera. > > > We could actually do a meson install --destdir= there. > > > > > > > the api-html/ and internal-api-html/ directories to the html/ directory. > > > > > > > > Fixing this is surprisingly difficult. The toctree links can't be > > > > changed to point to a path outside of the Sphinx document tree as this > > > > isn't supported by Sphinx. Using http URLs would link to the > > > > libcamera.org website inside of local documentation, which isn't > > > > acceptable. It may be possible to develop a Sphinx extension to patch > > > > the toctree after it gets parsed, but that would be complex and likely > > > > fragile. Modifying the install path of the Doxygen documentation to > > > > html/api-html/ and html/internal-api-html/ causes issues as the Sphinx > > > > documentation will then overwrite the Doxygen index.html files with the > > > > placeholder indexes. Creating symlinks from html/api-html/ to api-html/ > > > > in the installation directory causes similar problems if 'meson install' > > > > is run twice. Creating the symlinks in the build directory (which was > > > > attempted with a custom Sphinx extension) is also a no-go: starting with > > > > meson v1.8.0, installing symlinks to directories causes an exception due > > > > to a bug in meson. > > > > > > > > The right solution is probably to investigate usage of the doxysphinx > > > > extension. As that's no small amount of work, let's start with a > > > > non-perfect but simple improvement: configure doxylink based on the > > > > api-html/ and internal-api-html/ directories being children of the > > > > Sphinx html/ documentation, and move those two API documentation > > > > directories to html/ during installation with a post-install script. > > > > This fixes links in the installation directory. Links in the build > > > > directory remain broken, but that is not a regression. > > > > > > > > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> > > > > > > Phew, that was a ride. Now the initial cp moved into the install step. I > > > like that workaround. > > > > > > Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com> > > > > > > > --- > > > > Documentation/conf.py.in | 4 ++-- > > > > Documentation/install-doxygen.sh | 18 ++++++++++++++++++ > > > > Documentation/meson.build | 3 +++ > > > > 3 files changed, 23 insertions(+), 2 deletions(-) > > > > create mode 100755 Documentation/install-doxygen.sh > > > > > > > > diff --git a/Documentation/conf.py.in b/Documentation/conf.py.in > > > > index 34fa3956f49e..2c75a75799e6 100644 > > > > --- a/Documentation/conf.py.in > > > > +++ b/Documentation/conf.py.in > > > > @@ -75,11 +75,11 @@ pygments_style = None > > > > doxylink = { > > > > 'doxy-pub': ( > > > > '@TOP_BUILDDIR@/Documentation/api-html/tagfile.xml', > > > > - '../api-html/', > > > > + 'api-html/', > > > > ), > > > > 'doxy-int': ( > > > > '@TOP_BUILDDIR@/Documentation/internal-api-html/tagfile.xml', > > > > - '../internal-api-html/', > > > > + 'internal-api-html/', > > > > ), > > > > } > > > > > > > > diff --git a/Documentation/install-doxygen.sh b/Documentation/install-doxygen.sh > > > > new file mode 100755 > > > > index 000000000000..ea5a19dc8fda > > > > --- /dev/null > > > > +++ b/Documentation/install-doxygen.sh > > > > @@ -0,0 +1,18 @@ > > > > +#!/bin/sh > > > > +# SPDX-License-Identifier: GPL-2.0-or-later > > > > +# Copyright (C) 2025, Ideas on Board Oy > > > > +# > > > > +# Author: Laurent Pinchart <laurent.pinchart@ideasonboard.com> > > > > +# > > > > +# Move Doxygen-generated API documentation to correct location > > > > + > > > > +doc_dir="${MESON_INSTALL_DESTDIR_PREFIX}/$1" > > > > +shift > > > > +dirs="$*" > > > > + > > > > +echo "Moving API documentation" > > > > + > > > > +for dir in $dirs ; do > > > > + rm -r "${doc_dir}/html/${dir}" > > > > + mv "${doc_dir}/${dir}" "${doc_dir}/html/" > > > > +done > > > > diff --git a/Documentation/meson.build b/Documentation/meson.build > > > > index 8cf7775902f3..022770968fcf 100644 > > > > --- a/Documentation/meson.build > > > > +++ b/Documentation/meson.build > > > > @@ -193,6 +193,9 @@ if sphinx.found() > > > > install_dir : doc_install_dir, > > > > install_tag : 'doc') > > > > > > > > + meson.add_install_script('install-doxygen.sh', doc_install_dir, > > > > + 'api-html', 'internal-api-html') > > > > + > > > > custom_target('documentation-linkcheck', > > > > command : [sphinx, '-W', '-b', 'linkcheck', > > > > '-c', sphinx_conf_dir,
diff --git a/Documentation/conf.py.in b/Documentation/conf.py.in index 34fa3956f49e..2c75a75799e6 100644 --- a/Documentation/conf.py.in +++ b/Documentation/conf.py.in @@ -75,11 +75,11 @@ pygments_style = None doxylink = { 'doxy-pub': ( '@TOP_BUILDDIR@/Documentation/api-html/tagfile.xml', - '../api-html/', + 'api-html/', ), 'doxy-int': ( '@TOP_BUILDDIR@/Documentation/internal-api-html/tagfile.xml', - '../internal-api-html/', + 'internal-api-html/', ), } diff --git a/Documentation/install-doxygen.sh b/Documentation/install-doxygen.sh new file mode 100755 index 000000000000..ea5a19dc8fda --- /dev/null +++ b/Documentation/install-doxygen.sh @@ -0,0 +1,18 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (C) 2025, Ideas on Board Oy +# +# Author: Laurent Pinchart <laurent.pinchart@ideasonboard.com> +# +# Move Doxygen-generated API documentation to correct location + +doc_dir="${MESON_INSTALL_DESTDIR_PREFIX}/$1" +shift +dirs="$*" + +echo "Moving API documentation" + +for dir in $dirs ; do + rm -r "${doc_dir}/html/${dir}" + mv "${doc_dir}/${dir}" "${doc_dir}/html/" +done diff --git a/Documentation/meson.build b/Documentation/meson.build index 8cf7775902f3..022770968fcf 100644 --- a/Documentation/meson.build +++ b/Documentation/meson.build @@ -193,6 +193,9 @@ if sphinx.found() install_dir : doc_install_dir, install_tag : 'doc') + meson.add_install_script('install-doxygen.sh', doc_install_dir, + 'api-html', 'internal-api-html') + custom_target('documentation-linkcheck', command : [sphinx, '-W', '-b', 'linkcheck', '-c', sphinx_conf_dir,
The libcamera documentation comprises two parts: pages generated by Sphinx into the Documentation/html/ directory within the build tree, and API reference documentation generated by Doxygen into Documentation/internal-api-html/ and Documentation/api-html/. The two parts are generated separately, but link to each other. From Sphinx to Doxygen, we use the doxylink extension for Sphinx to generate links to the Doxygen pages corresponding to API elements. The extension needs to be configured with the paths to the Doxygen documentation, which are set based on the html/, api-html/ and internal-api-html/ directories being placed side by side in the same parent directory. Furthermore, we also want to link to the API documentation from the Sphinx toctree. As toctrees can only link to pages within the Sphinx documents tree (or to http URLs), we have placeholder .rst documents for api-html and internal-api-html in the Sphinx documentation tree. Those generate the Documentation/html/internal-api-html/index.html and Documentation/html/api-html/index.html placeholder files in the build tree. The other way around, the API documentation's introduction pagelinks to Sphinx pages using relative paths. Those paths are hardcoded based on the api-html/ and internal-api-html/ directories being children of the html/ directory. This results in links being broken in different ways in the build tree, as well as in the installation directory: the toctree links direct to the placeholder pages within the html/ directory instead of the Doxygen documentation in sibling directories, and the Doxygen introduction links to Sphinx are simply broken. When publishing documentation on the website we work around those issues by patching several files and moving the api-html/ and internal-api-html/ directories to the html/ directory. Fixing this is surprisingly difficult. The toctree links can't be changed to point to a path outside of the Sphinx document tree as this isn't supported by Sphinx. Using http URLs would link to the libcamera.org website inside of local documentation, which isn't acceptable. It may be possible to develop a Sphinx extension to patch the toctree after it gets parsed, but that would be complex and likely fragile. Modifying the install path of the Doxygen documentation to html/api-html/ and html/internal-api-html/ causes issues as the Sphinx documentation will then overwrite the Doxygen index.html files with the placeholder indexes. Creating symlinks from html/api-html/ to api-html/ in the installation directory causes similar problems if 'meson install' is run twice. Creating the symlinks in the build directory (which was attempted with a custom Sphinx extension) is also a no-go: starting with meson v1.8.0, installing symlinks to directories causes an exception due to a bug in meson. The right solution is probably to investigate usage of the doxysphinx extension. As that's no small amount of work, let's start with a non-perfect but simple improvement: configure doxylink based on the api-html/ and internal-api-html/ directories being children of the Sphinx html/ documentation, and move those two API documentation directories to html/ during installation with a post-install script. This fixes links in the installation directory. Links in the build directory remain broken, but that is not a regression. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> --- Documentation/conf.py.in | 4 ++-- Documentation/install-doxygen.sh | 18 ++++++++++++++++++ Documentation/meson.build | 3 +++ 3 files changed, 23 insertions(+), 2 deletions(-) create mode 100755 Documentation/install-doxygen.sh