[libcamera-devel,v3] Documentation: Add descriptions for env. variables

10336 diff mbox series

Describe the environment variables used in libcamera, excluded
variables are `LIBCAMERA_IPA_FORCE_C_API` and `LIBCAMERA_IPA_PROXY_PATH`,
the former because it is likely to be removed and the later because
it has no current use-case.

Add a brief explanation for the IPA configuration and IPA modules.
List all the available Log levels and categories and add a short guide
for how to use them for debugging.

Signed-off-by: Sebastian Fricke <sebastian.fricke.linux@gmail.com>

---

Changes since V2:
* replace bullet point list with definition list
* Remove the category list as it is too difficult to maintain, instead
  explain how the categories are set with the LOG_DECLARE_CATEGORY &
  LOG_DEFINE_CATEGORY macros
* Remove the short summary for the IPA module, as it can be misleading
  and probably not accurate enough
* Improve the notes for debugging part by linking to a full explaination
  while extending the description in the documentation as well
* Add an example for the Log level setting
* Remove the documentation from guides and add it to the documentation
  base path

Changes since V1:
* abandon the usage of tables as they are too clunky and difficult to
maintain
* Fix a wrong example that does not work on most distributions and setups
* Improve structure of log categories
---
 Documentation/environment_variables.rst | 129 ++++++++++++++++++++++++
 Documentation/index.rst                 |   1 +
 Documentation/meson.build               |   1 +
 3 files changed, 131 insertions(+)
 create mode 100644 Documentation/environment_variables.rst

Hi Sebastian,

On Wed, Nov 04, 2020 at 07:48:48PM +0100, Sebastian Fricke wrote:
> Describe the environment variables used in libcamera, excluded
> variables are `LIBCAMERA_IPA_FORCE_C_API` and `LIBCAMERA_IPA_PROXY_PATH`,
> the former because it is likely to be removed and the later because
> it has no current use-case.
>
> Add a brief explanation for the IPA configuration and IPA modules.
> List all the available Log levels and categories and add a short guide
> for how to use them for debugging.

s/for/on/

>
> Signed-off-by: Sebastian Fricke <sebastian.fricke.linux@gmail.com>
>
> ---
>
> Changes since V2:
> * replace bullet point list with definition list
> * Remove the category list as it is too difficult to maintain, instead
>   explain how the categories are set with the LOG_DECLARE_CATEGORY &
>   LOG_DEFINE_CATEGORY macros
> * Remove the short summary for the IPA module, as it can be misleading
>   and probably not accurate enough
> * Improve the notes for debugging part by linking to a full explaination
>   while extending the description in the documentation as well
> * Add an example for the Log level setting
> * Remove the documentation from guides and add it to the documentation
>   base path
>
> Changes since V1:
> * abandon the usage of tables as they are too clunky and difficult to
> maintain
> * Fix a wrong example that does not work on most distributions and setups
> * Improve structure of log categories
> ---
>  Documentation/environment_variables.rst | 129 ++++++++++++++++++++++++
>  Documentation/index.rst                 |   1 +
>  Documentation/meson.build               |   1 +
>  3 files changed, 131 insertions(+)
>  create mode 100644 Documentation/environment_variables.rst
>
> diff --git a/Documentation/environment_variables.rst b/Documentation/environment_variables.rst
> new file mode 100644
> index 0000000..a03412b
> --- /dev/null
> +++ b/Documentation/environment_variables.rst
> @@ -0,0 +1,129 @@
> +Environment variables
> +=====================
> +
> +List of variables
> +-----------------
> +
> +LIBCAMERA_LOG_FILE
> +   The custom destination for log output.
> +
> +   Example value: ``/home/{user}/camera_log.log``
> +
> +LIBCAMERA_LOG_LEVELS
> +   Configure the verbosity of log messages for different categories (`more <#log-levels>`__)
> +
> +   Example value: ``*:DEBUG``
> +
> +LIBCAMERA_IPA_CONFIG_PATH
> +   Define custom search locations for IPA configurations (`more <#ipa-configuration>`__)
> +
> +   Example value: ``/usr/path/one:/tmp/path/two``
> +
> +LIBCAMERA_IPA_MODULE_PATH
> +   Define custom search locations for IPA modules (`more <#ipa-module>`__)
> +
> +   Example value: ``/usr/path/one:/tmp/path/two``
> +
> +Further details
> +---------------
> +
> +Notes about debugging
> +~~~~~~~~~~~~~~~~~~~~~
> +
> +The environment variables `LIBCAMERA_LOG_FILE` and `LIBCAMERA_LOG_LEVELS`
> +are used to modify the destination and verbosity of messages provided by
> +the libcamera API.

s/the libcamera API/libcamera/
> +
> +The `LIBCAMERA_LOG_LEVELS` variable expects a comma-separated list of
> +'category:level' pairs. The `level <#log-levels>`__ part can either be given
> +as the index or the name in uppercase letters.

not sure about 'index' maybe

"can either be specified by name or by the associated numerical
index" ?

> +While `category <#log-categories>`__ should either be the exact name of the
> +category, the '*' wildcard symbol or no value, which targets every category.

We do pattern matching on categories.

I woudl say:

The `LIBCAMERA_LOG_LEVELS` variable accepts a comma-separated list of
'category:level' pairs.

The `level <#log-levels>`__ part is mandatory and can either be
specified by name or by numerical index associated to each level.

The optional `category <#log-categories>`__ part is a regular
expression which is matched against the categories defined by each
file in the source base using the logging infrastructure.

> +While `category <#log-categories>`__ should either be the exact name of the
> +category, the '*' wildcard symbol or no value, which targets every category.

> +

Double empty line

> +
> +For more information refer to the `API-documentation <http://libcamera.org/api-html/log_8h.html#details>`__
> +
> +Examples:
> +
> +Enable full debug output to a separate file, for every `category <#log-categories>`__
> +within a local environment:

I would drop 'within.. '

> +
> +.. code:: bash
> +
> +        :~$ LIBCAMERA_LOG_FILE='/tmp/example_log.log' \
> +            LIBCAMERA_LOG_LEVELS=0 \
> +            cam --list
> +
> +Enable full debug output for the categories Camera & V4L2 within a
> +global environment:

Ah I get what you mean with local/global.
That's pretty standard handling of environment variables, do we need
it specified here ? It doesn't hurt though, up to you

> +
> +.. code:: bash
> +
> +   :~$ export LIBCAMERA_LOG_LEVELS='Camera:DEBUG,V4L2:DEBUG'
> +   :~$ cam --list
> +
> +Log levels
> +~~~~~~~~~~~
> +
> +This is the list of available log levels, notice that all levels below
> +the chosen one are printed, while those above are discarded.
> +
> +-  DEBUG (0)
> +-  INFO (1)
> +-  WARN   (2)
           ^ double space. Intentional ?
> +-  ERROR (3)
> +-  FATAL (4)
> +
> +Example:
> +If you choose WARN (2), you will be able to see WARN (2), ERROR (3) & FATAL (4)
> +but not DEBUG (0) & INFO (1).
> +
> +Log categories
> +~~~~~~~~~~~~~~~
> +
> +Every category represent a specific area of the libcamera codebase,
> +the names can be located within the source code, for example:
> +`src/libcamera/camera_manager.cpp <https://git.libcamera.org/libcamera/libcamera.git/tree/src/libcamera/camera_manager.cpp#n35>`__
> +
> +.. code:: bash
> +
> +   LOG_DEFINE_CATEGORY(Camera)
> +
> +There are 2 available macros used to assign a category name to a part of the
> +libcamera API:

s/API/code base/

> +
> +LOG_DEFINE_CATEGORY
> +        This macro is required, in order to use the `LOGC` macro for a
> +	particular category.  It can only be used once for each category.
> +	If you want to create log messages within multiple compilation
> +	units for the same category utilize the `LOG_DECLARE_CATEGORY` macro,
> +	in every file except the definition file.
> +LOG_DECLARE_CATEGORY
> +        Used for sharing an already defined category in between multiple
> +	separately compiled files.

compilation units maybe ?

> +
> +Both macros have to be used within the namespace part of the C++ source code.

within the libcamera namespace.

This part is nice, but it's not much about environment variables but
on how to use the logging infrastructure.

> +

double empty line. Intentional /

> +
> +IPA configuration
> +~~~~~~~~~~~~~~~~~~
> +
> +The format and contents of the configuration file are specific to the
> +IPA (Image Processing Algorithm). It usually contains data that optimize

s/data that optimize... /tuning parameters for the IPA algorithms/
> +the behavior of the algorithms stored in JSON format. The
> +`LIBCAMERA_IPA_CONFIG_PATH` variable can be used to specify custom
> +storage locations to search for those configuration files.
> +
> +`Examples <https://git.libcamera.org/libcamera/libcamera.git/tree/src/ipa/raspberrypi/data>`__
> +
> +IPA module
> +~~~~~~~~~~~
> +
> +In order to locate the correct IPA module for your hardware, libcamera gathers
> +existing IPA modules from multiple locations. The default locations for this
> +operation are the installed system path (for example on Debian:
> +``/usr/local/x86_64-pc-linux-gnu/libcamera``) and the build directory.
> +With the `LIBCAMERA_IPA_MODULE_PATH`, you can specify a non-default
> +location to search for IPA modules.
> +
> +`Examples for existing IPA modules <https://git.libcamera.org/libcamera/libcamera.git/tree/src/ipa>`__

Thanks, very useful ;)

With the above fixed
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>

Thanks
  j

> diff --git a/Documentation/index.rst b/Documentation/index.rst
> index ff697d4..c49db18 100644
> --- a/Documentation/index.rst
> +++ b/Documentation/index.rst
> @@ -17,3 +17,4 @@
>     Application Writer's Guide <guides/application-developer>
>     Pipeline Handler Writer's Guide <guides/pipeline-handler>
>     Tracing guide <guides/tracing>
> +   Environment variables <environment_variables>
> diff --git a/Documentation/meson.build b/Documentation/meson.build
> index 26a12fc..8086abf 100644
> --- a/Documentation/meson.build
> +++ b/Documentation/meson.build
> @@ -53,6 +53,7 @@ if sphinx.found()
>          'contributing.rst',
>          'docs.rst',
>          'index.rst',
> +        'environment_variables.rst',
>          'guides/introduction.rst',
>          'guides/application-developer.rst',
>          'guides/pipeline-handler.rst',
> --
> 2.20.1
>
> _______________________________________________
> libcamera-devel mailing list
> libcamera-devel@lists.libcamera.org
> https://lists.libcamera.org/listinfo/libcamera-devel

On 17.11.2020 18:24, Jacopo Mondi wrote:
>Hi Sebastian,

Hi Jacopo,

thanks for the review!

>
>On Wed, Nov 04, 2020 at 07:48:48PM +0100, Sebastian Fricke wrote:
>> Describe the environment variables used in libcamera, excluded
>> variables are `LIBCAMERA_IPA_FORCE_C_API` and `LIBCAMERA_IPA_PROXY_PATH`,
>> the former because it is likely to be removed and the later because
>> it has no current use-case.
>>
>> Add a brief explanation for the IPA configuration and IPA modules.
>> List all the available Log levels and categories and add a short guide
>> for how to use them for debugging.
>
>s/for/on/
>
>>
>> Signed-off-by: Sebastian Fricke <sebastian.fricke.linux@gmail.com>
>>
>> ---
>>
>> Changes since V2:
>> * replace bullet point list with definition list
>> * Remove the category list as it is too difficult to maintain, instead
>>   explain how the categories are set with the LOG_DECLARE_CATEGORY &
>>   LOG_DEFINE_CATEGORY macros
>> * Remove the short summary for the IPA module, as it can be misleading
>>   and probably not accurate enough
>> * Improve the notes for debugging part by linking to a full explaination
>>   while extending the description in the documentation as well
>> * Add an example for the Log level setting
>> * Remove the documentation from guides and add it to the documentation
>>   base path
>>
>> Changes since V1:
>> * abandon the usage of tables as they are too clunky and difficult to
>> maintain
>> * Fix a wrong example that does not work on most distributions and setups
>> * Improve structure of log categories
>> ---
>>  Documentation/environment_variables.rst | 129 ++++++++++++++++++++++++
>>  Documentation/index.rst                 |   1 +
>>  Documentation/meson.build               |   1 +
>>  3 files changed, 131 insertions(+)
>>  create mode 100644 Documentation/environment_variables.rst
>>
>> diff --git a/Documentation/environment_variables.rst b/Documentation/environment_variables.rst
>> new file mode 100644
>> index 0000000..a03412b
>> --- /dev/null
>> +++ b/Documentation/environment_variables.rst
>> @@ -0,0 +1,129 @@
>> +Environment variables
>> +=====================
>> +
>> +List of variables
>> +-----------------
>> +
>> +LIBCAMERA_LOG_FILE
>> +   The custom destination for log output.
>> +
>> +   Example value: ``/home/{user}/camera_log.log``
>> +
>> +LIBCAMERA_LOG_LEVELS
>> +   Configure the verbosity of log messages for different categories (`more <#log-levels>`__)
>> +
>> +   Example value: ``*:DEBUG``
>> +
>> +LIBCAMERA_IPA_CONFIG_PATH
>> +   Define custom search locations for IPA configurations (`more <#ipa-configuration>`__)
>> +
>> +   Example value: ``/usr/path/one:/tmp/path/two``
>> +
>> +LIBCAMERA_IPA_MODULE_PATH
>> +   Define custom search locations for IPA modules (`more <#ipa-module>`__)
>> +
>> +   Example value: ``/usr/path/one:/tmp/path/two``
>> +
>> +Further details
>> +---------------
>> +
>> +Notes about debugging
>> +~~~~~~~~~~~~~~~~~~~~~
>> +
>> +The environment variables `LIBCAMERA_LOG_FILE` and `LIBCAMERA_LOG_LEVELS`
>> +are used to modify the destination and verbosity of messages provided by
>> +the libcamera API.
>
>s/the libcamera API/libcamera/
>> +
>> +The `LIBCAMERA_LOG_LEVELS` variable expects a comma-separated list of
>> +'category:level' pairs. The `level <#log-levels>`__ part can either be given
>> +as the index or the name in uppercase letters.
>
>not sure about 'index' maybe
>
>"can either be specified by name or by the associated numerical
>index" ?
>
>> +While `category <#log-categories>`__ should either be the exact name of the
>> +category, the '*' wildcard symbol or no value, which targets every category.
>
>We do pattern matching on categories.
>
>I woudl say:
>
>The `LIBCAMERA_LOG_LEVELS` variable accepts a comma-separated list of
>'category:level' pairs.
>
>The `level <#log-levels>`__ part is mandatory and can either be
>specified by name or by numerical index associated to each level.
>
>The optional `category <#log-categories>`__ part is a regular
>expression which is matched against the categories defined by each
>file in the source base using the logging infrastructure.
>
>> +While `category <#log-categories>`__ should either be the exact name of the
>> +category, the '*' wildcard symbol or no value, which targets every category.
>
>> +
>
>Double empty line
>
>> +
>> +For more information refer to the `API-documentation <http://libcamera.org/api-html/log_8h.html#details>`__
>> +
>> +Examples:
>> +
>> +Enable full debug output to a separate file, for every `category <#log-categories>`__
>> +within a local environment:
>
>I would drop 'within.. '
>
>> +
>> +.. code:: bash
>> +
>> +        :~$ LIBCAMERA_LOG_FILE='/tmp/example_log.log' \
>> +            LIBCAMERA_LOG_LEVELS=0 \
>> +            cam --list
>> +
>> +Enable full debug output for the categories Camera & V4L2 within a
>> +global environment:
>
>Ah I get what you mean with local/global.
>That's pretty standard handling of environment variables, do we need
>it specified here ? It doesn't hurt though, up to you

I am not too sure if we have to keep it, I just noticed while testing
around that it is sometimes quite handy to know that there are different
ways of doing something. And I know that there complete tutorials about
this stuff and we can not explain every thing about the unix environment
within this documnetation. My thought process for this was the
following:
If I am able to explain two variants of doing something within about the
same amount of words, then why not?

>
>> +
>> +.. code:: bash
>> +
>> +   :~$ export LIBCAMERA_LOG_LEVELS='Camera:DEBUG,V4L2:DEBUG'
>> +   :~$ cam --list
>> +
>> +Log levels
>> +~~~~~~~~~~~
>> +
>> +This is the list of available log levels, notice that all levels below
>> +the chosen one are printed, while those above are discarded.
>> +
>> +-  DEBUG (0)
>> +-  INFO (1)
>> +-  WARN   (2)
>           ^ double space. Intentional ?
>> +-  ERROR (3)
>> +-  FATAL (4)
>> +
>> +Example:
>> +If you choose WARN (2), you will be able to see WARN (2), ERROR (3) & FATAL (4)
>> +but not DEBUG (0) & INFO (1).
>> +
>> +Log categories
>> +~~~~~~~~~~~~~~~
>> +
>> +Every category represent a specific area of the libcamera codebase,
>> +the names can be located within the source code, for example:
>> +`src/libcamera/camera_manager.cpp <https://git.libcamera.org/libcamera/libcamera.git/tree/src/libcamera/camera_manager.cpp#n35>`__
>> +
>> +.. code:: bash
>> +
>> +   LOG_DEFINE_CATEGORY(Camera)
>> +
>> +There are 2 available macros used to assign a category name to a part of the
>> +libcamera API:
>
>s/API/code base/
>
>> +
>> +LOG_DEFINE_CATEGORY
>> +        This macro is required, in order to use the `LOGC` macro for a
>> +	particular category.  It can only be used once for each category.
>> +	If you want to create log messages within multiple compilation
>> +	units for the same category utilize the `LOG_DECLARE_CATEGORY` macro,
>> +	in every file except the definition file.
>> +LOG_DECLARE_CATEGORY
>> +        Used for sharing an already defined category in between multiple
>> +	separately compiled files.
>
>compilation units maybe ?
>
>> +
>> +Both macros have to be used within the namespace part of the C++ source code.
>
>within the libcamera namespace.
>
>This part is nice, but it's not much about environment variables but
>on how to use the logging infrastructure.

I've got the main idea from Laurent,
quote:
"Would it make sense to instead link to the Doxygen-generated documentation
(and possibly expanding it a little bit to mention LOG_DEFINE_CATEGORY and LOG_DECLARE_CATEGORY)"

And I think that it fits quite well in here as logging is very tightly
related to the environment variables, because you cannot activate the
logging without the variables. Also I think we do not have enough
content for a logging documentation on it's own.
I believe it helps using the env. variable, when you understand where
those names come from and how to expand the logging if one likes to.

>
>> +
>
>double empty line. Intentional /
>
>> +
>> +IPA configuration
>> +~~~~~~~~~~~~~~~~~~
>> +
>> +The format and contents of the configuration file are specific to the
>> +IPA (Image Processing Algorithm). It usually contains data that optimize
>
>s/data that optimize... /tuning parameters for the IPA algorithms/
>> +the behavior of the algorithms stored in JSON format. The
>> +`LIBCAMERA_IPA_CONFIG_PATH` variable can be used to specify custom
>> +storage locations to search for those configuration files.
>> +
>> +`Examples <https://git.libcamera.org/libcamera/libcamera.git/tree/src/ipa/raspberrypi/data>`__
>> +
>> +IPA module
>> +~~~~~~~~~~~
>> +
>> +In order to locate the correct IPA module for your hardware, libcamera gathers
>> +existing IPA modules from multiple locations. The default locations for this
>> +operation are the installed system path (for example on Debian:
>> +``/usr/local/x86_64-pc-linux-gnu/libcamera``) and the build directory.
>> +With the `LIBCAMERA_IPA_MODULE_PATH`, you can specify a non-default
>> +location to search for IPA modules.
>> +
>> +`Examples for existing IPA modules <https://git.libcamera.org/libcamera/libcamera.git/tree/src/ipa>`__
>
>Thanks, very useful ;)

Thank you very much :)

>
>With the above fixed
>Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
>
>Thanks
>  j
>
>> diff --git a/Documentation/index.rst b/Documentation/index.rst
>> index ff697d4..c49db18 100644
>> --- a/Documentation/index.rst
>> +++ b/Documentation/index.rst
>> @@ -17,3 +17,4 @@
>>     Application Writer's Guide <guides/application-developer>
>>     Pipeline Handler Writer's Guide <guides/pipeline-handler>
>>     Tracing guide <guides/tracing>
>> +   Environment variables <environment_variables>
>> diff --git a/Documentation/meson.build b/Documentation/meson.build
>> index 26a12fc..8086abf 100644
>> --- a/Documentation/meson.build
>> +++ b/Documentation/meson.build
>> @@ -53,6 +53,7 @@ if sphinx.found()
>>          'contributing.rst',
>>          'docs.rst',
>>          'index.rst',
>> +        'environment_variables.rst',
>>          'guides/introduction.rst',
>>          'guides/application-developer.rst',
>>          'guides/pipeline-handler.rst',
>> --
>> 2.20.1
>>
>> _______________________________________________
>> libcamera-devel mailing list
>> libcamera-devel@lists.libcamera.org
>> https://lists.libcamera.org/listinfo/libcamera-devel