[libcamera-devel,v1,0/6] External IPU3 IPA support
mbox series

Message ID 20210514075808.282479-1-umang.jain@ideasonboard.com
Headers show
Series
  • External IPU3 IPA support
Related show

Message

Umang Jain May 14, 2021, 7:58 a.m. UTC
This series targets towards supporting external IPA module
for IPU3. To present a high level view of what things needs
in-place:

I) Support in libcamera (this series)

II) Have a standalone repo for IPU3 IPA holding
     - TBD at: https://git.libcamera.org/libcamera/ipu3-ipa.git/

III) ChromeOS build integration
     - .ebuild file for II)

II) and III) are present locally on my system and I have been testing I)
against them. After satisfactory testing, I am posting the I) on this
list for reviews, while scrub things on II) and III) in parallel.

Kieran Bingham (1):
  libcamera: pipeline: ipu3: Pass request metadata to IPA

Umang Jain (5):
  ipa: mojom: Move CameraSensorInfo struct exclusively to IPA IPC
  ipa: meson: Install mojom generated headers to include paths
  ipa: ipu3: Introduce IPAConfigInfo in IPC
  meson: Add a configuration option to build IPAs
  meson: Generate a helper .so containing libcamera's internal headers

 Documentation/Doxyfile.in                     |   4 +-
 Documentation/meson.build                     |   1 +
 include/libcamera/internal/camera_sensor.h    |  19 +-
 include/libcamera/internal/meson.build        |   6 +
 include/libcamera/ipa/core.mojom              |   2 +-
 include/libcamera/ipa/core_ipa_interface.cpp  | 190 ++++++++++++++++++
 include/libcamera/ipa/ipa_interface.h         |   2 -
 include/libcamera/ipa/ipu3.mojom              |  10 +-
 include/libcamera/ipa/ipu3_ipa_interface.cpp  |  39 ++++
 include/libcamera/ipa/meson.build             |  13 +-
 include/libcamera/ipa/raspberrypi.mojom       |   2 +-
 include/libcamera/ipa/rkisp1.mojom            |   2 +-
 meson.build                                   |   8 +
 meson_options.txt                             |   5 +
 src/ipa/ipu3/ipu3.cpp                         |  14 +-
 src/ipa/ipu3/ipu3_agc.cpp                     |   2 +-
 src/ipa/ipu3/meson.build                      |   4 +
 src/ipa/meson.build                           |   2 +
 src/ipa/raspberrypi/meson.build               |   4 +
 src/ipa/raspberrypi/raspberrypi.cpp           |   9 +-
 src/ipa/rkisp1/meson.build                    |   4 +
 src/ipa/rkisp1/rkisp1.cpp                     |   6 +-
 src/ipa/vimc/meson.build                      |   4 +
 src/libcamera/camera_sensor.cpp               | 117 +----------
 src/libcamera/meson.build                     |   6 +
 src/libcamera/pipeline/ipu3/ipu3.cpp          |  19 +-
 .../pipeline/raspberrypi/raspberrypi.cpp      |   4 +-
 src/libcamera/pipeline/rkisp1/rkisp1.cpp      |   2 +-
 28 files changed, 333 insertions(+), 167 deletions(-)
 create mode 100644 include/libcamera/ipa/core_ipa_interface.cpp
 create mode 100644 include/libcamera/ipa/ipu3_ipa_interface.cpp

Comments

Kieran Bingham May 14, 2021, 9:33 a.m. UTC | #1
Hi Umang,

On 14/05/2021 08:58, Umang Jain wrote:
> Generated IPA headers from mojom files need to be installed sy

sy? Perhaps in? or to?

> $INCLUDE_PATH in order to be available system-wide. Without this,
> out-of-tree IPAs won't be able to link and build themselves.
> 
> Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>

Seems like the right things to do.

Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>

> ---
>  include/libcamera/ipa/meson.build | 8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)
> 
> diff --git a/include/libcamera/ipa/meson.build b/include/libcamera/ipa/meson.build
> index 69bc855e..da60aa68 100644
> --- a/include/libcamera/ipa/meson.build
> +++ b/include/libcamera/ipa/meson.build
> @@ -1,5 +1,7 @@
>  # SPDX-License-Identifier: CC0-1.0
>  
> +libcamera_ipa_include_dir = libcamera_include_dir / 'ipa'
> +
>  libcamera_ipa_headers = files([
>      'ipa_controls.h',
>      'ipa_interface.h',
> @@ -11,7 +13,7 @@ libcamera_ipa_docs = files([
>  ])
>  
>  install_headers(libcamera_ipa_headers,
> -                subdir: libcamera_include_dir / 'ipa')
> +                subdir: libcamera_ipa_include_dir)
>  
>  libcamera_generated_ipa_headers = []
>  
> @@ -35,6 +37,8 @@ libcamera_generated_ipa_headers += custom_target('core_ipa_interface_h',
>                    input : ipa_mojom_core,
>                    output : 'core_ipa_interface.h',
>                    depends : mojom_templates,
> +                  install : true,
> +                  install_dir : get_option('includedir') / libcamera_ipa_include_dir,
>                    command : [
>                        mojom_generator, 'generate',
>                        '-g', 'libcamera',
> @@ -97,6 +101,8 @@ foreach file : ipa_mojom_files
>                             input : mojom,
>                             output : name + '_ipa_interface.h',
>                             depends : mojom_templates,
> +                           install : true,
> +                           install_dir : get_option('includedir') / libcamera_ipa_include_dir,
>                             command : [
>                                 mojom_generator, 'generate',
>                                 '-g', 'libcamera',
>
Kieran Bingham May 14, 2021, 10:09 a.m. UTC | #2
Hi Umang,

On 14/05/2021 08:58, Umang Jain wrote:
> There can be multiple IPAs per pipeline-handler or platform.
> For e.g., the IPU3 platform has a open source IPA in-tree whereas
> it shall use a closed source one from a standalone separate remote
> for ChromeOS. In a case like this, there should be configure-time
> option whether to build the in-tree IPAs or not.

We don't need to state where external IPAs will be used. That's just
this current use case, and it may not be true.

They might use it. Or they might choose to use the open one. That's up
to them, not us.


> By default, all in-tree IPAs are built.



> 
> Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
> ---
>  meson.build                     | 1 +
>  meson_options.txt               | 5 +++++
>  src/ipa/ipu3/meson.build        | 4 ++++
>  src/ipa/meson.build             | 2 ++
>  src/ipa/raspberrypi/meson.build | 4 ++++
>  src/ipa/rkisp1/meson.build      | 4 ++++
>  src/ipa/vimc/meson.build        | 4 ++++
>  7 files changed, 24 insertions(+)
> 
> diff --git a/meson.build b/meson.build
> index fa2a62cf..b89797f3 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -167,6 +167,7 @@ py_mod.find_installation('python3', modules: py_modules)
>  ## Summarise Configurations
>  summary({
>              'Enabled pipelines': pipelines,
> +            'Enabled IPAs': ipas,
>              'Android support': android_enabled,
>              'GStreamer support': gst_enabled,
>              'V4L2 emulation support': v4l2_enabled,
> diff --git a/meson_options.txt b/meson_options.txt
> index 69f11f85..1fcbecc1 100644
> --- a/meson_options.txt
> +++ b/meson_options.txt
> @@ -25,6 +25,11 @@ option('gstreamer',
>          value : 'auto',
>          description : 'Compile libcamera GStreamer plugin')
>  
> +option('ipas',
> +        type : 'array',
> +        choices : ['ipu3', 'raspberrypi', 'rkisp1', 'vimc'],
> +        description : 'Select which internal IPA to include')

I know this description comes from the pipelines option, but 'include'
doesn't sound right to me.

This decides which ones get built. I'd prefer something like
	"Select which IPA modules to build"



> +
>  option('lc-compliance',
>          type : 'feature',
>          value : 'auto',
> diff --git a/src/ipa/ipu3/meson.build b/src/ipa/ipu3/meson.build
> index 0d843846..a9f5d0aa 100644
> --- a/src/ipa/ipu3/meson.build
> +++ b/src/ipa/ipu3/meson.build
> @@ -2,6 +2,10 @@
>  
>  ipa_name = 'ipa_ipu3'
>  
> +if 'ipu3' not in ipas
> +    subdir_done()
> +endif

You shouldn't need this.

If ipu3 isn't specified in get_option('ipas') then it won't go into the
subdir.


> +
>  ipu3_ipa_sources = files([
>      'ipu3.cpp',
>      'ipu3_agc.cpp',
> diff --git a/src/ipa/meson.build b/src/ipa/meson.build
> index 5b5684a1..fb687497 100644
> --- a/src/ipa/meson.build
> +++ b/src/ipa/meson.build
> @@ -22,6 +22,8 @@ ipa_sign = files('ipa-sign.sh')
>  ipas = ['ipu3', 'raspberrypi', 'rkisp1', 'vimc']
>  ipa_names = []
>  
> +ipas = get_option('ipas')

I think you forgot to remove the ipas list 3 lines above it.
This should replace that line.


> +
>  # The ipa-sign-install.sh script which uses the ipa_names variable will itself
>  # prepend MESON_INSTALL_DESTDIR_PREFIX to each ipa module name, therefore we
>  # must not include the prefix string here.
> diff --git a/src/ipa/raspberrypi/meson.build b/src/ipa/raspberrypi/meson.build
> index d1397a32..02c143b1 100644
> --- a/src/ipa/raspberrypi/meson.build
> +++ b/src/ipa/raspberrypi/meson.build
> @@ -2,6 +2,10 @@
>  
>  ipa_name = 'ipa_rpi'
>  
> +if 'rasberrypi' not in ipas
> +    subdir_done()
> +endif

This isn't needed.


> +
>  rpi_ipa_deps = [
>      libcamera_dep,
>      dependency('boost'),
> diff --git a/src/ipa/rkisp1/meson.build b/src/ipa/rkisp1/meson.build
> index 1a1c7159..eecc5877 100644
> --- a/src/ipa/rkisp1/meson.build
> +++ b/src/ipa/rkisp1/meson.build
> @@ -2,6 +2,10 @@
>  
>  ipa_name = 'ipa_rkisp1'
>  
> +if 'rkisp1' not in ipas
> +    subdir_done()
> +endif

same.

> +
>  mod = shared_module(ipa_name,
>                      ['rkisp1.cpp', libcamera_generated_ipa_headers],
>                      name_prefix : '',
> diff --git a/src/ipa/vimc/meson.build b/src/ipa/vimc/meson.build
> index a35825ae..a41a0407 100644
> --- a/src/ipa/vimc/meson.build
> +++ b/src/ipa/vimc/meson.build
> @@ -2,6 +2,10 @@
>  
>  ipa_name = 'ipa_vimc'
>  
> +if 'ipu3' not in ipas
> +    subdir_done()
> +endif


same.


> +
>  mod = shared_module(ipa_name,
>                      ['vimc.cpp', libcamera_generated_ipa_headers],
>                      name_prefix : '',
>
Kieran Bingham May 14, 2021, 10:34 a.m. UTC | #3
Hi Umang,

On 14/05/2021 08:58, Umang Jain wrote:
> The IPU3 closed source IPA shall live externally(out-of-tree) however,
> it needs access to some of the internal libcamera headers. These headers
> are not typically intended for public usage hence, shouldn't be placed
> as  headers in $includedir. In order to solve this issue, generate a
> helper shared library, which can be used to link with, by the external
> IPU3 closed source IPA.

This is a much bigger task than just sharing the headers.
Headers should be accompanied with implementation.



> Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
> ---
>  include/libcamera/internal/meson.build | 6 ++++++
>  meson.build                            | 7 +++++++
>  src/libcamera/meson.build              | 6 ++++++
>  3 files changed, 19 insertions(+)
> 
> diff --git a/include/libcamera/internal/meson.build b/include/libcamera/internal/meson.build
> index 6cff1b90..fcbf4212 100644
> --- a/include/libcamera/internal/meson.build
> +++ b/include/libcamera/internal/meson.build
> @@ -49,3 +49,9 @@ libcamera_internal_headers = files([
>      'v4l2_subdevice.h',
>      'v4l2_videodevice.h',
>  ])
> +
> +libcamera_helpers = files ([
> +    'buffer.h',
> +    'file.h',
> +    'log.h',
> +])
> diff --git a/meson.build b/meson.build
> index b89797f3..c48c5a66 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -160,6 +160,13 @@ pkg_mod.generate(libraries : libcamera,
>                   description : 'Complex Camera Support Library',
>                   subdirs : 'libcamera')
>  
> +# \todo Bikeshedding on camera_helper naming
> +pkg_mod.generate(libraries : libcamera_helper_lib,
> +                 version : '0.0',
> +                 name : 'libcamera_helper_lib',
> +                 filebase : 'camera_helper',
> +                 description : 'libcamera internal helper library')
> +
>  # Check for python installation and modules.
>  py_mod = import('python')
>  py_mod.find_installation('python3', modules: py_modules)
> diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build
> index 7bc59b84..788b767d 100644
> --- a/src/libcamera/meson.build
> +++ b/src/libcamera/meson.build
> @@ -162,4 +162,10 @@ libcamera_dep = declare_dependency(sources : [
>                                     include_directories : libcamera_includes,
>                                     link_with : libcamera)
>  
> +# libcamera helper shared library
> +libcamera_helper_lib = shared_library('camera_helper', [libcamera_helpers],
> +                                      install: true,
> +                                      install_dir: libcamera_libdir,
> +                                      dependencies: libcamera_dep)

As a separate library, it should be separated from src/libcamera

So dependant upon the name selected this should have it's own structure

 include/
	libcamera-support/
		file.h
		log.h
		buffer.h

 src/
	libcamera-support/
		file.cpp
		log.cpp
		buffer.cpp

And then libcamera should link against it. But things are not that
simple, and the dependencies between all these are quite convoluted.


Of course the library name is hard to choose.
It could be
	libcamera-helper,
	libcamera-utils,
	libcamera-support,
	libcamera-internal ...


And then the scope of what goes in there could be interesting too.

log and file are system helpers, accessing files and generic logging.
Buffers are (our) common buffer abstraction



But there's others to consider (in the future, not necessarily immediately)


## definitely libcamera internal?

byte_stream_buffer.h
camera_controls.h
camera_sensor.h
camera_sensor.h.orig
camera_sensor_properties.h
control_serializer.h
control_validator.h
delayed_controls.h
event_dispatcher.h
event_dispatcher_poll.h
event_notifier.h
ipa_data_serializer.h
ipa_manager.h
ipa_module.h
ipa_proxy.h
pipeline_handler.h
tracepoints
utils.h


# Linux V4L2/MediaController/Pixel Formats / buffers
device_enumerator.h
device_enumerator_sysfs.h
device_enumerator_udev.h
media_device.h
media_object.h
v4l2_controls.h
v4l2_device.h
v4l2_pixelformat.h
v4l2_subdevice.h
v4l2_videodevice.h


# Pixel Formats and buffers
bayer_format.h
buffer.h
formats.h


# Operating system specific abstractions ?
file.h
ipc_pipe.h
ipc_pipe_unixsocket.h
ipc_unixsocket.h
log.h
message.h
process.h
pub_key.h
semaphore.h
sysfs.h
thread.h
timer.h







> +
>  subdir('proxy/worker')
>
Laurent Pinchart May 15, 2021, 4:27 p.m. UTC | #4
On Fri, May 14, 2021 at 10:33:21AM +0100, Kieran Bingham wrote:
> On 14/05/2021 08:58, Umang Jain wrote:
> > Generated IPA headers from mojom files need to be installed sy
> 
> sy? Perhaps in? or to?
> 
> > $INCLUDE_PATH in order to be available system-wide. Without this,
> > out-of-tree IPAs won't be able to link and build themselves.
> > 
> > Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
> 
> Seems like the right things to do.
> 
> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>

Agreed,

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> > ---
> >  include/libcamera/ipa/meson.build | 8 +++++++-
> >  1 file changed, 7 insertions(+), 1 deletion(-)
> > 
> > diff --git a/include/libcamera/ipa/meson.build b/include/libcamera/ipa/meson.build
> > index 69bc855e..da60aa68 100644
> > --- a/include/libcamera/ipa/meson.build
> > +++ b/include/libcamera/ipa/meson.build
> > @@ -1,5 +1,7 @@
> >  # SPDX-License-Identifier: CC0-1.0
> >  
> > +libcamera_ipa_include_dir = libcamera_include_dir / 'ipa'
> > +
> >  libcamera_ipa_headers = files([
> >      'ipa_controls.h',
> >      'ipa_interface.h',
> > @@ -11,7 +13,7 @@ libcamera_ipa_docs = files([
> >  ])
> >  
> >  install_headers(libcamera_ipa_headers,
> > -                subdir: libcamera_include_dir / 'ipa')
> > +                subdir: libcamera_ipa_include_dir)
> >  
> >  libcamera_generated_ipa_headers = []
> >  
> > @@ -35,6 +37,8 @@ libcamera_generated_ipa_headers += custom_target('core_ipa_interface_h',
> >                    input : ipa_mojom_core,
> >                    output : 'core_ipa_interface.h',
> >                    depends : mojom_templates,
> > +                  install : true,
> > +                  install_dir : get_option('includedir') / libcamera_ipa_include_dir,
> >                    command : [
> >                        mojom_generator, 'generate',
> >                        '-g', 'libcamera',
> > @@ -97,6 +101,8 @@ foreach file : ipa_mojom_files
> >                             input : mojom,
> >                             output : name + '_ipa_interface.h',
> >                             depends : mojom_templates,
> > +                           install : true,
> > +                           install_dir : get_option('includedir') / libcamera_ipa_include_dir,
> >                             command : [
> >                                 mojom_generator, 'generate',
> >                                 '-g', 'libcamera',
Laurent Pinchart May 15, 2021, 4:35 p.m. UTC | #5
Hi Umang,

Thank you for the patch.

On Fri, May 14, 2021 at 11:09:18AM +0100, Kieran Bingham wrote:
> On 14/05/2021 08:58, Umang Jain wrote:
> > There can be multiple IPAs per pipeline-handler or platform.
> > For e.g., the IPU3 platform has a open source IPA in-tree whereas

s/a open/an open/

> > it shall use a closed source one from a standalone separate remote
> > for ChromeOS. In a case like this, there should be configure-time
> > option whether to build the in-tree IPAs or not.
> 
> We don't need to state where external IPAs will be used. That's just
> this current use case, and it may not be true.
> 
> They might use it. Or they might choose to use the open one. That's up
> to them, not us.
> 
> > By default, all in-tree IPAs are built.
> > 
> > Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
> > ---
> >  meson.build                     | 1 +
> >  meson_options.txt               | 5 +++++
> >  src/ipa/ipu3/meson.build        | 4 ++++
> >  src/ipa/meson.build             | 2 ++
> >  src/ipa/raspberrypi/meson.build | 4 ++++
> >  src/ipa/rkisp1/meson.build      | 4 ++++
> >  src/ipa/vimc/meson.build        | 4 ++++
> >  7 files changed, 24 insertions(+)
> > 
> > diff --git a/meson.build b/meson.build
> > index fa2a62cf..b89797f3 100644
> > --- a/meson.build
> > +++ b/meson.build
> > @@ -167,6 +167,7 @@ py_mod.find_installation('python3', modules: py_modules)
> >  ## Summarise Configurations
> >  summary({
> >              'Enabled pipelines': pipelines,
> > +            'Enabled IPAs': ipas,

I'd write 'Enabled IPA modules', and perhaps rename the ipas variable to
ipa_modules.

> >              'Android support': android_enabled,
> >              'GStreamer support': gst_enabled,
> >              'V4L2 emulation support': v4l2_enabled,
> > diff --git a/meson_options.txt b/meson_options.txt
> > index 69f11f85..1fcbecc1 100644
> > --- a/meson_options.txt
> > +++ b/meson_options.txt
> > @@ -25,6 +25,11 @@ option('gstreamer',
> >          value : 'auto',
> >          description : 'Compile libcamera GStreamer plugin')
> >  
> > +option('ipas',

We could also rename the option to ipa_modules.

> > +        type : 'array',
> > +        choices : ['ipu3', 'raspberrypi', 'rkisp1', 'vimc'],
> > +        description : 'Select which internal IPA to include')

'IPA modules' here too.

> I know this description comes from the pipelines option, but 'include'
> doesn't sound right to me.
> 
> This decides which ones get built. I'd prefer something like
> 	"Select which IPA modules to build"
> 
> > +
> >  option('lc-compliance',
> >          type : 'feature',
> >          value : 'auto',
> > diff --git a/src/ipa/ipu3/meson.build b/src/ipa/ipu3/meson.build
> > index 0d843846..a9f5d0aa 100644
> > --- a/src/ipa/ipu3/meson.build
> > +++ b/src/ipa/ipu3/meson.build
> > @@ -2,6 +2,10 @@
> >  
> >  ipa_name = 'ipa_ipu3'
> >  
> > +if 'ipu3' not in ipas
> > +    subdir_done()
> > +endif
> 
> You shouldn't need this.
> 
> If ipu3 isn't specified in get_option('ipas') then it won't go into the
> subdir.
> 
> > +
> >  ipu3_ipa_sources = files([
> >      'ipu3.cpp',
> >      'ipu3_agc.cpp',
> > diff --git a/src/ipa/meson.build b/src/ipa/meson.build
> > index 5b5684a1..fb687497 100644
> > --- a/src/ipa/meson.build
> > +++ b/src/ipa/meson.build
> > @@ -22,6 +22,8 @@ ipa_sign = files('ipa-sign.sh')
> >  ipas = ['ipu3', 'raspberrypi', 'rkisp1', 'vimc']
> >  ipa_names = []
> >  
> > +ipas = get_option('ipas')
> 
> I think you forgot to remove the ipas list 3 lines above it.
> This should replace that line.
> 
> > +
> >  # The ipa-sign-install.sh script which uses the ipa_names variable will itself
> >  # prepend MESON_INSTALL_DESTDIR_PREFIX to each ipa module name, therefore we
> >  # must not include the prefix string here.
> > diff --git a/src/ipa/raspberrypi/meson.build b/src/ipa/raspberrypi/meson.build
> > index d1397a32..02c143b1 100644
> > --- a/src/ipa/raspberrypi/meson.build
> > +++ b/src/ipa/raspberrypi/meson.build
> > @@ -2,6 +2,10 @@
> >  
> >  ipa_name = 'ipa_rpi'
> >  
> > +if 'rasberrypi' not in ipas
> > +    subdir_done()
> > +endif
> 
> This isn't needed.
> 
> > +
> >  rpi_ipa_deps = [
> >      libcamera_dep,
> >      dependency('boost'),
> > diff --git a/src/ipa/rkisp1/meson.build b/src/ipa/rkisp1/meson.build
> > index 1a1c7159..eecc5877 100644
> > --- a/src/ipa/rkisp1/meson.build
> > +++ b/src/ipa/rkisp1/meson.build
> > @@ -2,6 +2,10 @@
> >  
> >  ipa_name = 'ipa_rkisp1'
> >  
> > +if 'rkisp1' not in ipas
> > +    subdir_done()
> > +endif
> 
> same.
> 
> > +
> >  mod = shared_module(ipa_name,
> >                      ['rkisp1.cpp', libcamera_generated_ipa_headers],
> >                      name_prefix : '',
> > diff --git a/src/ipa/vimc/meson.build b/src/ipa/vimc/meson.build
> > index a35825ae..a41a0407 100644
> > --- a/src/ipa/vimc/meson.build
> > +++ b/src/ipa/vimc/meson.build
> > @@ -2,6 +2,10 @@
> >  
> >  ipa_name = 'ipa_vimc'
> >  
> > +if 'ipu3' not in ipas
> > +    subdir_done()
> > +endif
> 
> same.
> 
> > +
> >  mod = shared_module(ipa_name,
> >                      ['vimc.cpp', libcamera_generated_ipa_headers],
> >                      name_prefix : '',
Laurent Pinchart May 15, 2021, 4:46 p.m. UTC | #6
Hello,

On Fri, May 14, 2021 at 11:34:51AM +0100, Kieran Bingham wrote:
> On 14/05/2021 08:58, Umang Jain wrote:
> > The IPU3 closed source IPA shall live externally(out-of-tree) however,
> > it needs access to some of the internal libcamera headers. These headers
> > are not typically intended for public usage hence, shouldn't be placed
> > as  headers in $includedir. In order to solve this issue, generate a
> > helper shared library, which can be used to link with, by the external
> > IPU3 closed source IPA.
> 
> This is a much bigger task than just sharing the headers.
> Headers should be accompanied with implementation.
> 
> > Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
> > ---
> >  include/libcamera/internal/meson.build | 6 ++++++
> >  meson.build                            | 7 +++++++
> >  src/libcamera/meson.build              | 6 ++++++
> >  3 files changed, 19 insertions(+)
> > 
> > diff --git a/include/libcamera/internal/meson.build b/include/libcamera/internal/meson.build
> > index 6cff1b90..fcbf4212 100644
> > --- a/include/libcamera/internal/meson.build
> > +++ b/include/libcamera/internal/meson.build
> > @@ -49,3 +49,9 @@ libcamera_internal_headers = files([
> >      'v4l2_subdevice.h',
> >      'v4l2_videodevice.h',
> >  ])
> > +
> > +libcamera_helpers = files ([
> > +    'buffer.h',
> > +    'file.h',
> > +    'log.h',
> > +])
> > diff --git a/meson.build b/meson.build
> > index b89797f3..c48c5a66 100644
> > --- a/meson.build
> > +++ b/meson.build
> > @@ -160,6 +160,13 @@ pkg_mod.generate(libraries : libcamera,
> >                   description : 'Complex Camera Support Library',
> >                   subdirs : 'libcamera')
> >  
> > +# \todo Bikeshedding on camera_helper naming
> > +pkg_mod.generate(libraries : libcamera_helper_lib,
> > +                 version : '0.0',
> > +                 name : 'libcamera_helper_lib',
> > +                 filebase : 'camera_helper',
> > +                 description : 'libcamera internal helper library')
> > +
> >  # Check for python installation and modules.
> >  py_mod = import('python')
> >  py_mod.find_installation('python3', modules: py_modules)
> > diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build
> > index 7bc59b84..788b767d 100644
> > --- a/src/libcamera/meson.build
> > +++ b/src/libcamera/meson.build
> > @@ -162,4 +162,10 @@ libcamera_dep = declare_dependency(sources : [
> >                                     include_directories : libcamera_includes,
> >                                     link_with : libcamera)
> >  
> > +# libcamera helper shared library
> > +libcamera_helper_lib = shared_library('camera_helper', [libcamera_helpers],
> > +                                      install: true,
> > +                                      install_dir: libcamera_libdir,
> > +                                      dependencies: libcamera_dep)
> 
> As a separate library, it should be separated from src/libcamera
> 
> So dependant upon the name selected this should have it's own structure
> 
>  include/
> 	libcamera-support/
> 		file.h
> 		log.h
> 		buffer.h

I had envisioned include/libcamera/support/ (bikeshedding on the
'support' name aside).

>  src/
> 	libcamera-support/
> 		file.cpp
> 		log.cpp
> 		buffer.cpp

And possibly similarly here, although I wouldn't object keeping the
files in src/libcamera/

> And then libcamera should link against it. But things are not that
> simple, and the dependencies between all these are quite convoluted.
> 
> 
> Of course the library name is hard to choose.
> It could be
> 	libcamera-helper,
> 	libcamera-utils,
> 	libcamera-support,
> 	libcamera-internal ...

My initial preference is libcamera-utils.

> And then the scope of what goes in there could be interesting too.
> 
> log and file are system helpers, accessing files and generic logging.
> Buffers are (our) common buffer abstraction
> 
> 
> 
> But there's others to consider (in the future, not necessarily immediately)
> 
> 
> ## definitely libcamera internal?
> 
> byte_stream_buffer.h
> camera_controls.h
> camera_sensor.h
> camera_sensor.h.orig

I think we can skip this one ;-)

> camera_sensor_properties.h
> control_serializer.h
> control_validator.h
> delayed_controls.h
> event_dispatcher.h
> event_dispatcher_poll.h
> event_notifier.h
> ipa_data_serializer.h
> ipa_manager.h
> ipa_module.h
> ipa_proxy.h
> pipeline_handler.h
> tracepoints
> utils.h

utils.h sounds fairly useful for IPA modules.

> # Linux V4L2/MediaController/Pixel Formats / buffers
> device_enumerator.h
> device_enumerator_sysfs.h
> device_enumerator_udev.h
> media_device.h
> media_object.h
> v4l2_controls.h
> v4l2_device.h
> v4l2_pixelformat.h
> v4l2_subdevice.h
> v4l2_videodevice.h

All this is definitely libcamera internal, IPA modules must never access
devices.

> # Pixel Formats and buffers
> bayer_format.h
> buffer.h
> formats.h
> 
> 
> # Operating system specific abstractions ?
> file.h
> ipc_pipe.h
> ipc_pipe_unixsocket.h
> ipc_unixsocket.h

IPC should be handled transparently by the generated code for the IPA
module, so I'd skip it for now. We'll have to revisit this as part of
the IPC mechanism rework discussions (related to whether to keep
spawning a process directly, or use an external daemon), for now I'd
prefer only moving code that we know is of interest to IPA modules,
excluding the proxy.

> log.h
> message.h
> process.h
> pub_key.h

I'd also skip process.h and pub_key.h, I don't see a need for IPA
modules to use those.

> semaphore.h
> sysfs.h

Same for sysfs.h.

> thread.h
> timer.h

Timers and threads require an event dispatcher, so you would need to
also move event_dispatcher.h and event_dispatcher_poll.h, and, as a
consequence, event_notifier.h.

> > +
> >  subdir('proxy/worker')
> >