[libcamera-devel,0/8] ipa: rkisp1: Add autofocus algorithm
mbox series

Message ID 20220630143543.39599-1-dse@thaumatec.com
Headers show
Series
  • ipa: rkisp1: Add autofocus algorithm
Related show

Message

Daniel Semkowicz June 30, 2022, 2:35 p.m. UTC
Hello,

The following set of patches introduces an autofocus algorithm for
rkisp1 platforms. As there are common parts of code in the IPU3, RPi and
now they would be repeated in rkisp1, I tried to separate the main
control part of the AF algorithm in the new AfHillClimbing class.

These changes make use of the freshly introduced algorithm registration
mechanism. As there was a missing part for the easy way to get the
specific algorithm from the dynamically loaded list, I added 
getAlgorithm<AlgoType>() method to the Module class that. It was
implemented in the fastest way to get it working, but I think we should
improve it later. Maybe it would be good to extend algorithm creation
and store also the name of corresponding algorithm to allow calling it
by name instead of type. Especially that algorithms are loaded from
tuning file by name.

I am not sure if the directory structure I have chosen for the common
code is a good place, so please correct me if I should place it
somewhere else.

Doxygen documentation part should be, for sure, more detailed, but I
firstly want to wait for your comments about the code, just in case if
there were any changes to the design.

These changes depend on the "libcamera: rkisp1: Add lens control"
changes that are still in the review process.
Link: https://patchwork.libcamera.org/project/libcamera/list/?series=3228

I am waiting for your comments :)

Best regards
Daniel Semkowicz

Daniel Semkowicz (8):
  ipa: Add base class defining AF algorithm interface
  ipa: Add class that implements base AF control algorithm
  ipa: rkisp1: Add AF algorithm basing on common AfHillClimbing class
  pipeline: rkisp1: Add basic AF controls to the supported controls list
  ipa: module: Add getAlgorithm() method
  ipa: rkisp1: Pass requests setting AF controls to the AF algorithm
  ipa: rkisp1: Add OV5675 tuning file
  ipa: rkisp1: Add "Windows" Metering mode to auto focus algorithm

 src/ipa/libipa/algorithms/af_algorithm.cpp    |  77 ++++++
 src/ipa/libipa/algorithms/af_algorithm.h      |  39 +++
 .../libipa/algorithms/af_hill_climbing.cpp    |  89 +++++++
 src/ipa/libipa/algorithms/af_hill_climbing.h  | 251 ++++++++++++++++++
 src/ipa/libipa/algorithms/meson.build         |  11 +
 src/ipa/libipa/meson.build                    |   6 +
 src/ipa/libipa/module.cpp                     |   7 +
 src/ipa/libipa/module.h                       |  17 ++
 src/ipa/rkisp1/algorithms/af.cpp              | 132 +++++++++
 src/ipa/rkisp1/algorithms/af.h                |  41 +++
 src/ipa/rkisp1/algorithms/meson.build         |   1 +
 src/ipa/rkisp1/data/meson.build               |   1 +
 src/ipa/rkisp1/data/ov5675.yaml               |  12 +
 src/ipa/rkisp1/rkisp1.cpp                     |  72 ++++-
 src/libcamera/pipeline/rkisp1/rkisp1.cpp      |  12 +-
 15 files changed, 762 insertions(+), 6 deletions(-)
 create mode 100644 src/ipa/libipa/algorithms/af_algorithm.cpp
 create mode 100644 src/ipa/libipa/algorithms/af_algorithm.h
 create mode 100644 src/ipa/libipa/algorithms/af_hill_climbing.cpp
 create mode 100644 src/ipa/libipa/algorithms/af_hill_climbing.h
 create mode 100644 src/ipa/libipa/algorithms/meson.build
 create mode 100644 src/ipa/rkisp1/algorithms/af.cpp
 create mode 100644 src/ipa/rkisp1/algorithms/af.h
 create mode 100644 src/ipa/rkisp1/data/ov5675.yaml

Comments

Jacopo Mondi July 12, 2022, 9:17 a.m. UTC | #1
Hi Daniel,
   just a note to let you know this has not gone forgotten. I've very
keen in looking into that. I spent some time setting up a rockpi4
where I can connect an imx519 camera module which has a lens.

I'll keep working to test your series there!

Thanks
   j

On Thu, Jun 30, 2022 at 04:35:35PM +0200, Daniel Semkowicz via libcamera-devel wrote:
> Hello,
>
> The following set of patches introduces an autofocus algorithm for
> rkisp1 platforms. As there are common parts of code in the IPU3, RPi and
> now they would be repeated in rkisp1, I tried to separate the main
> control part of the AF algorithm in the new AfHillClimbing class.
>
> These changes make use of the freshly introduced algorithm registration
> mechanism. As there was a missing part for the easy way to get the
> specific algorithm from the dynamically loaded list, I added
> getAlgorithm<AlgoType>() method to the Module class that. It was
> implemented in the fastest way to get it working, but I think we should
> improve it later. Maybe it would be good to extend algorithm creation
> and store also the name of corresponding algorithm to allow calling it
> by name instead of type. Especially that algorithms are loaded from
> tuning file by name.
>
> I am not sure if the directory structure I have chosen for the common
> code is a good place, so please correct me if I should place it
> somewhere else.
>
> Doxygen documentation part should be, for sure, more detailed, but I
> firstly want to wait for your comments about the code, just in case if
> there were any changes to the design.
>
> These changes depend on the "libcamera: rkisp1: Add lens control"
> changes that are still in the review process.
> Link: https://patchwork.libcamera.org/project/libcamera/list/?series=3228
>
> I am waiting for your comments :)
>
> Best regards
> Daniel Semkowicz
>
> Daniel Semkowicz (8):
>   ipa: Add base class defining AF algorithm interface
>   ipa: Add class that implements base AF control algorithm
>   ipa: rkisp1: Add AF algorithm basing on common AfHillClimbing class
>   pipeline: rkisp1: Add basic AF controls to the supported controls list
>   ipa: module: Add getAlgorithm() method
>   ipa: rkisp1: Pass requests setting AF controls to the AF algorithm
>   ipa: rkisp1: Add OV5675 tuning file
>   ipa: rkisp1: Add "Windows" Metering mode to auto focus algorithm
>
>  src/ipa/libipa/algorithms/af_algorithm.cpp    |  77 ++++++
>  src/ipa/libipa/algorithms/af_algorithm.h      |  39 +++
>  .../libipa/algorithms/af_hill_climbing.cpp    |  89 +++++++
>  src/ipa/libipa/algorithms/af_hill_climbing.h  | 251 ++++++++++++++++++
>  src/ipa/libipa/algorithms/meson.build         |  11 +
>  src/ipa/libipa/meson.build                    |   6 +
>  src/ipa/libipa/module.cpp                     |   7 +
>  src/ipa/libipa/module.h                       |  17 ++
>  src/ipa/rkisp1/algorithms/af.cpp              | 132 +++++++++
>  src/ipa/rkisp1/algorithms/af.h                |  41 +++
>  src/ipa/rkisp1/algorithms/meson.build         |   1 +
>  src/ipa/rkisp1/data/meson.build               |   1 +
>  src/ipa/rkisp1/data/ov5675.yaml               |  12 +
>  src/ipa/rkisp1/rkisp1.cpp                     |  72 ++++-
>  src/libcamera/pipeline/rkisp1/rkisp1.cpp      |  12 +-
>  15 files changed, 762 insertions(+), 6 deletions(-)
>  create mode 100644 src/ipa/libipa/algorithms/af_algorithm.cpp
>  create mode 100644 src/ipa/libipa/algorithms/af_algorithm.h
>  create mode 100644 src/ipa/libipa/algorithms/af_hill_climbing.cpp
>  create mode 100644 src/ipa/libipa/algorithms/af_hill_climbing.h
>  create mode 100644 src/ipa/libipa/algorithms/meson.build
>  create mode 100644 src/ipa/rkisp1/algorithms/af.cpp
>  create mode 100644 src/ipa/rkisp1/algorithms/af.h
>  create mode 100644 src/ipa/rkisp1/data/ov5675.yaml
>
> --
> 2.34.1
>
Daniel Semkowicz July 12, 2022, 12:13 p.m. UTC | #2
Hi Jacopo,

Thank you for the information about the state. I see there is a lot of
traffic on libcamera mailing list, so I thought you probably have a
lot of work :)

In the mean time I tested my changes on different platform and found one
issue that needs an improvement, so I have two additional commits to
this series. Should I send them separately or resend this series as new
version with additional commits? There are no changes in already pushed
commits.

Best regards
Daniel

On Tue, Jul 12, 2022 at 11:17 AM Jacopo Mondi <jacopo@jmondi.org> wrote:
>
> Hi Daniel,
>    just a note to let you know this has not gone forgotten. I've very
> keen in looking into that. I spent some time setting up a rockpi4
> where I can connect an imx519 camera module which has a lens.
>
> I'll keep working to test your series there!
>
> Thanks
>    j
>
> On Thu, Jun 30, 2022 at 04:35:35PM +0200, Daniel Semkowicz via libcamera-devel wrote:
> > Hello,
> >
> > The following set of patches introduces an autofocus algorithm for
> > rkisp1 platforms. As there are common parts of code in the IPU3, RPi and
> > now they would be repeated in rkisp1, I tried to separate the main
> > control part of the AF algorithm in the new AfHillClimbing class.
> >
> > These changes make use of the freshly introduced algorithm registration
> > mechanism. As there was a missing part for the easy way to get the
> > specific algorithm from the dynamically loaded list, I added
> > getAlgorithm<AlgoType>() method to the Module class that. It was
> > implemented in the fastest way to get it working, but I think we should
> > improve it later. Maybe it would be good to extend algorithm creation
> > and store also the name of corresponding algorithm to allow calling it
> > by name instead of type. Especially that algorithms are loaded from
> > tuning file by name.
> >
> > I am not sure if the directory structure I have chosen for the common
> > code is a good place, so please correct me if I should place it
> > somewhere else.
> >
> > Doxygen documentation part should be, for sure, more detailed, but I
> > firstly want to wait for your comments about the code, just in case if
> > there were any changes to the design.
> >
> > These changes depend on the "libcamera: rkisp1: Add lens control"
> > changes that are still in the review process.
> > Link: https://patchwork.libcamera.org/project/libcamera/list/?series=3228
> >
> > I am waiting for your comments :)
> >
> > Best regards
> > Daniel Semkowicz
> >
> > Daniel Semkowicz (8):
> >   ipa: Add base class defining AF algorithm interface
> >   ipa: Add class that implements base AF control algorithm
> >   ipa: rkisp1: Add AF algorithm basing on common AfHillClimbing class
> >   pipeline: rkisp1: Add basic AF controls to the supported controls list
> >   ipa: module: Add getAlgorithm() method
> >   ipa: rkisp1: Pass requests setting AF controls to the AF algorithm
> >   ipa: rkisp1: Add OV5675 tuning file
> >   ipa: rkisp1: Add "Windows" Metering mode to auto focus algorithm
> >
> >  src/ipa/libipa/algorithms/af_algorithm.cpp    |  77 ++++++
> >  src/ipa/libipa/algorithms/af_algorithm.h      |  39 +++
> >  .../libipa/algorithms/af_hill_climbing.cpp    |  89 +++++++
> >  src/ipa/libipa/algorithms/af_hill_climbing.h  | 251 ++++++++++++++++++
> >  src/ipa/libipa/algorithms/meson.build         |  11 +
> >  src/ipa/libipa/meson.build                    |   6 +
> >  src/ipa/libipa/module.cpp                     |   7 +
> >  src/ipa/libipa/module.h                       |  17 ++
> >  src/ipa/rkisp1/algorithms/af.cpp              | 132 +++++++++
> >  src/ipa/rkisp1/algorithms/af.h                |  41 +++
> >  src/ipa/rkisp1/algorithms/meson.build         |   1 +
> >  src/ipa/rkisp1/data/meson.build               |   1 +
> >  src/ipa/rkisp1/data/ov5675.yaml               |  12 +
> >  src/ipa/rkisp1/rkisp1.cpp                     |  72 ++++-
> >  src/libcamera/pipeline/rkisp1/rkisp1.cpp      |  12 +-
> >  15 files changed, 762 insertions(+), 6 deletions(-)
> >  create mode 100644 src/ipa/libipa/algorithms/af_algorithm.cpp
> >  create mode 100644 src/ipa/libipa/algorithms/af_algorithm.h
> >  create mode 100644 src/ipa/libipa/algorithms/af_hill_climbing.cpp
> >  create mode 100644 src/ipa/libipa/algorithms/af_hill_climbing.h
> >  create mode 100644 src/ipa/libipa/algorithms/meson.build
> >  create mode 100644 src/ipa/rkisp1/algorithms/af.cpp
> >  create mode 100644 src/ipa/rkisp1/algorithms/af.h
> >  create mode 100644 src/ipa/rkisp1/data/ov5675.yaml
> >
> > --
> > 2.34.1
> >
Jacopo Mondi July 12, 2022, 1:07 p.m. UTC | #3
Hi Daniel,

On Tue, Jul 12, 2022 at 02:13:20PM +0200, Daniel Semkowicz wrote:
> Hi Jacopo,
>
> Thank you for the information about the state. I see there is a lot of
> traffic on libcamera mailing list, so I thought you probably have a
> lot of work :)
>
> In the mean time I tested my changes on different platform and found one
> issue that needs an improvement, so I have two additional commits to
> this series. Should I send them separately or resend this series as new
> version with additional commits? There are no changes in already pushed
> commits.

Please send a new series, so that the state is kept in a single
thread.

Thanks
  j

>
> Best regards
> Daniel
>
> On Tue, Jul 12, 2022 at 11:17 AM Jacopo Mondi <jacopo@jmondi.org> wrote:
> >
> > Hi Daniel,
> >    just a note to let you know this has not gone forgotten. I've very
> > keen in looking into that. I spent some time setting up a rockpi4
> > where I can connect an imx519 camera module which has a lens.
> >
> > I'll keep working to test your series there!
> >
> > Thanks
> >    j
> >
> > On Thu, Jun 30, 2022 at 04:35:35PM +0200, Daniel Semkowicz via libcamera-devel wrote:
> > > Hello,
> > >
> > > The following set of patches introduces an autofocus algorithm for
> > > rkisp1 platforms. As there are common parts of code in the IPU3, RPi and
> > > now they would be repeated in rkisp1, I tried to separate the main
> > > control part of the AF algorithm in the new AfHillClimbing class.
> > >
> > > These changes make use of the freshly introduced algorithm registration
> > > mechanism. As there was a missing part for the easy way to get the
> > > specific algorithm from the dynamically loaded list, I added
> > > getAlgorithm<AlgoType>() method to the Module class that. It was
> > > implemented in the fastest way to get it working, but I think we should
> > > improve it later. Maybe it would be good to extend algorithm creation
> > > and store also the name of corresponding algorithm to allow calling it
> > > by name instead of type. Especially that algorithms are loaded from
> > > tuning file by name.
> > >
> > > I am not sure if the directory structure I have chosen for the common
> > > code is a good place, so please correct me if I should place it
> > > somewhere else.
> > >
> > > Doxygen documentation part should be, for sure, more detailed, but I
> > > firstly want to wait for your comments about the code, just in case if
> > > there were any changes to the design.
> > >
> > > These changes depend on the "libcamera: rkisp1: Add lens control"
> > > changes that are still in the review process.
> > > Link: https://patchwork.libcamera.org/project/libcamera/list/?series=3228
> > >
> > > I am waiting for your comments :)
> > >
> > > Best regards
> > > Daniel Semkowicz
> > >
> > > Daniel Semkowicz (8):
> > >   ipa: Add base class defining AF algorithm interface
> > >   ipa: Add class that implements base AF control algorithm
> > >   ipa: rkisp1: Add AF algorithm basing on common AfHillClimbing class
> > >   pipeline: rkisp1: Add basic AF controls to the supported controls list
> > >   ipa: module: Add getAlgorithm() method
> > >   ipa: rkisp1: Pass requests setting AF controls to the AF algorithm
> > >   ipa: rkisp1: Add OV5675 tuning file
> > >   ipa: rkisp1: Add "Windows" Metering mode to auto focus algorithm
> > >
> > >  src/ipa/libipa/algorithms/af_algorithm.cpp    |  77 ++++++
> > >  src/ipa/libipa/algorithms/af_algorithm.h      |  39 +++
> > >  .../libipa/algorithms/af_hill_climbing.cpp    |  89 +++++++
> > >  src/ipa/libipa/algorithms/af_hill_climbing.h  | 251 ++++++++++++++++++
> > >  src/ipa/libipa/algorithms/meson.build         |  11 +
> > >  src/ipa/libipa/meson.build                    |   6 +
> > >  src/ipa/libipa/module.cpp                     |   7 +
> > >  src/ipa/libipa/module.h                       |  17 ++
> > >  src/ipa/rkisp1/algorithms/af.cpp              | 132 +++++++++
> > >  src/ipa/rkisp1/algorithms/af.h                |  41 +++
> > >  src/ipa/rkisp1/algorithms/meson.build         |   1 +
> > >  src/ipa/rkisp1/data/meson.build               |   1 +
> > >  src/ipa/rkisp1/data/ov5675.yaml               |  12 +
> > >  src/ipa/rkisp1/rkisp1.cpp                     |  72 ++++-
> > >  src/libcamera/pipeline/rkisp1/rkisp1.cpp      |  12 +-
> > >  15 files changed, 762 insertions(+), 6 deletions(-)
> > >  create mode 100644 src/ipa/libipa/algorithms/af_algorithm.cpp
> > >  create mode 100644 src/ipa/libipa/algorithms/af_algorithm.h
> > >  create mode 100644 src/ipa/libipa/algorithms/af_hill_climbing.cpp
> > >  create mode 100644 src/ipa/libipa/algorithms/af_hill_climbing.h
> > >  create mode 100644 src/ipa/libipa/algorithms/meson.build
> > >  create mode 100644 src/ipa/rkisp1/algorithms/af.cpp
> > >  create mode 100644 src/ipa/rkisp1/algorithms/af.h
> > >  create mode 100644 src/ipa/rkisp1/data/ov5675.yaml
> > >
> > > --
> > > 2.34.1
> > >