| Message ID | 20201002093309.109180-1-naush@raspberrypi.com |
|---|---|
| Headers | show |
| Series |
|
| Related | show |
Hi all, Gentle ping to get some comments on this one. If you prefer, I could submit as an official patch to review? Thanks, Naush On Fri, 2 Oct 2020 at 10:33, Naushir Patuck <naush@raspberrypi.com> wrote: > > Hi all, > > I would like to resurrect the topic of passing controls to the pipeline handlers and ipas on startup. We talked about this some time back, and unfortunately it drifted off my radar. > > Attached is some work I've done to resurrect this discussion. I have added the ability for an application to pass in controls through the camera::start() public API. This is similar to how controls are passed in though Requests, the only difference being that if we do pass it in the latter, we could incur multiple frames of delay before they are applied. This would be unsuitable for a lot of users I feel. This change gives us a huge jump in getting to feature parity with the existing Raspberry Pi camera stack. > > There are a couple of things to address in these changes: > - I have modified the camera::start() public API with the following signature: int start(ControlList *controls = nullptr). I know the convention is to use a reference here, but I wanted to keep API backward compatible, and a pointer can have a default value nullptr. I could also overload start and keep the old and new signature perhaps, but not sure on the policy for public methods having multiple signatures. > > - the IPA context wrapper code needs the ability to serialise IPAOperationData structures. This is needed for this code, as well as for configure() as part of an earlier set of changes. > > Please note, this patch set is for discussion to start with and not ready to submit as-is. Happy to hear your thoughts. > > Regards, > Naush > > Naushir Patuck (5): > pipeline: raspberrypi: Sensor flips should be applied unconditionally > libcamera: pipeline: Pass libcamera controls into > pipeline_handler::start() > libcamera: ipa: Pass a set of controls and return results from > ipa::start() > pipeline: ipa: raspberrypi: Pass controls to IPA on start > DNI: qcam: Simple test to pass controls on camera::start() > > Documentation/guides/pipeline-handler.rst | 4 +- > include/libcamera/camera.h | 2 +- > .../libcamera/internal/ipa_context_wrapper.h | 3 +- > include/libcamera/internal/pipeline_handler.h | 2 +- > include/libcamera/ipa/ipa_interface.h | 3 +- > include/libcamera/ipa/raspberrypi.h | 1 + > src/ipa/libipa/ipa_interface_wrapper.cpp | 4 +- > src/ipa/raspberrypi/raspberrypi.cpp | 50 ++++++++++++------- > src/ipa/rkisp1/rkisp1.cpp | 3 +- > src/ipa/vimc/vimc.cpp | 6 ++- > src/libcamera/camera.cpp | 11 ++-- > src/libcamera/ipa_context_wrapper.cpp | 6 ++- > src/libcamera/ipa_interface.cpp | 7 +++ > src/libcamera/pipeline/ipu3/ipu3.cpp | 4 +- > .../pipeline/raspberrypi/raspberrypi.cpp | 46 ++++++++++------- > src/libcamera/pipeline/rkisp1/rkisp1.cpp | 9 ++-- > src/libcamera/pipeline/simple/simple.cpp | 4 +- > src/libcamera/pipeline/uvcvideo/uvcvideo.cpp | 4 +- > src/libcamera/pipeline/vimc/vimc.cpp | 7 +-- > src/libcamera/pipeline_handler.cpp | 1 + > src/libcamera/proxy/ipa_proxy_linux.cpp | 3 +- > src/libcamera/proxy/ipa_proxy_thread.cpp | 13 +++-- > src/qcam/main_window.cpp | 7 ++- > test/ipa/ipa_interface_test.cpp | 3 +- > test/ipa/ipa_wrappers_test.cpp | 5 +- > 25 files changed, 134 insertions(+), 74 deletions(-) > > -- > 2.25.1 >
Hi Naush, On 19/10/2020 11:02, Naushir Patuck wrote: > Hi all, > > Gentle ping to get some comments on this one. If you prefer, I could > submit as an official patch to review? > > Thanks, > Naush > > On Fri, 2 Oct 2020 at 10:33, Naushir Patuck <naush@raspberrypi.com> wrote: >> >> Hi all, >> >> I would like to resurrect the topic of passing controls to the pipeline handlers and ipas on startup. We talked about this some time back, and unfortunately it drifted off my radar. >> >> Attached is some work I've done to resurrect this discussion. I have added the ability for an application to pass in controls through the camera::start() public API. This is similar to how controls are passed in though Requests, the only difference being that if we do pass it in the latter, we could incur multiple frames of delay before they are applied. This would be unsuitable for a lot of users I feel. This change gives us a huge jump in getting to feature parity with the existing Raspberry Pi camera stack. Passing this in through start sounds a bit odd to me. Should this be part of the camera->configure() phase? It's setting the initial configuration right? And we can add a ControlList to the CameraConfiguration without needing to update any function signatures. Then when a pipeline is called with configure(), it needs to apply any controls in the list at that point. And it looks like the CameraConfiguration is already passed through to: int RPiCameraData::configureIPA(const CameraConfiguration *config) So, then there is the information required at that phase too? I'm wondering if I've missed something obvious that would have prevented you from already using this approach? -- Kieran >> >> There are a couple of things to address in these changes: >> - I have modified the camera::start() public API with the following signature: int start(ControlList *controls = nullptr). I know the convention is to use a reference here, but I wanted to keep API backward compatible, and a pointer can have a default value nullptr. I could also overload start and keep the old and new signature perhaps, but not sure on the policy for public methods having multiple signatures. >> >> - the IPA context wrapper code needs the ability to serialise IPAOperationData structures. This is needed for this code, as well as for configure() as part of an earlier set of changes.>> Please note, this patch set is for discussion to start with and not ready to submit as-is. Happy to hear your thoughts. >> >> Regards, >> Naush >> >> Naushir Patuck (5): >> pipeline: raspberrypi: Sensor flips should be applied unconditionally >> libcamera: pipeline: Pass libcamera controls into >> pipeline_handler::start() >> libcamera: ipa: Pass a set of controls and return results from >> ipa::start() >> pipeline: ipa: raspberrypi: Pass controls to IPA on start >> DNI: qcam: Simple test to pass controls on camera::start() >> >> Documentation/guides/pipeline-handler.rst | 4 +- >> include/libcamera/camera.h | 2 +- >> .../libcamera/internal/ipa_context_wrapper.h | 3 +- >> include/libcamera/internal/pipeline_handler.h | 2 +- >> include/libcamera/ipa/ipa_interface.h | 3 +- >> include/libcamera/ipa/raspberrypi.h | 1 + >> src/ipa/libipa/ipa_interface_wrapper.cpp | 4 +- >> src/ipa/raspberrypi/raspberrypi.cpp | 50 ++++++++++++------- >> src/ipa/rkisp1/rkisp1.cpp | 3 +- >> src/ipa/vimc/vimc.cpp | 6 ++- >> src/libcamera/camera.cpp | 11 ++-- >> src/libcamera/ipa_context_wrapper.cpp | 6 ++- >> src/libcamera/ipa_interface.cpp | 7 +++ >> src/libcamera/pipeline/ipu3/ipu3.cpp | 4 +- >> .../pipeline/raspberrypi/raspberrypi.cpp | 46 ++++++++++------- >> src/libcamera/pipeline/rkisp1/rkisp1.cpp | 9 ++-- >> src/libcamera/pipeline/simple/simple.cpp | 4 +- >> src/libcamera/pipeline/uvcvideo/uvcvideo.cpp | 4 +- >> src/libcamera/pipeline/vimc/vimc.cpp | 7 +-- >> src/libcamera/pipeline_handler.cpp | 1 + >> src/libcamera/proxy/ipa_proxy_linux.cpp | 3 +- >> src/libcamera/proxy/ipa_proxy_thread.cpp | 13 +++-- >> src/qcam/main_window.cpp | 7 ++- >> test/ipa/ipa_interface_test.cpp | 3 +- >> test/ipa/ipa_wrappers_test.cpp | 5 +- >> 25 files changed, 134 insertions(+), 74 deletions(-) >> >> -- >> 2.25.1 >> > _______________________________________________ > libcamera-devel mailing list > libcamera-devel@lists.libcamera.org > https://lists.libcamera.org/listinfo/libcamera-devel >
Hi Kieran, On Mon, 19 Oct 2020 at 15:01, Kieran Bingham < kieran.bingham@ideasonboard.com> wrote: > Hi Naush, > > On 19/10/2020 11:02, Naushir Patuck wrote: > > Hi all, > > > > Gentle ping to get some comments on this one. If you prefer, I could > > submit as an official patch to review? > > > > Thanks, > > Naush > > > > On Fri, 2 Oct 2020 at 10:33, Naushir Patuck <naush@raspberrypi.com> > wrote: > >> > >> Hi all, > >> > >> I would like to resurrect the topic of passing controls to the pipeline > handlers and ipas on startup. We talked about this some time back, and > unfortunately it drifted off my radar. > >> > >> Attached is some work I've done to resurrect this discussion. I have > added the ability for an application to pass in controls through the > camera::start() public API. This is similar to how controls are passed in > though Requests, the only difference being that if we do pass it in the > latter, we could incur multiple frames of delay before they are applied. > This would be unsuitable for a lot of users I feel. This change gives us a > huge jump in getting to feature parity with the existing Raspberry Pi > camera stack. > > Passing this in through start sounds a bit odd to me. > > Should this be part of the camera->configure() phase? > > It's setting the initial configuration right? > > And we can add a ControlList to the CameraConfiguration without needing > to update any function signatures. > > Then when a pipeline is called with configure(), it needs to apply any > controls in the list at that point. And it looks like the > CameraConfiguration is already passed through to: > > int RPiCameraData::configureIPA(const CameraConfiguration *config) > > So, then there is the information required at that phase too? > > I'm wondering if I've missed something obvious that would have prevented > you from already using this approach? > Indeed, this was one of the approaches we discussed some time back. However, as you mentioned, CameraConfiguration gets set in the camera->configure() phase, and this may not necessarily be what we want. An application may want to set startup parameters without doing a camera->configure() in a run. With this change, we would do a sequence like start(), stop() start(new config parameters). If we were to do start(), stop() configure(new config parameters), start(), it may incur additional overheads in calling configure() only to setup some new startup params, where the configure() might be going to the kernel and calling the sensors to set itself up again, even though it is not needed. Hope that makes sense? Regards, Naush > > -- > Kieran > > > >> > >> There are a couple of things to address in these changes: > >> - I have modified the camera::start() public API with the following > signature: int start(ControlList *controls = nullptr). I know the > convention is to use a reference here, but I wanted to keep API backward > compatible, and a pointer can have a default value nullptr. I could also > overload start and keep the old and new signature perhaps, but not sure on > the policy for public methods having multiple signatures. > >> > >> - the IPA context wrapper code needs the ability to serialise > IPAOperationData structures. This is needed for this code, as well as for > configure() as part of an earlier set of changes.>> Please note, this patch > set is for discussion to start with and not > ready to submit as-is. Happy to hear your thoughts. > >> > >> Regards, > >> Naush > >> > >> Naushir Patuck (5): > >> pipeline: raspberrypi: Sensor flips should be applied unconditionally > >> libcamera: pipeline: Pass libcamera controls into > >> pipeline_handler::start() > >> libcamera: ipa: Pass a set of controls and return results from > >> ipa::start() > >> pipeline: ipa: raspberrypi: Pass controls to IPA on start > >> DNI: qcam: Simple test to pass controls on camera::start() > >> > >> Documentation/guides/pipeline-handler.rst | 4 +- > >> include/libcamera/camera.h | 2 +- > >> .../libcamera/internal/ipa_context_wrapper.h | 3 +- > >> include/libcamera/internal/pipeline_handler.h | 2 +- > >> include/libcamera/ipa/ipa_interface.h | 3 +- > >> include/libcamera/ipa/raspberrypi.h | 1 + > >> src/ipa/libipa/ipa_interface_wrapper.cpp | 4 +- > >> src/ipa/raspberrypi/raspberrypi.cpp | 50 ++++++++++++------- > >> src/ipa/rkisp1/rkisp1.cpp | 3 +- > >> src/ipa/vimc/vimc.cpp | 6 ++- > >> src/libcamera/camera.cpp | 11 ++-- > >> src/libcamera/ipa_context_wrapper.cpp | 6 ++- > >> src/libcamera/ipa_interface.cpp | 7 +++ > >> src/libcamera/pipeline/ipu3/ipu3.cpp | 4 +- > >> .../pipeline/raspberrypi/raspberrypi.cpp | 46 ++++++++++------- > >> src/libcamera/pipeline/rkisp1/rkisp1.cpp | 9 ++-- > >> src/libcamera/pipeline/simple/simple.cpp | 4 +- > >> src/libcamera/pipeline/uvcvideo/uvcvideo.cpp | 4 +- > >> src/libcamera/pipeline/vimc/vimc.cpp | 7 +-- > >> src/libcamera/pipeline_handler.cpp | 1 + > >> src/libcamera/proxy/ipa_proxy_linux.cpp | 3 +- > >> src/libcamera/proxy/ipa_proxy_thread.cpp | 13 +++-- > >> src/qcam/main_window.cpp | 7 ++- > >> test/ipa/ipa_interface_test.cpp | 3 +- > >> test/ipa/ipa_wrappers_test.cpp | 5 +- > >> 25 files changed, 134 insertions(+), 74 deletions(-) > >> > >> -- > >> 2.25.1 > >> > > _______________________________________________ > > libcamera-devel mailing list > > libcamera-devel@lists.libcamera.org > > https://lists.libcamera.org/listinfo/libcamera-devel > > > > -- > Regards > -- > Kieran >
Hi all, I would like to resurrect the topic of passing controls to the pipeline handlers and ipas on startup. We talked about this some time back, and unfortunately it drifted off my radar. Attached is some work I've done to resurrect this discussion. I have added the ability for an application to pass in controls through the camera::start() public API. This is similar to how controls are passed in though Requests, the only difference being that if we do pass it in the latter, we could incur multiple frames of delay before they are applied. This would be unsuitable for a lot of users I feel. This change gives us a huge jump in getting to feature parity with the existing Raspberry Pi camera stack. There are a couple of things to address in these changes: - I have modified the camera::start() public API with the following signature: int start(ControlList *controls = nullptr). I know the convention is to use a reference here, but I wanted to keep API backward compatible, and a pointer can have a default value nullptr. I could also overload start and keep the old and new signature perhaps, but not sure on the policy for public methods having multiple signatures. - the IPA context wrapper code needs the ability to serialise IPAOperationData structures. This is needed for this code, as well as for configure() as part of an earlier set of changes. Please note, this patch set is for discussion to start with and not ready to submit as-is. Happy to hear your thoughts. Regards, Naush Naushir Patuck (5): pipeline: raspberrypi: Sensor flips should be applied unconditionally libcamera: pipeline: Pass libcamera controls into pipeline_handler::start() libcamera: ipa: Pass a set of controls and return results from ipa::start() pipeline: ipa: raspberrypi: Pass controls to IPA on start DNI: qcam: Simple test to pass controls on camera::start() Documentation/guides/pipeline-handler.rst | 4 +- include/libcamera/camera.h | 2 +- .../libcamera/internal/ipa_context_wrapper.h | 3 +- include/libcamera/internal/pipeline_handler.h | 2 +- include/libcamera/ipa/ipa_interface.h | 3 +- include/libcamera/ipa/raspberrypi.h | 1 + src/ipa/libipa/ipa_interface_wrapper.cpp | 4 +- src/ipa/raspberrypi/raspberrypi.cpp | 50 ++++++++++++------- src/ipa/rkisp1/rkisp1.cpp | 3 +- src/ipa/vimc/vimc.cpp | 6 ++- src/libcamera/camera.cpp | 11 ++-- src/libcamera/ipa_context_wrapper.cpp | 6 ++- src/libcamera/ipa_interface.cpp | 7 +++ src/libcamera/pipeline/ipu3/ipu3.cpp | 4 +- .../pipeline/raspberrypi/raspberrypi.cpp | 46 ++++++++++------- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 9 ++-- src/libcamera/pipeline/simple/simple.cpp | 4 +- src/libcamera/pipeline/uvcvideo/uvcvideo.cpp | 4 +- src/libcamera/pipeline/vimc/vimc.cpp | 7 +-- src/libcamera/pipeline_handler.cpp | 1 + src/libcamera/proxy/ipa_proxy_linux.cpp | 3 +- src/libcamera/proxy/ipa_proxy_thread.cpp | 13 +++-- src/qcam/main_window.cpp | 7 ++- test/ipa/ipa_interface_test.cpp | 3 +- test/ipa/ipa_wrappers_test.cpp | 5 +- 25 files changed, 134 insertions(+), 74 deletions(-)