Message ID | 20240808102346.13065-2-naush@raspberrypi.com |
---|---|
State | Superseded, archived |
Headers | show |
Series |
|
Related | show |
Hi Naush On Thu, Aug 08, 2024 at 11:23:40AM GMT, Naushir Patuck wrote: > Add a vendor control rpi::ScalerCrops that is analogous to the current > core::ScalerCrop, but can apply a different crop to each configured > stream. > > This control takes a span of Rectangle structures - the order of > rectangles must match the order of streams configured by the application. > > Signed-off-by: Naushir Patuck <naush@raspberrypi.com> > --- > src/ipa/rpi/common/ipa_base.cpp | 2 ++ > src/libcamera/control_ids_rpi.yaml | 15 +++++++++++++++ > 2 files changed, 17 insertions(+) > > diff --git a/src/ipa/rpi/common/ipa_base.cpp b/src/ipa/rpi/common/ipa_base.cpp > index ee3848b54f21..463f6d384c9e 100644 > --- a/src/ipa/rpi/common/ipa_base.cpp > +++ b/src/ipa/rpi/common/ipa_base.cpp > @@ -71,6 +71,7 @@ const ControlInfoMap::Map ipaControls{ > { &controls::HdrMode, ControlInfo(controls::HdrModeValues) }, > { &controls::Sharpness, ControlInfo(0.0f, 16.0f, 1.0f) }, > { &controls::ScalerCrop, ControlInfo(Rectangle{}, Rectangle(65535, 65535, 65535, 65535), Rectangle{}) }, > + { &controls::rpi::ScalerCrops, ControlInfo(Rectangle{}, Rectangle(65535, 65535, 65535, 65535), Rectangle{}) }, Can this be registered only in the pisp IPA ? In this way you won't see the control on vc4 platforms, and you can safely keep the code handling it in pipeline_base, but it will only be exercized on pi5 > { &controls::FrameDurationLimits, ControlInfo(INT64_C(33333), INT64_C(120000)) }, > { &controls::draft::NoiseReductionMode, ControlInfo(controls::draft::NoiseReductionModeValues) }, > { &controls::rpi::StatsOutputEnable, ControlInfo(false, true, false) }, > @@ -1070,6 +1071,7 @@ void IpaBase::applyControls(const ControlList &controls) > break; > } > > + case controls::rpi::SCALER_CROPS: > case controls::SCALER_CROP: { > /* We do nothing with this, but should avoid the warning below. */ > break; > diff --git a/src/libcamera/control_ids_rpi.yaml b/src/libcamera/control_ids_rpi.yaml > index cb097f887e16..3d4a305dd8f5 100644 > --- a/src/libcamera/control_ids_rpi.yaml > +++ b/src/libcamera/control_ids_rpi.yaml > @@ -26,4 +26,19 @@ controls: > > \sa StatsOutputEnable > > + - ScalerCrops: > + type: Rectangle > + size: [n] > + description: | > + An array of rectangles, where each singular value has identical functionality > + to the ScalerCrop control. This control allows the Raspberry Pi pipeline > + handler to control individual scaler crops per output stream. > + > + The order of rectangles passed into the control must match the order of > + streams configured by the application. > + > + Note that using different crop rectangles for each output stream is only > + applicable on the Pi5/PiSP platform. > + > + \sa ScalerCrop > ... > -- > 2.34.1 >
Hi Jacopo, Thank you for the feedback. On Wed, 28 Aug 2024 at 10:10, Jacopo Mondi <jacopo.mondi@ideasonboard.com> wrote: > Hi Naush > > On Thu, Aug 08, 2024 at 11:23:40AM GMT, Naushir Patuck wrote: > > Add a vendor control rpi::ScalerCrops that is analogous to the current > > core::ScalerCrop, but can apply a different crop to each configured > > stream. > > > > This control takes a span of Rectangle structures - the order of > > rectangles must match the order of streams configured by the application. > > > > Signed-off-by: Naushir Patuck <naush@raspberrypi.com> > > --- > > src/ipa/rpi/common/ipa_base.cpp | 2 ++ > > src/libcamera/control_ids_rpi.yaml | 15 +++++++++++++++ > > 2 files changed, 17 insertions(+) > > > > diff --git a/src/ipa/rpi/common/ipa_base.cpp > b/src/ipa/rpi/common/ipa_base.cpp > > index ee3848b54f21..463f6d384c9e 100644 > > --- a/src/ipa/rpi/common/ipa_base.cpp > > +++ b/src/ipa/rpi/common/ipa_base.cpp > > @@ -71,6 +71,7 @@ const ControlInfoMap::Map ipaControls{ > > { &controls::HdrMode, ControlInfo(controls::HdrModeValues) }, > > { &controls::Sharpness, ControlInfo(0.0f, 16.0f, 1.0f) }, > > { &controls::ScalerCrop, ControlInfo(Rectangle{}, Rectangle(65535, > 65535, 65535, 65535), Rectangle{}) }, > > + { &controls::rpi::ScalerCrops, ControlInfo(Rectangle{}, > Rectangle(65535, 65535, 65535, 65535), Rectangle{}) }, > > Can this be registered only in the pisp IPA ? In this way you won't > see the control on vc4 platforms, and you can safely keep the code > handling it in pipeline_base, but it will only be exercized on pi5 > Yes this is possible. I can rework it for v2. Regards, Naush > > > { &controls::FrameDurationLimits, ControlInfo(INT64_C(33333), > INT64_C(120000)) }, > > { &controls::draft::NoiseReductionMode, > ControlInfo(controls::draft::NoiseReductionModeValues) }, > > { &controls::rpi::StatsOutputEnable, ControlInfo(false, true, > false) }, > > @@ -1070,6 +1071,7 @@ void IpaBase::applyControls(const ControlList > &controls) > > break; > > } > > > > + case controls::rpi::SCALER_CROPS: > > case controls::SCALER_CROP: { > > /* We do nothing with this, but should avoid the > warning below. */ > > break; > > diff --git a/src/libcamera/control_ids_rpi.yaml > b/src/libcamera/control_ids_rpi.yaml > > index cb097f887e16..3d4a305dd8f5 100644 > > --- a/src/libcamera/control_ids_rpi.yaml > > +++ b/src/libcamera/control_ids_rpi.yaml > > @@ -26,4 +26,19 @@ controls: > > > > \sa StatsOutputEnable > > > > + - ScalerCrops: > > + type: Rectangle > > + size: [n] > > + description: | > > + An array of rectangles, where each singular value has identical > functionality > > + to the ScalerCrop control. This control allows the Raspberry Pi > pipeline > > + handler to control individual scaler crops per output stream. > > + > > + The order of rectangles passed into the control must match the > order of > > + streams configured by the application. > > + > > + Note that using different crop rectangles for each output > stream is only > > + applicable on the Pi5/PiSP platform. > > + > > + \sa ScalerCrop > > ... > > -- > > 2.34.1 > > >
Hello, On Wed, Aug 28, 2024 at 10:58:55AM +0100, Naushir Patuck wrote: > On Wed, 28 Aug 2024 at 10:10, Jacopo Mondi wrote: > > On Thu, Aug 08, 2024 at 11:23:40AM GMT, Naushir Patuck wrote: > > > Add a vendor control rpi::ScalerCrops that is analogous to the current > > > core::ScalerCrop, but can apply a different crop to each configured > > > stream. > > > > > > This control takes a span of Rectangle structures - the order of > > > rectangles must match the order of streams configured by the application. > > > > > > Signed-off-by: Naushir Patuck <naush@raspberrypi.com> > > > --- > > > src/ipa/rpi/common/ipa_base.cpp | 2 ++ > > > src/libcamera/control_ids_rpi.yaml | 15 +++++++++++++++ > > > 2 files changed, 17 insertions(+) > > > > > > diff --git a/src/ipa/rpi/common/ipa_base.cpp b/src/ipa/rpi/common/ipa_base.cpp > > > index ee3848b54f21..463f6d384c9e 100644 > > > --- a/src/ipa/rpi/common/ipa_base.cpp > > > +++ b/src/ipa/rpi/common/ipa_base.cpp > > > @@ -71,6 +71,7 @@ const ControlInfoMap::Map ipaControls{ > > > { &controls::HdrMode, ControlInfo(controls::HdrModeValues) }, > > > { &controls::Sharpness, ControlInfo(0.0f, 16.0f, 1.0f) }, > > > { &controls::ScalerCrop, ControlInfo(Rectangle{}, Rectangle(65535, 65535, 65535, 65535), Rectangle{}) }, > > > + { &controls::rpi::ScalerCrops, ControlInfo(Rectangle{}, Rectangle(65535, 65535, 65535, 65535), Rectangle{}) }, > > > > Can this be registered only in the pisp IPA ? In this way you won't > > see the control on vc4 platforms, and you can safely keep the code > > handling it in pipeline_base, but it will only be exercized on pi5 > > Yes this is possible. I can rework it for v2. I would prefer that too. > > > { &controls::FrameDurationLimits, ControlInfo(INT64_C(33333), INT64_C(120000)) }, > > > { &controls::draft::NoiseReductionMode, ControlInfo(controls::draft::NoiseReductionModeValues) }, > > > { &controls::rpi::StatsOutputEnable, ControlInfo(false, true, false) }, > > > @@ -1070,6 +1071,7 @@ void IpaBase::applyControls(const ControlList &controls) > > > break; > > > } > > > > > > + case controls::rpi::SCALER_CROPS: > > > case controls::SCALER_CROP: { > > > /* We do nothing with this, but should avoid the warning below. */ > > > break; > > > diff --git a/src/libcamera/control_ids_rpi.yaml b/src/libcamera/control_ids_rpi.yaml > > > index cb097f887e16..3d4a305dd8f5 100644 > > > --- a/src/libcamera/control_ids_rpi.yaml > > > +++ b/src/libcamera/control_ids_rpi.yaml > > > @@ -26,4 +26,19 @@ controls: > > > > > > \sa StatsOutputEnable > > > > > > + - ScalerCrops: > > > + type: Rectangle > > > + size: [n] > > > + description: | > > > + An array of rectangles, where each singular value has identical functionality > > > + to the ScalerCrop control. This control allows the Raspberry Pi pipeline > > > + handler to control individual scaler crops per output stream. > > > + > > > + The order of rectangles passed into the control must match the order of > > > + streams configured by the application. How about the number of rectangles ? Can you specify less rectangles than the number of streams ? Or more ? What happens if you do, is the control completely ignored, or do part of the rectangles get applied ? Is there also a need to allow applications to change the scaler crop for a subset of the streams, without affecting other ones ? Something that isn't explained in this, and barely mentioned in the cover letter (it just says "preliminary" without elaborating) is that this vendor control is a work around for the lack of a per-stream controls API. Do we agree that this control will be replaced by per-stream controls in the future, and that it will then be dropped and not kept for backward compatibility ? If so, could you please record this here, with a \todo note ? Can I also volunteer you for a per-stream control RFC ? :-) > > > + > > > + Note that using different crop rectangles for each output stream is only > > > + applicable on the Pi5/PiSP platform. > > > + > > > + \sa ScalerCrop > > > ...
Hi Laurent, On Wed, 28 Aug 2024 at 17:17, Laurent Pinchart <laurent.pinchart@ideasonboard.com> wrote: > > Hello, > > On Wed, Aug 28, 2024 at 10:58:55AM +0100, Naushir Patuck wrote: > > On Wed, 28 Aug 2024 at 10:10, Jacopo Mondi wrote: > > > On Thu, Aug 08, 2024 at 11:23:40AM GMT, Naushir Patuck wrote: > > > > Add a vendor control rpi::ScalerCrops that is analogous to the current > > > > core::ScalerCrop, but can apply a different crop to each configured > > > > stream. > > > > > > > > This control takes a span of Rectangle structures - the order of > > > > rectangles must match the order of streams configured by the application. > > > > > > > > Signed-off-by: Naushir Patuck <naush@raspberrypi.com> > > > > --- > > > > src/ipa/rpi/common/ipa_base.cpp | 2 ++ > > > > src/libcamera/control_ids_rpi.yaml | 15 +++++++++++++++ > > > > 2 files changed, 17 insertions(+) > > > > > > > > diff --git a/src/ipa/rpi/common/ipa_base.cpp b/src/ipa/rpi/common/ipa_base.cpp > > > > index ee3848b54f21..463f6d384c9e 100644 > > > > --- a/src/ipa/rpi/common/ipa_base.cpp > > > > +++ b/src/ipa/rpi/common/ipa_base.cpp > > > > @@ -71,6 +71,7 @@ const ControlInfoMap::Map ipaControls{ > > > > { &controls::HdrMode, ControlInfo(controls::HdrModeValues) }, > > > > { &controls::Sharpness, ControlInfo(0.0f, 16.0f, 1.0f) }, > > > > { &controls::ScalerCrop, ControlInfo(Rectangle{}, Rectangle(65535, 65535, 65535, 65535), Rectangle{}) }, > > > > + { &controls::rpi::ScalerCrops, ControlInfo(Rectangle{}, Rectangle(65535, 65535, 65535, 65535), Rectangle{}) }, > > > > > > Can this be registered only in the pisp IPA ? In this way you won't > > > see the control on vc4 platforms, and you can safely keep the code > > > handling it in pipeline_base, but it will only be exercized on pi5 > > > > Yes this is possible. I can rework it for v2. > > I would prefer that too. Ack > > > > > { &controls::FrameDurationLimits, ControlInfo(INT64_C(33333), INT64_C(120000)) }, > > > > { &controls::draft::NoiseReductionMode, ControlInfo(controls::draft::NoiseReductionModeValues) }, > > > > { &controls::rpi::StatsOutputEnable, ControlInfo(false, true, false) }, > > > > @@ -1070,6 +1071,7 @@ void IpaBase::applyControls(const ControlList &controls) > > > > break; > > > > } > > > > > > > > + case controls::rpi::SCALER_CROPS: > > > > case controls::SCALER_CROP: { > > > > /* We do nothing with this, but should avoid the warning below. */ > > > > break; > > > > diff --git a/src/libcamera/control_ids_rpi.yaml b/src/libcamera/control_ids_rpi.yaml > > > > index cb097f887e16..3d4a305dd8f5 100644 > > > > --- a/src/libcamera/control_ids_rpi.yaml > > > > +++ b/src/libcamera/control_ids_rpi.yaml > > > > @@ -26,4 +26,19 @@ controls: > > > > > > > > \sa StatsOutputEnable > > > > > > > > + - ScalerCrops: > > > > + type: Rectangle > > > > + size: [n] > > > > + description: | > > > > + An array of rectangles, where each singular value has identical functionality > > > > + to the ScalerCrop control. This control allows the Raspberry Pi pipeline > > > > + handler to control individual scaler crops per output stream. > > > > + > > > > + The order of rectangles passed into the control must match the order of > > > > + streams configured by the application. > > How about the number of rectangles ? Can you specify less rectangles > than the number of streams ? Or more ? What happens if you do, is the > control completely ignored, or do part of the rectangles get applied ? > > Is there also a need to allow applications to change the scaler crop for > a subset of the streams, without affecting other ones ? Sadly this mechanism is not flexible at all. So the application must specify the correct number of scalar crop rectangles based on the number of configured streams unconditionally. A new control API will definitely need such flexibility. > > Something that isn't explained in this, and barely mentioned in the > cover letter (it just says "preliminary" without elaborating) is that > this vendor control is a work around for the lack of a per-stream > controls API. Do we agree that this control will be replaced by > per-stream controls in the future, and that it will then be dropped and > not kept for backward compatibility ? If so, could you please record > this here, with a \todo note ? > > Can I also volunteer you for a per-stream control RFC ? :-) This change is really a stop-gap until we have an official API for per-stream controls. I'll add a todo to note this. I will also make a RFC proposal for the changes and perhaps we can spend some time talking about it in the Vienna F2F. Regards, Naush > > > > > + > > > > + Note that using different crop rectangles for each output stream is only > > > > + applicable on the Pi5/PiSP platform. > > > > + > > > > + \sa ScalerCrop > > > > ... > > -- > Regards, > > Laurent Pinchart
On Fri, Aug 30, 2024 at 11:05:28AM +0100, Naushir Patuck wrote: > On Wed, 28 Aug 2024 at 17:17, Laurent Pinchart wrote: > > On Wed, Aug 28, 2024 at 10:58:55AM +0100, Naushir Patuck wrote: > > > On Wed, 28 Aug 2024 at 10:10, Jacopo Mondi wrote: > > > > On Thu, Aug 08, 2024 at 11:23:40AM GMT, Naushir Patuck wrote: > > > > > Add a vendor control rpi::ScalerCrops that is analogous to the current > > > > > core::ScalerCrop, but can apply a different crop to each configured > > > > > stream. > > > > > > > > > > This control takes a span of Rectangle structures - the order of > > > > > rectangles must match the order of streams configured by the application. > > > > > > > > > > Signed-off-by: Naushir Patuck <naush@raspberrypi.com> > > > > > --- > > > > > src/ipa/rpi/common/ipa_base.cpp | 2 ++ > > > > > src/libcamera/control_ids_rpi.yaml | 15 +++++++++++++++ > > > > > 2 files changed, 17 insertions(+) > > > > > > > > > > diff --git a/src/ipa/rpi/common/ipa_base.cpp b/src/ipa/rpi/common/ipa_base.cpp > > > > > index ee3848b54f21..463f6d384c9e 100644 > > > > > --- a/src/ipa/rpi/common/ipa_base.cpp > > > > > +++ b/src/ipa/rpi/common/ipa_base.cpp > > > > > @@ -71,6 +71,7 @@ const ControlInfoMap::Map ipaControls{ > > > > > { &controls::HdrMode, ControlInfo(controls::HdrModeValues) }, > > > > > { &controls::Sharpness, ControlInfo(0.0f, 16.0f, 1.0f) }, > > > > > { &controls::ScalerCrop, ControlInfo(Rectangle{}, Rectangle(65535, 65535, 65535, 65535), Rectangle{}) }, > > > > > + { &controls::rpi::ScalerCrops, ControlInfo(Rectangle{}, Rectangle(65535, 65535, 65535, 65535), Rectangle{}) }, > > > > > > > > Can this be registered only in the pisp IPA ? In this way you won't > > > > see the control on vc4 platforms, and you can safely keep the code > > > > handling it in pipeline_base, but it will only be exercized on pi5 > > > > > > Yes this is possible. I can rework it for v2. > > > > I would prefer that too. > > Ack > > > > > > { &controls::FrameDurationLimits, ControlInfo(INT64_C(33333), INT64_C(120000)) }, > > > > > { &controls::draft::NoiseReductionMode, ControlInfo(controls::draft::NoiseReductionModeValues) }, > > > > > { &controls::rpi::StatsOutputEnable, ControlInfo(false, true, false) }, > > > > > @@ -1070,6 +1071,7 @@ void IpaBase::applyControls(const ControlList &controls) > > > > > break; > > > > > } > > > > > > > > > > + case controls::rpi::SCALER_CROPS: > > > > > case controls::SCALER_CROP: { > > > > > /* We do nothing with this, but should avoid the warning below. */ > > > > > break; > > > > > diff --git a/src/libcamera/control_ids_rpi.yaml b/src/libcamera/control_ids_rpi.yaml > > > > > index cb097f887e16..3d4a305dd8f5 100644 > > > > > --- a/src/libcamera/control_ids_rpi.yaml > > > > > +++ b/src/libcamera/control_ids_rpi.yaml > > > > > @@ -26,4 +26,19 @@ controls: > > > > > > > > > > \sa StatsOutputEnable > > > > > > > > > > + - ScalerCrops: > > > > > + type: Rectangle > > > > > + size: [n] > > > > > + description: | > > > > > + An array of rectangles, where each singular value has identical functionality > > > > > + to the ScalerCrop control. This control allows the Raspberry Pi pipeline > > > > > + handler to control individual scaler crops per output stream. > > > > > + > > > > > + The order of rectangles passed into the control must match the order of > > > > > + streams configured by the application. > > > > How about the number of rectangles ? Can you specify less rectangles > > than the number of streams ? Or more ? What happens if you do, is the > > control completely ignored, or do part of the rectangles get applied ? > > > > Is there also a need to allow applications to change the scaler crop for > > a subset of the streams, without affecting other ones ? > > Sadly this mechanism is not flexible at all. So the application must > specify the correct number of scalar crop rectangles based on the > number of configured streams unconditionally. A new control API will > definitely need such flexibility. Fine with me. Could you please document the requirement here ? > > Something that isn't explained in this, and barely mentioned in the > > cover letter (it just says "preliminary" without elaborating) is that > > this vendor control is a work around for the lack of a per-stream > > controls API. Do we agree that this control will be replaced by > > per-stream controls in the future, and that it will then be dropped and > > not kept for backward compatibility ? If so, could you please record > > this here, with a \todo note ? > > > > Can I also volunteer you for a per-stream control RFC ? :-) > > This change is really a stop-gap until we have an official API for > per-stream controls. I'll add a todo to note this. I will also make > a RFC proposal for the changes and perhaps we can spend some time > talking about it in the Vienna F2F. Great, thank you ! > > > > > + > > > > > + Note that using different crop rectangles for each output stream is only > > > > > + applicable on the Pi5/PiSP platform. > > > > > + > > > > > + \sa ScalerCrop > > > > > ...
diff --git a/src/ipa/rpi/common/ipa_base.cpp b/src/ipa/rpi/common/ipa_base.cpp index ee3848b54f21..463f6d384c9e 100644 --- a/src/ipa/rpi/common/ipa_base.cpp +++ b/src/ipa/rpi/common/ipa_base.cpp @@ -71,6 +71,7 @@ const ControlInfoMap::Map ipaControls{ { &controls::HdrMode, ControlInfo(controls::HdrModeValues) }, { &controls::Sharpness, ControlInfo(0.0f, 16.0f, 1.0f) }, { &controls::ScalerCrop, ControlInfo(Rectangle{}, Rectangle(65535, 65535, 65535, 65535), Rectangle{}) }, + { &controls::rpi::ScalerCrops, ControlInfo(Rectangle{}, Rectangle(65535, 65535, 65535, 65535), Rectangle{}) }, { &controls::FrameDurationLimits, ControlInfo(INT64_C(33333), INT64_C(120000)) }, { &controls::draft::NoiseReductionMode, ControlInfo(controls::draft::NoiseReductionModeValues) }, { &controls::rpi::StatsOutputEnable, ControlInfo(false, true, false) }, @@ -1070,6 +1071,7 @@ void IpaBase::applyControls(const ControlList &controls) break; } + case controls::rpi::SCALER_CROPS: case controls::SCALER_CROP: { /* We do nothing with this, but should avoid the warning below. */ break; diff --git a/src/libcamera/control_ids_rpi.yaml b/src/libcamera/control_ids_rpi.yaml index cb097f887e16..3d4a305dd8f5 100644 --- a/src/libcamera/control_ids_rpi.yaml +++ b/src/libcamera/control_ids_rpi.yaml @@ -26,4 +26,19 @@ controls: \sa StatsOutputEnable + - ScalerCrops: + type: Rectangle + size: [n] + description: | + An array of rectangles, where each singular value has identical functionality + to the ScalerCrop control. This control allows the Raspberry Pi pipeline + handler to control individual scaler crops per output stream. + + The order of rectangles passed into the control must match the order of + streams configured by the application. + + Note that using different crop rectangles for each output stream is only + applicable on the Pi5/PiSP platform. + + \sa ScalerCrop ...
Add a vendor control rpi::ScalerCrops that is analogous to the current core::ScalerCrop, but can apply a different crop to each configured stream. This control takes a span of Rectangle structures - the order of rectangles must match the order of streams configured by the application. Signed-off-by: Naushir Patuck <naush@raspberrypi.com> --- src/ipa/rpi/common/ipa_base.cpp | 2 ++ src/libcamera/control_ids_rpi.yaml | 15 +++++++++++++++ 2 files changed, 17 insertions(+)