| Message ID | 20210907194107.803730-16-jacopo@jmondi.org |
|---|---|
| State | Superseded |
| Headers | show |
| Series |
|
| Related | show |
Hi Jacopo, On Tue, Sep 07, 2021 at 09:41:05PM +0200, Jacopo Mondi wrote: > Limit the reported minumum frame duration to 30 FPS. > > The reason to do is to bring the libcamra HAL in par with the Intel > HAL implementation on IPU3 platform, where 30FPS is the frame rate used > to perform quality tuning in the closed-source IPA module and has been > validated as the most efficient rate for the power/performace budget. > > This change bring into the HAL a platform specific constraints, which > might be opportune for most platforms but should rather be configurable > by system integrators. Record that with a \todo entry. > > Also record that, even if we report a lower frame rate, we currently > do not limit what the camera actually produce. Hm, now I'm starting to wonder if we should cap the reported min frame duration in the HAL, but allow libcamera to go as fast as any user wants. But then we also need a way to actually set the frame duration in libcamera... If the HAL reports max 30fps, then the HAL will only request up to 30fps, and then we can just set that on libcamera's side... that way we satisfy the HAL requirements while keeping the flexibility on libcamera's side. Or would it be better to limit libcamera's IPU3 as well, since 30fps has been confirmed to be the best? "We know what the user wants more than the user knows what they want" Paul > > Signed-off-by: Jacopo Mondi <jacopo@jmondi.org> > --- > src/android/camera_capabilities.cpp | 23 +++++++++++++++++++++++ > 1 file changed, 23 insertions(+) > > diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp > index deaec3282fd7..cff1e30a9567 100644 > --- a/src/android/camera_capabilities.cpp > +++ b/src/android/camera_capabilities.cpp > @@ -632,6 +632,29 @@ int CameraCapabilities::initializeStreamConfigurations() > > int64_t minFrameDuration = frameDurations->second.min().get<int64_t>() * 1000; > int64_t maxFrameDuration = frameDurations->second.max().get<int64_t>() * 1000; > + > + /* > + * Cap min frame duration to 30 FPS. > + * > + * 30 frames per second has been validated as the most > + * opportune frame rate for quality tuning, and power > + * vs performances budget on Intel IPU3. > + * > + * \todo This is a platform-specific decision that needs > + * to be abstracted and delegated to the configuration > + * file. > + * > + * \todo libcamera only allows to control frame duration > + * through the per-request controls::FrameDuration > + * control. If we cap the durations here, we should be > + * capable of configuring the camera to operate at such > + * duration without requiring to have the FrameDuration > + * control to be specified for each Request. Defer this > + * to the in-development configuration API rework. > + */ > + if (minFrameDuration < 1e9 / 30.0) > + minFrameDuration = 1e9 / 30.0; > + > streamConfigurations_.push_back({ > res, androidFormat, minFrameDuration, maxFrameDuration, > }); > -- > 2.32.0 >
Hello, On Wed, Sep 08, 2021 at 09:36:20AM +0900, paul.elder@ideasonboard.com wrote: > On Tue, Sep 07, 2021 at 09:41:05PM +0200, Jacopo Mondi wrote: > > Limit the reported minumum frame duration to 30 FPS. > > > > The reason to do is to bring the libcamra HAL in par with the Intel > > HAL implementation on IPU3 platform, where 30FPS is the frame rate used > > to perform quality tuning in the closed-source IPA module and has been > > validated as the most efficient rate for the power/performace budget. > > > > This change bring into the HAL a platform specific constraints, which > > might be opportune for most platforms but should rather be configurable > > by system integrators. Record that with a \todo entry. > > > > Also record that, even if we report a lower frame rate, we currently > > do not limit what the camera actually produce. > > Hm, now I'm starting to wonder if we should cap the reported min frame > duration in the HAL, but allow libcamera to go as fast as any user > wants. But then we also need a way to actually set the frame duration in > libcamera... > > If the HAL reports max 30fps, then the HAL will only request up to > 30fps, and then we can just set that on libcamera's side... that way we > satisfy the HAL requirements while keeping the flexibility on > libcamera's side. > > Or would it be better to limit libcamera's IPU3 as well, since 30fps has > been confirmed to be the best? "We know what the user wants more than > the user knows what they want" I'd rather go with this patch, I don't think libcamera should limit the frame rate artifically on IPU3. > > Signed-off-by: Jacopo Mondi <jacopo@jmondi.org> > > --- > > src/android/camera_capabilities.cpp | 23 +++++++++++++++++++++++ > > 1 file changed, 23 insertions(+) > > > > diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp > > index deaec3282fd7..cff1e30a9567 100644 > > --- a/src/android/camera_capabilities.cpp > > +++ b/src/android/camera_capabilities.cpp > > @@ -632,6 +632,29 @@ int CameraCapabilities::initializeStreamConfigurations() > > > > int64_t minFrameDuration = frameDurations->second.min().get<int64_t>() * 1000; > > int64_t maxFrameDuration = frameDurations->second.max().get<int64_t>() * 1000; > > + > > + /* > > + * Cap min frame duration to 30 FPS. > > + * > > + * 30 frames per second has been validated as the most > > + * opportune frame rate for quality tuning, and power > > + * vs performances budget on Intel IPU3. I'd say "on Intel IPU3-based Chromebooks", as this budget decision is device-specific. Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> > > + * > > + * \todo This is a platform-specific decision that needs > > + * to be abstracted and delegated to the configuration > > + * file. > > + * > > + * \todo libcamera only allows to control frame duration > > + * through the per-request controls::FrameDuration > > + * control. If we cap the durations here, we should be > > + * capable of configuring the camera to operate at such > > + * duration without requiring to have the FrameDuration > > + * control to be specified for each Request. Defer this > > + * to the in-development configuration API rework. > > + */ > > + if (minFrameDuration < 1e9 / 30.0) > > + minFrameDuration = 1e9 / 30.0; > > + > > streamConfigurations_.push_back({ > > res, androidFormat, minFrameDuration, maxFrameDuration, > > });
On Wed, Oct 06, 2021 at 04:53:57AM +0300, Laurent Pinchart wrote: > Hello, > > On Wed, Sep 08, 2021 at 09:36:20AM +0900, paul.elder@ideasonboard.com wrote: > > On Tue, Sep 07, 2021 at 09:41:05PM +0200, Jacopo Mondi wrote: > > > Limit the reported minumum frame duration to 30 FPS. > > > > > > The reason to do is to bring the libcamra HAL in par with the Intel > > > HAL implementation on IPU3 platform, where 30FPS is the frame rate used > > > to perform quality tuning in the closed-source IPA module and has been > > > validated as the most efficient rate for the power/performace budget. > > > > > > This change bring into the HAL a platform specific constraints, which > > > might be opportune for most platforms but should rather be configurable > > > by system integrators. Record that with a \todo entry. > > > > > > Also record that, even if we report a lower frame rate, we currently > > > do not limit what the camera actually produce. > > > > Hm, now I'm starting to wonder if we should cap the reported min frame > > duration in the HAL, but allow libcamera to go as fast as any user > > wants. But then we also need a way to actually set the frame duration in > > libcamera... > > > > If the HAL reports max 30fps, then the HAL will only request up to > > 30fps, and then we can just set that on libcamera's side... that way we > > satisfy the HAL requirements while keeping the flexibility on > > libcamera's side. > > The problem here is that we have currently no way to 'limit' the frame rate to 30FPS if not issuing a FrameDuration in each request. Ideally we should be able to set the desired duration as part of the StreamConfiguration as we do for the stream size and format. > > Or would it be better to limit libcamera's IPU3 as well, since 30fps has > > been confirmed to be the best? "We know what the user wants more than > > the user knows what they want" > > I'd rather go with this patch, I don't think libcamera should limit the > frame rate artifically on IPU3. Be aware that wihtout limiting the actual duration to what we report, we'll have tests failing due to a mismatch in the reported and calculated durations. Thanks j > > > > Signed-off-by: Jacopo Mondi <jacopo@jmondi.org> > > > --- > > > src/android/camera_capabilities.cpp | 23 +++++++++++++++++++++++ > > > 1 file changed, 23 insertions(+) > > > > > > diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp > > > index deaec3282fd7..cff1e30a9567 100644 > > > --- a/src/android/camera_capabilities.cpp > > > +++ b/src/android/camera_capabilities.cpp > > > @@ -632,6 +632,29 @@ int CameraCapabilities::initializeStreamConfigurations() > > > > > > int64_t minFrameDuration = frameDurations->second.min().get<int64_t>() * 1000; > > > int64_t maxFrameDuration = frameDurations->second.max().get<int64_t>() * 1000; > > > + > > > + /* > > > + * Cap min frame duration to 30 FPS. > > > + * > > > + * 30 frames per second has been validated as the most > > > + * opportune frame rate for quality tuning, and power > > > + * vs performances budget on Intel IPU3. > > I'd say "on Intel IPU3-based Chromebooks", as this budget decision is > device-specific. > > Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> > > > > + * > > > + * \todo This is a platform-specific decision that needs > > > + * to be abstracted and delegated to the configuration > > > + * file. > > > + * > > > + * \todo libcamera only allows to control frame duration > > > + * through the per-request controls::FrameDuration > > > + * control. If we cap the durations here, we should be > > > + * capable of configuring the camera to operate at such > > > + * duration without requiring to have the FrameDuration > > > + * control to be specified for each Request. Defer this > > > + * to the in-development configuration API rework. > > > + */ > > > + if (minFrameDuration < 1e9 / 30.0) > > > + minFrameDuration = 1e9 / 30.0; > > > + > > > streamConfigurations_.push_back({ > > > res, androidFormat, minFrameDuration, maxFrameDuration, > > > }); > > -- > Regards, > > Laurent Pinchart
diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp index deaec3282fd7..cff1e30a9567 100644 --- a/src/android/camera_capabilities.cpp +++ b/src/android/camera_capabilities.cpp @@ -632,6 +632,29 @@ int CameraCapabilities::initializeStreamConfigurations() int64_t minFrameDuration = frameDurations->second.min().get<int64_t>() * 1000; int64_t maxFrameDuration = frameDurations->second.max().get<int64_t>() * 1000; + + /* + * Cap min frame duration to 30 FPS. + * + * 30 frames per second has been validated as the most + * opportune frame rate for quality tuning, and power + * vs performances budget on Intel IPU3. + * + * \todo This is a platform-specific decision that needs + * to be abstracted and delegated to the configuration + * file. + * + * \todo libcamera only allows to control frame duration + * through the per-request controls::FrameDuration + * control. If we cap the durations here, we should be + * capable of configuring the camera to operate at such + * duration without requiring to have the FrameDuration + * control to be specified for each Request. Defer this + * to the in-development configuration API rework. + */ + if (minFrameDuration < 1e9 / 30.0) + minFrameDuration = 1e9 / 30.0; + streamConfigurations_.push_back({ res, androidFormat, minFrameDuration, maxFrameDuration, });
Limit the reported minumum frame duration to 30 FPS. The reason to do is to bring the libcamra HAL in par with the Intel HAL implementation on IPU3 platform, where 30FPS is the frame rate used to perform quality tuning in the closed-source IPA module and has been validated as the most efficient rate for the power/performace budget. This change bring into the HAL a platform specific constraints, which might be opportune for most platforms but should rather be configurable by system integrators. Record that with a \todo entry. Also record that, even if we report a lower frame rate, we currently do not limit what the camera actually produce. Signed-off-by: Jacopo Mondi <jacopo@jmondi.org> --- src/android/camera_capabilities.cpp | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+)