| Message ID | 20260107110544.16422-1-david.plowman@raspberrypi.com |
|---|---|
| State | New |
| Headers | show |
| Series |
|
| Related | show |
Hi 2026. 01. 07. 12:05 keltezéssel, David Plowman írta: > The controller min frame duration is used to rate limit how often we > run IPAs. Historically this has been set to 33333us, meaning that the > algorithms effectively skip frames when the camera is running faster > than 30fps. > > This patch adds a small amount of plumbing that allows this value to > be set in the Raspberry Pi configuration file. Some applications or > platforms (such as Pi 5) are easily capable of running these more > often, should there be a need to do so. > > Signed-off-by: David Plowman <david.plowman@raspberrypi.com> > --- > include/libcamera/ipa/raspberrypi.mojom | 1 + > src/ipa/rpi/common/ipa_base.cpp | 10 +++++++- > src/ipa/rpi/common/ipa_base.h | 2 ++ > .../pipeline/rpi/common/pipeline_base.cpp | 24 ++++++++++++++----- > .../pipeline/rpi/common/pipeline_base.h | 5 ++++ > .../pipeline/rpi/pisp/data/example.yaml | 6 +++++ > .../pipeline/rpi/vc4/data/example.yaml | 6 +++++ > 7 files changed, 47 insertions(+), 7 deletions(-) > > [...] > diff --git a/src/ipa/rpi/common/ipa_base.cpp b/src/ipa/rpi/common/ipa_base.cpp > index 14aba450..0a1fc0e9 100644 > --- a/src/ipa/rpi/common/ipa_base.cpp > +++ b/src/ipa/rpi/common/ipa_base.cpp > @@ -184,6 +184,14 @@ int32_t IpaBase::init(const IPASettings &settings, const InitParams ¶ms, Ini > > result->controlInfo = ControlInfoMap(std::move(ctrlMap), controls::controls); > > + /* > + * This determines the minimum allowable inter-frame duration to run the > + * controller algorithms. If the pipeline handler provider frames at a > + * rate higher than this, we rate-limit the controller Prepare() and > + * Process() calls to lower than or equal to this rate. > + */ > + controllerMinFrameDuration_ = params.controllerMinFrameDurationUs * 1us; > + > return platformInit(params, result); > } > > @@ -465,7 +473,7 @@ void IpaBase::prepareIsp(const PrepareParams ¶ms) > /* Allow a 10% margin on the comparison below. */ > Duration delta = (frameTimestamp - lastRunTimestamp_) * 1.0ns; > if (lastRunTimestamp_ && frameCount_ > invalidCount_ && > - delta < controllerMinFrameDuration * 0.9 && !hdrChange) { > + delta < controllerMinFrameDuration_ * 0.9 && !hdrChange) { I think the constant `controllerMinFrameDuration` in `ipa_base.cpp` can be removed, no? Regards, Barnabás Pőcze > /* > * Ensure we merge the previous frame's metadata with the current > * frame. This will not overwrite exposure/gain values for the > [...]
Hi Barnabas Thanks for the review. On Wed, 7 Jan 2026 at 14:45, Barnabás Pőcze <barnabas.pocze@ideasonboard.com> wrote: > > Hi > > 2026. 01. 07. 12:05 keltezéssel, David Plowman írta: > > The controller min frame duration is used to rate limit how often we > > run IPAs. Historically this has been set to 33333us, meaning that the > > algorithms effectively skip frames when the camera is running faster > > than 30fps. > > > > This patch adds a small amount of plumbing that allows this value to > > be set in the Raspberry Pi configuration file. Some applications or > > platforms (such as Pi 5) are easily capable of running these more > > often, should there be a need to do so. > > > > Signed-off-by: David Plowman <david.plowman@raspberrypi.com> > > --- > > include/libcamera/ipa/raspberrypi.mojom | 1 + > > src/ipa/rpi/common/ipa_base.cpp | 10 +++++++- > > src/ipa/rpi/common/ipa_base.h | 2 ++ > > .../pipeline/rpi/common/pipeline_base.cpp | 24 ++++++++++++++----- > > .../pipeline/rpi/common/pipeline_base.h | 5 ++++ > > .../pipeline/rpi/pisp/data/example.yaml | 6 +++++ > > .../pipeline/rpi/vc4/data/example.yaml | 6 +++++ > > 7 files changed, 47 insertions(+), 7 deletions(-) > > > > [...] > > diff --git a/src/ipa/rpi/common/ipa_base.cpp b/src/ipa/rpi/common/ipa_base.cpp > > index 14aba450..0a1fc0e9 100644 > > --- a/src/ipa/rpi/common/ipa_base.cpp > > +++ b/src/ipa/rpi/common/ipa_base.cpp > > @@ -184,6 +184,14 @@ int32_t IpaBase::init(const IPASettings &settings, const InitParams ¶ms, Ini > > > > result->controlInfo = ControlInfoMap(std::move(ctrlMap), controls::controls); > > > > + /* > > + * This determines the minimum allowable inter-frame duration to run the > > + * controller algorithms. If the pipeline handler provider frames at a > > + * rate higher than this, we rate-limit the controller Prepare() and > > + * Process() calls to lower than or equal to this rate. > > + */ > > + controllerMinFrameDuration_ = params.controllerMinFrameDurationUs * 1us; > > + > > return platformInit(params, result); > > } > > > > @@ -465,7 +473,7 @@ void IpaBase::prepareIsp(const PrepareParams ¶ms) > > /* Allow a 10% margin on the comparison below. */ > > Duration delta = (frameTimestamp - lastRunTimestamp_) * 1.0ns; > > if (lastRunTimestamp_ && frameCount_ > invalidCount_ && > > - delta < controllerMinFrameDuration * 0.9 && !hdrChange) { > > + delta < controllerMinFrameDuration_ * 0.9 && !hdrChange) { > > I think the constant `controllerMinFrameDuration` in `ipa_base.cpp` can be removed, no? Yes of course - thanks for spotting! David > > > Regards, > Barnabás Pőcze > > > /* > > * Ensure we merge the previous frame's metadata with the current > > * frame. This will not overwrite exposure/gain values for the > > [...]
diff --git a/include/libcamera/ipa/raspberrypi.mojom b/include/libcamera/ipa/raspberrypi.mojom index 12b083e9..1b7e0358 100644 --- a/include/libcamera/ipa/raspberrypi.mojom +++ b/include/libcamera/ipa/raspberrypi.mojom @@ -18,6 +18,7 @@ struct SensorConfig { struct InitParams { bool lensPresent; libcamera.IPACameraSensorInfo sensorInfo; + float controllerMinFrameDurationUs; /* PISP specific */ libcamera.SharedFD fe; libcamera.SharedFD be; diff --git a/src/ipa/rpi/common/ipa_base.cpp b/src/ipa/rpi/common/ipa_base.cpp index 14aba450..0a1fc0e9 100644 --- a/src/ipa/rpi/common/ipa_base.cpp +++ b/src/ipa/rpi/common/ipa_base.cpp @@ -184,6 +184,14 @@ int32_t IpaBase::init(const IPASettings &settings, const InitParams ¶ms, Ini result->controlInfo = ControlInfoMap(std::move(ctrlMap), controls::controls); + /* + * This determines the minimum allowable inter-frame duration to run the + * controller algorithms. If the pipeline handler provider frames at a + * rate higher than this, we rate-limit the controller Prepare() and + * Process() calls to lower than or equal to this rate. + */ + controllerMinFrameDuration_ = params.controllerMinFrameDurationUs * 1us; + return platformInit(params, result); } @@ -465,7 +473,7 @@ void IpaBase::prepareIsp(const PrepareParams ¶ms) /* Allow a 10% margin on the comparison below. */ Duration delta = (frameTimestamp - lastRunTimestamp_) * 1.0ns; if (lastRunTimestamp_ && frameCount_ > invalidCount_ && - delta < controllerMinFrameDuration * 0.9 && !hdrChange) { + delta < controllerMinFrameDuration_ * 0.9 && !hdrChange) { /* * Ensure we merge the previous frame's metadata with the current * frame. This will not overwrite exposure/gain values for the diff --git a/src/ipa/rpi/common/ipa_base.h b/src/ipa/rpi/common/ipa_base.h index 5348f2ea..90f018b2 100644 --- a/src/ipa/rpi/common/ipa_base.h +++ b/src/ipa/rpi/common/ipa_base.h @@ -142,6 +142,8 @@ private: } flickerState_; bool awbEnabled_; + + utils::Duration controllerMinFrameDuration_; }; } /* namespace ipa::RPi */ diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index fb8e466f..b7655d8d 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -33,6 +33,12 @@ LOG_DEFINE_CATEGORY(RPI) using StreamFlag = RPi::Stream::StreamFlag; +/* + * The IPA's algorithms will not be called more often than this many + * microseconds. The default corresponds to 30fps. + */ +constexpr float defaultControllerMinimumFrameDurationUs = 1000000.0 / 30.0; + namespace { constexpr unsigned int defaultRawBitDepth = 12; @@ -800,6 +806,12 @@ int PipelineHandlerBase::registerCamera(std::unique_ptr<RPi::CameraData> &camera if (!data->sensor_) return -EINVAL; + ret = data->loadPipelineConfiguration(); + if (ret) { + LOG(RPI, Error) << "Unable to load pipeline configuration"; + return ret; + } + /* Populate the map of sensor supported formats and sizes. */ for (auto const mbusCode : data->sensor_->mbusCodes()) data->sensorFormats_.emplace(mbusCode, @@ -859,12 +871,6 @@ int PipelineHandlerBase::registerCamera(std::unique_ptr<RPi::CameraData> &camera if (ret) return ret; - ret = data->loadPipelineConfiguration(); - if (ret) { - LOG(RPI, Error) << "Unable to load pipeline configuration"; - return ret; - } - /* Setup the general IPA signal handlers. */ data->frontendDevice()->dequeueTimeout.connect(data, &RPi::CameraData::cameraTimeout); data->frontendDevice()->frameStart.connect(data, &RPi::CameraData::frameStarted); @@ -1096,6 +1102,7 @@ int CameraData::loadPipelineConfiguration() { config_ = { .cameraTimeoutValue = 0, + .controllerMinFrameDurationUs = defaultControllerMinimumFrameDurationUs, }; /* Initial configuration of the platform, in case no config file is present */ @@ -1145,6 +1152,9 @@ int CameraData::loadPipelineConfiguration() frontendDevice()->setDequeueTimeout(config_.cameraTimeoutValue * 1ms); } + config_.controllerMinFrameDurationUs = + phConfig["controller_min_frame_duration_us"].get<double>(config_.controllerMinFrameDurationUs); + return platformPipelineConfigure(root); } @@ -1173,6 +1183,8 @@ int CameraData::loadIPA(ipa::RPi::InitResult *result) } params.lensPresent = !!sensor_->focusLens(); + params.controllerMinFrameDurationUs = config_.controllerMinFrameDurationUs; + ret = platformInitIpa(params); if (ret) return ret; diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index 6257a934..597eb587 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -169,6 +169,11 @@ public: * on frame durations. */ unsigned int cameraTimeoutValue; + /* + * The minimum frame duration between the IPA's calls to the + * algorithms themselves (in microseconds). + */ + float controllerMinFrameDurationUs; }; Config config_; diff --git a/src/libcamera/pipeline/rpi/pisp/data/example.yaml b/src/libcamera/pipeline/rpi/pisp/data/example.yaml index baf03be7..c5edbba0 100644 --- a/src/libcamera/pipeline/rpi/pisp/data/example.yaml +++ b/src/libcamera/pipeline/rpi/pisp/data/example.yaml @@ -36,5 +36,11 @@ # framebuffers required for its operation. # # "disable_hdr": false, + + # Limits the rate at which IPAs are called. The algorithms will + # be skipped until this many microseconds have elapsed since + # the last call. The default value represents a 30fps limit. + # + # "controller_min_frame_duration_us": 33333.333, } } diff --git a/src/libcamera/pipeline/rpi/vc4/data/example.yaml b/src/libcamera/pipeline/rpi/vc4/data/example.yaml index 27e54348..2ee2b864 100644 --- a/src/libcamera/pipeline/rpi/vc4/data/example.yaml +++ b/src/libcamera/pipeline/rpi/vc4/data/example.yaml @@ -37,5 +37,11 @@ # timeout value. # # "camera_timeout_value_ms": 0, + + # Limits the rate at which IPAs are called. The algorithms will + # be skipped until this many microseconds have elapsed since + # the last call. The default value represents a 30fps limit. + # + # "controller_min_frame_duration_us": 33333.333, } }
The controller min frame duration is used to rate limit how often we run IPAs. Historically this has been set to 33333us, meaning that the algorithms effectively skip frames when the camera is running faster than 30fps. This patch adds a small amount of plumbing that allows this value to be set in the Raspberry Pi configuration file. Some applications or platforms (such as Pi 5) are easily capable of running these more often, should there be a need to do so. Signed-off-by: David Plowman <david.plowman@raspberrypi.com> --- include/libcamera/ipa/raspberrypi.mojom | 1 + src/ipa/rpi/common/ipa_base.cpp | 10 +++++++- src/ipa/rpi/common/ipa_base.h | 2 ++ .../pipeline/rpi/common/pipeline_base.cpp | 24 ++++++++++++++----- .../pipeline/rpi/common/pipeline_base.h | 5 ++++ .../pipeline/rpi/pisp/data/example.yaml | 6 +++++ .../pipeline/rpi/vc4/data/example.yaml | 6 +++++ 7 files changed, 47 insertions(+), 7 deletions(-)