| Message ID | 20251204164918.83334-4-mzamazal@redhat.com |
|---|---|
| State | Accepted |
| Headers | show |
| Series |
|
| Related | show |
On 2025-12-04 22:19, Milan Zamazal wrote: > SimpleCameraConfiguration::validate() looks for the best configuration. > As part of enabling raw stream support, the method must consider raw > streams in addition to the processed streams. > > Raw streams are adjusted from the capture format and size. > > Configuration validation computes the maximum size of all the requested > streams and compares it to the output sizes. When e.g. only a raw > stream is requested then this may result in an invalid adjustment of its > size. This is because the output sizes are computed for processed > streams and may be smaller than capture sizes. If a raw stream with the > capture size is requested, it may then be wrongly adjusted to a larger > size because the output sizes, which are irrelevant for raw streams > anyway, are smaller than the requested capture size. The problem is > resolved by tracking raw and processed streams maximum sizes separately > and comparing raw stream sizes against capture rather than output sizes. > > Note that with both processed and raw streams, the requested sizes must > be mutually matching, including resizing due to debayer requirements. > For example, the following `cam' setup is valid for imx219 > > cam -s role=viewfinder,width=1920,height=1080 \ > -s role=raw,width=3280,height=2464 > > rather than > > cam -s role=viewfinder,width=1920,height=1080 \ > -s role=raw,width=1920,height=1080 > > due to the resolution of 1924x1080 actually selected for debayering to > 1920x1080. If the resolutions don't match mutually or don't match the > available sizes, validation adjusts them. > > Setting up the right configurations is still not enough to make the raw > streams working. Buffer handling must be changed in the simple > pipeline, which is addressed in followup patches. Not sure if I go in such detail for the commit message, it could be simply: ``` The maximum RAW stream size is tracked separate to ensure appropriate sensor capture size(or pipeConfig) selection. In case a processed stream is additionally requested, the validation can adjust the RAW stream size in order to satisfy the output of both stream sizes (including resizing due to debayer requirements). ``` > > Co-developed-by: Umang Jain <uajain@igalia.com> > Signed-off-by: Milan Zamazal <mzamazal@redhat.com> Not sure if my R-b is applicable here, but anyway: Reviewed-by: Umang Jain <uajain@igalia.com> > --- > src/libcamera/pipeline/simple/simple.cpp | 117 ++++++++++++++++------- > 1 file changed, 82 insertions(+), 35 deletions(-) > > diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp > index 6b83254fb..aaafe0571 100644 > --- a/src/libcamera/pipeline/simple/simple.cpp > +++ b/src/libcamera/pipeline/simple/simple.cpp > @@ -11,6 +11,7 @@ > #include <list> > #include <map> > #include <memory> > +#include <optional> > #include <queue> > #include <set> > #include <stdint.h> > @@ -27,6 +28,7 @@ > #include <libcamera/camera.h> > #include <libcamera/color_space.h> > #include <libcamera/control_ids.h> > +#include <libcamera/geometry.h> > #include <libcamera/pixel_format.h> > #include <libcamera/request.h> > #include <libcamera/stream.h> > @@ -1138,29 +1140,57 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() > status = Adjusted; > } > > - /* Find the largest stream size. */ > - Size maxStreamSize; > - for (const StreamConfiguration &cfg : config_) > - maxStreamSize.expandTo(cfg.size); > + /* Find the largest stream sizes. */ > + Size maxProcessedStreamSize; > + Size maxRawStreamSize; > + for (const StreamConfiguration &cfg : config_) { > + if (isRaw(cfg)) > + maxRawStreamSize.expandTo(cfg.size); > + else > + maxProcessedStreamSize.expandTo(cfg.size); > + } > > LOG(SimplePipeline, Debug) > - << "Largest stream size is " << maxStreamSize; > + << "Largest processed stream size is " << maxProcessedStreamSize; > + LOG(SimplePipeline, Debug) > + << "Largest raw stream size is " << maxRawStreamSize; > + > + /* Cap the number of raw stream configurations */ > + unsigned int rawCount = 0; > + PixelFormat requestedRawFormat; > + for (const StreamConfiguration &cfg : config_) { > + if (!isRaw(cfg)) > + continue; > + requestedRawFormat = cfg.pixelFormat; > + rawCount++; > + } > + > + if (rawCount > 1) { > + LOG(SimplePipeline, Error) > + << "Camera configuration with multiple raw streams not supported"; > + return Invalid; > + } > > /* > * Find the best configuration for the pipeline using a heuristic. > - * First select the pixel format based on the streams (which are > - * considered ordered from highest to lowest priority). Default to the > - * first pipeline configuration if no streams request a supported pixel > - * format. > + * First select the pixel format based on the raw streams followed by > + * non-raw streams (which are considered ordered from highest to lowest > + * priority). Default to the first pipeline configuration if no streams > + * request a supported pixel format. > */ > const std::vector<const SimpleCameraData::Configuration *> *configs = > &data_->formats_.begin()->second; > > - for (const StreamConfiguration &cfg : config_) { > - auto it = data_->formats_.find(cfg.pixelFormat); > - if (it != data_->formats_.end()) { > - configs = &it->second; > - break; > + auto rawIter = data_->formats_.find(requestedRawFormat); > + if (rawIter != data_->formats_.end()) { > + configs = &rawIter->second; > + } else { > + for (const StreamConfiguration &cfg : config_) { > + auto it = data_->formats_.find(cfg.pixelFormat); > + if (it != data_->formats_.end()) { > + configs = &it->second; > + break; > + } > } > } > > @@ -1182,8 +1212,10 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() > const Size &captureSize = pipeConfig->captureSize; > const Size &maxOutputSize = pipeConfig->outputSizes.max; > > - if (maxOutputSize.width >= maxStreamSize.width && > - maxOutputSize.height >= maxStreamSize.height) { > + if (maxOutputSize.width >= maxProcessedStreamSize.width && > + maxOutputSize.height >= maxProcessedStreamSize.height && > + captureSize.width >= maxRawStreamSize.width && > + captureSize.height >= maxRawStreamSize.height) { > if (!pipeConfig_ || captureSize < pipeConfig_->captureSize) > pipeConfig_ = pipeConfig; > } > @@ -1201,7 +1233,8 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() > << V4L2SubdeviceFormat{ pipeConfig_->code, pipeConfig_->sensorSize, {} } > << " -> " << pipeConfig_->captureSize > << "-" << pipeConfig_->captureFormat > - << " for max stream size " << maxStreamSize; > + << " for max processed stream size " << maxProcessedStreamSize > + << " and max raw stream size " << maxRawStreamSize; > > /* > * Adjust the requested streams. > @@ -1220,21 +1253,35 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() > > for (unsigned int i = 0; i < config_.size(); ++i) { > StreamConfiguration &cfg = config_[i]; > + const bool raw = isRaw(cfg); > > /* Adjust the pixel format and size. */ > - auto it = std::find(pipeConfig_->outputFormats.begin(), > - pipeConfig_->outputFormats.end(), > - cfg.pixelFormat); > - if (it == pipeConfig_->outputFormats.end()) > - it = pipeConfig_->outputFormats.begin(); > - > - PixelFormat pixelFormat = *it; > - if (cfg.pixelFormat != pixelFormat) { > - LOG(SimplePipeline, Debug) > - << "Adjusting pixel format from " > - << cfg.pixelFormat << " to " << pixelFormat; > - cfg.pixelFormat = pixelFormat; > - status = Adjusted; > + if (raw) { > + if (cfg.pixelFormat != pipeConfig_->captureFormat || > + cfg.size != pipeConfig_->captureSize) { > + cfg.pixelFormat = pipeConfig_->captureFormat; > + cfg.size = pipeConfig_->captureSize; > + > + LOG(SimplePipeline, Debug) > + << "Adjusting raw stream to " > + << cfg.toString(); > + status = Adjusted; > + } > + } else { > + auto it = std::find(pipeConfig_->outputFormats.begin(), > + pipeConfig_->outputFormats.end(), > + cfg.pixelFormat); > + if (it == pipeConfig_->outputFormats.end()) > + it = pipeConfig_->outputFormats.begin(); > + > + PixelFormat pixelFormat = *it; > + if (cfg.pixelFormat != pixelFormat) { > + LOG(SimplePipeline, Debug) > + << "Adjusting processed pixel format from " > + << cfg.pixelFormat << " to " << pixelFormat; > + cfg.pixelFormat = pixelFormat; > + status = Adjusted; > + } > } > > /* > @@ -1245,7 +1292,7 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() > * case, perform the standard pixel format based color space adjustment. > */ > if (!cfg.colorSpace) { > - const PixelFormatInfo &info = PixelFormatInfo::info(pixelFormat); > + const PixelFormatInfo &info = PixelFormatInfo::info(cfg.pixelFormat); > switch (info.colourEncoding) { > case PixelFormatInfo::ColourEncodingRGB: > cfg.colorSpace = ColorSpace::Srgb; > @@ -1265,9 +1312,9 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() > * adjusting a requested one, changes here shouldn't set the status > * to Adjusted. > */ > - cfg.colorSpace->adjust(pixelFormat); > + cfg.colorSpace->adjust(cfg.pixelFormat); > } else { > - if (cfg.colorSpace->adjust(pixelFormat)) { > + if (cfg.colorSpace->adjust(cfg.pixelFormat)) { > LOG(SimplePipeline, Debug) > << "Color space adjusted to " > << cfg.colorSpace.value().toString(); > @@ -1275,7 +1322,7 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() > } > } > > - if (!pipeConfig_->outputSizes.contains(cfg.size)) { > + if (!raw && !pipeConfig_->outputSizes.contains(cfg.size)) { > Size adjustedSize = pipeConfig_->captureSize; > /* > * The converter (when present) may not be able to output > @@ -1298,7 +1345,7 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() > needConversion_ = true; > > /* Set the stride and frameSize. */ > - if (needConversion_) { > + if (needConversion_ && !raw) { > std::tie(cfg.stride, cfg.frameSize) = > data_->converter_ > ? data_->converter_->strideAndFrameSize(cfg.pixelFormat,
Hi Umang, thank you for review. Umang Jain <uajain@igalia.com> writes: > On 2025-12-04 22:19, Milan Zamazal wrote: >> SimpleCameraConfiguration::validate() looks for the best configuration. >> As part of enabling raw stream support, the method must consider raw > >> streams in addition to the processed streams. >> >> Raw streams are adjusted from the capture format and size. >> >> Configuration validation computes the maximum size of all the requested >> streams and compares it to the output sizes. When e.g. only a raw >> stream is requested then this may result in an invalid adjustment of its >> size. This is because the output sizes are computed for processed >> streams and may be smaller than capture sizes. If a raw stream with the >> capture size is requested, it may then be wrongly adjusted to a larger >> size because the output sizes, which are irrelevant for raw streams >> anyway, are smaller than the requested capture size. The problem is >> resolved by tracking raw and processed streams maximum sizes separately >> and comparing raw stream sizes against capture rather than output sizes. >> >> Note that with both processed and raw streams, the requested sizes must >> be mutually matching, including resizing due to debayer requirements. >> For example, the following `cam' setup is valid for imx219 >> >> cam -s role=viewfinder,width=1920,height=1080 \ >> -s role=raw,width=3280,height=2464 >> >> rather than >> >> cam -s role=viewfinder,width=1920,height=1080 \ >> -s role=raw,width=1920,height=1080 >> >> due to the resolution of 1924x1080 actually selected for debayering to >> 1920x1080. If the resolutions don't match mutually or don't match the >> available sizes, validation adjusts them. >> >> Setting up the right configurations is still not enough to make the raw >> streams working. Buffer handling must be changed in the simple >> pipeline, which is addressed in followup patches. > > Not sure if I go in such detail for the commit message, it could be > simply: > > ``` > The maximum RAW stream size is tracked separate to ensure appropriate > sensor capture size(or pipeConfig) selection. In case a processed stream > is additionally requested, the validation can adjust the RAW stream size > in order to satisfy the output of both stream sizes (including resizing > due to debayer requirements). > ``` Thank you for the suggestion but I'm not sure the more concise description would be sufficiently clear to a casual code reader. I think it wouldn't be to me and I usually forget changes details much earlier than I think I will :-). Commit messages are often the only explanations available and there is no need to save space in them. >> >> Co-developed-by: Umang Jain <uajain@igalia.com> >> Signed-off-by: Milan Zamazal <mzamazal@redhat.com> > > Not sure if my R-b is applicable here, but anyway: > > Reviewed-by: Umang Jain <uajain@igalia.com> >> --- >> src/libcamera/pipeline/simple/simple.cpp | 117 ++++++++++++++++------- >> 1 file changed, 82 insertions(+), 35 deletions(-) >> >> diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp >> index 6b83254fb..aaafe0571 100644 >> --- a/src/libcamera/pipeline/simple/simple.cpp >> +++ b/src/libcamera/pipeline/simple/simple.cpp >> @@ -11,6 +11,7 @@ >> #include <list> >> #include <map> >> #include <memory> >> +#include <optional> >> #include <queue> >> #include <set> >> #include <stdint.h> >> @@ -27,6 +28,7 @@ >> #include <libcamera/camera.h> >> #include <libcamera/color_space.h> >> #include <libcamera/control_ids.h> >> +#include <libcamera/geometry.h> >> #include <libcamera/pixel_format.h> >> #include <libcamera/request.h> >> #include <libcamera/stream.h> >> @@ -1138,29 +1140,57 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() >> status = Adjusted; >> } >> >> - /* Find the largest stream size. */ >> - Size maxStreamSize; >> - for (const StreamConfiguration &cfg : config_) >> - maxStreamSize.expandTo(cfg.size); >> + /* Find the largest stream sizes. */ >> + Size maxProcessedStreamSize; >> + Size maxRawStreamSize; >> + for (const StreamConfiguration &cfg : config_) { >> + if (isRaw(cfg)) >> + maxRawStreamSize.expandTo(cfg.size); >> + else >> + maxProcessedStreamSize.expandTo(cfg.size); >> + } >> >> LOG(SimplePipeline, Debug) >> - << "Largest stream size is " << maxStreamSize; >> + << "Largest processed stream size is " << maxProcessedStreamSize; >> + LOG(SimplePipeline, Debug) >> + << "Largest raw stream size is " << maxRawStreamSize; >> + >> + /* Cap the number of raw stream configurations */ >> + unsigned int rawCount = 0; >> + PixelFormat requestedRawFormat; >> + for (const StreamConfiguration &cfg : config_) { >> + if (!isRaw(cfg)) >> + continue; >> + requestedRawFormat = cfg.pixelFormat; >> + rawCount++; >> + } >> + >> + if (rawCount > 1) { >> + LOG(SimplePipeline, Error) >> + << "Camera configuration with multiple raw streams not supported"; >> + return Invalid; >> + } >> >> /* >> * Find the best configuration for the pipeline using a heuristic. >> - * First select the pixel format based on the streams (which are >> - * considered ordered from highest to lowest priority). Default to the >> - * first pipeline configuration if no streams request a supported pixel >> - * format. >> + * First select the pixel format based on the raw streams followed by >> + * non-raw streams (which are considered ordered from highest to lowest >> + * priority). Default to the first pipeline configuration if no streams >> + * request a supported pixel format. >> */ >> const std::vector<const SimpleCameraData::Configuration *> *configs = >> &data_->formats_.begin()->second; >> >> - for (const StreamConfiguration &cfg : config_) { >> - auto it = data_->formats_.find(cfg.pixelFormat); >> - if (it != data_->formats_.end()) { >> - configs = &it->second; >> - break; >> + auto rawIter = data_->formats_.find(requestedRawFormat); >> + if (rawIter != data_->formats_.end()) { >> + configs = &rawIter->second; >> + } else { >> + for (const StreamConfiguration &cfg : config_) { >> + auto it = data_->formats_.find(cfg.pixelFormat); >> + if (it != data_->formats_.end()) { >> + configs = &it->second; >> + break; >> + } >> } >> } >> >> @@ -1182,8 +1212,10 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() >> const Size &captureSize = pipeConfig->captureSize; >> const Size &maxOutputSize = pipeConfig->outputSizes.max; >> >> - if (maxOutputSize.width >= maxStreamSize.width && >> - maxOutputSize.height >= maxStreamSize.height) { >> + if (maxOutputSize.width >= maxProcessedStreamSize.width && >> + maxOutputSize.height >= maxProcessedStreamSize.height && >> + captureSize.width >= maxRawStreamSize.width && >> + captureSize.height >= maxRawStreamSize.height) { >> if (!pipeConfig_ || captureSize < pipeConfig_->captureSize) >> pipeConfig_ = pipeConfig; >> } >> @@ -1201,7 +1233,8 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() >> << V4L2SubdeviceFormat{ pipeConfig_->code, pipeConfig_->sensorSize, {} } >> << " -> " << pipeConfig_->captureSize >> << "-" << pipeConfig_->captureFormat >> - << " for max stream size " << maxStreamSize; >> + << " for max processed stream size " << maxProcessedStreamSize >> + << " and max raw stream size " << maxRawStreamSize; >> >> /* >> * Adjust the requested streams. >> @@ -1220,21 +1253,35 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() >> >> for (unsigned int i = 0; i < config_.size(); ++i) { >> StreamConfiguration &cfg = config_[i]; >> + const bool raw = isRaw(cfg); >> >> /* Adjust the pixel format and size. */ >> - auto it = std::find(pipeConfig_->outputFormats.begin(), >> - pipeConfig_->outputFormats.end(), >> - cfg.pixelFormat); >> - if (it == pipeConfig_->outputFormats.end()) >> - it = pipeConfig_->outputFormats.begin(); >> - >> - PixelFormat pixelFormat = *it; >> - if (cfg.pixelFormat != pixelFormat) { >> - LOG(SimplePipeline, Debug) >> - << "Adjusting pixel format from " >> - << cfg.pixelFormat << " to " << pixelFormat; >> - cfg.pixelFormat = pixelFormat; >> - status = Adjusted; >> + if (raw) { >> + if (cfg.pixelFormat != pipeConfig_->captureFormat || >> + cfg.size != pipeConfig_->captureSize) { >> + cfg.pixelFormat = pipeConfig_->captureFormat; >> + cfg.size = pipeConfig_->captureSize; >> + >> + LOG(SimplePipeline, Debug) >> + << "Adjusting raw stream to " >> + << cfg.toString(); >> + status = Adjusted; >> + } >> + } else { >> + auto it = std::find(pipeConfig_->outputFormats.begin(), >> + pipeConfig_->outputFormats.end(), >> + cfg.pixelFormat); >> + if (it == pipeConfig_->outputFormats.end()) >> + it = pipeConfig_->outputFormats.begin(); >> + >> + PixelFormat pixelFormat = *it; >> + if (cfg.pixelFormat != pixelFormat) { >> + LOG(SimplePipeline, Debug) >> + << "Adjusting processed pixel format from " >> + << cfg.pixelFormat << " to " << pixelFormat; >> + cfg.pixelFormat = pixelFormat; >> + status = Adjusted; >> + } >> } >> >> /* >> @@ -1245,7 +1292,7 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() >> * case, perform the standard pixel format based color space adjustment. >> */ >> if (!cfg.colorSpace) { >> - const PixelFormatInfo &info = PixelFormatInfo::info(pixelFormat); >> + const PixelFormatInfo &info = PixelFormatInfo::info(cfg.pixelFormat); >> switch (info.colourEncoding) { >> case PixelFormatInfo::ColourEncodingRGB: >> cfg.colorSpace = ColorSpace::Srgb; >> @@ -1265,9 +1312,9 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() >> * adjusting a requested one, changes here shouldn't set the status >> * to Adjusted. >> */ >> - cfg.colorSpace->adjust(pixelFormat); >> + cfg.colorSpace->adjust(cfg.pixelFormat); >> } else { >> - if (cfg.colorSpace->adjust(pixelFormat)) { >> + if (cfg.colorSpace->adjust(cfg.pixelFormat)) { >> LOG(SimplePipeline, Debug) >> << "Color space adjusted to " >> << cfg.colorSpace.value().toString(); >> @@ -1275,7 +1322,7 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() >> } >> } >> >> - if (!pipeConfig_->outputSizes.contains(cfg.size)) { >> + if (!raw && !pipeConfig_->outputSizes.contains(cfg.size)) { >> Size adjustedSize = pipeConfig_->captureSize; >> /* >> * The converter (when present) may not be able to output >> @@ -1298,7 +1345,7 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() >> needConversion_ = true; >> >> /* Set the stride and frameSize. */ >> - if (needConversion_) { >> + if (needConversion_ && !raw) { >> std::tie(cfg.stride, cfg.frameSize) = >> data_->converter_ >> ? data_->converter_->strideAndFrameSize(cfg.pixelFormat,
On 2025-12-05 15:29, Milan Zamazal wrote: > Hi Umang, > > thank you for review. > > Umang Jain <uajain@igalia.com> writes: > >> On 2025-12-04 22:19, Milan Zamazal wrote: >>> SimpleCameraConfiguration::validate() looks for the best configuration. >>> As part of enabling raw stream support, the method must consider raw >> >>> streams in addition to the processed streams. >>> >>> Raw streams are adjusted from the capture format and size. >>> >>> Configuration validation computes the maximum size of all the requested >>> streams and compares it to the output sizes. When e.g. only a raw >>> stream is requested then this may result in an invalid adjustment of its >>> size. This is because the output sizes are computed for processed >>> streams and may be smaller than capture sizes. If a raw stream with the >>> capture size is requested, it may then be wrongly adjusted to a larger >>> size because the output sizes, which are irrelevant for raw streams >>> anyway, are smaller than the requested capture size. The problem is >>> resolved by tracking raw and processed streams maximum sizes separately >>> and comparing raw stream sizes against capture rather than output sizes. >>> >>> Note that with both processed and raw streams, the requested sizes must >>> be mutually matching, including resizing due to debayer requirements. >>> For example, the following `cam' setup is valid for imx219 >>> >>> cam -s role=viewfinder,width=1920,height=1080 \ >>> -s role=raw,width=3280,height=2464 >>> >>> rather than >>> >>> cam -s role=viewfinder,width=1920,height=1080 \ >>> -s role=raw,width=1920,height=1080 >>> >>> due to the resolution of 1924x1080 actually selected for debayering to >>> 1920x1080. If the resolutions don't match mutually or don't match the >>> available sizes, validation adjusts them. >>> >>> Setting up the right configurations is still not enough to make the raw >>> streams working. Buffer handling must be changed in the simple >>> pipeline, which is addressed in followup patches. >> >> Not sure if I go in such detail for the commit message, it could be >> simply: >> >> ``` >> The maximum RAW stream size is tracked separate to ensure appropriate >> sensor capture size(or pipeConfig) selection. In case a processed stream >> is additionally requested, the validation can adjust the RAW stream size >> in order to satisfy the output of both stream sizes (including resizing >> due to debayer requirements). >> ``` > > Thank you for the suggestion but I'm not sure the more concise > description would be sufficiently clear to a casual code reader. I > think it wouldn't be to me and I usually forget changes details much > earlier than I think I will :-). Commit messages are often the only > explanations available and there is no need to save space in them. > Not against detailed commit message but in this case, the material looks like it is geared with interaction of Robert's output sizes comparison patch. If this series would have merged and there was a patch on top (like in v16) the commit message interaction detail made full sense. I am OK keeping you as you like, was just giving PoV. >>> >>> Co-developed-by: Umang Jain <uajain@igalia.com> >>> Signed-off-by: Milan Zamazal <mzamazal@redhat.com> >> >> Not sure if my R-b is applicable here, but anyway: >> >> Reviewed-by: Umang Jain <uajain@igalia.com> >>> --- >>> src/libcamera/pipeline/simple/simple.cpp | 117 ++++++++++++++++------- >>> 1 file changed, 82 insertions(+), 35 deletions(-) >>> >>> diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp >>> index 6b83254fb..aaafe0571 100644 >>> --- a/src/libcamera/pipeline/simple/simple.cpp >>> +++ b/src/libcamera/pipeline/simple/simple.cpp >>> @@ -11,6 +11,7 @@ >>> #include <list> >>> #include <map> >>> #include <memory> >>> +#include <optional> >>> #include <queue> >>> #include <set> >>> #include <stdint.h> >>> @@ -27,6 +28,7 @@ >>> #include <libcamera/camera.h> >>> #include <libcamera/color_space.h> >>> #include <libcamera/control_ids.h> >>> +#include <libcamera/geometry.h> >>> #include <libcamera/pixel_format.h> >>> #include <libcamera/request.h> >>> #include <libcamera/stream.h> >>> @@ -1138,29 +1140,57 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() >>> status = Adjusted; >>> } >>> >>> - /* Find the largest stream size. */ >>> - Size maxStreamSize; >>> - for (const StreamConfiguration &cfg : config_) >>> - maxStreamSize.expandTo(cfg.size); >>> + /* Find the largest stream sizes. */ >>> + Size maxProcessedStreamSize; >>> + Size maxRawStreamSize; >>> + for (const StreamConfiguration &cfg : config_) { >>> + if (isRaw(cfg)) >>> + maxRawStreamSize.expandTo(cfg.size); >>> + else >>> + maxProcessedStreamSize.expandTo(cfg.size); >>> + } >>> >>> LOG(SimplePipeline, Debug) >>> - << "Largest stream size is " << maxStreamSize; >>> + << "Largest processed stream size is " << maxProcessedStreamSize; >>> + LOG(SimplePipeline, Debug) >>> + << "Largest raw stream size is " << maxRawStreamSize; >>> + >>> + /* Cap the number of raw stream configurations */ >>> + unsigned int rawCount = 0; >>> + PixelFormat requestedRawFormat; >>> + for (const StreamConfiguration &cfg : config_) { >>> + if (!isRaw(cfg)) >>> + continue; >>> + requestedRawFormat = cfg.pixelFormat; >>> + rawCount++; >>> + } >>> + >>> + if (rawCount > 1) { >>> + LOG(SimplePipeline, Error) >>> + << "Camera configuration with multiple raw streams not supported"; >>> + return Invalid; >>> + } >>> >>> /* >>> * Find the best configuration for the pipeline using a heuristic. >>> - * First select the pixel format based on the streams (which are >>> - * considered ordered from highest to lowest priority). Default to the >>> - * first pipeline configuration if no streams request a supported pixel >>> - * format. >>> + * First select the pixel format based on the raw streams followed by >>> + * non-raw streams (which are considered ordered from highest to lowest >>> + * priority). Default to the first pipeline configuration if no streams >>> + * request a supported pixel format. >>> */ >>> const std::vector<const SimpleCameraData::Configuration *> *configs = >>> &data_->formats_.begin()->second; >>> >>> - for (const StreamConfiguration &cfg : config_) { >>> - auto it = data_->formats_.find(cfg.pixelFormat); >>> - if (it != data_->formats_.end()) { >>> - configs = &it->second; >>> - break; >>> + auto rawIter = data_->formats_.find(requestedRawFormat); >>> + if (rawIter != data_->formats_.end()) { >>> + configs = &rawIter->second; >>> + } else { >>> + for (const StreamConfiguration &cfg : config_) { >>> + auto it = data_->formats_.find(cfg.pixelFormat); >>> + if (it != data_->formats_.end()) { >>> + configs = &it->second; >>> + break; >>> + } >>> } >>> } >>> >>> @@ -1182,8 +1212,10 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() >>> const Size &captureSize = pipeConfig->captureSize; >>> const Size &maxOutputSize = pipeConfig->outputSizes.max; >>> >>> - if (maxOutputSize.width >= maxStreamSize.width && >>> - maxOutputSize.height >= maxStreamSize.height) { >>> + if (maxOutputSize.width >= maxProcessedStreamSize.width && >>> + maxOutputSize.height >= maxProcessedStreamSize.height && >>> + captureSize.width >= maxRawStreamSize.width && >>> + captureSize.height >= maxRawStreamSize.height) { >>> if (!pipeConfig_ || captureSize < pipeConfig_->captureSize) >>> pipeConfig_ = pipeConfig; >>> } >>> @@ -1201,7 +1233,8 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() >>> << V4L2SubdeviceFormat{ pipeConfig_->code, pipeConfig_->sensorSize, {} } >>> << " -> " << pipeConfig_->captureSize >>> << "-" << pipeConfig_->captureFormat >>> - << " for max stream size " << maxStreamSize; >>> + << " for max processed stream size " << maxProcessedStreamSize >>> + << " and max raw stream size " << maxRawStreamSize; >>> >>> /* >>> * Adjust the requested streams. >>> @@ -1220,21 +1253,35 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() >>> >>> for (unsigned int i = 0; i < config_.size(); ++i) { >>> StreamConfiguration &cfg = config_[i]; >>> + const bool raw = isRaw(cfg); >>> >>> /* Adjust the pixel format and size. */ >>> - auto it = std::find(pipeConfig_->outputFormats.begin(), >>> - pipeConfig_->outputFormats.end(), >>> - cfg.pixelFormat); >>> - if (it == pipeConfig_->outputFormats.end()) >>> - it = pipeConfig_->outputFormats.begin(); >>> - >>> - PixelFormat pixelFormat = *it; >>> - if (cfg.pixelFormat != pixelFormat) { >>> - LOG(SimplePipeline, Debug) >>> - << "Adjusting pixel format from " >>> - << cfg.pixelFormat << " to " << pixelFormat; >>> - cfg.pixelFormat = pixelFormat; >>> - status = Adjusted; >>> + if (raw) { >>> + if (cfg.pixelFormat != pipeConfig_->captureFormat || >>> + cfg.size != pipeConfig_->captureSize) { >>> + cfg.pixelFormat = pipeConfig_->captureFormat; >>> + cfg.size = pipeConfig_->captureSize; >>> + >>> + LOG(SimplePipeline, Debug) >>> + << "Adjusting raw stream to " >>> + << cfg.toString(); >>> + status = Adjusted; >>> + } >>> + } else { >>> + auto it = std::find(pipeConfig_->outputFormats.begin(), >>> + pipeConfig_->outputFormats.end(), >>> + cfg.pixelFormat); >>> + if (it == pipeConfig_->outputFormats.end()) >>> + it = pipeConfig_->outputFormats.begin(); >>> + >>> + PixelFormat pixelFormat = *it; >>> + if (cfg.pixelFormat != pixelFormat) { >>> + LOG(SimplePipeline, Debug) >>> + << "Adjusting processed pixel format from " >>> + << cfg.pixelFormat << " to " << pixelFormat; >>> + cfg.pixelFormat = pixelFormat; >>> + status = Adjusted; >>> + } >>> } >>> >>> /* >>> @@ -1245,7 +1292,7 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() >>> * case, perform the standard pixel format based color space adjustment. >>> */ >>> if (!cfg.colorSpace) { >>> - const PixelFormatInfo &info = PixelFormatInfo::info(pixelFormat); >>> + const PixelFormatInfo &info = PixelFormatInfo::info(cfg.pixelFormat); >>> switch (info.colourEncoding) { >>> case PixelFormatInfo::ColourEncodingRGB: >>> cfg.colorSpace = ColorSpace::Srgb; >>> @@ -1265,9 +1312,9 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() >>> * adjusting a requested one, changes here shouldn't set the status >>> * to Adjusted. >>> */ >>> - cfg.colorSpace->adjust(pixelFormat); >>> + cfg.colorSpace->adjust(cfg.pixelFormat); >>> } else { >>> - if (cfg.colorSpace->adjust(pixelFormat)) { >>> + if (cfg.colorSpace->adjust(cfg.pixelFormat)) { >>> LOG(SimplePipeline, Debug) >>> << "Color space adjusted to " >>> << cfg.colorSpace.value().toString(); >>> @@ -1275,7 +1322,7 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() >>> } >>> } >>> >>> - if (!pipeConfig_->outputSizes.contains(cfg.size)) { >>> + if (!raw && !pipeConfig_->outputSizes.contains(cfg.size)) { >>> Size adjustedSize = pipeConfig_->captureSize; >>> /* >>> * The converter (when present) may not be able to output >>> @@ -1298,7 +1345,7 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() >>> needConversion_ = true; >>> >>> /* Set the stride and frameSize. */ >>> - if (needConversion_) { >>> + if (needConversion_ && !raw) { >>> std::tie(cfg.stride, cfg.frameSize) = >>> data_->converter_ >>> ? data_->converter_->strideAndFrameSize(cfg.pixelFormat,
On 2025-12-04 22:19, Milan Zamazal wrote: > SimpleCameraConfiguration::validate() looks for the best configuration. > As part of enabling raw stream support, the method must consider raw > streams in addition to the processed streams. > > Raw streams are adjusted from the capture format and size. > > Configuration validation computes the maximum size of all the requested > streams and compares it to the output sizes. When e.g. only a raw > stream is requested then this may result in an invalid adjustment of its > size. This is because the output sizes are computed for processed > streams and may be smaller than capture sizes. If a raw stream with the > capture size is requested, it may then be wrongly adjusted to a larger > size because the output sizes, which are irrelevant for raw streams > anyway, are smaller than the requested capture size. The problem is > resolved by tracking raw and processed streams maximum sizes separately > and comparing raw stream sizes against capture rather than output sizes. > > Note that with both processed and raw streams, the requested sizes must > be mutually matching, including resizing due to debayer requirements. > For example, the following `cam' setup is valid for imx219 > > cam -s role=viewfinder,width=1920,height=1080 \ > -s role=raw,width=3280,height=2464 > > rather than > > cam -s role=viewfinder,width=1920,height=1080 \ > -s role=raw,width=1920,height=1080 > > due to the resolution of 1924x1080 actually selected for debayering to > 1920x1080. If the resolutions don't match mutually or don't match the > available sizes, validation adjusts them. > > Setting up the right configurations is still not enough to make the raw > streams working. Buffer handling must be changed in the simple > pipeline, which is addressed in followup patches. > > Co-developed-by: Umang Jain <uajain@igalia.com> > Signed-off-by: Milan Zamazal <mzamazal@redhat.com> Signed-off-by: Umang Jain <uajain@igalia.com> > --- > src/libcamera/pipeline/simple/simple.cpp | 117 ++++++++++++++++------- > 1 file changed, 82 insertions(+), 35 deletions(-) > > diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp > index 6b83254fb..aaafe0571 100644 > --- a/src/libcamera/pipeline/simple/simple.cpp > +++ b/src/libcamera/pipeline/simple/simple.cpp > @@ -11,6 +11,7 @@ > #include <list> > #include <map> > #include <memory> > +#include <optional> > #include <queue> > #include <set> > #include <stdint.h> > @@ -27,6 +28,7 @@ > #include <libcamera/camera.h> > #include <libcamera/color_space.h> > #include <libcamera/control_ids.h> > +#include <libcamera/geometry.h> > #include <libcamera/pixel_format.h> > #include <libcamera/request.h> > #include <libcamera/stream.h> > @@ -1138,29 +1140,57 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() > status = Adjusted; > } > > - /* Find the largest stream size. */ > - Size maxStreamSize; > - for (const StreamConfiguration &cfg : config_) > - maxStreamSize.expandTo(cfg.size); > + /* Find the largest stream sizes. */ > + Size maxProcessedStreamSize; > + Size maxRawStreamSize; > + for (const StreamConfiguration &cfg : config_) { > + if (isRaw(cfg)) > + maxRawStreamSize.expandTo(cfg.size); > + else > + maxProcessedStreamSize.expandTo(cfg.size); > + } > > LOG(SimplePipeline, Debug) > - << "Largest stream size is " << maxStreamSize; > + << "Largest processed stream size is " << maxProcessedStreamSize; > + LOG(SimplePipeline, Debug) > + << "Largest raw stream size is " << maxRawStreamSize; > + > + /* Cap the number of raw stream configurations */ > + unsigned int rawCount = 0; > + PixelFormat requestedRawFormat; > + for (const StreamConfiguration &cfg : config_) { > + if (!isRaw(cfg)) > + continue; > + requestedRawFormat = cfg.pixelFormat; > + rawCount++; > + } > + > + if (rawCount > 1) { > + LOG(SimplePipeline, Error) > + << "Camera configuration with multiple raw streams not supported"; > + return Invalid; > + } > > /* > * Find the best configuration for the pipeline using a heuristic. > - * First select the pixel format based on the streams (which are > - * considered ordered from highest to lowest priority). Default to the > - * first pipeline configuration if no streams request a supported pixel > - * format. > + * First select the pixel format based on the raw streams followed by > + * non-raw streams (which are considered ordered from highest to lowest > + * priority). Default to the first pipeline configuration if no streams > + * request a supported pixel format. > */ > const std::vector<const SimpleCameraData::Configuration *> *configs = > &data_->formats_.begin()->second; > > - for (const StreamConfiguration &cfg : config_) { > - auto it = data_->formats_.find(cfg.pixelFormat); > - if (it != data_->formats_.end()) { > - configs = &it->second; > - break; > + auto rawIter = data_->formats_.find(requestedRawFormat); > + if (rawIter != data_->formats_.end()) { > + configs = &rawIter->second; > + } else { > + for (const StreamConfiguration &cfg : config_) { > + auto it = data_->formats_.find(cfg.pixelFormat); > + if (it != data_->formats_.end()) { > + configs = &it->second; > + break; > + } > } > } > > @@ -1182,8 +1212,10 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() > const Size &captureSize = pipeConfig->captureSize; > const Size &maxOutputSize = pipeConfig->outputSizes.max; > > - if (maxOutputSize.width >= maxStreamSize.width && > - maxOutputSize.height >= maxStreamSize.height) { > + if (maxOutputSize.width >= maxProcessedStreamSize.width && > + maxOutputSize.height >= maxProcessedStreamSize.height && > + captureSize.width >= maxRawStreamSize.width && > + captureSize.height >= maxRawStreamSize.height) { > if (!pipeConfig_ || captureSize < pipeConfig_->captureSize) > pipeConfig_ = pipeConfig; > } > @@ -1201,7 +1233,8 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() > << V4L2SubdeviceFormat{ pipeConfig_->code, pipeConfig_->sensorSize, {} } > << " -> " << pipeConfig_->captureSize > << "-" << pipeConfig_->captureFormat > - << " for max stream size " << maxStreamSize; > + << " for max processed stream size " << maxProcessedStreamSize > + << " and max raw stream size " << maxRawStreamSize; > > /* > * Adjust the requested streams. > @@ -1220,21 +1253,35 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() > > for (unsigned int i = 0; i < config_.size(); ++i) { > StreamConfiguration &cfg = config_[i]; > + const bool raw = isRaw(cfg); > > /* Adjust the pixel format and size. */ > - auto it = std::find(pipeConfig_->outputFormats.begin(), > - pipeConfig_->outputFormats.end(), > - cfg.pixelFormat); > - if (it == pipeConfig_->outputFormats.end()) > - it = pipeConfig_->outputFormats.begin(); > - > - PixelFormat pixelFormat = *it; > - if (cfg.pixelFormat != pixelFormat) { > - LOG(SimplePipeline, Debug) > - << "Adjusting pixel format from " > - << cfg.pixelFormat << " to " << pixelFormat; > - cfg.pixelFormat = pixelFormat; > - status = Adjusted; > + if (raw) { > + if (cfg.pixelFormat != pipeConfig_->captureFormat || > + cfg.size != pipeConfig_->captureSize) { > + cfg.pixelFormat = pipeConfig_->captureFormat; > + cfg.size = pipeConfig_->captureSize; > + > + LOG(SimplePipeline, Debug) > + << "Adjusting raw stream to " > + << cfg.toString(); > + status = Adjusted; > + } > + } else { > + auto it = std::find(pipeConfig_->outputFormats.begin(), > + pipeConfig_->outputFormats.end(), > + cfg.pixelFormat); > + if (it == pipeConfig_->outputFormats.end()) > + it = pipeConfig_->outputFormats.begin(); > + > + PixelFormat pixelFormat = *it; > + if (cfg.pixelFormat != pixelFormat) { > + LOG(SimplePipeline, Debug) > + << "Adjusting processed pixel format from " > + << cfg.pixelFormat << " to " << pixelFormat; > + cfg.pixelFormat = pixelFormat; > + status = Adjusted; > + } > } > > /* > @@ -1245,7 +1292,7 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() > * case, perform the standard pixel format based color space adjustment. > */ > if (!cfg.colorSpace) { > - const PixelFormatInfo &info = PixelFormatInfo::info(pixelFormat); > + const PixelFormatInfo &info = PixelFormatInfo::info(cfg.pixelFormat); > switch (info.colourEncoding) { > case PixelFormatInfo::ColourEncodingRGB: > cfg.colorSpace = ColorSpace::Srgb; > @@ -1265,9 +1312,9 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() > * adjusting a requested one, changes here shouldn't set the status > * to Adjusted. > */ > - cfg.colorSpace->adjust(pixelFormat); > + cfg.colorSpace->adjust(cfg.pixelFormat); > } else { > - if (cfg.colorSpace->adjust(pixelFormat)) { > + if (cfg.colorSpace->adjust(cfg.pixelFormat)) { > LOG(SimplePipeline, Debug) > << "Color space adjusted to " > << cfg.colorSpace.value().toString(); > @@ -1275,7 +1322,7 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() > } > } > > - if (!pipeConfig_->outputSizes.contains(cfg.size)) { > + if (!raw && !pipeConfig_->outputSizes.contains(cfg.size)) { > Size adjustedSize = pipeConfig_->captureSize; > /* > * The converter (when present) may not be able to output > @@ -1298,7 +1345,7 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() > needConversion_ = true; > > /* Set the stride and frameSize. */ > - if (needConversion_) { > + if (needConversion_ && !raw) { > std::tie(cfg.stride, cfg.frameSize) = > data_->converter_ > ? data_->converter_->strideAndFrameSize(cfg.pixelFormat,
diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 6b83254fb..aaafe0571 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -11,6 +11,7 @@ #include <list> #include <map> #include <memory> +#include <optional> #include <queue> #include <set> #include <stdint.h> @@ -27,6 +28,7 @@ #include <libcamera/camera.h> #include <libcamera/color_space.h> #include <libcamera/control_ids.h> +#include <libcamera/geometry.h> #include <libcamera/pixel_format.h> #include <libcamera/request.h> #include <libcamera/stream.h> @@ -1138,29 +1140,57 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() status = Adjusted; } - /* Find the largest stream size. */ - Size maxStreamSize; - for (const StreamConfiguration &cfg : config_) - maxStreamSize.expandTo(cfg.size); + /* Find the largest stream sizes. */ + Size maxProcessedStreamSize; + Size maxRawStreamSize; + for (const StreamConfiguration &cfg : config_) { + if (isRaw(cfg)) + maxRawStreamSize.expandTo(cfg.size); + else + maxProcessedStreamSize.expandTo(cfg.size); + } LOG(SimplePipeline, Debug) - << "Largest stream size is " << maxStreamSize; + << "Largest processed stream size is " << maxProcessedStreamSize; + LOG(SimplePipeline, Debug) + << "Largest raw stream size is " << maxRawStreamSize; + + /* Cap the number of raw stream configurations */ + unsigned int rawCount = 0; + PixelFormat requestedRawFormat; + for (const StreamConfiguration &cfg : config_) { + if (!isRaw(cfg)) + continue; + requestedRawFormat = cfg.pixelFormat; + rawCount++; + } + + if (rawCount > 1) { + LOG(SimplePipeline, Error) + << "Camera configuration with multiple raw streams not supported"; + return Invalid; + } /* * Find the best configuration for the pipeline using a heuristic. - * First select the pixel format based on the streams (which are - * considered ordered from highest to lowest priority). Default to the - * first pipeline configuration if no streams request a supported pixel - * format. + * First select the pixel format based on the raw streams followed by + * non-raw streams (which are considered ordered from highest to lowest + * priority). Default to the first pipeline configuration if no streams + * request a supported pixel format. */ const std::vector<const SimpleCameraData::Configuration *> *configs = &data_->formats_.begin()->second; - for (const StreamConfiguration &cfg : config_) { - auto it = data_->formats_.find(cfg.pixelFormat); - if (it != data_->formats_.end()) { - configs = &it->second; - break; + auto rawIter = data_->formats_.find(requestedRawFormat); + if (rawIter != data_->formats_.end()) { + configs = &rawIter->second; + } else { + for (const StreamConfiguration &cfg : config_) { + auto it = data_->formats_.find(cfg.pixelFormat); + if (it != data_->formats_.end()) { + configs = &it->second; + break; + } } } @@ -1182,8 +1212,10 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() const Size &captureSize = pipeConfig->captureSize; const Size &maxOutputSize = pipeConfig->outputSizes.max; - if (maxOutputSize.width >= maxStreamSize.width && - maxOutputSize.height >= maxStreamSize.height) { + if (maxOutputSize.width >= maxProcessedStreamSize.width && + maxOutputSize.height >= maxProcessedStreamSize.height && + captureSize.width >= maxRawStreamSize.width && + captureSize.height >= maxRawStreamSize.height) { if (!pipeConfig_ || captureSize < pipeConfig_->captureSize) pipeConfig_ = pipeConfig; } @@ -1201,7 +1233,8 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() << V4L2SubdeviceFormat{ pipeConfig_->code, pipeConfig_->sensorSize, {} } << " -> " << pipeConfig_->captureSize << "-" << pipeConfig_->captureFormat - << " for max stream size " << maxStreamSize; + << " for max processed stream size " << maxProcessedStreamSize + << " and max raw stream size " << maxRawStreamSize; /* * Adjust the requested streams. @@ -1220,21 +1253,35 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() for (unsigned int i = 0; i < config_.size(); ++i) { StreamConfiguration &cfg = config_[i]; + const bool raw = isRaw(cfg); /* Adjust the pixel format and size. */ - auto it = std::find(pipeConfig_->outputFormats.begin(), - pipeConfig_->outputFormats.end(), - cfg.pixelFormat); - if (it == pipeConfig_->outputFormats.end()) - it = pipeConfig_->outputFormats.begin(); - - PixelFormat pixelFormat = *it; - if (cfg.pixelFormat != pixelFormat) { - LOG(SimplePipeline, Debug) - << "Adjusting pixel format from " - << cfg.pixelFormat << " to " << pixelFormat; - cfg.pixelFormat = pixelFormat; - status = Adjusted; + if (raw) { + if (cfg.pixelFormat != pipeConfig_->captureFormat || + cfg.size != pipeConfig_->captureSize) { + cfg.pixelFormat = pipeConfig_->captureFormat; + cfg.size = pipeConfig_->captureSize; + + LOG(SimplePipeline, Debug) + << "Adjusting raw stream to " + << cfg.toString(); + status = Adjusted; + } + } else { + auto it = std::find(pipeConfig_->outputFormats.begin(), + pipeConfig_->outputFormats.end(), + cfg.pixelFormat); + if (it == pipeConfig_->outputFormats.end()) + it = pipeConfig_->outputFormats.begin(); + + PixelFormat pixelFormat = *it; + if (cfg.pixelFormat != pixelFormat) { + LOG(SimplePipeline, Debug) + << "Adjusting processed pixel format from " + << cfg.pixelFormat << " to " << pixelFormat; + cfg.pixelFormat = pixelFormat; + status = Adjusted; + } } /* @@ -1245,7 +1292,7 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() * case, perform the standard pixel format based color space adjustment. */ if (!cfg.colorSpace) { - const PixelFormatInfo &info = PixelFormatInfo::info(pixelFormat); + const PixelFormatInfo &info = PixelFormatInfo::info(cfg.pixelFormat); switch (info.colourEncoding) { case PixelFormatInfo::ColourEncodingRGB: cfg.colorSpace = ColorSpace::Srgb; @@ -1265,9 +1312,9 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() * adjusting a requested one, changes here shouldn't set the status * to Adjusted. */ - cfg.colorSpace->adjust(pixelFormat); + cfg.colorSpace->adjust(cfg.pixelFormat); } else { - if (cfg.colorSpace->adjust(pixelFormat)) { + if (cfg.colorSpace->adjust(cfg.pixelFormat)) { LOG(SimplePipeline, Debug) << "Color space adjusted to " << cfg.colorSpace.value().toString(); @@ -1275,7 +1322,7 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() } } - if (!pipeConfig_->outputSizes.contains(cfg.size)) { + if (!raw && !pipeConfig_->outputSizes.contains(cfg.size)) { Size adjustedSize = pipeConfig_->captureSize; /* * The converter (when present) may not be able to output @@ -1298,7 +1345,7 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() needConversion_ = true; /* Set the stride and frameSize. */ - if (needConversion_) { + if (needConversion_ && !raw) { std::tie(cfg.stride, cfg.frameSize) = data_->converter_ ? data_->converter_->strideAndFrameSize(cfg.pixelFormat,
SimpleCameraConfiguration::validate() looks for the best configuration. As part of enabling raw stream support, the method must consider raw streams in addition to the processed streams. Raw streams are adjusted from the capture format and size. Configuration validation computes the maximum size of all the requested streams and compares it to the output sizes. When e.g. only a raw stream is requested then this may result in an invalid adjustment of its size. This is because the output sizes are computed for processed streams and may be smaller than capture sizes. If a raw stream with the capture size is requested, it may then be wrongly adjusted to a larger size because the output sizes, which are irrelevant for raw streams anyway, are smaller than the requested capture size. The problem is resolved by tracking raw and processed streams maximum sizes separately and comparing raw stream sizes against capture rather than output sizes. Note that with both processed and raw streams, the requested sizes must be mutually matching, including resizing due to debayer requirements. For example, the following `cam' setup is valid for imx219 cam -s role=viewfinder,width=1920,height=1080 \ -s role=raw,width=3280,height=2464 rather than cam -s role=viewfinder,width=1920,height=1080 \ -s role=raw,width=1920,height=1080 due to the resolution of 1924x1080 actually selected for debayering to 1920x1080. If the resolutions don't match mutually or don't match the available sizes, validation adjusts them. Setting up the right configurations is still not enough to make the raw streams working. Buffer handling must be changed in the simple pipeline, which is addressed in followup patches. Co-developed-by: Umang Jain <uajain@igalia.com> Signed-off-by: Milan Zamazal <mzamazal@redhat.com> --- src/libcamera/pipeline/simple/simple.cpp | 117 ++++++++++++++++------- 1 file changed, 82 insertions(+), 35 deletions(-)