| Message ID | 20250305192617.18361-6-mzamazal@redhat.com |
|---|---|
| State | New |
| Headers | show |
| Series |
|
| Related | show |
Quoting Milan Zamazal (2025-03-05 19:26:12) > When generating simple pipeline configuration, raw output configurations > are generated but later filtered out. Let's include processed and/or > raw output configurations depending on the requested stream roles. > > Raw and processed formats are handled separately. The intention is that > in case both raw and processed formats are requested then the raw > formats should be left intact to the extent possible and not be > influenced by the processed formats (implying that raw parameters > compatible with the processed output requirements must be requested by > the application). > > Raw output configurations are marked by setting their colorSpace to > ColorSpace::Raw, which is the only place there suitable for the purpose. > > This is another preparatory patch without making raw outputs working. I've identified two interesting effects which I've bisected to this patch: First: Without this patch on x13s: [libcamera] kbingham@charm:~/iob/libcamera/build/gcc$ cam -c1 --metadata -C10 [1:44:39.464209549] [37696] INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/kbingham/iob/libcamera/build/gcc/src/ipa' to the IPA search path [1:44:39.471858239] [37696] INFO Camera camera_manager.cpp:327 libcamera v0.4.0+149-628e8c80 [1:44:39.497573824] [37697] WARN CameraSensor camera_sensor_legacy.cpp:616 'ov5675 24-0010': Rotation control not available, default to 0 degrees [1:44:39.502155101] [37697] INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data' [1:44:39.502180672] [37697] WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'ov5675.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml' [1:44:39.502468206] [37697] INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data' [1:44:39.512547578] [37697] INFO Pipeline pipeline_handler.cpp:618 libcamera is not installed. Loading platform configuration file from '/home/kbingham/iob/libcamera/src/libcamera/pipeline/virtual/data/virtual.yaml' Using camera /base/soc@0/cci@ac4c000/i2c-bus@1/camera@10 as cam0 [1:44:39.514127115] [37696] INFO Camera camera.cpp:1203 configuring streams: (0) 2584x1944-ABGR8888 [1:44:39.520536247] [37697] WARN CameraSensor camera_sensor_legacy.cpp:501 'ov5675 24-0010': No sensor delays found in static properties. Assuming unverified defaults. [1:44:39.520899922] [37697] INFO IPASoft soft_simple.cpp:251 IPASoft: Exposure 4-2016, gain 1-15.9922 (0.149922) cam0: Capture 10 frames 6279.664529 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 20093184 SensorTimestamp = 6279664529000 6279.697607 (30.23 fps) cam0-stream0 seq: 000001 bytesused: 20093184 SensorTimestamp = 6279697607000 6279.731336 (29.65 fps) cam0-stream0 seq: 000002 bytesused: 20093184 SensorTimestamp = 6279731336000 6279.831491 (9.98 fps) cam0-stream0 seq: 000003 bytesused: 20093184 SensorTimestamp = 6279831491000 6279.898368 (14.95 fps) cam0-stream0 seq: 000004 bytesused: 20093184 SensorTimestamp = 6279898368000 6279.965133 (14.98 fps) cam0-stream0 seq: 000005 bytesused: 20093184 SensorTimestamp = 6279965133000 6280.031911 (14.97 fps) cam0-stream0 seq: 000006 bytesused: 20093184 SensorTimestamp = 6280031911000 6280.098577 (15.00 fps) cam0-stream0 seq: 000007 bytesused: 20093184 SensorTimestamp = 6280098577000 6280.165320 (14.98 fps) cam0-stream0 seq: 000008 bytesused: 20093184 SensorTimestamp = 6280165320000 6280.232225 (14.95 fps) cam0-stream0 seq: 000009 bytesused: 20093184 SensorTimestamp = 6280232225000 (Note the 15 FPS), and pipewire/gstreamer/cheese are all 'streaming'... But with this patch the rate jumps to a more reasonable 30 FPS seen by qcam and cam: [libcamera] kbingham@charm:~/iob/libcamera/build/gcc$ cam -c1 --metadata -C10 [1:46:59.040578435] [38346] INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/kbingham/iob/libcamera/build/gcc/src/ipa' to the IPA search path [1:46:59.048420757] [38346] INFO Camera camera_manager.cpp:327 libcamera v0.4.0+150-d0e1caa0 [1:46:59.074259713] [38347] WARN CameraSensor camera_sensor_legacy.cpp:616 'ov5675 24-0010': Rotation control not available, default to 0 degrees [1:46:59.078796773] [38347] INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data' [1:46:59.078822605] [38347] WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'ov5675.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml' [1:46:59.079110243] [38347] INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data' [1:46:59.087987917] [38347] INFO Pipeline pipeline_handler.cpp:618 libcamera is not installed. Loading platform configuration file from '/home/kbingham/iob/libcamera/src/libcamera/pipeline/virtual/data/virtual.yaml' Using camera /base/soc@0/cci@ac4c000/i2c-bus@1/camera@10 as cam0 [1:46:59.089522196] [38346] INFO Camera camera.cpp:1203 configuring streams: (0) 1288x972-ABGR8888 [1:46:59.096422962] [38347] WARN CameraSensor camera_sensor_legacy.cpp:501 'ov5675 24-0010': No sensor delays found in static properties. Assuming unverified defaults. [1:46:59.096786533] [38347] INFO IPASoft soft_simple.cpp:251 IPASoft: Exposure 4-2016, gain 1-15.9922 (0.149922) cam0: Capture 10 frames 6419.234779 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 5007744 SensorTimestamp = 6419234779000 6419.268213 (29.91 fps) cam0-stream0 seq: 000001 bytesused: 5007744 SensorTimestamp = 6419268213000 6419.301359 (30.17 fps) cam0-stream0 seq: 000002 bytesused: 5007744 SensorTimestamp = 6419301359000 6419.334744 (29.95 fps) cam0-stream0 seq: 000003 bytesused: 5007744 SensorTimestamp = 6419334744000 6419.368129 (29.95 fps) cam0-stream0 seq: 000004 bytesused: 5007744 SensorTimestamp = 6419368129000 6419.401516 (29.95 fps) cam0-stream0 seq: 000005 bytesused: 5007744 SensorTimestamp = 6419401516000 6419.434902 (29.95 fps) cam0-stream0 seq: 000006 bytesused: 5007744 SensorTimestamp = 6419434902000 6419.468289 (29.95 fps) cam0-stream0 seq: 000007 bytesused: 5007744 SensorTimestamp = 6419468289000 6419.501673 (29.95 fps) cam0-stream0 seq: 000008 bytesused: 5007744 SensorTimestamp = 6419501673000 6419.535058 (29.95 fps) cam0-stream0 seq: 000009 bytesused: 5007744 SensorTimestamp = 6419535058000 But alas I note that now gstreamer (and cheese, and pipewire) now fail: [libcamera] kbingham@charm:~/iob/libcamera/build/gcc$ gst-launch-1.0 libcamerasrc ! autovideosink Setting pipeline to PAUSED ... [1:46:09.897725501] [38325] INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/kbingham/iob/libcamera/build/gcc/src/ipa' to the IPA search path [1:46:09.903176776] [38325] INFO Camera camera_manager.cpp:327 libcamera v0.4.0+150-d0e1caa0 [1:46:09.935374613] [38330] WARN CameraSensor camera_sensor_legacy.cpp:616 'ov5675 24-0010': Rotation control not available, default to 0 degrees [1:46:09.941677813] [38330] INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data' [1:46:09.941734112] [38330] WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'ov5675.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml' [1:46:09.942172574] [38330] INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data' [1:46:09.949343846] [38330] INFO Pipeline pipeline_handler.cpp:618 libcamera is not installed. Loading platform configuration file from '/home/kbingham/iob/libcamera/src/libcamera/pipeline/virtual/data/virtual.yaml' Pipeline is live and does not need PREROLL ... Pipeline is PREROLLED ... Setting pipeline to PLAYING ... New clock: GstSystemClock [1:46:09.950895937] [38331] ERROR Stream stream.cpp:215 Range format is ambiguous [1:46:09.950940309] [38331] ERROR Stream stream.cpp:215 Range format is ambiguous [1:46:09.950962495] [38331] ERROR Stream stream.cpp:215 Range format is ambiguous [1:46:09.950982129] [38331] ERROR Stream stream.cpp:215 Range format is ambiguous [1:46:09.951001711] [38331] ERROR Stream stream.cpp:215 Range format is ambiguous [1:46:09.951020564] [38331] ERROR Stream stream.cpp:215 Range format is ambiguous ERROR: from element /GstPipeline:pipeline0/GstLibcameraSrc:libcamerasrc0: Internal data stream error. Additional debug info: ../../src/gstreamer/gstlibcamerasrc.cpp(662): gst_libcamera_src_task_enter (): /GstPipeline:pipeline0/GstLibcameraSrc:libcamerasrc0: streaming stopped, reason not-negotiated (-4) Execution ended after 0:00:00.000438306 Setting pipeline to NULL ... Freeing pipeline ... Debugging a little, at stream.ccp:215 diff --git a/src/libcamera/stream.cpp b/src/libcamera/stream.cpp index 978d72752b00..22f348c2df87 100644 --- a/src/libcamera/stream.cpp +++ b/src/libcamera/stream.cpp @@ -213,6 +213,8 @@ std::vector<Size> StreamFormats::sizes(const PixelFormat &pixelformat) const if (!discrete) { if (ranges.size() != 1) { LOG(Stream, Error) << "Range format is ambiguous"; + for (auto range: ranges) + LOG(Stream, Error) << range; return {}; } Reports: [1:51:01.855863590] [40484] ERROR Stream stream.cpp:215 Range format is ambiguous [1:51:01.855901452] [40484] ERROR Stream stream.cpp:217 (4x2)-(1288x972)/(+4,+2) [1:51:01.855920305] [40484] ERROR Stream stream.cpp:217 (4x2)-(2584x1944)/(+4,+2) So we're somehow setting up the stream capabilities incorrectly here which breaks things... (note that ranges.size is 2, not 1?) Perhaps we're incorrectly adding the raw stream sizes to the processed stream sizes ? -- Kieran > > Signed-off-by: Milan Zamazal <mzamazal@redhat.com> > --- > src/libcamera/pipeline/simple/simple.cpp | 72 ++++++++++++++++-------- > 1 file changed, 49 insertions(+), 23 deletions(-) > > diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp > index 0c6e0f44..92b9608b 100644 > --- a/src/libcamera/pipeline/simple/simple.cpp > +++ b/src/libcamera/pipeline/simple/simple.cpp > @@ -25,6 +25,7 @@ > #include <libcamera/base/log.h> > > #include <libcamera/camera.h> > +#include <libcamera/color_space.h> > #include <libcamera/control_ids.h> > #include <libcamera/pixel_format.h> > #include <libcamera/request.h> > @@ -389,6 +390,8 @@ private: > const MediaPad *acquirePipeline(SimpleCameraData *data); > void releasePipeline(SimpleCameraData *data); > > + void setUpFormatSizes(std::map<PixelFormat, std::vector<SizeRange>> formats); > + > std::map<const MediaEntity *, EntityData> entities_; > > MediaDevice *converter_; > @@ -1191,15 +1194,56 @@ SimplePipelineHandler::generateConfiguration(Camera *camera, Span<const StreamRo > data->processedRequested_ = true; > } > > - /* Create the formats map. */ > - std::map<PixelFormat, std::vector<SizeRange>> formats; > + /* Create the formats maps. */ > + std::map<PixelFormat, std::vector<SizeRange>> processedFormats; > + std::map<PixelFormat, std::vector<SizeRange>> rawFormats; > > for (const SimpleCameraData::Configuration &cfg : data->configs_) > - if (!cfg.raw) > - for (PixelFormat format : cfg.outputFormats) > - formats[format].push_back(cfg.outputSizes); > + for (PixelFormat format : cfg.outputFormats) > + (cfg.raw ? rawFormats : processedFormats)[format].push_back(cfg.outputSizes); > + > + if (data->processedRequested_ && processedFormats.empty()) { > + LOG(SimplePipeline, Error) > + << "Processed stream requsted but no corresponding output configuration found"; > + return nullptr; > + } > + if (data->rawRequested_ && rawFormats.empty()) { > + LOG(SimplePipeline, Error) > + << "Raw stream requsted but no corresponding output configuration found"; > + return nullptr; > + } > + > + setUpFormatSizes(processedFormats); > + setUpFormatSizes(rawFormats); > + > + /* > + * Create the stream configurations. Take the first entry in the formats > + * map as the default, for lack of a better option. > + * > + * \todo Implement a better way to pick the default format > + */ > + for (StreamRole role : roles) { > + bool raw = (role == StreamRole::Raw); > + auto formats = (raw ? rawFormats : processedFormats); > + StreamConfiguration cfg{ StreamFormats{ formats } }; > + cfg.pixelFormat = formats.begin()->first; > + cfg.size = formats.begin()->second[0].max; > + if (raw) > + cfg.colorSpace = ColorSpace::Raw; > + > + config->addConfiguration(cfg); > + } > + > + config->validate(); > + > + return config; > +} > > +void SimplePipelineHandler::setUpFormatSizes( > + std::map<PixelFormat, std::vector<SizeRange>> formats) > +{ > /* Sort the sizes and merge any consecutive overlapping ranges. */ > + > for (auto &[format, sizes] : formats) { > std::sort(sizes.begin(), sizes.end(), > [](SizeRange &a, SizeRange &b) { > @@ -1219,24 +1263,6 @@ SimplePipelineHandler::generateConfiguration(Camera *camera, Span<const StreamRo > > sizes.erase(++cur, sizes.end()); > } > - > - /* > - * Create the stream configurations. Take the first entry in the formats > - * map as the default, for lack of a better option. > - * > - * \todo Implement a better way to pick the default format > - */ > - for ([[maybe_unused]] StreamRole role : roles) { > - StreamConfiguration cfg{ StreamFormats{ formats } }; > - cfg.pixelFormat = formats.begin()->first; > - cfg.size = formats.begin()->second[0].max; > - > - config->addConfiguration(cfg); > - } > - > - config->validate(); > - > - return config; > } > > int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c) > -- > 2.48.1 >
I just tried the series on the Librem5 (in order to check if Camshark) and run into the same issue, starting with this commit: > purism-librem5:~/git/libcamera$ cam -c 1 -I > [0:12:04.495109800] [3837] INFO Camera camera_manager.cpp:329 > libcamera v0.4.0+194-06e2de17 > [0:12:04.611452658] [3838] WARN IPASoft soft_simple.cpp:98 IPASoft: > Failed to create camera sensor helper for hi846 > [0:12:04.622258198] [3838] WARN CameraSensorProperties > camera_sensor_properties.cpp:463 No static properties available for > 's5k3l6xx' > [0:12:04.622611614] [3838] WARN CameraSensorProperties > camera_sensor_properties.cpp:465 Please consider updating the camera > sensor properties database > [0:12:04.623478774] [3838] WARN CameraSensor > camera_sensor_legacy.cpp:669 's5k3l6xx 3-002d': Unsupported ancillary > entity function 131074 > [0:12:04.635553373] [3838] WARN IPASoft soft_simple.cpp:98 IPASoft: > Failed to create camera sensor helper for s5k3l6xx > Using camera /base/soc@0/bus@30800000/i2c@30a40000/camera@20 as cam0 > 0: 636x480-ABGR8888 > * Pixelformat: ABGR8888 (2x2)-(1628x1224)/(+0,+0) > [0:12:04.643438097] [3837] ERROR Stream stream.cpp:215 Range format is > ambiguous > * Pixelformat: XBGR8888 (2x2)-(1628x1224)/(+0,+0) > [0:12:04.643696229] [3837] ERROR Stream stream.cpp:215 Range format is > ambiguous > * Pixelformat: BGR888 (2x2)-(1628x1224)/(+0,+0) > [0:12:04.643748792] [3837] ERROR Stream stream.cpp:215 Range format is > ambiguous > * Pixelformat: RGB888 (2x2)-(1628x1224)/(+0,+0) > [0:12:04.643861237] [3837] ERROR Stream stream.cpp:215 Range format is > ambiguous > * Pixelformat: ARGB8888 (2x2)-(1628x1224)/(+0,+0) > [0:12:04.643918479] [3837] ERROR Stream stream.cpp:215 Range format is > ambiguous > * Pixelformat: XRGB8888 (2x2)-(1628x1224)/(+0,+0) > [0:12:04.643971162] [3837] ERROR Stream stream.cpp:215 Range format is > ambiguous Interestingly I don't see the same on the OP6 - i.e. not every sensor appears to be affected. On 20.03.25 23:43, Kieran Bingham wrote: > Quoting Milan Zamazal (2025-03-05 19:26:12) >> When generating simple pipeline configuration, raw output configurations >> are generated but later filtered out. Let's include processed and/or >> raw output configurations depending on the requested stream roles. >> >> Raw and processed formats are handled separately. The intention is that >> in case both raw and processed formats are requested then the raw >> formats should be left intact to the extent possible and not be >> influenced by the processed formats (implying that raw parameters >> compatible with the processed output requirements must be requested by >> the application). >> >> Raw output configurations are marked by setting their colorSpace to >> ColorSpace::Raw, which is the only place there suitable for the purpose. >> >> This is another preparatory patch without making raw outputs working. > I've identified two interesting effects which I've bisected to this > patch: > > First: > > Without this patch on x13s: > > [libcamera] kbingham@charm:~/iob/libcamera/build/gcc$ cam -c1 --metadata -C10 > [1:44:39.464209549] [37696] INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/kbingham/iob/libcamera/build/gcc/src/ipa' to the IPA search path > [1:44:39.471858239] [37696] INFO Camera camera_manager.cpp:327 libcamera v0.4.0+149-628e8c80 > [1:44:39.497573824] [37697] WARN CameraSensor camera_sensor_legacy.cpp:616 'ov5675 24-0010': Rotation control not available, default to 0 degrees > [1:44:39.502155101] [37697] INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data' > [1:44:39.502180672] [37697] WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'ov5675.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml' > [1:44:39.502468206] [37697] INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data' > [1:44:39.512547578] [37697] INFO Pipeline pipeline_handler.cpp:618 libcamera is not installed. Loading platform configuration file from '/home/kbingham/iob/libcamera/src/libcamera/pipeline/virtual/data/virtual.yaml' > Using camera /base/soc@0/cci@ac4c000/i2c-bus@1/camera@10 as cam0 > [1:44:39.514127115] [37696] INFO Camera camera.cpp:1203 configuring streams: (0) 2584x1944-ABGR8888 > [1:44:39.520536247] [37697] WARN CameraSensor camera_sensor_legacy.cpp:501 'ov5675 24-0010': No sensor delays found in static properties. Assuming unverified defaults. > [1:44:39.520899922] [37697] INFO IPASoft soft_simple.cpp:251 IPASoft: Exposure 4-2016, gain 1-15.9922 (0.149922) > cam0: Capture 10 frames > 6279.664529 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 20093184 > SensorTimestamp = 6279664529000 > 6279.697607 (30.23 fps) cam0-stream0 seq: 000001 bytesused: 20093184 > SensorTimestamp = 6279697607000 > 6279.731336 (29.65 fps) cam0-stream0 seq: 000002 bytesused: 20093184 > SensorTimestamp = 6279731336000 > 6279.831491 (9.98 fps) cam0-stream0 seq: 000003 bytesused: 20093184 > SensorTimestamp = 6279831491000 > 6279.898368 (14.95 fps) cam0-stream0 seq: 000004 bytesused: 20093184 > SensorTimestamp = 6279898368000 > 6279.965133 (14.98 fps) cam0-stream0 seq: 000005 bytesused: 20093184 > SensorTimestamp = 6279965133000 > 6280.031911 (14.97 fps) cam0-stream0 seq: 000006 bytesused: 20093184 > SensorTimestamp = 6280031911000 > 6280.098577 (15.00 fps) cam0-stream0 seq: 000007 bytesused: 20093184 > SensorTimestamp = 6280098577000 > 6280.165320 (14.98 fps) cam0-stream0 seq: 000008 bytesused: 20093184 > SensorTimestamp = 6280165320000 > 6280.232225 (14.95 fps) cam0-stream0 seq: 000009 bytesused: 20093184 > SensorTimestamp = 6280232225000 > > (Note the 15 FPS), and pipewire/gstreamer/cheese are all 'streaming'... > > > But with this patch the rate jumps to a more reasonable 30 FPS seen by > qcam and cam: > > [libcamera] kbingham@charm:~/iob/libcamera/build/gcc$ cam -c1 --metadata -C10 > [1:46:59.040578435] [38346] INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/kbingham/iob/libcamera/build/gcc/src/ipa' to the IPA search path > [1:46:59.048420757] [38346] INFO Camera camera_manager.cpp:327 libcamera v0.4.0+150-d0e1caa0 > [1:46:59.074259713] [38347] WARN CameraSensor camera_sensor_legacy.cpp:616 'ov5675 24-0010': Rotation control not available, default to 0 degrees > [1:46:59.078796773] [38347] INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data' > [1:46:59.078822605] [38347] WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'ov5675.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml' > [1:46:59.079110243] [38347] INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data' > [1:46:59.087987917] [38347] INFO Pipeline pipeline_handler.cpp:618 libcamera is not installed. Loading platform configuration file from '/home/kbingham/iob/libcamera/src/libcamera/pipeline/virtual/data/virtual.yaml' > Using camera /base/soc@0/cci@ac4c000/i2c-bus@1/camera@10 as cam0 > [1:46:59.089522196] [38346] INFO Camera camera.cpp:1203 configuring streams: (0) 1288x972-ABGR8888 > [1:46:59.096422962] [38347] WARN CameraSensor camera_sensor_legacy.cpp:501 'ov5675 24-0010': No sensor delays found in static properties. Assuming unverified defaults. > [1:46:59.096786533] [38347] INFO IPASoft soft_simple.cpp:251 IPASoft: Exposure 4-2016, gain 1-15.9922 (0.149922) > cam0: Capture 10 frames > 6419.234779 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 5007744 > SensorTimestamp = 6419234779000 > 6419.268213 (29.91 fps) cam0-stream0 seq: 000001 bytesused: 5007744 > SensorTimestamp = 6419268213000 > 6419.301359 (30.17 fps) cam0-stream0 seq: 000002 bytesused: 5007744 > SensorTimestamp = 6419301359000 > 6419.334744 (29.95 fps) cam0-stream0 seq: 000003 bytesused: 5007744 > SensorTimestamp = 6419334744000 > 6419.368129 (29.95 fps) cam0-stream0 seq: 000004 bytesused: 5007744 > SensorTimestamp = 6419368129000 > 6419.401516 (29.95 fps) cam0-stream0 seq: 000005 bytesused: 5007744 > SensorTimestamp = 6419401516000 > 6419.434902 (29.95 fps) cam0-stream0 seq: 000006 bytesused: 5007744 > SensorTimestamp = 6419434902000 > 6419.468289 (29.95 fps) cam0-stream0 seq: 000007 bytesused: 5007744 > SensorTimestamp = 6419468289000 > 6419.501673 (29.95 fps) cam0-stream0 seq: 000008 bytesused: 5007744 > SensorTimestamp = 6419501673000 > 6419.535058 (29.95 fps) cam0-stream0 seq: 000009 bytesused: 5007744 > SensorTimestamp = 6419535058000 > > > But alas I note that now gstreamer (and cheese, and pipewire) now fail: > > [libcamera] kbingham@charm:~/iob/libcamera/build/gcc$ gst-launch-1.0 libcamerasrc ! autovideosink > Setting pipeline to PAUSED ... > [1:46:09.897725501] [38325] INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/kbingham/iob/libcamera/build/gcc/src/ipa' to the IPA search path > [1:46:09.903176776] [38325] INFO Camera camera_manager.cpp:327 libcamera v0.4.0+150-d0e1caa0 > [1:46:09.935374613] [38330] WARN CameraSensor camera_sensor_legacy.cpp:616 'ov5675 24-0010': Rotation control not available, default to 0 degrees > [1:46:09.941677813] [38330] INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data' > [1:46:09.941734112] [38330] WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'ov5675.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml' > [1:46:09.942172574] [38330] INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data' > [1:46:09.949343846] [38330] INFO Pipeline pipeline_handler.cpp:618 libcamera is not installed. Loading platform configuration file from '/home/kbingham/iob/libcamera/src/libcamera/pipeline/virtual/data/virtual.yaml' > Pipeline is live and does not need PREROLL ... > Pipeline is PREROLLED ... > Setting pipeline to PLAYING ... > New clock: GstSystemClock > [1:46:09.950895937] [38331] ERROR Stream stream.cpp:215 Range format is ambiguous > [1:46:09.950940309] [38331] ERROR Stream stream.cpp:215 Range format is ambiguous > [1:46:09.950962495] [38331] ERROR Stream stream.cpp:215 Range format is ambiguous > [1:46:09.950982129] [38331] ERROR Stream stream.cpp:215 Range format is ambiguous > [1:46:09.951001711] [38331] ERROR Stream stream.cpp:215 Range format is ambiguous > [1:46:09.951020564] [38331] ERROR Stream stream.cpp:215 Range format is ambiguous > ERROR: from element /GstPipeline:pipeline0/GstLibcameraSrc:libcamerasrc0: Internal data stream error. > Additional debug info: > ../../src/gstreamer/gstlibcamerasrc.cpp(662): gst_libcamera_src_task_enter (): /GstPipeline:pipeline0/GstLibcameraSrc:libcamerasrc0: > streaming stopped, reason not-negotiated (-4) > Execution ended after 0:00:00.000438306 > Setting pipeline to NULL ... > Freeing pipeline ... > > Debugging a little, at stream.ccp:215 > > diff --git a/src/libcamera/stream.cpp b/src/libcamera/stream.cpp > index 978d72752b00..22f348c2df87 100644 > --- a/src/libcamera/stream.cpp > +++ b/src/libcamera/stream.cpp > @@ -213,6 +213,8 @@ std::vector<Size> StreamFormats::sizes(const PixelFormat &pixelformat) const > if (!discrete) { > if (ranges.size() != 1) { > LOG(Stream, Error) << "Range format is ambiguous"; > + for (auto range: ranges) > + LOG(Stream, Error) << range; > return {}; > } > > Reports: > > [1:51:01.855863590] [40484] ERROR Stream stream.cpp:215 Range format is ambiguous > [1:51:01.855901452] [40484] ERROR Stream stream.cpp:217 (4x2)-(1288x972)/(+4,+2) > [1:51:01.855920305] [40484] ERROR Stream stream.cpp:217 (4x2)-(2584x1944)/(+4,+2) > > So we're somehow setting up the stream capabilities incorrectly here > which breaks things... (note that ranges.size is 2, not 1?) > > Perhaps we're incorrectly adding the raw stream sizes to the processed stream > sizes ? > > -- > Kieran > > >> Signed-off-by: Milan Zamazal <mzamazal@redhat.com> >> --- >> src/libcamera/pipeline/simple/simple.cpp | 72 ++++++++++++++++-------- >> 1 file changed, 49 insertions(+), 23 deletions(-) >> >> diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp >> index 0c6e0f44..92b9608b 100644 >> --- a/src/libcamera/pipeline/simple/simple.cpp >> +++ b/src/libcamera/pipeline/simple/simple.cpp >> @@ -25,6 +25,7 @@ >> #include <libcamera/base/log.h> >> >> #include <libcamera/camera.h> >> +#include <libcamera/color_space.h> >> #include <libcamera/control_ids.h> >> #include <libcamera/pixel_format.h> >> #include <libcamera/request.h> >> @@ -389,6 +390,8 @@ private: >> const MediaPad *acquirePipeline(SimpleCameraData *data); >> void releasePipeline(SimpleCameraData *data); >> >> + void setUpFormatSizes(std::map<PixelFormat, std::vector<SizeRange>> formats); >> + >> std::map<const MediaEntity *, EntityData> entities_; >> >> MediaDevice *converter_; >> @@ -1191,15 +1194,56 @@ SimplePipelineHandler::generateConfiguration(Camera *camera, Span<const StreamRo >> data->processedRequested_ = true; >> } >> >> - /* Create the formats map. */ >> - std::map<PixelFormat, std::vector<SizeRange>> formats; >> + /* Create the formats maps. */ >> + std::map<PixelFormat, std::vector<SizeRange>> processedFormats; >> + std::map<PixelFormat, std::vector<SizeRange>> rawFormats; >> >> for (const SimpleCameraData::Configuration &cfg : data->configs_) >> - if (!cfg.raw) >> - for (PixelFormat format : cfg.outputFormats) >> - formats[format].push_back(cfg.outputSizes); >> + for (PixelFormat format : cfg.outputFormats) >> + (cfg.raw ? rawFormats : processedFormats)[format].push_back(cfg.outputSizes); >> + >> + if (data->processedRequested_ && processedFormats.empty()) { >> + LOG(SimplePipeline, Error) >> + << "Processed stream requsted but no corresponding output configuration found"; >> + return nullptr; >> + } >> + if (data->rawRequested_ && rawFormats.empty()) { >> + LOG(SimplePipeline, Error) >> + << "Raw stream requsted but no corresponding output configuration found"; >> + return nullptr; >> + } >> + >> + setUpFormatSizes(processedFormats); >> + setUpFormatSizes(rawFormats); >> + >> + /* >> + * Create the stream configurations. Take the first entry in the formats >> + * map as the default, for lack of a better option. >> + * >> + * \todo Implement a better way to pick the default format >> + */ >> + for (StreamRole role : roles) { >> + bool raw = (role == StreamRole::Raw); >> + auto formats = (raw ? rawFormats : processedFormats); >> + StreamConfiguration cfg{ StreamFormats{ formats } }; >> + cfg.pixelFormat = formats.begin()->first; >> + cfg.size = formats.begin()->second[0].max; >> + if (raw) >> + cfg.colorSpace = ColorSpace::Raw; >> + >> + config->addConfiguration(cfg); >> + } >> + >> + config->validate(); >> + >> + return config; >> +} >> >> +void SimplePipelineHandler::setUpFormatSizes( >> + std::map<PixelFormat, std::vector<SizeRange>> formats) >> +{ >> /* Sort the sizes and merge any consecutive overlapping ranges. */ >> + >> for (auto &[format, sizes] : formats) { >> std::sort(sizes.begin(), sizes.end(), >> [](SizeRange &a, SizeRange &b) { >> @@ -1219,24 +1263,6 @@ SimplePipelineHandler::generateConfiguration(Camera *camera, Span<const StreamRo >> >> sizes.erase(++cur, sizes.end()); >> } >> - >> - /* >> - * Create the stream configurations. Take the first entry in the formats >> - * map as the default, for lack of a better option. >> - * >> - * \todo Implement a better way to pick the default format >> - */ >> - for ([[maybe_unused]] StreamRole role : roles) { >> - StreamConfiguration cfg{ StreamFormats{ formats } }; >> - cfg.pixelFormat = formats.begin()->first; >> - cfg.size = formats.begin()->second[0].max; >> - >> - config->addConfiguration(cfg); >> - } >> - >> - config->validate(); >> - >> - return config; >> } >> >> int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c) >> -- >> 2.48.1 >>
Hi Kieran, thank you for testing. Kieran Bingham <kieran.bingham@ideasonboard.com> writes: > Quoting Milan Zamazal (2025-03-05 19:26:12) >> When generating simple pipeline configuration, raw output configurations >> are generated but later filtered out. Let's include processed and/or >> raw output configurations depending on the requested stream roles. >> >> Raw and processed formats are handled separately. The intention is that >> in case both raw and processed formats are requested then the raw >> formats should be left intact to the extent possible and not be >> influenced by the processed formats (implying that raw parameters >> compatible with the processed output requirements must be requested by >> the application). >> >> Raw output configurations are marked by setting their colorSpace to >> ColorSpace::Raw, which is the only place there suitable for the purpose. >> >> This is another preparatory patch without making raw outputs working. > > I've identified two interesting effects which I've bisected to this > patch: > > First: > > Without this patch on x13s: > > [libcamera] kbingham@charm:~/iob/libcamera/build/gcc$ cam -c1 --metadata -C10 > [1:44:39.464209549] [37696] INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/kbingham/iob/libcamera/build/gcc/src/ipa' to the IPA search path > [1:44:39.471858239] [37696] INFO Camera camera_manager.cpp:327 libcamera v0.4.0+149-628e8c80 > [1:44:39.497573824] [37697] WARN CameraSensor camera_sensor_legacy.cpp:616 'ov5675 24-0010': Rotation control not available, default to 0 degrees > [1:44:39.502155101] [37697] INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data' > [1:44:39.502180672] [37697] WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'ov5675.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml' > [1:44:39.502468206] [37697] INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data' > [1:44:39.512547578] [37697] INFO Pipeline pipeline_handler.cpp:618 libcamera is not installed. Loading platform configuration file from '/home/kbingham/iob/libcamera/src/libcamera/pipeline/virtual/data/virtual.yaml' > Using camera /base/soc@0/cci@ac4c000/i2c-bus@1/camera@10 as cam0 > [1:44:39.514127115] [37696] INFO Camera camera.cpp:1203 configuring streams: (0) 2584x1944-ABGR8888 > [1:44:39.520536247] [37697] WARN CameraSensor camera_sensor_legacy.cpp:501 'ov5675 24-0010': No sensor delays found in static properties. Assuming unverified defaults. > [1:44:39.520899922] [37697] INFO IPASoft soft_simple.cpp:251 IPASoft: Exposure 4-2016, gain 1-15.9922 (0.149922) > cam0: Capture 10 frames > 6279.664529 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 20093184 > SensorTimestamp = 6279664529000 > 6279.697607 (30.23 fps) cam0-stream0 seq: 000001 bytesused: 20093184 > SensorTimestamp = 6279697607000 > 6279.731336 (29.65 fps) cam0-stream0 seq: 000002 bytesused: 20093184 > SensorTimestamp = 6279731336000 > 6279.831491 (9.98 fps) cam0-stream0 seq: 000003 bytesused: 20093184 > SensorTimestamp = 6279831491000 > 6279.898368 (14.95 fps) cam0-stream0 seq: 000004 bytesused: 20093184 > SensorTimestamp = 6279898368000 > 6279.965133 (14.98 fps) cam0-stream0 seq: 000005 bytesused: 20093184 > SensorTimestamp = 6279965133000 > 6280.031911 (14.97 fps) cam0-stream0 seq: 000006 bytesused: 20093184 > SensorTimestamp = 6280031911000 > 6280.098577 (15.00 fps) cam0-stream0 seq: 000007 bytesused: 20093184 > SensorTimestamp = 6280098577000 > 6280.165320 (14.98 fps) cam0-stream0 seq: 000008 bytesused: 20093184 > SensorTimestamp = 6280165320000 > 6280.232225 (14.95 fps) cam0-stream0 seq: 000009 bytesused: 20093184 > SensorTimestamp = 6280232225000 > > (Note the 15 FPS), and pipewire/gstreamer/cheese are all 'streaming'... > > > But with this patch the rate jumps to a more reasonable 30 FPS seen by > qcam and cam: Well, clearly because of the lower resolution. I don't have an explanation why the patch should make it different. > [libcamera] kbingham@charm:~/iob/libcamera/build/gcc$ cam -c1 --metadata -C10 > [1:46:59.040578435] [38346] INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/kbingham/iob/libcamera/build/gcc/src/ipa' to the IPA search path > [1:46:59.048420757] [38346] INFO Camera camera_manager.cpp:327 libcamera v0.4.0+150-d0e1caa0 > [1:46:59.074259713] [38347] WARN CameraSensor camera_sensor_legacy.cpp:616 'ov5675 24-0010': Rotation control not available, default to 0 degrees > [1:46:59.078796773] [38347] INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data' > [1:46:59.078822605] [38347] WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'ov5675.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml' > [1:46:59.079110243] [38347] INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data' > [1:46:59.087987917] [38347] INFO Pipeline pipeline_handler.cpp:618 libcamera is not installed. Loading platform configuration file from '/home/kbingham/iob/libcamera/src/libcamera/pipeline/virtual/data/virtual.yaml' > Using camera /base/soc@0/cci@ac4c000/i2c-bus@1/camera@10 as cam0 > [1:46:59.089522196] [38346] INFO Camera camera.cpp:1203 configuring streams: (0) 1288x972-ABGR8888 > [1:46:59.096422962] [38347] WARN CameraSensor camera_sensor_legacy.cpp:501 'ov5675 24-0010': No sensor delays found in static properties. Assuming unverified defaults. > [1:46:59.096786533] [38347] INFO IPASoft soft_simple.cpp:251 IPASoft: Exposure 4-2016, gain 1-15.9922 (0.149922) > cam0: Capture 10 frames > 6419.234779 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 5007744 > SensorTimestamp = 6419234779000 > 6419.268213 (29.91 fps) cam0-stream0 seq: 000001 bytesused: 5007744 > SensorTimestamp = 6419268213000 > 6419.301359 (30.17 fps) cam0-stream0 seq: 000002 bytesused: 5007744 > SensorTimestamp = 6419301359000 > 6419.334744 (29.95 fps) cam0-stream0 seq: 000003 bytesused: 5007744 > SensorTimestamp = 6419334744000 > 6419.368129 (29.95 fps) cam0-stream0 seq: 000004 bytesused: 5007744 > SensorTimestamp = 6419368129000 > 6419.401516 (29.95 fps) cam0-stream0 seq: 000005 bytesused: 5007744 > SensorTimestamp = 6419401516000 > 6419.434902 (29.95 fps) cam0-stream0 seq: 000006 bytesused: 5007744 > SensorTimestamp = 6419434902000 > 6419.468289 (29.95 fps) cam0-stream0 seq: 000007 bytesused: 5007744 > SensorTimestamp = 6419468289000 > 6419.501673 (29.95 fps) cam0-stream0 seq: 000008 bytesused: 5007744 > SensorTimestamp = 6419501673000 > 6419.535058 (29.95 fps) cam0-stream0 seq: 000009 bytesused: 5007744 > SensorTimestamp = 6419535058000 > > > But alas I note that now gstreamer (and cheese, and pipewire) now fail: > > [libcamera] kbingham@charm:~/iob/libcamera/build/gcc$ gst-launch-1.0 libcamerasrc ! autovideosink > Setting pipeline to PAUSED ... > [1:46:09.897725501] [38325] INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/kbingham/iob/libcamera/build/gcc/src/ipa' to the IPA search path > [1:46:09.903176776] [38325] INFO Camera camera_manager.cpp:327 libcamera v0.4.0+150-d0e1caa0 > [1:46:09.935374613] [38330] WARN CameraSensor camera_sensor_legacy.cpp:616 'ov5675 24-0010': Rotation control not available, default to 0 degrees > [1:46:09.941677813] [38330] INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data' > [1:46:09.941734112] [38330] WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'ov5675.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml' > [1:46:09.942172574] [38330] INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data' > [1:46:09.949343846] [38330] INFO Pipeline pipeline_handler.cpp:618 libcamera is not installed. Loading platform configuration file from '/home/kbingham/iob/libcamera/src/libcamera/pipeline/virtual/data/virtual.yaml' > Pipeline is live and does not need PREROLL ... > Pipeline is PREROLLED ... > Setting pipeline to PLAYING ... > New clock: GstSystemClock > [1:46:09.950895937] [38331] ERROR Stream stream.cpp:215 Range format is ambiguous > [1:46:09.950940309] [38331] ERROR Stream stream.cpp:215 Range format is ambiguous > [1:46:09.950962495] [38331] ERROR Stream stream.cpp:215 Range format is ambiguous > [1:46:09.950982129] [38331] ERROR Stream stream.cpp:215 Range format is ambiguous > [1:46:09.951001711] [38331] ERROR Stream stream.cpp:215 Range format is ambiguous > [1:46:09.951020564] [38331] ERROR Stream stream.cpp:215 Range format is ambiguous > ERROR: from element /GstPipeline:pipeline0/GstLibcameraSrc:libcamerasrc0: Internal data stream error. > Additional debug info: > ../../src/gstreamer/gstlibcamerasrc.cpp(662): gst_libcamera_src_task_enter (): /GstPipeline:pipeline0/GstLibcameraSrc:libcamerasrc0: > streaming stopped, reason not-negotiated (-4) > Execution ended after 0:00:00.000438306 > Setting pipeline to NULL ... > Freeing pipeline ... > > Debugging a little, at stream.ccp:215 > > diff --git a/src/libcamera/stream.cpp b/src/libcamera/stream.cpp > index 978d72752b00..22f348c2df87 100644 > --- a/src/libcamera/stream.cpp > +++ b/src/libcamera/stream.cpp > @@ -213,6 +213,8 @@ std::vector<Size> StreamFormats::sizes(const PixelFormat &pixelformat) const > if (!discrete) { > if (ranges.size() != 1) { > LOG(Stream, Error) << "Range format is ambiguous"; > + for (auto range: ranges) > + LOG(Stream, Error) << range; > return {}; > } > > Reports: > > [1:51:01.855863590] [40484] ERROR Stream stream.cpp:215 Range format is ambiguous > [1:51:01.855901452] [40484] ERROR Stream stream.cpp:217 (4x2)-(1288x972)/(+4,+2) > [1:51:01.855920305] [40484] ERROR Stream stream.cpp:217 (4x2)-(2584x1944)/(+4,+2) > > So we're somehow setting up the stream capabilities incorrectly here > which breaks things... (note that ranges.size is 2, not 1?) > > Perhaps we're incorrectly adding the raw stream sizes to the processed stream > sizes ? No, it's a stupid (and not immediately visible) bug: passing formats to setUpFormatSizes by value rather than by reference, see below. > -- > Kieran > > >> >> Signed-off-by: Milan Zamazal <mzamazal@redhat.com> >> --- >> src/libcamera/pipeline/simple/simple.cpp | 72 ++++++++++++++++-------- >> 1 file changed, 49 insertions(+), 23 deletions(-) >> >> diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp >> index 0c6e0f44..92b9608b 100644 >> --- a/src/libcamera/pipeline/simple/simple.cpp >> +++ b/src/libcamera/pipeline/simple/simple.cpp >> @@ -25,6 +25,7 @@ >> #include <libcamera/base/log.h> >> >> #include <libcamera/camera.h> >> +#include <libcamera/color_space.h> >> #include <libcamera/control_ids.h> >> #include <libcamera/pixel_format.h> >> #include <libcamera/request.h> >> @@ -389,6 +390,8 @@ private: >> const MediaPad *acquirePipeline(SimpleCameraData *data); >> void releasePipeline(SimpleCameraData *data); >> >> + void setUpFormatSizes(std::map<PixelFormat, std::vector<SizeRange>> formats); Should be &formats. >> + >> std::map<const MediaEntity *, EntityData> entities_; >> >> MediaDevice *converter_; >> @@ -1191,15 +1194,56 @@ SimplePipelineHandler::generateConfiguration(Camera *camera, Span<const StreamRo >> data->processedRequested_ = true; >> } >> >> - /* Create the formats map. */ >> - std::map<PixelFormat, std::vector<SizeRange>> formats; >> + /* Create the formats maps. */ >> + std::map<PixelFormat, std::vector<SizeRange>> processedFormats; >> + std::map<PixelFormat, std::vector<SizeRange>> rawFormats; >> >> for (const SimpleCameraData::Configuration &cfg : data->configs_) >> - if (!cfg.raw) >> - for (PixelFormat format : cfg.outputFormats) >> - formats[format].push_back(cfg.outputSizes); >> + for (PixelFormat format : cfg.outputFormats) >> + (cfg.raw ? rawFormats : processedFormats)[format].push_back(cfg.outputSizes); >> + >> + if (data->processedRequested_ && processedFormats.empty()) { >> + LOG(SimplePipeline, Error) >> + << "Processed stream requsted but no corresponding output configuration found"; >> + return nullptr; >> + } >> + if (data->rawRequested_ && rawFormats.empty()) { >> + LOG(SimplePipeline, Error) >> + << "Raw stream requsted but no corresponding output configuration found"; >> + return nullptr; >> + } >> + >> + setUpFormatSizes(processedFormats); >> + setUpFormatSizes(rawFormats); >> + >> + /* >> + * Create the stream configurations. Take the first entry in the formats >> + * map as the default, for lack of a better option. >> + * >> + * \todo Implement a better way to pick the default format >> + */ >> + for (StreamRole role : roles) { >> + bool raw = (role == StreamRole::Raw); >> + auto formats = (raw ? rawFormats : processedFormats); >> + StreamConfiguration cfg{ StreamFormats{ formats } }; >> + cfg.pixelFormat = formats.begin()->first; >> + cfg.size = formats.begin()->second[0].max; >> + if (raw) >> + cfg.colorSpace = ColorSpace::Raw; >> + >> + config->addConfiguration(cfg); >> + } >> + >> + config->validate(); >> + >> + return config; >> +} >> >> +void SimplePipelineHandler::setUpFormatSizes( >> + std::map<PixelFormat, std::vector<SizeRange>> formats) &formats >> +{ >> /* Sort the sizes and merge any consecutive overlapping ranges. */ >> + >> for (auto &[format, sizes] : formats) { >> std::sort(sizes.begin(), sizes.end(), >> [](SizeRange &a, SizeRange &b) { >> @@ -1219,24 +1263,6 @@ SimplePipelineHandler::generateConfiguration(Camera *camera, Span<const StreamRo >> >> sizes.erase(++cur, sizes.end()); >> } >> - >> - /* >> - * Create the stream configurations. Take the first entry in the formats >> - * map as the default, for lack of a better option. >> - * >> - * \todo Implement a better way to pick the default format >> - */ >> - for ([[maybe_unused]] StreamRole role : roles) { >> - StreamConfiguration cfg{ StreamFormats{ formats } }; >> - cfg.pixelFormat = formats.begin()->first; >> - cfg.size = formats.begin()->second[0].max; >> - >> - config->addConfiguration(cfg); >> - } >> - >> - config->validate(); >> - >> - return config; >> } >> >> int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c) >> -- >> 2.48.1 >>
Hi Robert, thank you for testing, see my response to Kieran what's the problem. Robert Mader <robert.mader@collabora.com> writes: > I just tried the series on the Librem5 (in order to check if Camshark) and run into the same issue, starting with this commit: > >> purism-librem5:~/git/libcamera$ cam -c 1 -I A really easy reproducer, thanks. :-) >> [0:12:04.495109800] [3837] INFO Camera camera_manager.cpp:329 libcamera v0.4.0+194-06e2de17 >> [0:12:04.611452658] [3838] WARN IPASoft soft_simple.cpp:98 IPASoft: Failed to create camera sensor helper for hi846 >> [0:12:04.622258198] [3838] WARN CameraSensorProperties camera_sensor_properties.cpp:463 No static properties available for 's5k3l6xx' >> [0:12:04.622611614] [3838] WARN CameraSensorProperties camera_sensor_properties.cpp:465 Please consider updating the camera sensor properties database >> [0:12:04.623478774] [3838] WARN CameraSensor camera_sensor_legacy.cpp:669 's5k3l6xx 3-002d': Unsupported ancillary entity function 131074 >> [0:12:04.635553373] [3838] WARN IPASoft soft_simple.cpp:98 IPASoft: Failed to create camera sensor helper for s5k3l6xx >> Using camera /base/soc@0/bus@30800000/i2c@30a40000/camera@20 as cam0 >> 0: 636x480-ABGR8888 >> * Pixelformat: ABGR8888 (2x2)-(1628x1224)/(+0,+0) >> [0:12:04.643438097] [3837] ERROR Stream stream.cpp:215 Range format is ambiguous >> * Pixelformat: XBGR8888 (2x2)-(1628x1224)/(+0,+0) >> [0:12:04.643696229] [3837] ERROR Stream stream.cpp:215 Range format is ambiguous >> * Pixelformat: BGR888 (2x2)-(1628x1224)/(+0,+0) >> [0:12:04.643748792] [3837] ERROR Stream stream.cpp:215 Range format is ambiguous >> * Pixelformat: RGB888 (2x2)-(1628x1224)/(+0,+0) >> [0:12:04.643861237] [3837] ERROR Stream stream.cpp:215 Range format is ambiguous >> * Pixelformat: ARGB8888 (2x2)-(1628x1224)/(+0,+0) >> [0:12:04.643918479] [3837] ERROR Stream stream.cpp:215 Range format is ambiguous >> * Pixelformat: XRGB8888 (2x2)-(1628x1224)/(+0,+0) >> [0:12:04.643971162] [3837] ERROR Stream stream.cpp:215 Range format is ambiguous > > Interestingly I don't see the same on the OP6 - i.e. not every sensor appears to be affected. > > On 20.03.25 23:43, Kieran Bingham wrote: >> Quoting Milan Zamazal (2025-03-05 19:26:12) >>> When generating simple pipeline configuration, raw output configurations >>> are generated but later filtered out. Let's include processed and/or >>> raw output configurations depending on the requested stream roles. >>> >>> Raw and processed formats are handled separately. The intention is that >>> in case both raw and processed formats are requested then the raw >>> formats should be left intact to the extent possible and not be >>> influenced by the processed formats (implying that raw parameters >>> compatible with the processed output requirements must be requested by >>> the application). >>> >>> Raw output configurations are marked by setting their colorSpace to >>> ColorSpace::Raw, which is the only place there suitable for the purpose. >>> >>> This is another preparatory patch without making raw outputs working. >> I've identified two interesting effects which I've bisected to this >> patch: >> >> First: >> >> Without this patch on x13s: >> >> [libcamera] kbingham@charm:~/iob/libcamera/build/gcc$ cam -c1 --metadata -C10 >> [1:44:39.464209549] [37696] INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/kbingham/iob/libcamera/build/gcc/src/ipa' to the IPA search path >> [1:44:39.471858239] [37696] INFO Camera camera_manager.cpp:327 libcamera v0.4.0+149-628e8c80 >> [1:44:39.497573824] [37697] WARN CameraSensor camera_sensor_legacy.cpp:616 'ov5675 24-0010': Rotation control not available, default to 0 degrees >> [1:44:39.502155101] [37697] INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data' >> [1:44:39.502180672] [37697] WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'ov5675.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml' >> [1:44:39.502468206] [37697] INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data' >> [1:44:39.512547578] [37697] INFO Pipeline pipeline_handler.cpp:618 libcamera is not installed. Loading platform configuration file from '/home/kbingham/iob/libcamera/src/libcamera/pipeline/virtual/data/virtual.yaml' >> Using camera /base/soc@0/cci@ac4c000/i2c-bus@1/camera@10 as cam0 >> [1:44:39.514127115] [37696] INFO Camera camera.cpp:1203 configuring streams: (0) 2584x1944-ABGR8888 >> [1:44:39.520536247] [37697] WARN CameraSensor camera_sensor_legacy.cpp:501 'ov5675 24-0010': No sensor delays found in static properties. Assuming unverified defaults. >> [1:44:39.520899922] [37697] INFO IPASoft soft_simple.cpp:251 IPASoft: Exposure 4-2016, gain 1-15.9922 (0.149922) >> cam0: Capture 10 frames >> 6279.664529 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 20093184 >> SensorTimestamp = 6279664529000 >> 6279.697607 (30.23 fps) cam0-stream0 seq: 000001 bytesused: 20093184 >> SensorTimestamp = 6279697607000 >> 6279.731336 (29.65 fps) cam0-stream0 seq: 000002 bytesused: 20093184 >> SensorTimestamp = 6279731336000 >> 6279.831491 (9.98 fps) cam0-stream0 seq: 000003 bytesused: 20093184 >> SensorTimestamp = 6279831491000 >> 6279.898368 (14.95 fps) cam0-stream0 seq: 000004 bytesused: 20093184 >> SensorTimestamp = 6279898368000 >> 6279.965133 (14.98 fps) cam0-stream0 seq: 000005 bytesused: 20093184 >> SensorTimestamp = 6279965133000 >> 6280.031911 (14.97 fps) cam0-stream0 seq: 000006 bytesused: 20093184 >> SensorTimestamp = 6280031911000 >> 6280.098577 (15.00 fps) cam0-stream0 seq: 000007 bytesused: 20093184 >> SensorTimestamp = 6280098577000 >> 6280.165320 (14.98 fps) cam0-stream0 seq: 000008 bytesused: 20093184 >> SensorTimestamp = 6280165320000 >> 6280.232225 (14.95 fps) cam0-stream0 seq: 000009 bytesused: 20093184 >> SensorTimestamp = 6280232225000 >> >> (Note the 15 FPS), and pipewire/gstreamer/cheese are all 'streaming'... >> >> >> But with this patch the rate jumps to a more reasonable 30 FPS seen by >> qcam and cam: >> >> [libcamera] kbingham@charm:~/iob/libcamera/build/gcc$ cam -c1 --metadata -C10 >> [1:46:59.040578435] [38346] INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/kbingham/iob/libcamera/build/gcc/src/ipa' to the IPA search path >> [1:46:59.048420757] [38346] INFO Camera camera_manager.cpp:327 libcamera v0.4.0+150-d0e1caa0 >> [1:46:59.074259713] [38347] WARN CameraSensor camera_sensor_legacy.cpp:616 'ov5675 24-0010': Rotation control not available, default to 0 degrees >> [1:46:59.078796773] [38347] INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data' >> [1:46:59.078822605] [38347] WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'ov5675.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml' >> [1:46:59.079110243] [38347] INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data' >> [1:46:59.087987917] [38347] INFO Pipeline pipeline_handler.cpp:618 libcamera is not installed. Loading platform configuration file from '/home/kbingham/iob/libcamera/src/libcamera/pipeline/virtual/data/virtual.yaml' >> Using camera /base/soc@0/cci@ac4c000/i2c-bus@1/camera@10 as cam0 >> [1:46:59.089522196] [38346] INFO Camera camera.cpp:1203 configuring streams: (0) 1288x972-ABGR8888 >> [1:46:59.096422962] [38347] WARN CameraSensor camera_sensor_legacy.cpp:501 'ov5675 24-0010': No sensor delays found in static properties. Assuming unverified defaults. >> [1:46:59.096786533] [38347] INFO IPASoft soft_simple.cpp:251 IPASoft: Exposure 4-2016, gain 1-15.9922 (0.149922) >> cam0: Capture 10 frames >> 6419.234779 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 5007744 >> SensorTimestamp = 6419234779000 >> 6419.268213 (29.91 fps) cam0-stream0 seq: 000001 bytesused: 5007744 >> SensorTimestamp = 6419268213000 >> 6419.301359 (30.17 fps) cam0-stream0 seq: 000002 bytesused: 5007744 >> SensorTimestamp = 6419301359000 >> 6419.334744 (29.95 fps) cam0-stream0 seq: 000003 bytesused: 5007744 >> SensorTimestamp = 6419334744000 >> 6419.368129 (29.95 fps) cam0-stream0 seq: 000004 bytesused: 5007744 >> SensorTimestamp = 6419368129000 >> 6419.401516 (29.95 fps) cam0-stream0 seq: 000005 bytesused: 5007744 >> SensorTimestamp = 6419401516000 >> 6419.434902 (29.95 fps) cam0-stream0 seq: 000006 bytesused: 5007744 >> SensorTimestamp = 6419434902000 >> 6419.468289 (29.95 fps) cam0-stream0 seq: 000007 bytesused: 5007744 >> SensorTimestamp = 6419468289000 >> 6419.501673 (29.95 fps) cam0-stream0 seq: 000008 bytesused: 5007744 >> SensorTimestamp = 6419501673000 >> 6419.535058 (29.95 fps) cam0-stream0 seq: 000009 bytesused: 5007744 >> SensorTimestamp = 6419535058000 >> >> >> But alas I note that now gstreamer (and cheese, and pipewire) now fail: >> >> [libcamera] kbingham@charm:~/iob/libcamera/build/gcc$ gst-launch-1.0 libcamerasrc ! autovideosink >> Setting pipeline to PAUSED ... >> [1:46:09.897725501] [38325] INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/kbingham/iob/libcamera/build/gcc/src/ipa' to the IPA search path >> [1:46:09.903176776] [38325] INFO Camera camera_manager.cpp:327 libcamera v0.4.0+150-d0e1caa0 >> [1:46:09.935374613] [38330] WARN CameraSensor camera_sensor_legacy.cpp:616 'ov5675 24-0010': Rotation control not available, default to 0 degrees >> [1:46:09.941677813] [38330] INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data' >> [1:46:09.941734112] [38330] WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'ov5675.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml' >> [1:46:09.942172574] [38330] INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data' >> [1:46:09.949343846] [38330] INFO Pipeline pipeline_handler.cpp:618 libcamera is not installed. Loading platform configuration file from '/home/kbingham/iob/libcamera/src/libcamera/pipeline/virtual/data/virtual.yaml' >> Pipeline is live and does not need PREROLL ... >> Pipeline is PREROLLED ... >> Setting pipeline to PLAYING ... >> New clock: GstSystemClock >> [1:46:09.950895937] [38331] ERROR Stream stream.cpp:215 Range format is ambiguous >> [1:46:09.950940309] [38331] ERROR Stream stream.cpp:215 Range format is ambiguous >> [1:46:09.950962495] [38331] ERROR Stream stream.cpp:215 Range format is ambiguous >> [1:46:09.950982129] [38331] ERROR Stream stream.cpp:215 Range format is ambiguous >> [1:46:09.951001711] [38331] ERROR Stream stream.cpp:215 Range format is ambiguous >> [1:46:09.951020564] [38331] ERROR Stream stream.cpp:215 Range format is ambiguous >> ERROR: from element /GstPipeline:pipeline0/GstLibcameraSrc:libcamerasrc0: Internal data stream error. >> Additional debug info: >> ../../src/gstreamer/gstlibcamerasrc.cpp(662): gst_libcamera_src_task_enter (): /GstPipeline:pipeline0/GstLibcameraSrc:libcamerasrc0: >> streaming stopped, reason not-negotiated (-4) >> Execution ended after 0:00:00.000438306 >> Setting pipeline to NULL ... >> Freeing pipeline ... >> >> Debugging a little, at stream.ccp:215 >> >> diff --git a/src/libcamera/stream.cpp b/src/libcamera/stream.cpp >> index 978d72752b00..22f348c2df87 100644 >> --- a/src/libcamera/stream.cpp >> +++ b/src/libcamera/stream.cpp >> @@ -213,6 +213,8 @@ std::vector<Size> StreamFormats::sizes(const PixelFormat &pixelformat) const >> if (!discrete) { >> if (ranges.size() != 1) { >> LOG(Stream, Error) << "Range format is ambiguous"; >> + for (auto range: ranges) >> + LOG(Stream, Error) << range; >> return {}; >> } >> >> Reports: >> >> [1:51:01.855863590] [40484] ERROR Stream stream.cpp:215 Range format is ambiguous >> [1:51:01.855901452] [40484] ERROR Stream stream.cpp:217 (4x2)-(1288x972)/(+4,+2) >> [1:51:01.855920305] [40484] ERROR Stream stream.cpp:217 (4x2)-(2584x1944)/(+4,+2) >> >> So we're somehow setting up the stream capabilities incorrectly here >> which breaks things... (note that ranges.size is 2, not 1?) >> >> Perhaps we're incorrectly adding the raw stream sizes to the processed stream >> sizes ? >> >> -- >> Kieran >> >> >>> Signed-off-by: Milan Zamazal <mzamazal@redhat.com> >>> --- >>> src/libcamera/pipeline/simple/simple.cpp | 72 ++++++++++++++++-------- >>> 1 file changed, 49 insertions(+), 23 deletions(-) >>> >>> diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp >>> index 0c6e0f44..92b9608b 100644 >>> --- a/src/libcamera/pipeline/simple/simple.cpp >>> +++ b/src/libcamera/pipeline/simple/simple.cpp >>> @@ -25,6 +25,7 @@ >>> #include <libcamera/base/log.h> >>> #include <libcamera/camera.h> >>> +#include <libcamera/color_space.h> >>> #include <libcamera/control_ids.h> >>> #include <libcamera/pixel_format.h> >>> #include <libcamera/request.h> >>> @@ -389,6 +390,8 @@ private: >>> const MediaPad *acquirePipeline(SimpleCameraData *data); >>> void releasePipeline(SimpleCameraData *data); >>> + void setUpFormatSizes(std::map<PixelFormat, std::vector<SizeRange>> formats); >>> + >>> std::map<const MediaEntity *, EntityData> entities_; >>> MediaDevice *converter_; >>> @@ -1191,15 +1194,56 @@ SimplePipelineHandler::generateConfiguration(Camera *camera, Span<const StreamRo >>> data->processedRequested_ = true; >>> } >>> - /* Create the formats map. */ >>> - std::map<PixelFormat, std::vector<SizeRange>> formats; >>> + /* Create the formats maps. */ >>> + std::map<PixelFormat, std::vector<SizeRange>> processedFormats; >>> + std::map<PixelFormat, std::vector<SizeRange>> rawFormats; >>> for (const SimpleCameraData::Configuration &cfg : data->configs_) >>> - if (!cfg.raw) >>> - for (PixelFormat format : cfg.outputFormats) >>> - formats[format].push_back(cfg.outputSizes); >>> + for (PixelFormat format : cfg.outputFormats) >>> + (cfg.raw ? rawFormats : processedFormats)[format].push_back(cfg.outputSizes); >>> + >>> + if (data->processedRequested_ && processedFormats.empty()) { >>> + LOG(SimplePipeline, Error) >>> + << "Processed stream requsted but no corresponding output configuration found"; >>> + return nullptr; >>> + } >>> + if (data->rawRequested_ && rawFormats.empty()) { >>> + LOG(SimplePipeline, Error) >>> + << "Raw stream requsted but no corresponding output configuration found"; >>> + return nullptr; >>> + } >>> + >>> + setUpFormatSizes(processedFormats); >>> + setUpFormatSizes(rawFormats); >>> + >>> + /* >>> + * Create the stream configurations. Take the first entry in the formats >>> + * map as the default, for lack of a better option. >>> + * >>> + * \todo Implement a better way to pick the default format >>> + */ >>> + for (StreamRole role : roles) { >>> + bool raw = (role == StreamRole::Raw); >>> + auto formats = (raw ? rawFormats : processedFormats); >>> + StreamConfiguration cfg{ StreamFormats{ formats } }; >>> + cfg.pixelFormat = formats.begin()->first; >>> + cfg.size = formats.begin()->second[0].max; >>> + if (raw) >>> + cfg.colorSpace = ColorSpace::Raw; >>> + >>> + config->addConfiguration(cfg); >>> + } >>> + >>> + config->validate(); >>> + >>> + return config; >>> +} >>> +void SimplePipelineHandler::setUpFormatSizes( >>> + std::map<PixelFormat, std::vector<SizeRange>> formats) >>> +{ >>> /* Sort the sizes and merge any consecutive overlapping ranges. */ >>> + >>> for (auto &[format, sizes] : formats) { >>> std::sort(sizes.begin(), sizes.end(), >>> [](SizeRange &a, SizeRange &b) { >>> @@ -1219,24 +1263,6 @@ SimplePipelineHandler::generateConfiguration(Camera *camera, Span<const StreamRo >>> sizes.erase(++cur, sizes.end()); >>> } >>> - >>> - /* >>> - * Create the stream configurations. Take the first entry in the formats >>> - * map as the default, for lack of a better option. >>> - * >>> - * \todo Implement a better way to pick the default format >>> - */ >>> - for ([[maybe_unused]] StreamRole role : roles) { >>> - StreamConfiguration cfg{ StreamFormats{ formats } }; >>> - cfg.pixelFormat = formats.begin()->first; >>> - cfg.size = formats.begin()->second[0].max; >>> - >>> - config->addConfiguration(cfg); >>> - } >>> - >>> - config->validate(); >>> - >>> - return config; >>> } >>> int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c) >>> -- 2.48.1 >>>
Quoting Milan Zamazal (2025-04-02 19:31:02) > Hi Kieran, > > thank you for testing. > > Kieran Bingham <kieran.bingham@ideasonboard.com> writes: > > > Quoting Milan Zamazal (2025-03-05 19:26:12) > >> When generating simple pipeline configuration, raw output configurations > >> are generated but later filtered out. Let's include processed and/or > >> raw output configurations depending on the requested stream roles. > >> > >> Raw and processed formats are handled separately. The intention is that > >> in case both raw and processed formats are requested then the raw > >> formats should be left intact to the extent possible and not be > >> influenced by the processed formats (implying that raw parameters > >> compatible with the processed output requirements must be requested by > >> the application). > >> > >> Raw output configurations are marked by setting their colorSpace to > >> ColorSpace::Raw, which is the only place there suitable for the purpose. > >> > >> This is another preparatory patch without making raw outputs working. > > > > I've identified two interesting effects which I've bisected to this > > patch: > > > > First: > > > > Without this patch on x13s: > > > > [libcamera] kbingham@charm:~/iob/libcamera/build/gcc$ cam -c1 --metadata -C10 > > [1:44:39.464209549] [37696] INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/kbingham/iob/libcamera/build/gcc/src/ipa' to the IPA search path > > [1:44:39.471858239] [37696] INFO Camera camera_manager.cpp:327 libcamera v0.4.0+149-628e8c80 > > [1:44:39.497573824] [37697] WARN CameraSensor camera_sensor_legacy.cpp:616 'ov5675 24-0010': Rotation control not available, default to 0 degrees > > [1:44:39.502155101] [37697] INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data' > > [1:44:39.502180672] [37697] WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'ov5675.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml' > > [1:44:39.502468206] [37697] INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data' > > [1:44:39.512547578] [37697] INFO Pipeline pipeline_handler.cpp:618 libcamera is not installed. Loading platform configuration file from '/home/kbingham/iob/libcamera/src/libcamera/pipeline/virtual/data/virtual.yaml' > > Using camera /base/soc@0/cci@ac4c000/i2c-bus@1/camera@10 as cam0 > > [1:44:39.514127115] [37696] INFO Camera camera.cpp:1203 configuring streams: (0) 2584x1944-ABGR8888 > > [1:44:39.520536247] [37697] WARN CameraSensor camera_sensor_legacy.cpp:501 'ov5675 24-0010': No sensor delays found in static properties. Assuming unverified defaults. > > [1:44:39.520899922] [37697] INFO IPASoft soft_simple.cpp:251 IPASoft: Exposure 4-2016, gain 1-15.9922 (0.149922) > > cam0: Capture 10 frames > > 6279.664529 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 20093184 > > SensorTimestamp = 6279664529000 > > 6279.697607 (30.23 fps) cam0-stream0 seq: 000001 bytesused: 20093184 > > SensorTimestamp = 6279697607000 > > 6279.731336 (29.65 fps) cam0-stream0 seq: 000002 bytesused: 20093184 > > SensorTimestamp = 6279731336000 > > 6279.831491 (9.98 fps) cam0-stream0 seq: 000003 bytesused: 20093184 > > SensorTimestamp = 6279831491000 > > 6279.898368 (14.95 fps) cam0-stream0 seq: 000004 bytesused: 20093184 > > SensorTimestamp = 6279898368000 > > 6279.965133 (14.98 fps) cam0-stream0 seq: 000005 bytesused: 20093184 > > SensorTimestamp = 6279965133000 > > 6280.031911 (14.97 fps) cam0-stream0 seq: 000006 bytesused: 20093184 > > SensorTimestamp = 6280031911000 > > 6280.098577 (15.00 fps) cam0-stream0 seq: 000007 bytesused: 20093184 > > SensorTimestamp = 6280098577000 > > 6280.165320 (14.98 fps) cam0-stream0 seq: 000008 bytesused: 20093184 > > SensorTimestamp = 6280165320000 > > 6280.232225 (14.95 fps) cam0-stream0 seq: 000009 bytesused: 20093184 > > SensorTimestamp = 6280232225000 > > > > (Note the 15 FPS), and pipewire/gstreamer/cheese are all 'streaming'... > > > > > > But with this patch the rate jumps to a more reasonable 30 FPS seen by > > qcam and cam: > > Well, clearly because of the lower resolution. I don't have an > explanation why the patch should make it different. Aha - I completley missed that! But yes, that's a good reason at least for the speed difference ;-) > > > [libcamera] kbingham@charm:~/iob/libcamera/build/gcc$ cam -c1 --metadata -C10 > > [1:46:59.040578435] [38346] INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/kbingham/iob/libcamera/build/gcc/src/ipa' to the IPA search path > > [1:46:59.048420757] [38346] INFO Camera camera_manager.cpp:327 libcamera v0.4.0+150-d0e1caa0 > > [1:46:59.074259713] [38347] WARN CameraSensor camera_sensor_legacy.cpp:616 'ov5675 24-0010': Rotation control not available, default to 0 degrees > > [1:46:59.078796773] [38347] INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data' > > [1:46:59.078822605] [38347] WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'ov5675.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml' > > [1:46:59.079110243] [38347] INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data' > > [1:46:59.087987917] [38347] INFO Pipeline pipeline_handler.cpp:618 libcamera is not installed. Loading platform configuration file from '/home/kbingham/iob/libcamera/src/libcamera/pipeline/virtual/data/virtual.yaml' > > Using camera /base/soc@0/cci@ac4c000/i2c-bus@1/camera@10 as cam0 > > [1:46:59.089522196] [38346] INFO Camera camera.cpp:1203 configuring streams: (0) 1288x972-ABGR8888 > > [1:46:59.096422962] [38347] WARN CameraSensor camera_sensor_legacy.cpp:501 'ov5675 24-0010': No sensor delays found in static properties. Assuming unverified defaults. > > [1:46:59.096786533] [38347] INFO IPASoft soft_simple.cpp:251 IPASoft: Exposure 4-2016, gain 1-15.9922 (0.149922) > > cam0: Capture 10 frames > > 6419.234779 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 5007744 > > SensorTimestamp = 6419234779000 > > 6419.268213 (29.91 fps) cam0-stream0 seq: 000001 bytesused: 5007744 > > SensorTimestamp = 6419268213000 > > 6419.301359 (30.17 fps) cam0-stream0 seq: 000002 bytesused: 5007744 > > SensorTimestamp = 6419301359000 > > 6419.334744 (29.95 fps) cam0-stream0 seq: 000003 bytesused: 5007744 > > SensorTimestamp = 6419334744000 > > 6419.368129 (29.95 fps) cam0-stream0 seq: 000004 bytesused: 5007744 > > SensorTimestamp = 6419368129000 > > 6419.401516 (29.95 fps) cam0-stream0 seq: 000005 bytesused: 5007744 > > SensorTimestamp = 6419401516000 > > 6419.434902 (29.95 fps) cam0-stream0 seq: 000006 bytesused: 5007744 > > SensorTimestamp = 6419434902000 > > 6419.468289 (29.95 fps) cam0-stream0 seq: 000007 bytesused: 5007744 > > SensorTimestamp = 6419468289000 > > 6419.501673 (29.95 fps) cam0-stream0 seq: 000008 bytesused: 5007744 > > SensorTimestamp = 6419501673000 > > 6419.535058 (29.95 fps) cam0-stream0 seq: 000009 bytesused: 5007744 > > SensorTimestamp = 6419535058000 > > > > > > But alas I note that now gstreamer (and cheese, and pipewire) now fail: > > > > [libcamera] kbingham@charm:~/iob/libcamera/build/gcc$ gst-launch-1.0 libcamerasrc ! autovideosink > > Setting pipeline to PAUSED ... > > [1:46:09.897725501] [38325] INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/kbingham/iob/libcamera/build/gcc/src/ipa' to the IPA search path > > [1:46:09.903176776] [38325] INFO Camera camera_manager.cpp:327 libcamera v0.4.0+150-d0e1caa0 > > [1:46:09.935374613] [38330] WARN CameraSensor camera_sensor_legacy.cpp:616 'ov5675 24-0010': Rotation control not available, default to 0 degrees > > [1:46:09.941677813] [38330] INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data' > > [1:46:09.941734112] [38330] WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'ov5675.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml' > > [1:46:09.942172574] [38330] INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data' > > [1:46:09.949343846] [38330] INFO Pipeline pipeline_handler.cpp:618 libcamera is not installed. Loading platform configuration file from '/home/kbingham/iob/libcamera/src/libcamera/pipeline/virtual/data/virtual.yaml' > > Pipeline is live and does not need PREROLL ... > > Pipeline is PREROLLED ... > > Setting pipeline to PLAYING ... > > New clock: GstSystemClock > > [1:46:09.950895937] [38331] ERROR Stream stream.cpp:215 Range format is ambiguous > > [1:46:09.950940309] [38331] ERROR Stream stream.cpp:215 Range format is ambiguous > > [1:46:09.950962495] [38331] ERROR Stream stream.cpp:215 Range format is ambiguous > > [1:46:09.950982129] [38331] ERROR Stream stream.cpp:215 Range format is ambiguous > > [1:46:09.951001711] [38331] ERROR Stream stream.cpp:215 Range format is ambiguous > > [1:46:09.951020564] [38331] ERROR Stream stream.cpp:215 Range format is ambiguous > > ERROR: from element /GstPipeline:pipeline0/GstLibcameraSrc:libcamerasrc0: Internal data stream error. > > Additional debug info: > > ../../src/gstreamer/gstlibcamerasrc.cpp(662): gst_libcamera_src_task_enter (): /GstPipeline:pipeline0/GstLibcameraSrc:libcamerasrc0: > > streaming stopped, reason not-negotiated (-4) > > Execution ended after 0:00:00.000438306 > > Setting pipeline to NULL ... > > Freeing pipeline ... > > > > Debugging a little, at stream.ccp:215 > > > > diff --git a/src/libcamera/stream.cpp b/src/libcamera/stream.cpp > > index 978d72752b00..22f348c2df87 100644 > > --- a/src/libcamera/stream.cpp > > +++ b/src/libcamera/stream.cpp > > @@ -213,6 +213,8 @@ std::vector<Size> StreamFormats::sizes(const PixelFormat &pixelformat) const > > if (!discrete) { > > if (ranges.size() != 1) { > > LOG(Stream, Error) << "Range format is ambiguous"; > > + for (auto range: ranges) > > + LOG(Stream, Error) << range; > > return {}; > > } > > > > Reports: > > > > [1:51:01.855863590] [40484] ERROR Stream stream.cpp:215 Range format is ambiguous > > [1:51:01.855901452] [40484] ERROR Stream stream.cpp:217 (4x2)-(1288x972)/(+4,+2) > > [1:51:01.855920305] [40484] ERROR Stream stream.cpp:217 (4x2)-(2584x1944)/(+4,+2) > > > > So we're somehow setting up the stream capabilities incorrectly here > > which breaks things... (note that ranges.size is 2, not 1?) > > > > Perhaps we're incorrectly adding the raw stream sizes to the processed stream > > sizes ? > > No, it's a stupid (and not immediately visible) bug: passing formats to > setUpFormatSizes by value rather than by reference, see below. > > > -- > > Kieran > > > > > >> > >> Signed-off-by: Milan Zamazal <mzamazal@redhat.com> > >> --- > >> src/libcamera/pipeline/simple/simple.cpp | 72 ++++++++++++++++-------- > >> 1 file changed, 49 insertions(+), 23 deletions(-) > >> > >> diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp > >> index 0c6e0f44..92b9608b 100644 > >> --- a/src/libcamera/pipeline/simple/simple.cpp > >> +++ b/src/libcamera/pipeline/simple/simple.cpp > >> @@ -25,6 +25,7 @@ > >> #include <libcamera/base/log.h> > >> > >> #include <libcamera/camera.h> > >> +#include <libcamera/color_space.h> > >> #include <libcamera/control_ids.h> > >> #include <libcamera/pixel_format.h> > >> #include <libcamera/request.h> > >> @@ -389,6 +390,8 @@ private: > >> const MediaPad *acquirePipeline(SimpleCameraData *data); > >> void releasePipeline(SimpleCameraData *data); > >> > >> + void setUpFormatSizes(std::map<PixelFormat, std::vector<SizeRange>> formats); > > Should be &formats. Yikes - yeah - I wouldn't have noticed that either ;-) I'll retest my side this evening too. -- Kieran > > >> + > >> std::map<const MediaEntity *, EntityData> entities_; > >> > >> MediaDevice *converter_; > >> @@ -1191,15 +1194,56 @@ SimplePipelineHandler::generateConfiguration(Camera *camera, Span<const StreamRo > >> data->processedRequested_ = true; > >> } > >> > >> - /* Create the formats map. */ > >> - std::map<PixelFormat, std::vector<SizeRange>> formats; > >> + /* Create the formats maps. */ > >> + std::map<PixelFormat, std::vector<SizeRange>> processedFormats; > >> + std::map<PixelFormat, std::vector<SizeRange>> rawFormats; > >> > >> for (const SimpleCameraData::Configuration &cfg : data->configs_) > >> - if (!cfg.raw) > >> - for (PixelFormat format : cfg.outputFormats) > >> - formats[format].push_back(cfg.outputSizes); > >> + for (PixelFormat format : cfg.outputFormats) > >> + (cfg.raw ? rawFormats : processedFormats)[format].push_back(cfg.outputSizes); > >> + > >> + if (data->processedRequested_ && processedFormats.empty()) { > >> + LOG(SimplePipeline, Error) > >> + << "Processed stream requsted but no corresponding output configuration found"; > >> + return nullptr; > >> + } > >> + if (data->rawRequested_ && rawFormats.empty()) { > >> + LOG(SimplePipeline, Error) > >> + << "Raw stream requsted but no corresponding output configuration found"; > >> + return nullptr; > >> + } > >> + > >> + setUpFormatSizes(processedFormats); > >> + setUpFormatSizes(rawFormats); > >> + > >> + /* > >> + * Create the stream configurations. Take the first entry in the formats > >> + * map as the default, for lack of a better option. > >> + * > >> + * \todo Implement a better way to pick the default format > >> + */ > >> + for (StreamRole role : roles) { > >> + bool raw = (role == StreamRole::Raw); > >> + auto formats = (raw ? rawFormats : processedFormats); > >> + StreamConfiguration cfg{ StreamFormats{ formats } }; > >> + cfg.pixelFormat = formats.begin()->first; > >> + cfg.size = formats.begin()->second[0].max; > >> + if (raw) > >> + cfg.colorSpace = ColorSpace::Raw; > >> + > >> + config->addConfiguration(cfg); > >> + } > >> + > >> + config->validate(); > >> + > >> + return config; > >> +} > >> > >> +void SimplePipelineHandler::setUpFormatSizes( > >> + std::map<PixelFormat, std::vector<SizeRange>> formats) > > &formats > > >> +{ > >> /* Sort the sizes and merge any consecutive overlapping ranges. */ > >> + > >> for (auto &[format, sizes] : formats) { > >> std::sort(sizes.begin(), sizes.end(), > >> [](SizeRange &a, SizeRange &b) { > >> @@ -1219,24 +1263,6 @@ SimplePipelineHandler::generateConfiguration(Camera *camera, Span<const StreamRo > >> > >> sizes.erase(++cur, sizes.end()); > >> } > >> - > >> - /* > >> - * Create the stream configurations. Take the first entry in the formats > >> - * map as the default, for lack of a better option. > >> - * > >> - * \todo Implement a better way to pick the default format > >> - */ > >> - for ([[maybe_unused]] StreamRole role : roles) { > >> - StreamConfiguration cfg{ StreamFormats{ formats } }; > >> - cfg.pixelFormat = formats.begin()->first; > >> - cfg.size = formats.begin()->second[0].max; > >> - > >> - config->addConfiguration(cfg); > >> - } > >> - > >> - config->validate(); > >> - > >> - return config; > >> } > >> > >> int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c) > >> -- > >> 2.48.1 > >> >
diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 0c6e0f44..92b9608b 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -25,6 +25,7 @@ #include <libcamera/base/log.h> #include <libcamera/camera.h> +#include <libcamera/color_space.h> #include <libcamera/control_ids.h> #include <libcamera/pixel_format.h> #include <libcamera/request.h> @@ -389,6 +390,8 @@ private: const MediaPad *acquirePipeline(SimpleCameraData *data); void releasePipeline(SimpleCameraData *data); + void setUpFormatSizes(std::map<PixelFormat, std::vector<SizeRange>> formats); + std::map<const MediaEntity *, EntityData> entities_; MediaDevice *converter_; @@ -1191,15 +1194,56 @@ SimplePipelineHandler::generateConfiguration(Camera *camera, Span<const StreamRo data->processedRequested_ = true; } - /* Create the formats map. */ - std::map<PixelFormat, std::vector<SizeRange>> formats; + /* Create the formats maps. */ + std::map<PixelFormat, std::vector<SizeRange>> processedFormats; + std::map<PixelFormat, std::vector<SizeRange>> rawFormats; for (const SimpleCameraData::Configuration &cfg : data->configs_) - if (!cfg.raw) - for (PixelFormat format : cfg.outputFormats) - formats[format].push_back(cfg.outputSizes); + for (PixelFormat format : cfg.outputFormats) + (cfg.raw ? rawFormats : processedFormats)[format].push_back(cfg.outputSizes); + + if (data->processedRequested_ && processedFormats.empty()) { + LOG(SimplePipeline, Error) + << "Processed stream requsted but no corresponding output configuration found"; + return nullptr; + } + if (data->rawRequested_ && rawFormats.empty()) { + LOG(SimplePipeline, Error) + << "Raw stream requsted but no corresponding output configuration found"; + return nullptr; + } + + setUpFormatSizes(processedFormats); + setUpFormatSizes(rawFormats); + + /* + * Create the stream configurations. Take the first entry in the formats + * map as the default, for lack of a better option. + * + * \todo Implement a better way to pick the default format + */ + for (StreamRole role : roles) { + bool raw = (role == StreamRole::Raw); + auto formats = (raw ? rawFormats : processedFormats); + StreamConfiguration cfg{ StreamFormats{ formats } }; + cfg.pixelFormat = formats.begin()->first; + cfg.size = formats.begin()->second[0].max; + if (raw) + cfg.colorSpace = ColorSpace::Raw; + + config->addConfiguration(cfg); + } + + config->validate(); + + return config; +} +void SimplePipelineHandler::setUpFormatSizes( + std::map<PixelFormat, std::vector<SizeRange>> formats) +{ /* Sort the sizes and merge any consecutive overlapping ranges. */ + for (auto &[format, sizes] : formats) { std::sort(sizes.begin(), sizes.end(), [](SizeRange &a, SizeRange &b) { @@ -1219,24 +1263,6 @@ SimplePipelineHandler::generateConfiguration(Camera *camera, Span<const StreamRo sizes.erase(++cur, sizes.end()); } - - /* - * Create the stream configurations. Take the first entry in the formats - * map as the default, for lack of a better option. - * - * \todo Implement a better way to pick the default format - */ - for ([[maybe_unused]] StreamRole role : roles) { - StreamConfiguration cfg{ StreamFormats{ formats } }; - cfg.pixelFormat = formats.begin()->first; - cfg.size = formats.begin()->second[0].max; - - config->addConfiguration(cfg); - } - - config->validate(); - - return config; } int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c)
When generating simple pipeline configuration, raw output configurations are generated but later filtered out. Let's include processed and/or raw output configurations depending on the requested stream roles. Raw and processed formats are handled separately. The intention is that in case both raw and processed formats are requested then the raw formats should be left intact to the extent possible and not be influenced by the processed formats (implying that raw parameters compatible with the processed output requirements must be requested by the application). Raw output configurations are marked by setting their colorSpace to ColorSpace::Raw, which is the only place there suitable for the purpose. This is another preparatory patch without making raw outputs working. Signed-off-by: Milan Zamazal <mzamazal@redhat.com> --- src/libcamera/pipeline/simple/simple.cpp | 72 ++++++++++++++++-------- 1 file changed, 49 insertions(+), 23 deletions(-)