[RFC,0/6] libcamera: simple: Enable Raw capture
mbox series

Message ID 20250716142027.236277-1-uajain@igalia.com
Headers show
Series
  • libcamera: simple: Enable Raw capture
Related show

Message

Umang Jain July 16, 2025, 2:20 p.m. UTC
RFC series to enable raw capture on simple pipeline handler with
SoftISP enabled. I had a few thoughts on design of the features
when I was reviewing Milan's series [1]. Hence, I decided to crystallise
my thoughts and posted as RFC to gather more feedback (and some real
testing on supportedDevices[]).

I have left few things to include from [1] for e.g. colorspace issues.
My goal currently is to address the core issue of raw capture with this
RFC.

Testing branch [2]
Setup: RPi 3b + IMX219
Cases tested with cam:
	- Single raw output
	- Single processed output
	- 1 Raw + 1 processed output

One can propose more use-cases as part of their testing. If they turn
out to be complex, we can probably document it as \todo, to get
addressed at a later stage.

cam Output:
===========

uajain1@uajain:~/libcamera$ ./build-soft/src/apps/cam/cam -c1 -srole=raw,width=1920,height=1080 -C5
[27:26:59.248233501] [2384]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/uajain1/libcamera/build-soft/src/ipa' to the IPA search path
[27:26:59.262004021] [2384]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+63-c33be195
[27:26:59.593703813] [2390]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
[27:26:59.594050532] [2390]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'imx219.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
[27:26:59.596620063] [2390]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
Using camera /base/soc/i2c0mux/i2c@1/imx219@10 as cam0
[27:26:59.625957459] [2384]  INFO Camera camera.cpp:1205 configuring streams: (0) 1920x1080-SRGGB10/RAW
[27:26:59.627666834] [2390]  INFO SimplePipeline simple.cpp:1471 data->useConversion_ : 0
cam0: Capture 5 frames
98819.750177 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 4147200
98819.783504 (30.01 fps) cam0-stream0 seq: 000001 bytesused: 4147200
98819.816829 (30.01 fps) cam0-stream0 seq: 000002 bytesused: 4147200
98819.850156 (30.01 fps) cam0-stream0 seq: 000003 bytesused: 4147200
98819.883484 (30.00 fps) cam0-stream0 seq: 000004 bytesused: 4147200


uajain1@uajain:~/libcamera$ ./build-soft/src/apps/cam/cam -c1 -srole=viewfinder,width=1920,height=1080 -C5
[27:27:09.663699747] [2391]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/uajain1/libcamera/build-soft/src/ipa' to the IPA search path
[27:27:09.677708965] [2391]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+63-c33be195
[27:27:09.933555736] [2392]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
[27:27:09.933821778] [2392]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'imx219.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
[27:27:09.936308080] [2392]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
Camera configuration adjusted
Using camera /base/soc/i2c0mux/i2c@1/imx219@10 as cam0
[27:27:09.965424746] [2391]  INFO Camera camera.cpp:1205 configuring streams: (0) 1916x1080-XRGB8888/Unset
[27:27:09.967118705] [2392]  INFO SimplePipeline simple.cpp:1471 data->useConversion_ : 1
[27:27:09.970895528] [2392]  INFO IPASoft soft_simple.cpp:264 IPASoft: Exposure 4-1759, gain 1-10.6667 (0.0966667)
cam0: Capture 5 frames
98830.200120 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 8277120
98830.233445 (30.01 fps) cam0-stream0 seq: 000001 bytesused: 8277120
98830.266770 (30.01 fps) cam0-stream0 seq: 000002 bytesused: 8277120
98831.033291 (1.30 fps) cam0-stream0 seq: 000025 bytesused: 8277120
98831.799810 (1.30 fps) cam0-stream0 seq: 000048 bytesused: 8277120


uajain1@uajain:~/libcamera$ ./build-soft/src/apps/cam/cam -c1 -srole=raw,width=1920,height=1080 -srole=viewfinder,width=1920,height=1080 -C10
[27:27:26.239476459] [2395]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/uajain1/libcamera/build-soft/src/ipa' to the IPA search path
[27:27:26.253694063] [2395]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+63-c33be195
[27:27:26.572276146] [2396]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
[27:27:26.572539792] [2396]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'imx219.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
[27:27:26.574856667] [2396]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
Camera configuration adjusted
Using camera /base/soc/i2c0mux/i2c@1/imx219@10 as cam0
[27:27:26.603776146] [2395]  INFO Camera camera.cpp:1205 configuring streams: (0) 1920x1080-SRGGB10/RAW (1) 1916x1080-XRGB8888/Unset
[27:27:26.605656303] [2396]  INFO SimplePipeline simple.cpp:1471 data->useConversion_ : 1
[27:27:26.609448803] [2396]  INFO IPASoft soft_simple.cpp:264 IPASoft: Exposure 4-1759, gain 1-10.6667 (0.0966667)
cam0: Capture 10 frames
98846.739621 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 4147200 cam0-stream1 seq: 000000 bytesused: 8277120
98846.772946 (30.01 fps) cam0-stream0 seq: 000001 bytesused: 4147200 cam0-stream1 seq: 000001 bytesused: 8277120
98846.806274 (30.00 fps) cam0-stream0 seq: 000002 bytesused: 4147200 cam0-stream1 seq: 000002 bytesused: 8277120
98846.839601 (30.01 fps) cam0-stream0 seq: 000003 bytesused: 4147200 cam0-stream1 seq: 000003 bytesused: 8277120
98847.539465 (1.43 fps) cam0-stream0 seq: 000024 bytesused: 4147200 cam0-stream1 seq: 000024 bytesused: 8277120
98848.206006 (1.50 fps) cam0-stream0 seq: 000044 bytesused: 4147200 cam0-stream1 seq: 000044 bytesused: 8277120
98848.872546 (1.50 fps) cam0-stream0 seq: 000064 bytesused: 4147200 cam0-stream1 seq: 000064 bytesused: 8277120
98849.539084 (1.50 fps) cam0-stream0 seq: 000084 bytesused: 4147200 cam0-stream1 seq: 000084 bytesused: 8277120
98850.205624 (1.50 fps) cam0-stream0 seq: 000104 bytesused: 4147200 cam0-stream1 seq: 000104 bytesused: 8277120
98850.872164 (1.50 fps) cam0-stream0 seq: 000124 bytesused: 4147200 cam0-stream1 seq: 000124 bytesused: 8277120

[1]: https://patchwork.libcamera.org/project/libcamera/list/?series=5281
[2]: https://gitlab.freedesktop.org/uajain/libcamera/-/commits/uajain/simple/raw

Milan Zamazal (3):
  libcamera: simple: Set the number of software ISP streams to 2
  libcamera: simple: Exclude raw configurations from output conversions
  libcamera: simple: Require metadata only when software ISP is used

Umang Jain (3):
  libcamera: simple: Support StreamRole::Raw in generateConfiguration()
  libcamera: simple: Validate raw streams if requested
  libcamera: simple: Enable capturing of raw streams

 src/libcamera/pipeline/simple/simple.cpp | 166 ++++++++++++++++++-----
 1 file changed, 131 insertions(+), 35 deletions(-)

Comments

Milan Zamazal July 17, 2025, 2:04 p.m. UTC | #1
Hi Umang,

thank you for posting the patches.  How do you suggest to proceed with
both the patch series (yours and mine) to avoid duplicate work and get
closer to merging?

From the technical standpoint, I wonder how it works without changing
buffer handling in the simple pipeline.  Do you think
https://patchwork.libcamera.org/patch/23785/ is not needed?

Umang Jain <uajain@igalia.com> writes:

> RFC series to enable raw capture on simple pipeline handler with
> SoftISP enabled. I had a few thoughts on design of the features
> when I was reviewing Milan's series [1]. Hence, I decided to crystallise
> my thoughts and posted as RFC to gather more feedback (and some real
> testing on supportedDevices[]).
>
> I have left few things to include from [1] for e.g. colorspace issues.
> My goal currently is to address the core issue of raw capture with this
> RFC.
>
> Testing branch [2]
> Setup: RPi 3b + IMX219
> Cases tested with cam:
> 	- Single raw output
> 	- Single processed output
> 	- 1 Raw + 1 processed output
>
> One can propose more use-cases as part of their testing. If they turn
> out to be complex, we can probably document it as \todo, to get
> addressed at a later stage.
>
> cam Output:
> ===========
>
> uajain1@uajain:~/libcamera$ ./build-soft/src/apps/cam/cam -c1 -srole=raw,width=1920,height=1080 -C5
> [27:26:59.248233501] [2384]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/uajain1/libcamera/build-soft/src/ipa' to the IPA search path
> [27:26:59.262004021] [2384]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+63-c33be195
> [27:26:59.593703813] [2390]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
> [27:26:59.594050532] [2390]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'imx219.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
> [27:26:59.596620063] [2390]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
> Using camera /base/soc/i2c0mux/i2c@1/imx219@10 as cam0
> [27:26:59.625957459] [2384]  INFO Camera camera.cpp:1205 configuring streams: (0) 1920x1080-SRGGB10/RAW
> [27:26:59.627666834] [2390]  INFO SimplePipeline simple.cpp:1471 data->useConversion_ : 0
> cam0: Capture 5 frames
> 98819.750177 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 4147200
> 98819.783504 (30.01 fps) cam0-stream0 seq: 000001 bytesused: 4147200
> 98819.816829 (30.01 fps) cam0-stream0 seq: 000002 bytesused: 4147200
> 98819.850156 (30.01 fps) cam0-stream0 seq: 000003 bytesused: 4147200
> 98819.883484 (30.00 fps) cam0-stream0 seq: 000004 bytesused: 4147200
>
>
> uajain1@uajain:~/libcamera$ ./build-soft/src/apps/cam/cam -c1 -srole=viewfinder,width=1920,height=1080 -C5
> [27:27:09.663699747] [2391]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/uajain1/libcamera/build-soft/src/ipa' to the IPA search path
> [27:27:09.677708965] [2391]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+63-c33be195
> [27:27:09.933555736] [2392]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
> [27:27:09.933821778] [2392]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'imx219.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
> [27:27:09.936308080] [2392]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
> Camera configuration adjusted
> Using camera /base/soc/i2c0mux/i2c@1/imx219@10 as cam0
> [27:27:09.965424746] [2391]  INFO Camera camera.cpp:1205 configuring streams: (0) 1916x1080-XRGB8888/Unset
> [27:27:09.967118705] [2392]  INFO SimplePipeline simple.cpp:1471 data->useConversion_ : 1
> [27:27:09.970895528] [2392]  INFO IPASoft soft_simple.cpp:264 IPASoft: Exposure 4-1759, gain 1-10.6667 (0.0966667)
> cam0: Capture 5 frames
> 98830.200120 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 8277120
> 98830.233445 (30.01 fps) cam0-stream0 seq: 000001 bytesused: 8277120
> 98830.266770 (30.01 fps) cam0-stream0 seq: 000002 bytesused: 8277120
> 98831.033291 (1.30 fps) cam0-stream0 seq: 000025 bytesused: 8277120
> 98831.799810 (1.30 fps) cam0-stream0 seq: 000048 bytesused: 8277120
>
>
> uajain1@uajain:~/libcamera$ ./build-soft/src/apps/cam/cam -c1 -srole=raw,width=1920,height=1080 -srole=viewfinder,width=1920,height=1080 -C10
> [27:27:26.239476459] [2395]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/uajain1/libcamera/build-soft/src/ipa' to the IPA search path
> [27:27:26.253694063] [2395]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+63-c33be195
> [27:27:26.572276146] [2396]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
> [27:27:26.572539792] [2396]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'imx219.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
> [27:27:26.574856667] [2396]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
> Camera configuration adjusted
> Using camera /base/soc/i2c0mux/i2c@1/imx219@10 as cam0
> [27:27:26.603776146] [2395]  INFO Camera camera.cpp:1205 configuring streams: (0) 1920x1080-SRGGB10/RAW (1) 1916x1080-XRGB8888/Unset
> [27:27:26.605656303] [2396]  INFO SimplePipeline simple.cpp:1471 data->useConversion_ : 1
> [27:27:26.609448803] [2396]  INFO IPASoft soft_simple.cpp:264 IPASoft: Exposure 4-1759, gain 1-10.6667 (0.0966667)
> cam0: Capture 10 frames
> 98846.739621 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 4147200 cam0-stream1 seq: 000000 bytesused: 8277120
> 98846.772946 (30.01 fps) cam0-stream0 seq: 000001 bytesused: 4147200 cam0-stream1 seq: 000001 bytesused: 8277120
> 98846.806274 (30.00 fps) cam0-stream0 seq: 000002 bytesused: 4147200 cam0-stream1 seq: 000002 bytesused: 8277120
> 98846.839601 (30.01 fps) cam0-stream0 seq: 000003 bytesused: 4147200 cam0-stream1 seq: 000003 bytesused: 8277120
> 98847.539465 (1.43 fps) cam0-stream0 seq: 000024 bytesused: 4147200 cam0-stream1 seq: 000024 bytesused: 8277120
> 98848.206006 (1.50 fps) cam0-stream0 seq: 000044 bytesused: 4147200 cam0-stream1 seq: 000044 bytesused: 8277120
> 98848.872546 (1.50 fps) cam0-stream0 seq: 000064 bytesused: 4147200 cam0-stream1 seq: 000064 bytesused: 8277120
> 98849.539084 (1.50 fps) cam0-stream0 seq: 000084 bytesused: 4147200 cam0-stream1 seq: 000084 bytesused: 8277120
> 98850.205624 (1.50 fps) cam0-stream0 seq: 000104 bytesused: 4147200 cam0-stream1 seq: 000104 bytesused: 8277120
> 98850.872164 (1.50 fps) cam0-stream0 seq: 000124 bytesused: 4147200 cam0-stream1 seq: 000124 bytesused: 8277120
>
> [1]: https://patchwork.libcamera.org/project/libcamera/list/?series=5281
> [2]: https://gitlab.freedesktop.org/uajain/libcamera/-/commits/uajain/simple/raw
>
> Milan Zamazal (3):
>   libcamera: simple: Set the number of software ISP streams to 2
>   libcamera: simple: Exclude raw configurations from output conversions
>   libcamera: simple: Require metadata only when software ISP is used
>
> Umang Jain (3):
>   libcamera: simple: Support StreamRole::Raw in generateConfiguration()
>   libcamera: simple: Validate raw streams if requested
>   libcamera: simple: Enable capturing of raw streams
>
>  src/libcamera/pipeline/simple/simple.cpp | 166 ++++++++++++++++++-----
>  1 file changed, 131 insertions(+), 35 deletions(-)
Umang Jain July 18, 2025, 4 a.m. UTC | #2
Hi Milan,

On Thu, Jul 17, 2025 at 04:04:58PM +0200, Milan Zamazal wrote:
> Hi Umang,
> 
> thank you for posting the patches.  How do you suggest to proceed with
> both the patch series (yours and mine) to avoid duplicate work and get
> closer to merging?

Good question. I would suggest to test this series and see if it
addresses use-cases for the raw capture scenarios (maybe, I missed
something/corner-cases).

We can cross review each other patches for starters. I've picked 3 of
your patches and based my patches on that. Going forward, either of us
can collect the relevant reviewed patches and post a single series as
collaborative efforts.

I would also suggest to split off parallel patches (for e.g. colorspace
patches from your series) and see if we can progress them independently.
These are my thoughts, if you want to take an alternate approach - I am
willing to hear it as well!

> 
> From the technical standpoint, I wonder how it works without changing
> buffer handling in the simple pipeline.  Do you think
> https://patchwork.libcamera.org/patch/23785/ is not needed?

It's changed in 6/6.

> 
> Umang Jain <uajain@igalia.com> writes:
> 
> > RFC series to enable raw capture on simple pipeline handler with
> > SoftISP enabled. I had a few thoughts on design of the features
> > when I was reviewing Milan's series [1]. Hence, I decided to crystallise
> > my thoughts and posted as RFC to gather more feedback (and some real
> > testing on supportedDevices[]).
> >
> > I have left few things to include from [1] for e.g. colorspace issues.
> > My goal currently is to address the core issue of raw capture with this
> > RFC.
> >
> > Testing branch [2]
> > Setup: RPi 3b + IMX219
> > Cases tested with cam:
> > 	- Single raw output
> > 	- Single processed output
> > 	- 1 Raw + 1 processed output
> >
> > One can propose more use-cases as part of their testing. If they turn
> > out to be complex, we can probably document it as \todo, to get
> > addressed at a later stage.
> >
> > cam Output:
> > ===========
> >
> > uajain1@uajain:~/libcamera$ ./build-soft/src/apps/cam/cam -c1 -srole=raw,width=1920,height=1080 -C5
> > [27:26:59.248233501] [2384]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/uajain1/libcamera/build-soft/src/ipa' to the IPA search path
> > [27:26:59.262004021] [2384]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+63-c33be195
> > [27:26:59.593703813] [2390]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
> > [27:26:59.594050532] [2390]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'imx219.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
> > [27:26:59.596620063] [2390]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
> > Using camera /base/soc/i2c0mux/i2c@1/imx219@10 as cam0
> > [27:26:59.625957459] [2384]  INFO Camera camera.cpp:1205 configuring streams: (0) 1920x1080-SRGGB10/RAW
> > [27:26:59.627666834] [2390]  INFO SimplePipeline simple.cpp:1471 data->useConversion_ : 0
> > cam0: Capture 5 frames
> > 98819.750177 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 4147200
> > 98819.783504 (30.01 fps) cam0-stream0 seq: 000001 bytesused: 4147200
> > 98819.816829 (30.01 fps) cam0-stream0 seq: 000002 bytesused: 4147200
> > 98819.850156 (30.01 fps) cam0-stream0 seq: 000003 bytesused: 4147200
> > 98819.883484 (30.00 fps) cam0-stream0 seq: 000004 bytesused: 4147200
> >
> >
> > uajain1@uajain:~/libcamera$ ./build-soft/src/apps/cam/cam -c1 -srole=viewfinder,width=1920,height=1080 -C5
> > [27:27:09.663699747] [2391]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/uajain1/libcamera/build-soft/src/ipa' to the IPA search path
> > [27:27:09.677708965] [2391]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+63-c33be195
> > [27:27:09.933555736] [2392]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
> > [27:27:09.933821778] [2392]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'imx219.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
> > [27:27:09.936308080] [2392]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
> > Camera configuration adjusted
> > Using camera /base/soc/i2c0mux/i2c@1/imx219@10 as cam0
> > [27:27:09.965424746] [2391]  INFO Camera camera.cpp:1205 configuring streams: (0) 1916x1080-XRGB8888/Unset
> > [27:27:09.967118705] [2392]  INFO SimplePipeline simple.cpp:1471 data->useConversion_ : 1
> > [27:27:09.970895528] [2392]  INFO IPASoft soft_simple.cpp:264 IPASoft: Exposure 4-1759, gain 1-10.6667 (0.0966667)
> > cam0: Capture 5 frames
> > 98830.200120 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 8277120
> > 98830.233445 (30.01 fps) cam0-stream0 seq: 000001 bytesused: 8277120
> > 98830.266770 (30.01 fps) cam0-stream0 seq: 000002 bytesused: 8277120
> > 98831.033291 (1.30 fps) cam0-stream0 seq: 000025 bytesused: 8277120
> > 98831.799810 (1.30 fps) cam0-stream0 seq: 000048 bytesused: 8277120
> >
> >
> > uajain1@uajain:~/libcamera$ ./build-soft/src/apps/cam/cam -c1 -srole=raw,width=1920,height=1080 -srole=viewfinder,width=1920,height=1080 -C10
> > [27:27:26.239476459] [2395]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/uajain1/libcamera/build-soft/src/ipa' to the IPA search path
> > [27:27:26.253694063] [2395]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+63-c33be195
> > [27:27:26.572276146] [2396]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
> > [27:27:26.572539792] [2396]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'imx219.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
> > [27:27:26.574856667] [2396]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
> > Camera configuration adjusted
> > Using camera /base/soc/i2c0mux/i2c@1/imx219@10 as cam0
> > [27:27:26.603776146] [2395]  INFO Camera camera.cpp:1205 configuring streams: (0) 1920x1080-SRGGB10/RAW (1) 1916x1080-XRGB8888/Unset
> > [27:27:26.605656303] [2396]  INFO SimplePipeline simple.cpp:1471 data->useConversion_ : 1
> > [27:27:26.609448803] [2396]  INFO IPASoft soft_simple.cpp:264 IPASoft: Exposure 4-1759, gain 1-10.6667 (0.0966667)
> > cam0: Capture 10 frames
> > 98846.739621 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 4147200 cam0-stream1 seq: 000000 bytesused: 8277120
> > 98846.772946 (30.01 fps) cam0-stream0 seq: 000001 bytesused: 4147200 cam0-stream1 seq: 000001 bytesused: 8277120
> > 98846.806274 (30.00 fps) cam0-stream0 seq: 000002 bytesused: 4147200 cam0-stream1 seq: 000002 bytesused: 8277120
> > 98846.839601 (30.01 fps) cam0-stream0 seq: 000003 bytesused: 4147200 cam0-stream1 seq: 000003 bytesused: 8277120
> > 98847.539465 (1.43 fps) cam0-stream0 seq: 000024 bytesused: 4147200 cam0-stream1 seq: 000024 bytesused: 8277120
> > 98848.206006 (1.50 fps) cam0-stream0 seq: 000044 bytesused: 4147200 cam0-stream1 seq: 000044 bytesused: 8277120
> > 98848.872546 (1.50 fps) cam0-stream0 seq: 000064 bytesused: 4147200 cam0-stream1 seq: 000064 bytesused: 8277120
> > 98849.539084 (1.50 fps) cam0-stream0 seq: 000084 bytesused: 4147200 cam0-stream1 seq: 000084 bytesused: 8277120
> > 98850.205624 (1.50 fps) cam0-stream0 seq: 000104 bytesused: 4147200 cam0-stream1 seq: 000104 bytesused: 8277120
> > 98850.872164 (1.50 fps) cam0-stream0 seq: 000124 bytesused: 4147200 cam0-stream1 seq: 000124 bytesused: 8277120
> >
> > [1]: https://patchwork.libcamera.org/project/libcamera/list/?series=5281
> > [2]: https://gitlab.freedesktop.org/uajain/libcamera/-/commits/uajain/simple/raw
> >
> > Milan Zamazal (3):
> >   libcamera: simple: Set the number of software ISP streams to 2
> >   libcamera: simple: Exclude raw configurations from output conversions
> >   libcamera: simple: Require metadata only when software ISP is used
> >
> > Umang Jain (3):
> >   libcamera: simple: Support StreamRole::Raw in generateConfiguration()
> >   libcamera: simple: Validate raw streams if requested
> >   libcamera: simple: Enable capturing of raw streams
> >
> >  src/libcamera/pipeline/simple/simple.cpp | 166 ++++++++++++++++++-----
> >  1 file changed, 131 insertions(+), 35 deletions(-)
>
Milan Zamazal July 18, 2025, 11:35 a.m. UTC | #3
Umang Jain <uajain@igalia.com> writes:

> Hi Milan,
>
> On Thu, Jul 17, 2025 at 04:04:58PM +0200, Milan Zamazal wrote:
>> Hi Umang,
>> 
>> thank you for posting the patches.  How do you suggest to proceed with
>> both the patch series (yours and mine) to avoid duplicate work and get
>> closer to merging?
>
> Good question. I would suggest to test this series and see if it
> addresses use-cases for the raw capture scenarios (maybe, I missed
> something/corner-cases).
>
> We can cross review each other patches for starters. I've picked 3 of
> your patches and based my patches on that. Going forward, either of us
> can collect the relevant reviewed patches and post a single series as
> collaborative efforts.
>
> I would also suggest to split off parallel patches (for e.g. colorspace
> patches from your series) and see if we can progress them independently.
> These are my thoughts, if you want to take an alternate approach - I am
> willing to hear it as well!

OK for me.

>> From the technical standpoint, I wonder how it works without changing
>> buffer handling in the simple pipeline.  Do you think
>> https://patchwork.libcamera.org/patch/23785/ is not needed?
>
> It's changed in 6/6.

Ah, sorry, I was looking at patches from patchwork and that one was
omitted there.

>> Umang Jain <uajain@igalia.com> writes:
>> 
>> > RFC series to enable raw capture on simple pipeline handler with
>> > SoftISP enabled. I had a few thoughts on design of the features
>> > when I was reviewing Milan's series [1]. Hence, I decided to crystallise
>> > my thoughts and posted as RFC to gather more feedback (and some real
>> > testing on supportedDevices[]).
>> >
>> > I have left few things to include from [1] for e.g. colorspace issues.
>> > My goal currently is to address the core issue of raw capture with this
>> > RFC.
>> >
>> > Testing branch [2]
>> > Setup: RPi 3b + IMX219
>> > Cases tested with cam:
>> > 	- Single raw output
>> > 	- Single processed output
>> > 	- 1 Raw + 1 processed output
>> >
>> > One can propose more use-cases as part of their testing. If they turn
>> > out to be complex, we can probably document it as \todo, to get
>> > addressed at a later stage.
>> >
>> > cam Output:
>> > ===========
>> >
>> > uajain1@uajain:~/libcamera$ ./build-soft/src/apps/cam/cam -c1 -srole=raw,width=1920,height=1080 -C5
>> > [27:26:59.248233501] [2384]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/uajain1/libcamera/build-soft/src/ipa' to the IPA search path
>> > [27:26:59.262004021] [2384]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+63-c33be195
>> > [27:26:59.593703813] [2390]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
>> > [27:26:59.594050532] [2390]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'imx219.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
>> > [27:26:59.596620063] [2390]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
>> > Using camera /base/soc/i2c0mux/i2c@1/imx219@10 as cam0
>> > [27:26:59.625957459] [2384]  INFO Camera camera.cpp:1205 configuring streams: (0) 1920x1080-SRGGB10/RAW
>> > [27:26:59.627666834] [2390]  INFO SimplePipeline simple.cpp:1471 data->useConversion_ : 0
>> > cam0: Capture 5 frames
>> > 98819.750177 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 4147200
>> > 98819.783504 (30.01 fps) cam0-stream0 seq: 000001 bytesused: 4147200
>> > 98819.816829 (30.01 fps) cam0-stream0 seq: 000002 bytesused: 4147200
>> > 98819.850156 (30.01 fps) cam0-stream0 seq: 000003 bytesused: 4147200
>> > 98819.883484 (30.00 fps) cam0-stream0 seq: 000004 bytesused: 4147200
>> >
>> >
>> > uajain1@uajain:~/libcamera$ ./build-soft/src/apps/cam/cam -c1 -srole=viewfinder,width=1920,height=1080 -C5
>> > [27:27:09.663699747] [2391]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/uajain1/libcamera/build-soft/src/ipa' to the IPA search path
>> > [27:27:09.677708965] [2391]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+63-c33be195
>> > [27:27:09.933555736] [2392]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
>> > [27:27:09.933821778] [2392]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'imx219.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
>> > [27:27:09.936308080] [2392]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
>> > Camera configuration adjusted
>> > Using camera /base/soc/i2c0mux/i2c@1/imx219@10 as cam0
>> > [27:27:09.965424746] [2391]  INFO Camera camera.cpp:1205 configuring streams: (0) 1916x1080-XRGB8888/Unset
>> > [27:27:09.967118705] [2392]  INFO SimplePipeline simple.cpp:1471 data->useConversion_ : 1
>> > [27:27:09.970895528] [2392]  INFO IPASoft soft_simple.cpp:264 IPASoft: Exposure 4-1759, gain 1-10.6667 (0.0966667)
>> > cam0: Capture 5 frames
>> > 98830.200120 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 8277120
>> > 98830.233445 (30.01 fps) cam0-stream0 seq: 000001 bytesused: 8277120
>> > 98830.266770 (30.01 fps) cam0-stream0 seq: 000002 bytesused: 8277120
>> > 98831.033291 (1.30 fps) cam0-stream0 seq: 000025 bytesused: 8277120
>> > 98831.799810 (1.30 fps) cam0-stream0 seq: 000048 bytesused: 8277120
>> >
>> >
>> > uajain1@uajain:~/libcamera$ ./build-soft/src/apps/cam/cam -c1 -srole=raw,width=1920,height=1080 -srole=viewfinder,width=1920,height=1080 -C10
>> > [27:27:26.239476459] [2395]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/uajain1/libcamera/build-soft/src/ipa' to the IPA search path
>> > [27:27:26.253694063] [2395]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+63-c33be195
>> > [27:27:26.572276146] [2396]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
>> > [27:27:26.572539792] [2396]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'imx219.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
>> > [27:27:26.574856667] [2396]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
>> > Camera configuration adjusted
>> > Using camera /base/soc/i2c0mux/i2c@1/imx219@10 as cam0
>> > [27:27:26.603776146] [2395]  INFO Camera camera.cpp:1205 configuring streams: (0) 1920x1080-SRGGB10/RAW (1) 1916x1080-XRGB8888/Unset
>> > [27:27:26.605656303] [2396]  INFO SimplePipeline simple.cpp:1471 data->useConversion_ : 1
>> > [27:27:26.609448803] [2396]  INFO IPASoft soft_simple.cpp:264 IPASoft: Exposure 4-1759, gain 1-10.6667 (0.0966667)
>> > cam0: Capture 10 frames
>> > 98846.739621 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 4147200 cam0-stream1 seq: 000000 bytesused: 8277120
>> > 98846.772946 (30.01 fps) cam0-stream0 seq: 000001 bytesused: 4147200 cam0-stream1 seq: 000001 bytesused: 8277120
>> > 98846.806274 (30.00 fps) cam0-stream0 seq: 000002 bytesused: 4147200 cam0-stream1 seq: 000002 bytesused: 8277120
>> > 98846.839601 (30.01 fps) cam0-stream0 seq: 000003 bytesused: 4147200 cam0-stream1 seq: 000003 bytesused: 8277120
>> > 98847.539465 (1.43 fps) cam0-stream0 seq: 000024 bytesused: 4147200 cam0-stream1 seq: 000024 bytesused: 8277120
>> > 98848.206006 (1.50 fps) cam0-stream0 seq: 000044 bytesused: 4147200 cam0-stream1 seq: 000044 bytesused: 8277120
>> > 98848.872546 (1.50 fps) cam0-stream0 seq: 000064 bytesused: 4147200 cam0-stream1 seq: 000064 bytesused: 8277120
>> > 98849.539084 (1.50 fps) cam0-stream0 seq: 000084 bytesused: 4147200 cam0-stream1 seq: 000084 bytesused: 8277120
>> > 98850.205624 (1.50 fps) cam0-stream0 seq: 000104 bytesused: 4147200 cam0-stream1 seq: 000104 bytesused: 8277120
>> > 98850.872164 (1.50 fps) cam0-stream0 seq: 000124 bytesused: 4147200 cam0-stream1 seq: 000124 bytesused: 8277120
>> >
>> > [1]: https://patchwork.libcamera.org/project/libcamera/list/?series=5281
>> > [2]: https://gitlab.freedesktop.org/uajain/libcamera/-/commits/uajain/simple/raw
>> >
>> > Milan Zamazal (3):
>> >   libcamera: simple: Set the number of software ISP streams to 2
>> >   libcamera: simple: Exclude raw configurations from output conversions
>> >   libcamera: simple: Require metadata only when software ISP is used
>> >
>> > Umang Jain (3):
>> >   libcamera: simple: Support StreamRole::Raw in generateConfiguration()
>> >   libcamera: simple: Validate raw streams if requested
>> >   libcamera: simple: Enable capturing of raw streams
>> >
>> >  src/libcamera/pipeline/simple/simple.cpp | 166 ++++++++++++++++++-----
>> >  1 file changed, 131 insertions(+), 35 deletions(-)
>>
Kieran Bingham July 20, 2025, 3:45 p.m. UTC | #4
Hi All,

I'm trying to understand what the plan is on these multiple RAW series
now.

Quoting Milan Zamazal (2025-07-18 12:35:04)
> Umang Jain <uajain@igalia.com> writes:
> 
> > Hi Milan,
> >
> > On Thu, Jul 17, 2025 at 04:04:58PM +0200, Milan Zamazal wrote:
> >> Hi Umang,
> >> 
> >> thank you for posting the patches.  How do you suggest to proceed with
> >> both the patch series (yours and mine) to avoid duplicate work and get
> >> closer to merging?
> >
> > Good question. I would suggest to test this series and see if it
> > addresses use-cases for the raw capture scenarios (maybe, I missed
> > something/corner-cases).
> >
> > We can cross review each other patches for starters. I've picked 3 of
> > your patches and based my patches on that. Going forward, either of us
> > can collect the relevant reviewed patches and post a single series as
> > collaborative efforts.
> >
> > I would also suggest to split off parallel patches (for e.g. colorspace
> > patches from your series) and see if we can progress them independently.
> > These are my thoughts, if you want to take an alternate approach - I am
> > willing to hear it as well!
> 
> OK for me.

There's some curious differences between the two versions now ? On x13s
- if I run Milans' branch I get a smaller resolution by default (and
better performance because of that I guess) and also the RAW streams are
filtering better? 

kbingham@charm:~/iob/libcamera$ ./build/gcc/src/apps/cam/cam -c1 -I -srole=raw
[1:18:54.693658992] [15767]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/kbingham/iob/libcamera/build/gcc/src/ipa' to the IPA search path
[1:18:54.698672581] [15767]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+82-1f6b50aa
[1:18:54.736613408] [15768]  WARN CameraSensor camera_sensor_legacy.cpp:616 'ov5675 24-0010': Rotation control not available, default to 0 degrees
[1:18:54.736743817] [15768]  WARN CameraSensor camera_sensor_legacy.cpp:501 'ov5675 24-0010': No sensor delays found in static properties. Assuming unverified defaults.
[1:18:54.743968255] [15768]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data'
[1:18:54.744016950] [15768]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'ov5675.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
[1:18:54.744431351] [15768]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data'
[1:18:54.753486194] [15768]  INFO Pipeline pipeline_handler.cpp:646 libcamera is not installed. Loading platform configuration file from '/home/kbingham/iob/libcamera/src/libcamera/pipeline/virtual/data/virtual.yaml'
Using camera /base/soc@0/cci@ac4c000/i2c-bus@1/camera@10 as cam0
0: 1296x972-SGRBG10_CSI2P/RAW
 * Pixelformat: SGRBG10_CSI2P (1296x972)-(2592x1944)/(+0,+0)
  - 1296x972
  - 2592x1944


vs Umangs branch:

kbingham@charm:~/iob/libcamera$ ./build/gcc/src/apps/cam/cam -c1 -I -srole=raw | grep -i pixelformat
[1:20:02.718240596] [16578]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/kbingham/iob/libcamera/build/gcc/src/ipa' to the IPA search path
[1:20:02.725663876] [16578]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+81-d9b7a3bf
[1:20:02.762329159] [16580]  WARN CameraSensor camera_sensor_legacy.cpp:616 'ov5675 24-0010': Rotation control not available, default to 0 degrees
[1:20:02.762467588] [16580]  WARN CameraSensor camera_sensor_legacy.cpp:501 'ov5675 24-0010': No sensor delays found in static properties. Assuming unverified defaults.
[1:20:02.769917481] [16580]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data'
[1:20:02.769956802] [16580]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'ov5675.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
[1:20:02.770373234] [16580]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data'
[1:20:02.780852414] [16580]  INFO Pipeline pipeline_handler.cpp:646 libcamera is not installed. Loading platform configuration file from '/home/kbingham/iob/libcamera/src/libcamera/pipeline/virtual/data/virtual.yaml'
 * Pixelformat: SGRBG10_CSI2P (4x2)-(2584x1944)/(+4,+2)
 * Pixelformat: ABGR8888 (4x2)-(2584x1944)/(+4,+2)
 * Pixelformat: XBGR8888 (4x2)-(2584x1944)/(+4,+2)
 * Pixelformat: BGR888 (4x2)-(2584x1944)/(+4,+2)
 * Pixelformat: RGB888 (4x2)-(2584x1944)/(+4,+2)
 * Pixelformat: ARGB8888 (4x2)-(2584x1944)/(+4,+2)
 * Pixelformat: XRGB8888 (4x2)-(2584x1944)/(+4,+2)

So I'm afraid I think Umangs' branch is doing something wrong at the
moment?

I would like to target merging this RAW support by the end of this week
prior to the next release point - could you let me know what the best
approach is please ?

Robert, could you test on your devices please ? Ultimately the RAW
support is the important step we need to enable to start being able to
capture image cailbration images and tuning for devices...

Once this is merged we can work on enabling manual controls  for
exposure and gain which will be the next requirements for tuning.

--
Kieran



> 
> >> From the technical standpoint, I wonder how it works without changing
> >> buffer handling in the simple pipeline.  Do you think
> >> https://patchwork.libcamera.org/patch/23785/ is not needed?
> >
> > It's changed in 6/6.
> 
> Ah, sorry, I was looking at patches from patchwork and that one was
> omitted there.
> 
> >> Umang Jain <uajain@igalia.com> writes:
> >> 
> >> > RFC series to enable raw capture on simple pipeline handler with
> >> > SoftISP enabled. I had a few thoughts on design of the features
> >> > when I was reviewing Milan's series [1]. Hence, I decided to crystallise
> >> > my thoughts and posted as RFC to gather more feedback (and some real
> >> > testing on supportedDevices[]).
> >> >
> >> > I have left few things to include from [1] for e.g. colorspace issues.
> >> > My goal currently is to address the core issue of raw capture with this
> >> > RFC.
> >> >
> >> > Testing branch [2]
> >> > Setup: RPi 3b + IMX219
> >> > Cases tested with cam:
> >> >    - Single raw output
> >> >    - Single processed output
> >> >    - 1 Raw + 1 processed output
> >> >
> >> > One can propose more use-cases as part of their testing. If they turn
> >> > out to be complex, we can probably document it as \todo, to get
> >> > addressed at a later stage.
> >> >
> >> > cam Output:
> >> > ===========
> >> >
> >> > uajain1@uajain:~/libcamera$ ./build-soft/src/apps/cam/cam -c1 -srole=raw,width=1920,height=1080 -C5
> >> > [27:26:59.248233501] [2384]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/uajain1/libcamera/build-soft/src/ipa' to the IPA search path
> >> > [27:26:59.262004021] [2384]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+63-c33be195
> >> > [27:26:59.593703813] [2390]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
> >> > [27:26:59.594050532] [2390]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'imx219.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
> >> > [27:26:59.596620063] [2390]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
> >> > Using camera /base/soc/i2c0mux/i2c@1/imx219@10 as cam0
> >> > [27:26:59.625957459] [2384]  INFO Camera camera.cpp:1205 configuring streams: (0) 1920x1080-SRGGB10/RAW
> >> > [27:26:59.627666834] [2390]  INFO SimplePipeline simple.cpp:1471 data->useConversion_ : 0
> >> > cam0: Capture 5 frames
> >> > 98819.750177 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 4147200
> >> > 98819.783504 (30.01 fps) cam0-stream0 seq: 000001 bytesused: 4147200
> >> > 98819.816829 (30.01 fps) cam0-stream0 seq: 000002 bytesused: 4147200
> >> > 98819.850156 (30.01 fps) cam0-stream0 seq: 000003 bytesused: 4147200
> >> > 98819.883484 (30.00 fps) cam0-stream0 seq: 000004 bytesused: 4147200
> >> >
> >> >
> >> > uajain1@uajain:~/libcamera$ ./build-soft/src/apps/cam/cam -c1 -srole=viewfinder,width=1920,height=1080 -C5
> >> > [27:27:09.663699747] [2391]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/uajain1/libcamera/build-soft/src/ipa' to the IPA search path
> >> > [27:27:09.677708965] [2391]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+63-c33be195
> >> > [27:27:09.933555736] [2392]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
> >> > [27:27:09.933821778] [2392]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'imx219.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
> >> > [27:27:09.936308080] [2392]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
> >> > Camera configuration adjusted
> >> > Using camera /base/soc/i2c0mux/i2c@1/imx219@10 as cam0
> >> > [27:27:09.965424746] [2391]  INFO Camera camera.cpp:1205 configuring streams: (0) 1916x1080-XRGB8888/Unset
> >> > [27:27:09.967118705] [2392]  INFO SimplePipeline simple.cpp:1471 data->useConversion_ : 1
> >> > [27:27:09.970895528] [2392]  INFO IPASoft soft_simple.cpp:264 IPASoft: Exposure 4-1759, gain 1-10.6667 (0.0966667)
> >> > cam0: Capture 5 frames
> >> > 98830.200120 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 8277120
> >> > 98830.233445 (30.01 fps) cam0-stream0 seq: 000001 bytesused: 8277120
> >> > 98830.266770 (30.01 fps) cam0-stream0 seq: 000002 bytesused: 8277120
> >> > 98831.033291 (1.30 fps) cam0-stream0 seq: 000025 bytesused: 8277120
> >> > 98831.799810 (1.30 fps) cam0-stream0 seq: 000048 bytesused: 8277120
> >> >
> >> >
> >> > uajain1@uajain:~/libcamera$ ./build-soft/src/apps/cam/cam -c1 -srole=raw,width=1920,height=1080 -srole=viewfinder,width=1920,height=1080 -C10
> >> > [27:27:26.239476459] [2395]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/uajain1/libcamera/build-soft/src/ipa' to the IPA search path
> >> > [27:27:26.253694063] [2395]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+63-c33be195
> >> > [27:27:26.572276146] [2396]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
> >> > [27:27:26.572539792] [2396]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'imx219.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
> >> > [27:27:26.574856667] [2396]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
> >> > Camera configuration adjusted
> >> > Using camera /base/soc/i2c0mux/i2c@1/imx219@10 as cam0
> >> > [27:27:26.603776146] [2395]  INFO Camera camera.cpp:1205 configuring streams: (0) 1920x1080-SRGGB10/RAW (1) 1916x1080-XRGB8888/Unset
> >> > [27:27:26.605656303] [2396]  INFO SimplePipeline simple.cpp:1471 data->useConversion_ : 1
> >> > [27:27:26.609448803] [2396]  INFO IPASoft soft_simple.cpp:264 IPASoft: Exposure 4-1759, gain 1-10.6667 (0.0966667)
> >> > cam0: Capture 10 frames
> >> > 98846.739621 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 4147200 cam0-stream1 seq: 000000 bytesused: 8277120
> >> > 98846.772946 (30.01 fps) cam0-stream0 seq: 000001 bytesused: 4147200 cam0-stream1 seq: 000001 bytesused: 8277120
> >> > 98846.806274 (30.00 fps) cam0-stream0 seq: 000002 bytesused: 4147200 cam0-stream1 seq: 000002 bytesused: 8277120
> >> > 98846.839601 (30.01 fps) cam0-stream0 seq: 000003 bytesused: 4147200 cam0-stream1 seq: 000003 bytesused: 8277120
> >> > 98847.539465 (1.43 fps) cam0-stream0 seq: 000024 bytesused: 4147200 cam0-stream1 seq: 000024 bytesused: 8277120
> >> > 98848.206006 (1.50 fps) cam0-stream0 seq: 000044 bytesused: 4147200 cam0-stream1 seq: 000044 bytesused: 8277120
> >> > 98848.872546 (1.50 fps) cam0-stream0 seq: 000064 bytesused: 4147200 cam0-stream1 seq: 000064 bytesused: 8277120
> >> > 98849.539084 (1.50 fps) cam0-stream0 seq: 000084 bytesused: 4147200 cam0-stream1 seq: 000084 bytesused: 8277120
> >> > 98850.205624 (1.50 fps) cam0-stream0 seq: 000104 bytesused: 4147200 cam0-stream1 seq: 000104 bytesused: 8277120
> >> > 98850.872164 (1.50 fps) cam0-stream0 seq: 000124 bytesused: 4147200 cam0-stream1 seq: 000124 bytesused: 8277120
> >> >
> >> > [1]: https://patchwork.libcamera.org/project/libcamera/list/?series=5281
> >> > [2]: https://gitlab.freedesktop.org/uajain/libcamera/-/commits/uajain/simple/raw
> >> >
> >> > Milan Zamazal (3):
> >> >   libcamera: simple: Set the number of software ISP streams to 2
> >> >   libcamera: simple: Exclude raw configurations from output conversions
> >> >   libcamera: simple: Require metadata only when software ISP is used
> >> >
> >> > Umang Jain (3):
> >> >   libcamera: simple: Support StreamRole::Raw in generateConfiguration()
> >> >   libcamera: simple: Validate raw streams if requested
> >> >   libcamera: simple: Enable capturing of raw streams
> >> >
> >> >  src/libcamera/pipeline/simple/simple.cpp | 166 ++++++++++++++++++-----
> >> >  1 file changed, 131 insertions(+), 35 deletions(-)
> >> 
>
Umang Jain July 21, 2025, 4:17 a.m. UTC | #5
Hi Kieran,

On Sun, Jul 20, 2025 at 04:45:39PM +0100, Kieran Bingham wrote:
> Hi All,
> 
> I'm trying to understand what the plan is on these multiple RAW series
> now.

I think I stated out a plan already and Milan seems to agree. There is a
need for cross-review/test cycles and one of us then need to take the lead
towards merging the series.
> 
> Quoting Milan Zamazal (2025-07-18 12:35:04)
> > Umang Jain <uajain@igalia.com> writes:
> > 
> > > Hi Milan,
> > >
> > > On Thu, Jul 17, 2025 at 04:04:58PM +0200, Milan Zamazal wrote:
> > >> Hi Umang,
> > >> 
> > >> thank you for posting the patches.  How do you suggest to proceed with
> > >> both the patch series (yours and mine) to avoid duplicate work and get
> > >> closer to merging?
> > >
> > > Good question. I would suggest to test this series and see if it
> > > addresses use-cases for the raw capture scenarios (maybe, I missed
> > > something/corner-cases).
> > >
> > > We can cross review each other patches for starters. I've picked 3 of
> > > your patches and based my patches on that. Going forward, either of us
> > > can collect the relevant reviewed patches and post a single series as
> > > collaborative efforts.
> > >
> > > I would also suggest to split off parallel patches (for e.g. colorspace
> > > patches from your series) and see if we can progress them independently.
> > > These are my thoughts, if you want to take an alternate approach - I am
> > > willing to hear it as well!
> > 
> > OK for me.
> 
> There's some curious differences between the two versions now ? On x13s
> - if I run Milans' branch I get a smaller resolution by default (and
> better performance because of that I guess) and also the RAW streams are
> filtering better? 

If I take a look at current master branch - the status-quo I think is
that the filtering needs/defaults pickups needs work:

[~/src/libcamera]$ git grep \todo | grep -i 'pipeline/simple'
src/libcamera/pipeline/simple/simple.cpp:	 * \todo The sensor timestamp should be better estimated by connecting
src/libcamera/pipeline/simple/simple.cpp:	 * \todo Applying controls directly not only increases the risk of
src/libcamera/pipeline/simple/simple.cpp:	 * \todo Pick the best sensor output media bus format when the
src/libcamera/pipeline/simple/simple.cpp:		/* \todo Create a libcamera core class to group format and size */
src/libcamera/pipeline/simple/simple.cpp:	 * \todo Implement a better way to pick the default format

Currently, it's just picking up the first entry in a given map etc, so I
am not surprised there.

> 
> kbingham@charm:~/iob/libcamera$ ./build/gcc/src/apps/cam/cam -c1 -I -srole=raw
> [1:18:54.693658992] [15767]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/kbingham/iob/libcamera/build/gcc/src/ipa' to the IPA search path
> [1:18:54.698672581] [15767]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+82-1f6b50aa
> [1:18:54.736613408] [15768]  WARN CameraSensor camera_sensor_legacy.cpp:616 'ov5675 24-0010': Rotation control not available, default to 0 degrees
> [1:18:54.736743817] [15768]  WARN CameraSensor camera_sensor_legacy.cpp:501 'ov5675 24-0010': No sensor delays found in static properties. Assuming unverified defaults.
> [1:18:54.743968255] [15768]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data'
> [1:18:54.744016950] [15768]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'ov5675.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
> [1:18:54.744431351] [15768]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data'
> [1:18:54.753486194] [15768]  INFO Pipeline pipeline_handler.cpp:646 libcamera is not installed. Loading platform configuration file from '/home/kbingham/iob/libcamera/src/libcamera/pipeline/virtual/data/virtual.yaml'
> Using camera /base/soc@0/cci@ac4c000/i2c-bus@1/camera@10 as cam0
> 0: 1296x972-SGRBG10_CSI2P/RAW
>  * Pixelformat: SGRBG10_CSI2P (1296x972)-(2592x1944)/(+0,+0)
>   - 1296x972
>   - 2592x1944
> 
> 
> vs Umangs branch:
> 
> kbingham@charm:~/iob/libcamera$ ./build/gcc/src/apps/cam/cam -c1 -I -srole=raw | grep -i pixelformat
> [1:20:02.718240596] [16578]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/kbingham/iob/libcamera/build/gcc/src/ipa' to the IPA search path
> [1:20:02.725663876] [16578]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+81-d9b7a3bf
> [1:20:02.762329159] [16580]  WARN CameraSensor camera_sensor_legacy.cpp:616 'ov5675 24-0010': Rotation control not available, default to 0 degrees
> [1:20:02.762467588] [16580]  WARN CameraSensor camera_sensor_legacy.cpp:501 'ov5675 24-0010': No sensor delays found in static properties. Assuming unverified defaults.
> [1:20:02.769917481] [16580]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data'
> [1:20:02.769956802] [16580]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'ov5675.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
> [1:20:02.770373234] [16580]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data'
> [1:20:02.780852414] [16580]  INFO Pipeline pipeline_handler.cpp:646 libcamera is not installed. Loading platform configuration file from '/home/kbingham/iob/libcamera/src/libcamera/pipeline/virtual/data/virtual.yaml'
>  * Pixelformat: SGRBG10_CSI2P (4x2)-(2584x1944)/(+4,+2)
>  * Pixelformat: ABGR8888 (4x2)-(2584x1944)/(+4,+2)
>  * Pixelformat: XBGR8888 (4x2)-(2584x1944)/(+4,+2)
>  * Pixelformat: BGR888 (4x2)-(2584x1944)/(+4,+2)
>  * Pixelformat: RGB888 (4x2)-(2584x1944)/(+4,+2)
>  * Pixelformat: ARGB8888 (4x2)-(2584x1944)/(+4,+2)
>  * Pixelformat: XRGB8888 (4x2)-(2584x1944)/(+4,+2)
> 
> So I'm afraid I think Umangs' branch is doing something wrong at the
> moment?

I mostly have tested capturing points and see if the correct format/sensor
size are picked or not, for a given config, before sending out the RFC.

There might be some rough corners still, for e.g. you posted above where
the reporting of -srole=raw is giving out processed formats as well. So,
those rough corners needs some polishing before I post v1.

> 
> I would like to target merging this RAW support by the end of this week
> prior to the next release point - could you let me know what the best
> approach is please ?

I am not too sure, a week for proper testing, cross-reviews etc. seems
short. Milan what do you think ?

> 
> Robert, could you test on your devices please ? Ultimately the RAW
> support is the important step we need to enable to start being able to
> capture image cailbration images and tuning for devices...
> 
> Once this is merged we can work on enabling manual controls  for
> exposure and gain which will be the next requirements for tuning.
> 
> --
> Kieran
> 
> 
> 
> > 
> > >> From the technical standpoint, I wonder how it works without changing
> > >> buffer handling in the simple pipeline.  Do you think
> > >> https://patchwork.libcamera.org/patch/23785/ is not needed?
> > >
> > > It's changed in 6/6.
> > 
> > Ah, sorry, I was looking at patches from patchwork and that one was
> > omitted there.
> > 
> > >> Umang Jain <uajain@igalia.com> writes:
> > >> 
> > >> > RFC series to enable raw capture on simple pipeline handler with
> > >> > SoftISP enabled. I had a few thoughts on design of the features
> > >> > when I was reviewing Milan's series [1]. Hence, I decided to crystallise
> > >> > my thoughts and posted as RFC to gather more feedback (and some real
> > >> > testing on supportedDevices[]).
> > >> >
> > >> > I have left few things to include from [1] for e.g. colorspace issues.
> > >> > My goal currently is to address the core issue of raw capture with this
> > >> > RFC.
> > >> >
> > >> > Testing branch [2]
> > >> > Setup: RPi 3b + IMX219
> > >> > Cases tested with cam:
> > >> >    - Single raw output
> > >> >    - Single processed output
> > >> >    - 1 Raw + 1 processed output
> > >> >
> > >> > One can propose more use-cases as part of their testing. If they turn
> > >> > out to be complex, we can probably document it as \todo, to get
> > >> > addressed at a later stage.
> > >> >
> > >> > cam Output:
> > >> > ===========
> > >> >
> > >> > uajain1@uajain:~/libcamera$ ./build-soft/src/apps/cam/cam -c1 -srole=raw,width=1920,height=1080 -C5
> > >> > [27:26:59.248233501] [2384]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/uajain1/libcamera/build-soft/src/ipa' to the IPA search path
> > >> > [27:26:59.262004021] [2384]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+63-c33be195
> > >> > [27:26:59.593703813] [2390]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
> > >> > [27:26:59.594050532] [2390]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'imx219.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
> > >> > [27:26:59.596620063] [2390]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
> > >> > Using camera /base/soc/i2c0mux/i2c@1/imx219@10 as cam0
> > >> > [27:26:59.625957459] [2384]  INFO Camera camera.cpp:1205 configuring streams: (0) 1920x1080-SRGGB10/RAW
> > >> > [27:26:59.627666834] [2390]  INFO SimplePipeline simple.cpp:1471 data->useConversion_ : 0
> > >> > cam0: Capture 5 frames
> > >> > 98819.750177 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 4147200
> > >> > 98819.783504 (30.01 fps) cam0-stream0 seq: 000001 bytesused: 4147200
> > >> > 98819.816829 (30.01 fps) cam0-stream0 seq: 000002 bytesused: 4147200
> > >> > 98819.850156 (30.01 fps) cam0-stream0 seq: 000003 bytesused: 4147200
> > >> > 98819.883484 (30.00 fps) cam0-stream0 seq: 000004 bytesused: 4147200
> > >> >
> > >> >
> > >> > uajain1@uajain:~/libcamera$ ./build-soft/src/apps/cam/cam -c1 -srole=viewfinder,width=1920,height=1080 -C5
> > >> > [27:27:09.663699747] [2391]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/uajain1/libcamera/build-soft/src/ipa' to the IPA search path
> > >> > [27:27:09.677708965] [2391]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+63-c33be195
> > >> > [27:27:09.933555736] [2392]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
> > >> > [27:27:09.933821778] [2392]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'imx219.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
> > >> > [27:27:09.936308080] [2392]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
> > >> > Camera configuration adjusted
> > >> > Using camera /base/soc/i2c0mux/i2c@1/imx219@10 as cam0
> > >> > [27:27:09.965424746] [2391]  INFO Camera camera.cpp:1205 configuring streams: (0) 1916x1080-XRGB8888/Unset
> > >> > [27:27:09.967118705] [2392]  INFO SimplePipeline simple.cpp:1471 data->useConversion_ : 1
> > >> > [27:27:09.970895528] [2392]  INFO IPASoft soft_simple.cpp:264 IPASoft: Exposure 4-1759, gain 1-10.6667 (0.0966667)
> > >> > cam0: Capture 5 frames
> > >> > 98830.200120 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 8277120
> > >> > 98830.233445 (30.01 fps) cam0-stream0 seq: 000001 bytesused: 8277120
> > >> > 98830.266770 (30.01 fps) cam0-stream0 seq: 000002 bytesused: 8277120
> > >> > 98831.033291 (1.30 fps) cam0-stream0 seq: 000025 bytesused: 8277120
> > >> > 98831.799810 (1.30 fps) cam0-stream0 seq: 000048 bytesused: 8277120
> > >> >
> > >> >
> > >> > uajain1@uajain:~/libcamera$ ./build-soft/src/apps/cam/cam -c1 -srole=raw,width=1920,height=1080 -srole=viewfinder,width=1920,height=1080 -C10
> > >> > [27:27:26.239476459] [2395]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/uajain1/libcamera/build-soft/src/ipa' to the IPA search path
> > >> > [27:27:26.253694063] [2395]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+63-c33be195
> > >> > [27:27:26.572276146] [2396]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
> > >> > [27:27:26.572539792] [2396]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'imx219.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
> > >> > [27:27:26.574856667] [2396]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
> > >> > Camera configuration adjusted
> > >> > Using camera /base/soc/i2c0mux/i2c@1/imx219@10 as cam0
> > >> > [27:27:26.603776146] [2395]  INFO Camera camera.cpp:1205 configuring streams: (0) 1920x1080-SRGGB10/RAW (1) 1916x1080-XRGB8888/Unset
> > >> > [27:27:26.605656303] [2396]  INFO SimplePipeline simple.cpp:1471 data->useConversion_ : 1
> > >> > [27:27:26.609448803] [2396]  INFO IPASoft soft_simple.cpp:264 IPASoft: Exposure 4-1759, gain 1-10.6667 (0.0966667)
> > >> > cam0: Capture 10 frames
> > >> > 98846.739621 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 4147200 cam0-stream1 seq: 000000 bytesused: 8277120
> > >> > 98846.772946 (30.01 fps) cam0-stream0 seq: 000001 bytesused: 4147200 cam0-stream1 seq: 000001 bytesused: 8277120
> > >> > 98846.806274 (30.00 fps) cam0-stream0 seq: 000002 bytesused: 4147200 cam0-stream1 seq: 000002 bytesused: 8277120
> > >> > 98846.839601 (30.01 fps) cam0-stream0 seq: 000003 bytesused: 4147200 cam0-stream1 seq: 000003 bytesused: 8277120
> > >> > 98847.539465 (1.43 fps) cam0-stream0 seq: 000024 bytesused: 4147200 cam0-stream1 seq: 000024 bytesused: 8277120
> > >> > 98848.206006 (1.50 fps) cam0-stream0 seq: 000044 bytesused: 4147200 cam0-stream1 seq: 000044 bytesused: 8277120
> > >> > 98848.872546 (1.50 fps) cam0-stream0 seq: 000064 bytesused: 4147200 cam0-stream1 seq: 000064 bytesused: 8277120
> > >> > 98849.539084 (1.50 fps) cam0-stream0 seq: 000084 bytesused: 4147200 cam0-stream1 seq: 000084 bytesused: 8277120
> > >> > 98850.205624 (1.50 fps) cam0-stream0 seq: 000104 bytesused: 4147200 cam0-stream1 seq: 000104 bytesused: 8277120
> > >> > 98850.872164 (1.50 fps) cam0-stream0 seq: 000124 bytesused: 4147200 cam0-stream1 seq: 000124 bytesused: 8277120
> > >> >
> > >> > [1]: https://patchwork.libcamera.org/project/libcamera/list/?series=5281
> > >> > [2]: https://gitlab.freedesktop.org/uajain/libcamera/-/commits/uajain/simple/raw
> > >> >
> > >> > Milan Zamazal (3):
> > >> >   libcamera: simple: Set the number of software ISP streams to 2
> > >> >   libcamera: simple: Exclude raw configurations from output conversions
> > >> >   libcamera: simple: Require metadata only when software ISP is used
> > >> >
> > >> > Umang Jain (3):
> > >> >   libcamera: simple: Support StreamRole::Raw in generateConfiguration()
> > >> >   libcamera: simple: Validate raw streams if requested
> > >> >   libcamera: simple: Enable capturing of raw streams
> > >> >
> > >> >  src/libcamera/pipeline/simple/simple.cpp | 166 ++++++++++++++++++-----
> > >> >  1 file changed, 131 insertions(+), 35 deletions(-)
> > >> 
> >
Umang Jain July 21, 2025, 10:38 a.m. UTC | #6
On Mon, Jul 21, 2025 at 09:47:30AM +0530, Umang Jain wrote:
> Hi Kieran,
> 
> On Sun, Jul 20, 2025 at 04:45:39PM +0100, Kieran Bingham wrote:
> > Hi All,
> > 
> > I'm trying to understand what the plan is on these multiple RAW series
> > now.
> 
> I think I stated out a plan already and Milan seems to agree. There is a
> need for cross-review/test cycles and one of us then need to take the lead
> towards merging the series.
> > 
> > Quoting Milan Zamazal (2025-07-18 12:35:04)
> > > Umang Jain <uajain@igalia.com> writes:
> > > 
> > > > Hi Milan,
> > > >
> > > > On Thu, Jul 17, 2025 at 04:04:58PM +0200, Milan Zamazal wrote:
> > > >> Hi Umang,
> > > >> 
> > > >> thank you for posting the patches.  How do you suggest to proceed with
> > > >> both the patch series (yours and mine) to avoid duplicate work and get
> > > >> closer to merging?
> > > >
> > > > Good question. I would suggest to test this series and see if it
> > > > addresses use-cases for the raw capture scenarios (maybe, I missed
> > > > something/corner-cases).
> > > >
> > > > We can cross review each other patches for starters. I've picked 3 of
> > > > your patches and based my patches on that. Going forward, either of us
> > > > can collect the relevant reviewed patches and post a single series as
> > > > collaborative efforts.
> > > >
> > > > I would also suggest to split off parallel patches (for e.g. colorspace
> > > > patches from your series) and see if we can progress them independently.
> > > > These are my thoughts, if you want to take an alternate approach - I am
> > > > willing to hear it as well!
> > > 
> > > OK for me.
> > 
> > There's some curious differences between the two versions now ? On x13s
> > - if I run Milans' branch I get a smaller resolution by default (and
> > better performance because of that I guess) and also the RAW streams are
> > filtering better? 
> 
> If I take a look at current master branch - the status-quo I think is
> that the filtering needs/defaults pickups needs work:
> 
> [~/src/libcamera]$ git grep \todo | grep -i 'pipeline/simple'
> src/libcamera/pipeline/simple/simple.cpp:	 * \todo The sensor timestamp should be better estimated by connecting
> src/libcamera/pipeline/simple/simple.cpp:	 * \todo Applying controls directly not only increases the risk of
> src/libcamera/pipeline/simple/simple.cpp:	 * \todo Pick the best sensor output media bus format when the
> src/libcamera/pipeline/simple/simple.cpp:		/* \todo Create a libcamera core class to group format and size */
> src/libcamera/pipeline/simple/simple.cpp:	 * \todo Implement a better way to pick the default format
> 
> Currently, it's just picking up the first entry in a given map etc, so I
> am not surprised there.
> 
> > 
> > kbingham@charm:~/iob/libcamera$ ./build/gcc/src/apps/cam/cam -c1 -I -srole=raw
> > [1:18:54.693658992] [15767]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/kbingham/iob/libcamera/build/gcc/src/ipa' to the IPA search path
> > [1:18:54.698672581] [15767]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+82-1f6b50aa
> > [1:18:54.736613408] [15768]  WARN CameraSensor camera_sensor_legacy.cpp:616 'ov5675 24-0010': Rotation control not available, default to 0 degrees
> > [1:18:54.736743817] [15768]  WARN CameraSensor camera_sensor_legacy.cpp:501 'ov5675 24-0010': No sensor delays found in static properties. Assuming unverified defaults.
> > [1:18:54.743968255] [15768]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data'
> > [1:18:54.744016950] [15768]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'ov5675.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
> > [1:18:54.744431351] [15768]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data'
> > [1:18:54.753486194] [15768]  INFO Pipeline pipeline_handler.cpp:646 libcamera is not installed. Loading platform configuration file from '/home/kbingham/iob/libcamera/src/libcamera/pipeline/virtual/data/virtual.yaml'
> > Using camera /base/soc@0/cci@ac4c000/i2c-bus@1/camera@10 as cam0
> > 0: 1296x972-SGRBG10_CSI2P/RAW
> >  * Pixelformat: SGRBG10_CSI2P (1296x972)-(2592x1944)/(+0,+0)
> >   - 1296x972
> >   - 2592x1944
> > 
> > 
> > vs Umangs branch:
> > 
> > kbingham@charm:~/iob/libcamera$ ./build/gcc/src/apps/cam/cam -c1 -I -srole=raw | grep -i pixelformat
> > [1:20:02.718240596] [16578]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/kbingham/iob/libcamera/build/gcc/src/ipa' to the IPA search path
> > [1:20:02.725663876] [16578]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+81-d9b7a3bf
> > [1:20:02.762329159] [16580]  WARN CameraSensor camera_sensor_legacy.cpp:616 'ov5675 24-0010': Rotation control not available, default to 0 degrees
> > [1:20:02.762467588] [16580]  WARN CameraSensor camera_sensor_legacy.cpp:501 'ov5675 24-0010': No sensor delays found in static properties. Assuming unverified defaults.
> > [1:20:02.769917481] [16580]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data'
> > [1:20:02.769956802] [16580]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'ov5675.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
> > [1:20:02.770373234] [16580]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data'
> > [1:20:02.780852414] [16580]  INFO Pipeline pipeline_handler.cpp:646 libcamera is not installed. Loading platform configuration file from '/home/kbingham/iob/libcamera/src/libcamera/pipeline/virtual/data/virtual.yaml'
> >  * Pixelformat: SGRBG10_CSI2P (4x2)-(2584x1944)/(+4,+2)
> >  * Pixelformat: ABGR8888 (4x2)-(2584x1944)/(+4,+2)
> >  * Pixelformat: XBGR8888 (4x2)-(2584x1944)/(+4,+2)
> >  * Pixelformat: BGR888 (4x2)-(2584x1944)/(+4,+2)
> >  * Pixelformat: RGB888 (4x2)-(2584x1944)/(+4,+2)
> >  * Pixelformat: ARGB8888 (4x2)-(2584x1944)/(+4,+2)
> >  * Pixelformat: XRGB8888 (4x2)-(2584x1944)/(+4,+2)
> > 
> > So I'm afraid I think Umangs' branch is doing something wrong at the
> > moment?
> 
> I mostly have tested capturing points and see if the correct format/sensor
> size are picked or not, for a given config, before sending out the RFC.
> 
> There might be some rough corners still, for e.g. you posted above where
> the reporting of -srole=raw is giving out processed formats as well. So,
> those rough corners needs some polishing before I post v1.
> 

fwiw, I fixed up the RAW reporting and did some more initial testing, Output
here: https://paste.debian.net/1387016/

on my branch:
https://gitlab.freedesktop.org/uajain/libcamera/-/tree/uajain/simple/raw
> > 
> > I would like to target merging this RAW support by the end of this week
> > prior to the next release point - could you let me know what the best
> > approach is please ?
> 
> I am not too sure, a week for proper testing, cross-reviews etc. seems
> short. Milan what do you think ?
> 
> > 
> > Robert, could you test on your devices please ? Ultimately the RAW
> > support is the important step we need to enable to start being able to
> > capture image cailbration images and tuning for devices...
> > 
> > Once this is merged we can work on enabling manual controls  for
> > exposure and gain which will be the next requirements for tuning.
> > 
> > --
> > Kieran
> > 
> > 
> > 
> > > 
> > > >> From the technical standpoint, I wonder how it works without changing
> > > >> buffer handling in the simple pipeline.  Do you think
> > > >> https://patchwork.libcamera.org/patch/23785/ is not needed?
> > > >
> > > > It's changed in 6/6.
> > > 
> > > Ah, sorry, I was looking at patches from patchwork and that one was
> > > omitted there.
> > > 
> > > >> Umang Jain <uajain@igalia.com> writes:
> > > >> 
> > > >> > RFC series to enable raw capture on simple pipeline handler with
> > > >> > SoftISP enabled. I had a few thoughts on design of the features
> > > >> > when I was reviewing Milan's series [1]. Hence, I decided to crystallise
> > > >> > my thoughts and posted as RFC to gather more feedback (and some real
> > > >> > testing on supportedDevices[]).
> > > >> >
> > > >> > I have left few things to include from [1] for e.g. colorspace issues.
> > > >> > My goal currently is to address the core issue of raw capture with this
> > > >> > RFC.
> > > >> >
> > > >> > Testing branch [2]
> > > >> > Setup: RPi 3b + IMX219
> > > >> > Cases tested with cam:
> > > >> >    - Single raw output
> > > >> >    - Single processed output
> > > >> >    - 1 Raw + 1 processed output
> > > >> >
> > > >> > One can propose more use-cases as part of their testing. If they turn
> > > >> > out to be complex, we can probably document it as \todo, to get
> > > >> > addressed at a later stage.
> > > >> >
> > > >> > cam Output:
> > > >> > ===========
> > > >> >
> > > >> > uajain1@uajain:~/libcamera$ ./build-soft/src/apps/cam/cam -c1 -srole=raw,width=1920,height=1080 -C5
> > > >> > [27:26:59.248233501] [2384]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/uajain1/libcamera/build-soft/src/ipa' to the IPA search path
> > > >> > [27:26:59.262004021] [2384]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+63-c33be195
> > > >> > [27:26:59.593703813] [2390]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
> > > >> > [27:26:59.594050532] [2390]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'imx219.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
> > > >> > [27:26:59.596620063] [2390]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
> > > >> > Using camera /base/soc/i2c0mux/i2c@1/imx219@10 as cam0
> > > >> > [27:26:59.625957459] [2384]  INFO Camera camera.cpp:1205 configuring streams: (0) 1920x1080-SRGGB10/RAW
> > > >> > [27:26:59.627666834] [2390]  INFO SimplePipeline simple.cpp:1471 data->useConversion_ : 0
> > > >> > cam0: Capture 5 frames
> > > >> > 98819.750177 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 4147200
> > > >> > 98819.783504 (30.01 fps) cam0-stream0 seq: 000001 bytesused: 4147200
> > > >> > 98819.816829 (30.01 fps) cam0-stream0 seq: 000002 bytesused: 4147200
> > > >> > 98819.850156 (30.01 fps) cam0-stream0 seq: 000003 bytesused: 4147200
> > > >> > 98819.883484 (30.00 fps) cam0-stream0 seq: 000004 bytesused: 4147200
> > > >> >
> > > >> >
> > > >> > uajain1@uajain:~/libcamera$ ./build-soft/src/apps/cam/cam -c1 -srole=viewfinder,width=1920,height=1080 -C5
> > > >> > [27:27:09.663699747] [2391]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/uajain1/libcamera/build-soft/src/ipa' to the IPA search path
> > > >> > [27:27:09.677708965] [2391]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+63-c33be195
> > > >> > [27:27:09.933555736] [2392]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
> > > >> > [27:27:09.933821778] [2392]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'imx219.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
> > > >> > [27:27:09.936308080] [2392]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
> > > >> > Camera configuration adjusted
> > > >> > Using camera /base/soc/i2c0mux/i2c@1/imx219@10 as cam0
> > > >> > [27:27:09.965424746] [2391]  INFO Camera camera.cpp:1205 configuring streams: (0) 1916x1080-XRGB8888/Unset
> > > >> > [27:27:09.967118705] [2392]  INFO SimplePipeline simple.cpp:1471 data->useConversion_ : 1
> > > >> > [27:27:09.970895528] [2392]  INFO IPASoft soft_simple.cpp:264 IPASoft: Exposure 4-1759, gain 1-10.6667 (0.0966667)
> > > >> > cam0: Capture 5 frames
> > > >> > 98830.200120 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 8277120
> > > >> > 98830.233445 (30.01 fps) cam0-stream0 seq: 000001 bytesused: 8277120
> > > >> > 98830.266770 (30.01 fps) cam0-stream0 seq: 000002 bytesused: 8277120
> > > >> > 98831.033291 (1.30 fps) cam0-stream0 seq: 000025 bytesused: 8277120
> > > >> > 98831.799810 (1.30 fps) cam0-stream0 seq: 000048 bytesused: 8277120
> > > >> >
> > > >> >
> > > >> > uajain1@uajain:~/libcamera$ ./build-soft/src/apps/cam/cam -c1 -srole=raw,width=1920,height=1080 -srole=viewfinder,width=1920,height=1080 -C10
> > > >> > [27:27:26.239476459] [2395]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/uajain1/libcamera/build-soft/src/ipa' to the IPA search path
> > > >> > [27:27:26.253694063] [2395]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+63-c33be195
> > > >> > [27:27:26.572276146] [2396]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
> > > >> > [27:27:26.572539792] [2396]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'imx219.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
> > > >> > [27:27:26.574856667] [2396]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
> > > >> > Camera configuration adjusted
> > > >> > Using camera /base/soc/i2c0mux/i2c@1/imx219@10 as cam0
> > > >> > [27:27:26.603776146] [2395]  INFO Camera camera.cpp:1205 configuring streams: (0) 1920x1080-SRGGB10/RAW (1) 1916x1080-XRGB8888/Unset
> > > >> > [27:27:26.605656303] [2396]  INFO SimplePipeline simple.cpp:1471 data->useConversion_ : 1
> > > >> > [27:27:26.609448803] [2396]  INFO IPASoft soft_simple.cpp:264 IPASoft: Exposure 4-1759, gain 1-10.6667 (0.0966667)
> > > >> > cam0: Capture 10 frames
> > > >> > 98846.739621 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 4147200 cam0-stream1 seq: 000000 bytesused: 8277120
> > > >> > 98846.772946 (30.01 fps) cam0-stream0 seq: 000001 bytesused: 4147200 cam0-stream1 seq: 000001 bytesused: 8277120
> > > >> > 98846.806274 (30.00 fps) cam0-stream0 seq: 000002 bytesused: 4147200 cam0-stream1 seq: 000002 bytesused: 8277120
> > > >> > 98846.839601 (30.01 fps) cam0-stream0 seq: 000003 bytesused: 4147200 cam0-stream1 seq: 000003 bytesused: 8277120
> > > >> > 98847.539465 (1.43 fps) cam0-stream0 seq: 000024 bytesused: 4147200 cam0-stream1 seq: 000024 bytesused: 8277120
> > > >> > 98848.206006 (1.50 fps) cam0-stream0 seq: 000044 bytesused: 4147200 cam0-stream1 seq: 000044 bytesused: 8277120
> > > >> > 98848.872546 (1.50 fps) cam0-stream0 seq: 000064 bytesused: 4147200 cam0-stream1 seq: 000064 bytesused: 8277120
> > > >> > 98849.539084 (1.50 fps) cam0-stream0 seq: 000084 bytesused: 4147200 cam0-stream1 seq: 000084 bytesused: 8277120
> > > >> > 98850.205624 (1.50 fps) cam0-stream0 seq: 000104 bytesused: 4147200 cam0-stream1 seq: 000104 bytesused: 8277120
> > > >> > 98850.872164 (1.50 fps) cam0-stream0 seq: 000124 bytesused: 4147200 cam0-stream1 seq: 000124 bytesused: 8277120
> > > >> >
> > > >> > [1]: https://patchwork.libcamera.org/project/libcamera/list/?series=5281
> > > >> > [2]: https://gitlab.freedesktop.org/uajain/libcamera/-/commits/uajain/simple/raw
> > > >> >
> > > >> > Milan Zamazal (3):
> > > >> >   libcamera: simple: Set the number of software ISP streams to 2
> > > >> >   libcamera: simple: Exclude raw configurations from output conversions
> > > >> >   libcamera: simple: Require metadata only when software ISP is used
> > > >> >
> > > >> > Umang Jain (3):
> > > >> >   libcamera: simple: Support StreamRole::Raw in generateConfiguration()
> > > >> >   libcamera: simple: Validate raw streams if requested
> > > >> >   libcamera: simple: Enable capturing of raw streams
> > > >> >
> > > >> >  src/libcamera/pipeline/simple/simple.cpp | 166 ++++++++++++++++++-----
> > > >> >  1 file changed, 131 insertions(+), 35 deletions(-)
> > > >> 
> > >
Kieran Bingham July 21, 2025, 11:19 a.m. UTC | #7
Quoting Umang Jain (2025-07-21 11:38:06)
> On Mon, Jul 21, 2025 at 09:47:30AM +0530, Umang Jain wrote:
> > Hi Kieran,
> > 
> > On Sun, Jul 20, 2025 at 04:45:39PM +0100, Kieran Bingham wrote:
> > > Hi All,
> > > 
> > > I'm trying to understand what the plan is on these multiple RAW series
> > > now.
> > 
> > I think I stated out a plan already and Milan seems to agree. There is a
> > need for cross-review/test cycles and one of us then need to take the lead
> > towards merging the series.
> > > 
> > > Quoting Milan Zamazal (2025-07-18 12:35:04)
> > > > Umang Jain <uajain@igalia.com> writes:
> > > > 
> > > > > Hi Milan,
> > > > >
> > > > > On Thu, Jul 17, 2025 at 04:04:58PM +0200, Milan Zamazal wrote:
> > > > >> Hi Umang,
> > > > >> 
> > > > >> thank you for posting the patches.  How do you suggest to proceed with
> > > > >> both the patch series (yours and mine) to avoid duplicate work and get
> > > > >> closer to merging?
> > > > >
> > > > > Good question. I would suggest to test this series and see if it
> > > > > addresses use-cases for the raw capture scenarios (maybe, I missed
> > > > > something/corner-cases).
> > > > >
> > > > > We can cross review each other patches for starters. I've picked 3 of
> > > > > your patches and based my patches on that. Going forward, either of us
> > > > > can collect the relevant reviewed patches and post a single series as
> > > > > collaborative efforts.
> > > > >
> > > > > I would also suggest to split off parallel patches (for e.g. colorspace
> > > > > patches from your series) and see if we can progress them independently.
> > > > > These are my thoughts, if you want to take an alternate approach - I am
> > > > > willing to hear it as well!

If there are multiple separate topics in a branch - splitting can help
indeed. At the moment I'm only testing RAW support... as that's the
feature we need to be able to progress doing the calibrations for
Soft {CPU/GPU}ISP based devices.


> > > > 
> > > > OK for me.
> > > 
> > > There's some curious differences between the two versions now ? On x13s
> > > - if I run Milans' branch I get a smaller resolution by default (and
> > > better performance because of that I guess) and also the RAW streams are
> > > filtering better? 
> > 
> > If I take a look at current master branch - the status-quo I think is
> > that the filtering needs/defaults pickups needs work:
> > 
> > [~/src/libcamera]$ git grep \todo | grep -i 'pipeline/simple'
> > src/libcamera/pipeline/simple/simple.cpp:      * \todo The sensor timestamp should be better estimated by connecting
> > src/libcamera/pipeline/simple/simple.cpp:      * \todo Applying controls directly not only increases the risk of
> > src/libcamera/pipeline/simple/simple.cpp:      * \todo Pick the best sensor output media bus format when the
> > src/libcamera/pipeline/simple/simple.cpp:             /* \todo Create a libcamera core class to group format and size */
> > src/libcamera/pipeline/simple/simple.cpp:      * \todo Implement a better way to pick the default format
> > 
> > Currently, it's just picking up the first entry in a given map etc, so I
> > am not surprised there.
> > 
> > > 
> > > kbingham@charm:~/iob/libcamera$ ./build/gcc/src/apps/cam/cam -c1 -I -srole=raw
> > > [1:18:54.693658992] [15767]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/kbingham/iob/libcamera/build/gcc/src/ipa' to the IPA search path
> > > [1:18:54.698672581] [15767]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+82-1f6b50aa
> > > [1:18:54.736613408] [15768]  WARN CameraSensor camera_sensor_legacy.cpp:616 'ov5675 24-0010': Rotation control not available, default to 0 degrees
> > > [1:18:54.736743817] [15768]  WARN CameraSensor camera_sensor_legacy.cpp:501 'ov5675 24-0010': No sensor delays found in static properties. Assuming unverified defaults.
> > > [1:18:54.743968255] [15768]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data'
> > > [1:18:54.744016950] [15768]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'ov5675.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
> > > [1:18:54.744431351] [15768]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data'
> > > [1:18:54.753486194] [15768]  INFO Pipeline pipeline_handler.cpp:646 libcamera is not installed. Loading platform configuration file from '/home/kbingham/iob/libcamera/src/libcamera/pipeline/virtual/data/virtual.yaml'
> > > Using camera /base/soc@0/cci@ac4c000/i2c-bus@1/camera@10 as cam0
> > > 0: 1296x972-SGRBG10_CSI2P/RAW
> > >  * Pixelformat: SGRBG10_CSI2P (1296x972)-(2592x1944)/(+0,+0)
> > >   - 1296x972
> > >   - 2592x1944
> > > 
> > > 
> > > vs Umangs branch:
> > > 
> > > kbingham@charm:~/iob/libcamera$ ./build/gcc/src/apps/cam/cam -c1 -I -srole=raw | grep -i pixelformat
> > > [1:20:02.718240596] [16578]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/kbingham/iob/libcamera/build/gcc/src/ipa' to the IPA search path
> > > [1:20:02.725663876] [16578]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+81-d9b7a3bf
> > > [1:20:02.762329159] [16580]  WARN CameraSensor camera_sensor_legacy.cpp:616 'ov5675 24-0010': Rotation control not available, default to 0 degrees
> > > [1:20:02.762467588] [16580]  WARN CameraSensor camera_sensor_legacy.cpp:501 'ov5675 24-0010': No sensor delays found in static properties. Assuming unverified defaults.
> > > [1:20:02.769917481] [16580]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data'
> > > [1:20:02.769956802] [16580]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'ov5675.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
> > > [1:20:02.770373234] [16580]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data'
> > > [1:20:02.780852414] [16580]  INFO Pipeline pipeline_handler.cpp:646 libcamera is not installed. Loading platform configuration file from '/home/kbingham/iob/libcamera/src/libcamera/pipeline/virtual/data/virtual.yaml'
> > >  * Pixelformat: SGRBG10_CSI2P (4x2)-(2584x1944)/(+4,+2)
> > >  * Pixelformat: ABGR8888 (4x2)-(2584x1944)/(+4,+2)
> > >  * Pixelformat: XBGR8888 (4x2)-(2584x1944)/(+4,+2)
> > >  * Pixelformat: BGR888 (4x2)-(2584x1944)/(+4,+2)
> > >  * Pixelformat: RGB888 (4x2)-(2584x1944)/(+4,+2)
> > >  * Pixelformat: ARGB8888 (4x2)-(2584x1944)/(+4,+2)
> > >  * Pixelformat: XRGB8888 (4x2)-(2584x1944)/(+4,+2)
> > > 
> > > So I'm afraid I think Umangs' branch is doing something wrong at the
> > > moment?
> > 
> > I mostly have tested capturing points and see if the correct format/sensor
> > size are picked or not, for a given config, before sending out the RFC.
> > 
> > There might be some rough corners still, for e.g. you posted above where
> > the reporting of -srole=raw is giving out processed formats as well. So,
> > those rough corners needs some polishing before I post v1.
> > 
> 
> fwiw, I fixed up the RAW reporting and did some more initial testing, Output
> here: https://paste.debian.net/1387016/
> 
> on my branch:
> https://gitlab.freedesktop.org/uajain/libcamera/-/tree/uajain/simple/raw
> > > 
> > > I would like to target merging this RAW support by the end of this week

Sorry - I didn't mean 'this series' by saying 'this RAW' here.. I mean
the feature.

I was looking at the point that Milan is now at v10 and was working -
and passed on CI
(https://gitlab.freedesktop.org/camera/libcamera/-/pipelines/1470120) -
so I wanted to know how we get it merged - but now there is a competing
series ?

> > > prior to the next release point - could you let me know what the best
> > > approach is please ?
> > 
> > I am not too sure, a week for proper testing, cross-reviews etc. seems
> > short. Milan what do you think ?

Which leaves me confused. Is this series supposed to superceed Milan's?
Shouldn't it then have the same title and be v11 ?

If there are issues in Milan's series - would it make sense to post
fixup patches to show the changes against that series?

> > > Robert, could you test on your devices please ? Ultimately the RAW
> > > support is the important step we need to enable to start being able to
> > > capture image cailbration images and tuning for devices...
> > > 
> > > Once this is merged we can work on enabling manual controls  for

I said 'this' again while on the 'new' series... Perhaps I should have
continued the discussion on the 'first' (milans) series?

I'll wait to hear what you two decide is correct for merging RAW support
to simple pipeline handler. At this stage a clear 'v11' might help make
things clearer. But please coordinate if there are two developers
working on the same topic.

--
Kieran


> > > exposure and gain which will be the next requirements for tuning.
> > > 
> > > --
> > > Kieran
> > > 
> > > 
> > > 
> > > > 
> > > > >> From the technical standpoint, I wonder how it works without changing
> > > > >> buffer handling in the simple pipeline.  Do you think
> > > > >> https://patchwork.libcamera.org/patch/23785/ is not needed?
> > > > >
> > > > > It's changed in 6/6.
> > > > 
> > > > Ah, sorry, I was looking at patches from patchwork and that one was
> > > > omitted there.
> > > > 
> > > > >> Umang Jain <uajain@igalia.com> writes:
> > > > >> 
> > > > >> > RFC series to enable raw capture on simple pipeline handler with
> > > > >> > SoftISP enabled. I had a few thoughts on design of the features
> > > > >> > when I was reviewing Milan's series [1]. Hence, I decided to crystallise
> > > > >> > my thoughts and posted as RFC to gather more feedback (and some real
> > > > >> > testing on supportedDevices[]).
> > > > >> >
> > > > >> > I have left few things to include from [1] for e.g. colorspace issues.
> > > > >> > My goal currently is to address the core issue of raw capture with this
> > > > >> > RFC.
> > > > >> >
> > > > >> > Testing branch [2]
> > > > >> > Setup: RPi 3b + IMX219
> > > > >> > Cases tested with cam:
> > > > >> >    - Single raw output
> > > > >> >    - Single processed output
> > > > >> >    - 1 Raw + 1 processed output
> > > > >> >
> > > > >> > One can propose more use-cases as part of their testing. If they turn
> > > > >> > out to be complex, we can probably document it as \todo, to get
> > > > >> > addressed at a later stage.
> > > > >> >
> > > > >> > cam Output:
> > > > >> > ===========
> > > > >> >
> > > > >> > uajain1@uajain:~/libcamera$ ./build-soft/src/apps/cam/cam -c1 -srole=raw,width=1920,height=1080 -C5
> > > > >> > [27:26:59.248233501] [2384]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/uajain1/libcamera/build-soft/src/ipa' to the IPA search path
> > > > >> > [27:26:59.262004021] [2384]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+63-c33be195
> > > > >> > [27:26:59.593703813] [2390]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
> > > > >> > [27:26:59.594050532] [2390]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'imx219.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
> > > > >> > [27:26:59.596620063] [2390]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
> > > > >> > Using camera /base/soc/i2c0mux/i2c@1/imx219@10 as cam0
> > > > >> > [27:26:59.625957459] [2384]  INFO Camera camera.cpp:1205 configuring streams: (0) 1920x1080-SRGGB10/RAW
> > > > >> > [27:26:59.627666834] [2390]  INFO SimplePipeline simple.cpp:1471 data->useConversion_ : 0
> > > > >> > cam0: Capture 5 frames
> > > > >> > 98819.750177 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 4147200
> > > > >> > 98819.783504 (30.01 fps) cam0-stream0 seq: 000001 bytesused: 4147200
> > > > >> > 98819.816829 (30.01 fps) cam0-stream0 seq: 000002 bytesused: 4147200
> > > > >> > 98819.850156 (30.01 fps) cam0-stream0 seq: 000003 bytesused: 4147200
> > > > >> > 98819.883484 (30.00 fps) cam0-stream0 seq: 000004 bytesused: 4147200
> > > > >> >
> > > > >> >
> > > > >> > uajain1@uajain:~/libcamera$ ./build-soft/src/apps/cam/cam -c1 -srole=viewfinder,width=1920,height=1080 -C5
> > > > >> > [27:27:09.663699747] [2391]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/uajain1/libcamera/build-soft/src/ipa' to the IPA search path
> > > > >> > [27:27:09.677708965] [2391]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+63-c33be195
> > > > >> > [27:27:09.933555736] [2392]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
> > > > >> > [27:27:09.933821778] [2392]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'imx219.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
> > > > >> > [27:27:09.936308080] [2392]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
> > > > >> > Camera configuration adjusted
> > > > >> > Using camera /base/soc/i2c0mux/i2c@1/imx219@10 as cam0
> > > > >> > [27:27:09.965424746] [2391]  INFO Camera camera.cpp:1205 configuring streams: (0) 1916x1080-XRGB8888/Unset
> > > > >> > [27:27:09.967118705] [2392]  INFO SimplePipeline simple.cpp:1471 data->useConversion_ : 1
> > > > >> > [27:27:09.970895528] [2392]  INFO IPASoft soft_simple.cpp:264 IPASoft: Exposure 4-1759, gain 1-10.6667 (0.0966667)
> > > > >> > cam0: Capture 5 frames
> > > > >> > 98830.200120 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 8277120
> > > > >> > 98830.233445 (30.01 fps) cam0-stream0 seq: 000001 bytesused: 8277120
> > > > >> > 98830.266770 (30.01 fps) cam0-stream0 seq: 000002 bytesused: 8277120
> > > > >> > 98831.033291 (1.30 fps) cam0-stream0 seq: 000025 bytesused: 8277120
> > > > >> > 98831.799810 (1.30 fps) cam0-stream0 seq: 000048 bytesused: 8277120
> > > > >> >
> > > > >> >
> > > > >> > uajain1@uajain:~/libcamera$ ./build-soft/src/apps/cam/cam -c1 -srole=raw,width=1920,height=1080 -srole=viewfinder,width=1920,height=1080 -C10
> > > > >> > [27:27:26.239476459] [2395]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/uajain1/libcamera/build-soft/src/ipa' to the IPA search path
> > > > >> > [27:27:26.253694063] [2395]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+63-c33be195
> > > > >> > [27:27:26.572276146] [2396]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
> > > > >> > [27:27:26.572539792] [2396]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'imx219.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
> > > > >> > [27:27:26.574856667] [2396]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
> > > > >> > Camera configuration adjusted
> > > > >> > Using camera /base/soc/i2c0mux/i2c@1/imx219@10 as cam0
> > > > >> > [27:27:26.603776146] [2395]  INFO Camera camera.cpp:1205 configuring streams: (0) 1920x1080-SRGGB10/RAW (1) 1916x1080-XRGB8888/Unset
> > > > >> > [27:27:26.605656303] [2396]  INFO SimplePipeline simple.cpp:1471 data->useConversion_ : 1
> > > > >> > [27:27:26.609448803] [2396]  INFO IPASoft soft_simple.cpp:264 IPASoft: Exposure 4-1759, gain 1-10.6667 (0.0966667)
> > > > >> > cam0: Capture 10 frames
> > > > >> > 98846.739621 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 4147200 cam0-stream1 seq: 000000 bytesused: 8277120
> > > > >> > 98846.772946 (30.01 fps) cam0-stream0 seq: 000001 bytesused: 4147200 cam0-stream1 seq: 000001 bytesused: 8277120
> > > > >> > 98846.806274 (30.00 fps) cam0-stream0 seq: 000002 bytesused: 4147200 cam0-stream1 seq: 000002 bytesused: 8277120
> > > > >> > 98846.839601 (30.01 fps) cam0-stream0 seq: 000003 bytesused: 4147200 cam0-stream1 seq: 000003 bytesused: 8277120
> > > > >> > 98847.539465 (1.43 fps) cam0-stream0 seq: 000024 bytesused: 4147200 cam0-stream1 seq: 000024 bytesused: 8277120
> > > > >> > 98848.206006 (1.50 fps) cam0-stream0 seq: 000044 bytesused: 4147200 cam0-stream1 seq: 000044 bytesused: 8277120
> > > > >> > 98848.872546 (1.50 fps) cam0-stream0 seq: 000064 bytesused: 4147200 cam0-stream1 seq: 000064 bytesused: 8277120
> > > > >> > 98849.539084 (1.50 fps) cam0-stream0 seq: 000084 bytesused: 4147200 cam0-stream1 seq: 000084 bytesused: 8277120
> > > > >> > 98850.205624 (1.50 fps) cam0-stream0 seq: 000104 bytesused: 4147200 cam0-stream1 seq: 000104 bytesused: 8277120
> > > > >> > 98850.872164 (1.50 fps) cam0-stream0 seq: 000124 bytesused: 4147200 cam0-stream1 seq: 000124 bytesused: 8277120
> > > > >> >
> > > > >> > [1]: https://patchwork.libcamera.org/project/libcamera/list/?series=5281
> > > > >> > [2]: https://gitlab.freedesktop.org/uajain/libcamera/-/commits/uajain/simple/raw
> > > > >> >
> > > > >> > Milan Zamazal (3):
> > > > >> >   libcamera: simple: Set the number of software ISP streams to 2
> > > > >> >   libcamera: simple: Exclude raw configurations from output conversions
> > > > >> >   libcamera: simple: Require metadata only when software ISP is used
> > > > >> >
> > > > >> > Umang Jain (3):
> > > > >> >   libcamera: simple: Support StreamRole::Raw in generateConfiguration()
> > > > >> >   libcamera: simple: Validate raw streams if requested
> > > > >> >   libcamera: simple: Enable capturing of raw streams
> > > > >> >
> > > > >> >  src/libcamera/pipeline/simple/simple.cpp | 166 ++++++++++++++++++-----
> > > > >> >  1 file changed, 131 insertions(+), 35 deletions(-)
> > > > >> 
> > > >
Milan Zamazal July 21, 2025, 8:46 p.m. UTC | #8
Hi Kieran,

Kieran Bingham <kieran.bingham@ideasonboard.com> writes:

> Quoting Umang Jain (2025-07-21 11:38:06)
>> On Mon, Jul 21, 2025 at 09:47:30AM +0530, Umang Jain wrote:
>> > Hi Kieran,
>
>> > 
>> > On Sun, Jul 20, 2025 at 04:45:39PM +0100, Kieran Bingham wrote:
>> > > Hi All,
>> > > 
>> > > I'm trying to understand what the plan is on these multiple RAW series
>> > > now.
>> > 
>> > I think I stated out a plan already and Milan seems to agree. There is a
>> > need for cross-review/test cycles and one of us then need to take the lead
>> > towards merging the series.
>> > > 
>> > > Quoting Milan Zamazal (2025-07-18 12:35:04)
>> > > > Umang Jain <uajain@igalia.com> writes:
>> > > > 
>> > > > > Hi Milan,
>> > > > >
>> > > > > On Thu, Jul 17, 2025 at 04:04:58PM +0200, Milan Zamazal wrote:
>> > > > >> Hi Umang,
>> > > > >> 
>> > > > >> thank you for posting the patches.  How do you suggest to proceed with
>> > > > >> both the patch series (yours and mine) to avoid duplicate work and get
>> > > > >> closer to merging?
>> > > > >
>> > > > > Good question. I would suggest to test this series and see if it
>> > > > > addresses use-cases for the raw capture scenarios (maybe, I missed
>> > > > > something/corner-cases).
>> > > > >
>> > > > > We can cross review each other patches for starters. I've picked 3 of
>> > > > > your patches and based my patches on that. Going forward, either of us
>> > > > > can collect the relevant reviewed patches and post a single series as
>> > > > > collaborative efforts.
>> > > > >
>> > > > > I would also suggest to split off parallel patches (for e.g. colorspace
>> > > > > patches from your series) and see if we can progress them independently.
>> > > > > These are my thoughts, if you want to take an alternate approach - I am
>> > > > > willing to hear it as well!
>
> If there are multiple separate topics in a branch - splitting can help
> indeed. At the moment I'm only testing RAW support... as that's the
> feature we need to be able to progress doing the calibrations for
> Soft {CPU/GPU}ISP based devices.
>
>
>> > > > 
>> > > > OK for me.
>> > > 
>> > > There's some curious differences between the two versions now ? On x13s
>> > > - if I run Milans' branch I get a smaller resolution by default (and
>> > > better performance because of that I guess) and also the RAW streams are
>> > > filtering better? 
>> > 
>> > If I take a look at current master branch - the status-quo I think is
>> > that the filtering needs/defaults pickups needs work:
>> > 
>> > [~/src/libcamera]$ git grep \todo | grep -i 'pipeline/simple'
>> > src/libcamera/pipeline/simple/simple.cpp: * \todo The sensor timestamp should be better estimated by connecting
>> > src/libcamera/pipeline/simple/simple.cpp: * \todo Applying controls directly not only increases the risk of
>> > src/libcamera/pipeline/simple/simple.cpp: * \todo Pick the best sensor output media bus format when the
>> > src/libcamera/pipeline/simple/simple.cpp: /* \todo Create a libcamera core class to group format and size */
>> > src/libcamera/pipeline/simple/simple.cpp: * \todo Implement a better way to pick the default format
>> > 
>> > Currently, it's just picking up the first entry in a given map etc, so I
>> > am not surprised there.
>> > 
>> > > 
>> > > kbingham@charm:~/iob/libcamera$ ./build/gcc/src/apps/cam/cam -c1 -I -srole=raw
>> > > [1:18:54.693658992] [15767]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/kbingham/iob/libcamera/build/gcc/src/ipa' to the IPA search path
>> > > [1:18:54.698672581] [15767]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+82-1f6b50aa
>> > > [1:18:54.736613408] [15768]  WARN CameraSensor camera_sensor_legacy.cpp:616 'ov5675 24-0010': Rotation control not available, default to 0 degrees
>> > > [1:18:54.736743817] [15768]  WARN CameraSensor camera_sensor_legacy.cpp:501 'ov5675 24-0010': No sensor delays found in static properties. Assuming unverified defaults.
>> > > [1:18:54.743968255] [15768]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data'
>> > > [1:18:54.744016950] [15768]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'ov5675.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
>> > > [1:18:54.744431351] [15768]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data'
>> > > [1:18:54.753486194] [15768]  INFO Pipeline pipeline_handler.cpp:646 libcamera is not installed. Loading platform configuration file from '/home/kbingham/iob/libcamera/src/libcamera/pipeline/virtual/data/virtual.yaml'
>> > > Using camera /base/soc@0/cci@ac4c000/i2c-bus@1/camera@10 as cam0
>> > > 0: 1296x972-SGRBG10_CSI2P/RAW
>> > >  * Pixelformat: SGRBG10_CSI2P (1296x972)-(2592x1944)/(+0,+0)
>> > >   - 1296x972
>> > >   - 2592x1944
>> > > 
>> > > 
>> > > vs Umangs branch:
>> > > 
>> > > kbingham@charm:~/iob/libcamera$ ./build/gcc/src/apps/cam/cam -c1 -I -srole=raw | grep -i pixelformat
>> > > [1:20:02.718240596] [16578]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/kbingham/iob/libcamera/build/gcc/src/ipa' to the IPA search path
>> > > [1:20:02.725663876] [16578]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+81-d9b7a3bf
>> > > [1:20:02.762329159] [16580]  WARN CameraSensor camera_sensor_legacy.cpp:616 'ov5675 24-0010': Rotation control not available, default to 0 degrees
>> > > [1:20:02.762467588] [16580]  WARN CameraSensor camera_sensor_legacy.cpp:501 'ov5675 24-0010': No sensor delays found in static properties. Assuming unverified defaults.
>> > > [1:20:02.769917481] [16580]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data'
>> > > [1:20:02.769956802] [16580]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'ov5675.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
>> > > [1:20:02.770373234] [16580]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data'
>> > > [1:20:02.780852414] [16580]  INFO Pipeline pipeline_handler.cpp:646 libcamera is not installed. Loading platform configuration file from '/home/kbingham/iob/libcamera/src/libcamera/pipeline/virtual/data/virtual.yaml'
>> > >  * Pixelformat: SGRBG10_CSI2P (4x2)-(2584x1944)/(+4,+2)
>> > >  * Pixelformat: ABGR8888 (4x2)-(2584x1944)/(+4,+2)
>> > >  * Pixelformat: XBGR8888 (4x2)-(2584x1944)/(+4,+2)
>> > >  * Pixelformat: BGR888 (4x2)-(2584x1944)/(+4,+2)
>> > >  * Pixelformat: RGB888 (4x2)-(2584x1944)/(+4,+2)
>> > >  * Pixelformat: ARGB8888 (4x2)-(2584x1944)/(+4,+2)
>> > >  * Pixelformat: XRGB8888 (4x2)-(2584x1944)/(+4,+2)
>> > > 
>> > > So I'm afraid I think Umangs' branch is doing something wrong at the
>> > > moment?
>> > 
>> > I mostly have tested capturing points and see if the correct format/sensor
>> > size are picked or not, for a given config, before sending out the RFC.
>> > 
>> > There might be some rough corners still, for e.g. you posted above where
>> > the reporting of -srole=raw is giving out processed formats as well. So,
>> > those rough corners needs some polishing before I post v1.
>> > 
>> 
>> fwiw, I fixed up the RAW reporting and did some more initial testing, Output
>> here: https://paste.debian.net/1387016/
>> 
>> on my branch:
>> https://gitlab.freedesktop.org/uajain/libcamera/-/tree/uajain/simple/raw
>> > > 
>> > > I would like to target merging this RAW support by the end of this week
>
> Sorry - I didn't mean 'this series' by saying 'this RAW' here.. I mean
> the feature.

We can try but let's be careful not to break anything what hasn't been
broken before.

> I was looking at the point that Milan is now at v10 and was working -
> and passed on CI
> (https://gitlab.freedesktop.org/camera/libcamera/-/pipelines/1470120) -
> so I wanted to know how we get it merged - but now there is a competing
> series ?

I understand Umang's series as a way to demonstrate an alternative
approach to do some of the things and my patches need improvements in
some of those areas.  It's indeed hard to track this for everybody but
the recent Umang's comments to my patches help.

>> > > prior to the next release point - could you let me know what the best
>> > > approach is please ?
>> > 
>> > I am not too sure, a week for proper testing, cross-reviews etc. seems
>> > short. Milan what do you think ?
>
> Which leaves me confused. Is this series supposed to superceed Milan's?
> Shouldn't it then have the same title and be v11 ?
>
> If there are issues in Milan's series - would it make sense to post
> fixup patches to show the changes against that series?
>
>> > > Robert, could you test on your devices please ? Ultimately the RAW
>> > > support is the important step we need to enable to start being able to
>> > > capture image cailbration images and tuning for devices...
>> > > 
>> > > Once this is merged we can work on enabling manual controls  for
>
> I said 'this' again while on the 'new' series... Perhaps I should have
> continued the discussion on the 'first' (milans) series?
>
> I'll wait to hear what you two decide is correct for merging RAW support
> to simple pipeline handler. At this stage a clear 'v11' might help make
> things clearer. But please coordinate if there are two developers
> working on the same topic.

I can work on v11 tomorrow, hopefully it brings both the series closer
to each other.  I think simply continuing with my series is the best, as
it evolves from an already partially reviewed and tested code and
changes to it can be tracked without much trouble.

>
> --
> Kieran
>
>
>> > > exposure and gain which will be the next requirements for tuning.
>> > > 
>> > > --
>> > > Kieran
>> > > 
>> > > 
>> > > 
>> > > > 
>> > > > >> From the technical standpoint, I wonder how it works without changing
>> > > > >> buffer handling in the simple pipeline.  Do you think
>> > > > >> https://patchwork.libcamera.org/patch/23785/ is not needed?
>> > > > >
>> > > > > It's changed in 6/6.
>> > > > 
>> > > > Ah, sorry, I was looking at patches from patchwork and that one was
>> > > > omitted there.
>> > > > 
>> > > > >> Umang Jain <uajain@igalia.com> writes:
>> > > > >> 
>> > > > >> > RFC series to enable raw capture on simple pipeline handler with
>> > > > >> > SoftISP enabled. I had a few thoughts on design of the features
>> > > > >> > when I was reviewing Milan's series [1]. Hence, I decided to crystallise
>> > > > >> > my thoughts and posted as RFC to gather more feedback (and some real
>> > > > >> > testing on supportedDevices[]).
>> > > > >> >
>> > > > >> > I have left few things to include from [1] for e.g. colorspace issues.
>> > > > >> > My goal currently is to address the core issue of raw capture with this
>> > > > >> > RFC.
>> > > > >> >
>> > > > >> > Testing branch [2]
>> > > > >> > Setup: RPi 3b + IMX219
>> > > > >> > Cases tested with cam:
>> > > > >> >    - Single raw output
>> > > > >> >    - Single processed output
>> > > > >> >    - 1 Raw + 1 processed output
>> > > > >> >
>> > > > >> > One can propose more use-cases as part of their testing. If they turn
>> > > > >> > out to be complex, we can probably document it as \todo, to get
>> > > > >> > addressed at a later stage.
>> > > > >> >
>> > > > >> > cam Output:
>> > > > >> > ===========
>> > > > >> >
>> > > > >> > uajain1@uajain:~/libcamera$ ./build-soft/src/apps/cam/cam -c1 -srole=raw,width=1920,height=1080 -C5
>> > > > >> > [27:26:59.248233501] [2384]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/uajain1/libcamera/build-soft/src/ipa' to the IPA search path
>> > > > >> > [27:26:59.262004021] [2384]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+63-c33be195
>> > > > >> > [27:26:59.593703813] [2390]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
>> > > > >> > [27:26:59.594050532] [2390]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'imx219.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
>> > > > >> > [27:26:59.596620063] [2390]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
>> > > > >> > Using camera /base/soc/i2c0mux/i2c@1/imx219@10 as cam0
>> > > > >> > [27:26:59.625957459] [2384]  INFO Camera camera.cpp:1205 configuring streams: (0) 1920x1080-SRGGB10/RAW
>> > > > >> > [27:26:59.627666834] [2390]  INFO SimplePipeline simple.cpp:1471 data->useConversion_ : 0
>> > > > >> > cam0: Capture 5 frames
>> > > > >> > 98819.750177 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 4147200
>> > > > >> > 98819.783504 (30.01 fps) cam0-stream0 seq: 000001 bytesused: 4147200
>> > > > >> > 98819.816829 (30.01 fps) cam0-stream0 seq: 000002 bytesused: 4147200
>> > > > >> > 98819.850156 (30.01 fps) cam0-stream0 seq: 000003 bytesused: 4147200
>> > > > >> > 98819.883484 (30.00 fps) cam0-stream0 seq: 000004 bytesused: 4147200
>> > > > >> >
>> > > > >> >
>> > > > >> > uajain1@uajain:~/libcamera$ ./build-soft/src/apps/cam/cam -c1 -srole=viewfinder,width=1920,height=1080 -C5
>> > > > >> > [27:27:09.663699747] [2391]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/uajain1/libcamera/build-soft/src/ipa' to the IPA search path
>> > > > >> > [27:27:09.677708965] [2391]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+63-c33be195
>> > > > >> > [27:27:09.933555736] [2392]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
>> > > > >> > [27:27:09.933821778] [2392]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'imx219.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
>> > > > >> > [27:27:09.936308080] [2392]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
>> > > > >> > Camera configuration adjusted
>> > > > >> > Using camera /base/soc/i2c0mux/i2c@1/imx219@10 as cam0
>> > > > >> > [27:27:09.965424746] [2391]  INFO Camera camera.cpp:1205 configuring streams: (0) 1916x1080-XRGB8888/Unset
>> > > > >> > [27:27:09.967118705] [2392]  INFO SimplePipeline simple.cpp:1471 data->useConversion_ : 1
>> > > > >> > [27:27:09.970895528] [2392]  INFO IPASoft soft_simple.cpp:264 IPASoft: Exposure 4-1759, gain 1-10.6667 (0.0966667)
>> > > > >> > cam0: Capture 5 frames
>> > > > >> > 98830.200120 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 8277120
>> > > > >> > 98830.233445 (30.01 fps) cam0-stream0 seq: 000001 bytesused: 8277120
>> > > > >> > 98830.266770 (30.01 fps) cam0-stream0 seq: 000002 bytesused: 8277120
>> > > > >> > 98831.033291 (1.30 fps) cam0-stream0 seq: 000025 bytesused: 8277120
>> > > > >> > 98831.799810 (1.30 fps) cam0-stream0 seq: 000048 bytesused: 8277120
>> > > > >> >
>> > > > >> >
>> > > > >> > uajain1@uajain:~/libcamera$ ./build-soft/src/apps/cam/cam -c1 -srole=raw,width=1920,height=1080 -srole=viewfinder,width=1920,height=1080 -C10
>> > > > >> > [27:27:26.239476459] [2395]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/uajain1/libcamera/build-soft/src/ipa' to the IPA search path
>> > > > >> > [27:27:26.253694063] [2395]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+63-c33be195
>> > > > >> > [27:27:26.572276146] [2396]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
>> > > > >> > [27:27:26.572539792] [2396]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'imx219.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
>> > > > >> > [27:27:26.574856667] [2396]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
>> > > > >> > Camera configuration adjusted
>> > > > >> > Using camera /base/soc/i2c0mux/i2c@1/imx219@10 as cam0
>> > > > >> > [27:27:26.603776146] [2395]  INFO Camera camera.cpp:1205 configuring streams: (0) 1920x1080-SRGGB10/RAW (1) 1916x1080-XRGB8888/Unset
>> > > > >> > [27:27:26.605656303] [2396]  INFO SimplePipeline simple.cpp:1471 data->useConversion_ : 1
>> > > > >> > [27:27:26.609448803] [2396]  INFO IPASoft soft_simple.cpp:264 IPASoft: Exposure 4-1759, gain 1-10.6667 (0.0966667)
>> > > > >> > cam0: Capture 10 frames
>> > > > >> > 98846.739621 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 4147200 cam0-stream1 seq: 000000 bytesused: 8277120
>> > > > >> > 98846.772946 (30.01 fps) cam0-stream0 seq: 000001 bytesused: 4147200 cam0-stream1 seq: 000001 bytesused: 8277120
>> > > > >> > 98846.806274 (30.00 fps) cam0-stream0 seq: 000002 bytesused: 4147200 cam0-stream1 seq: 000002 bytesused: 8277120
>> > > > >> > 98846.839601 (30.01 fps) cam0-stream0 seq: 000003 bytesused: 4147200 cam0-stream1 seq: 000003 bytesused: 8277120
>> > > > >> > 98847.539465 (1.43 fps) cam0-stream0 seq: 000024 bytesused: 4147200 cam0-stream1 seq: 000024 bytesused: 8277120
>> > > > >> > 98848.206006 (1.50 fps) cam0-stream0 seq: 000044 bytesused: 4147200 cam0-stream1 seq: 000044 bytesused: 8277120
>> > > > >> > 98848.872546 (1.50 fps) cam0-stream0 seq: 000064 bytesused: 4147200 cam0-stream1 seq: 000064 bytesused: 8277120
>> > > > >> > 98849.539084 (1.50 fps) cam0-stream0 seq: 000084 bytesused: 4147200 cam0-stream1 seq: 000084 bytesused: 8277120
>> > > > >> > 98850.205624 (1.50 fps) cam0-stream0 seq: 000104 bytesused: 4147200 cam0-stream1 seq: 000104 bytesused: 8277120
>> > > > >> > 98850.872164 (1.50 fps) cam0-stream0 seq: 000124 bytesused: 4147200 cam0-stream1 seq: 000124 bytesused: 8277120
>> > > > >> >
>> > > > >> > [1]: https://patchwork.libcamera.org/project/libcamera/list/?series=5281
>> > > > >> > [2]: https://gitlab.freedesktop.org/uajain/libcamera/-/commits/uajain/simple/raw
>> > > > >> >
>> > > > >> > Milan Zamazal (3):
>> > > > >> >   libcamera: simple: Set the number of software ISP streams to 2
>> > > > >> >   libcamera: simple: Exclude raw configurations from output conversions
>> > > > >> >   libcamera: simple: Require metadata only when software ISP is used
>> > > > >> >
>> > > > >> > Umang Jain (3):
>> > > > >> >   libcamera: simple: Support StreamRole::Raw in generateConfiguration()
>> > > > >> >   libcamera: simple: Validate raw streams if requested
>> > > > >> >   libcamera: simple: Enable capturing of raw streams
>> > > > >> >
>> > > > >> >  src/libcamera/pipeline/simple/simple.cpp | 166 ++++++++++++++++++-----
>> > > > >> >  1 file changed, 131 insertions(+), 35 deletions(-)
>> > > > >> 
>> > > >
Umang Jain July 22, 2025, 4:49 a.m. UTC | #9
On Mon, Jul 21, 2025 at 10:46:26PM +0200, Milan Zamazal wrote:
> Hi Kieran,
> 
> Kieran Bingham <kieran.bingham@ideasonboard.com> writes:
> 
> > Quoting Umang Jain (2025-07-21 11:38:06)
> >> On Mon, Jul 21, 2025 at 09:47:30AM +0530, Umang Jain wrote:
> >> > Hi Kieran,
> >
> >> > 
> >> > On Sun, Jul 20, 2025 at 04:45:39PM +0100, Kieran Bingham wrote:
> >> > > Hi All,
> >> > > 
> >> > > I'm trying to understand what the plan is on these multiple RAW series
> >> > > now.
> >> > 
> >> > I think I stated out a plan already and Milan seems to agree. There is a
> >> > need for cross-review/test cycles and one of us then need to take the lead
> >> > towards merging the series.
> >> > > 
> >> > > Quoting Milan Zamazal (2025-07-18 12:35:04)
> >> > > > Umang Jain <uajain@igalia.com> writes:
> >> > > > 
> >> > > > > Hi Milan,
> >> > > > >
> >> > > > > On Thu, Jul 17, 2025 at 04:04:58PM +0200, Milan Zamazal wrote:
> >> > > > >> Hi Umang,
> >> > > > >> 
> >> > > > >> thank you for posting the patches.  How do you suggest to proceed with
> >> > > > >> both the patch series (yours and mine) to avoid duplicate work and get
> >> > > > >> closer to merging?
> >> > > > >
> >> > > > > Good question. I would suggest to test this series and see if it
> >> > > > > addresses use-cases for the raw capture scenarios (maybe, I missed
> >> > > > > something/corner-cases).
> >> > > > >
> >> > > > > We can cross review each other patches for starters. I've picked 3 of
> >> > > > > your patches and based my patches on that. Going forward, either of us
> >> > > > > can collect the relevant reviewed patches and post a single series as
> >> > > > > collaborative efforts.
> >> > > > >
> >> > > > > I would also suggest to split off parallel patches (for e.g. colorspace
> >> > > > > patches from your series) and see if we can progress them independently.
> >> > > > > These are my thoughts, if you want to take an alternate approach - I am
> >> > > > > willing to hear it as well!
> >
> > If there are multiple separate topics in a branch - splitting can help
> > indeed. At the moment I'm only testing RAW support... as that's the
> > feature we need to be able to progress doing the calibrations for
> > Soft {CPU/GPU}ISP based devices.
> >
> >
> >> > > > 
> >> > > > OK for me.
> >> > > 
> >> > > There's some curious differences between the two versions now ? On x13s
> >> > > - if I run Milans' branch I get a smaller resolution by default (and
> >> > > better performance because of that I guess) and also the RAW streams are
> >> > > filtering better? 
> >> > 
> >> > If I take a look at current master branch - the status-quo I think is
> >> > that the filtering needs/defaults pickups needs work:
> >> > 
> >> > [~/src/libcamera]$ git grep \todo | grep -i 'pipeline/simple'
> >> > src/libcamera/pipeline/simple/simple.cpp: * \todo The sensor timestamp should be better estimated by connecting
> >> > src/libcamera/pipeline/simple/simple.cpp: * \todo Applying controls directly not only increases the risk of
> >> > src/libcamera/pipeline/simple/simple.cpp: * \todo Pick the best sensor output media bus format when the
> >> > src/libcamera/pipeline/simple/simple.cpp: /* \todo Create a libcamera core class to group format and size */
> >> > src/libcamera/pipeline/simple/simple.cpp: * \todo Implement a better way to pick the default format
> >> > 
> >> > Currently, it's just picking up the first entry in a given map etc, so I
> >> > am not surprised there.
> >> > 
> >> > > 
> >> > > kbingham@charm:~/iob/libcamera$ ./build/gcc/src/apps/cam/cam -c1 -I -srole=raw
> >> > > [1:18:54.693658992] [15767]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/kbingham/iob/libcamera/build/gcc/src/ipa' to the IPA search path
> >> > > [1:18:54.698672581] [15767]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+82-1f6b50aa
> >> > > [1:18:54.736613408] [15768]  WARN CameraSensor camera_sensor_legacy.cpp:616 'ov5675 24-0010': Rotation control not available, default to 0 degrees
> >> > > [1:18:54.736743817] [15768]  WARN CameraSensor camera_sensor_legacy.cpp:501 'ov5675 24-0010': No sensor delays found in static properties. Assuming unverified defaults.
> >> > > [1:18:54.743968255] [15768]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data'
> >> > > [1:18:54.744016950] [15768]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'ov5675.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
> >> > > [1:18:54.744431351] [15768]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data'
> >> > > [1:18:54.753486194] [15768]  INFO Pipeline pipeline_handler.cpp:646 libcamera is not installed. Loading platform configuration file from '/home/kbingham/iob/libcamera/src/libcamera/pipeline/virtual/data/virtual.yaml'
> >> > > Using camera /base/soc@0/cci@ac4c000/i2c-bus@1/camera@10 as cam0
> >> > > 0: 1296x972-SGRBG10_CSI2P/RAW
> >> > >  * Pixelformat: SGRBG10_CSI2P (1296x972)-(2592x1944)/(+0,+0)
> >> > >   - 1296x972
> >> > >   - 2592x1944
> >> > > 
> >> > > 
> >> > > vs Umangs branch:
> >> > > 
> >> > > kbingham@charm:~/iob/libcamera$ ./build/gcc/src/apps/cam/cam -c1 -I -srole=raw | grep -i pixelformat
> >> > > [1:20:02.718240596] [16578]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/kbingham/iob/libcamera/build/gcc/src/ipa' to the IPA search path
> >> > > [1:20:02.725663876] [16578]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+81-d9b7a3bf
> >> > > [1:20:02.762329159] [16580]  WARN CameraSensor camera_sensor_legacy.cpp:616 'ov5675 24-0010': Rotation control not available, default to 0 degrees
> >> > > [1:20:02.762467588] [16580]  WARN CameraSensor camera_sensor_legacy.cpp:501 'ov5675 24-0010': No sensor delays found in static properties. Assuming unverified defaults.
> >> > > [1:20:02.769917481] [16580]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data'
> >> > > [1:20:02.769956802] [16580]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'ov5675.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
> >> > > [1:20:02.770373234] [16580]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data'
> >> > > [1:20:02.780852414] [16580]  INFO Pipeline pipeline_handler.cpp:646 libcamera is not installed. Loading platform configuration file from '/home/kbingham/iob/libcamera/src/libcamera/pipeline/virtual/data/virtual.yaml'
> >> > >  * Pixelformat: SGRBG10_CSI2P (4x2)-(2584x1944)/(+4,+2)
> >> > >  * Pixelformat: ABGR8888 (4x2)-(2584x1944)/(+4,+2)
> >> > >  * Pixelformat: XBGR8888 (4x2)-(2584x1944)/(+4,+2)
> >> > >  * Pixelformat: BGR888 (4x2)-(2584x1944)/(+4,+2)
> >> > >  * Pixelformat: RGB888 (4x2)-(2584x1944)/(+4,+2)
> >> > >  * Pixelformat: ARGB8888 (4x2)-(2584x1944)/(+4,+2)
> >> > >  * Pixelformat: XRGB8888 (4x2)-(2584x1944)/(+4,+2)
> >> > > 
> >> > > So I'm afraid I think Umangs' branch is doing something wrong at the
> >> > > moment?
> >> > 
> >> > I mostly have tested capturing points and see if the correct format/sensor
> >> > size are picked or not, for a given config, before sending out the RFC.
> >> > 
> >> > There might be some rough corners still, for e.g. you posted above where
> >> > the reporting of -srole=raw is giving out processed formats as well. So,
> >> > those rough corners needs some polishing before I post v1.
> >> > 
> >> 
> >> fwiw, I fixed up the RAW reporting and did some more initial testing, Output
> >> here: https://paste.debian.net/1387016/
> >> 
> >> on my branch:
> >> https://gitlab.freedesktop.org/uajain/libcamera/-/tree/uajain/simple/raw
> >> > > 
> >> > > I would like to target merging this RAW support by the end of this week
> >
> > Sorry - I didn't mean 'this series' by saying 'this RAW' here.. I mean
> > the feature.
> 
> We can try but let's be careful not to break anything what hasn't been
> broken before.

Precisely why I asked for testing from h/w platforms in
supported_devices[].

> 
> > I was looking at the point that Milan is now at v10 and was working -
> > and passed on CI
> > (https://gitlab.freedesktop.org/camera/libcamera/-/pipelines/1470120) -
> > so I wanted to know how we get it merged - but now there is a competing
> > series ?
> 
> I understand Umang's series as a way to demonstrate an alternative
> approach to do some of the things and my patches need improvements in
> some of those areas.  It's indeed hard to track this for everybody but
> the recent Umang's comments to my patches help.
> 
> >> > > prior to the next release point - could you let me know what the best
> >> > > approach is please ?
> >> > 
> >> > I am not too sure, a week for proper testing, cross-reviews etc. seems
> >> > short. Milan what do you think ?
> >
> > Which leaves me confused. Is this series supposed to superceed Milan's?
> > Shouldn't it then have the same title and be v11 ?
> >
> > If there are issues in Milan's series - would it make sense to post
> > fixup patches to show the changes against that series?
> >
> >> > > Robert, could you test on your devices please ? Ultimately the RAW
> >> > > support is the important step we need to enable to start being able to
> >> > > capture image cailbration images and tuning for devices...
> >> > > 
> >> > > Once this is merged we can work on enabling manual controls  for
> >
> > I said 'this' again while on the 'new' series... Perhaps I should have
> > continued the discussion on the 'first' (milans) series?
> >
> > I'll wait to hear what you two decide is correct for merging RAW support
> > to simple pipeline handler. At this stage a clear 'v11' might help make
> > things clearer. But please coordinate if there are two developers
> > working on the same topic.
> 
> I can work on v11 tomorrow, hopefully it brings both the series closer
> to each other.  I think simply continuing with my series is the best, as
> it evolves from an already partially reviewed and tested code and
> changes to it can be tracked without much trouble.

Feel free to cherry-pick patches from my branch and expand/fix as per
your needs.
> 
> >
> > --
> > Kieran
> >
> >
> >> > > exposure and gain which will be the next requirements for tuning.
> >> > > 
> >> > > --
> >> > > Kieran
> >> > > 
> >> > > 
> >> > > 
> >> > > > 
> >> > > > >> From the technical standpoint, I wonder how it works without changing
> >> > > > >> buffer handling in the simple pipeline.  Do you think
> >> > > > >> https://patchwork.libcamera.org/patch/23785/ is not needed?
> >> > > > >
> >> > > > > It's changed in 6/6.
> >> > > > 
> >> > > > Ah, sorry, I was looking at patches from patchwork and that one was
> >> > > > omitted there.
> >> > > > 
> >> > > > >> Umang Jain <uajain@igalia.com> writes:
> >> > > > >> 
> >> > > > >> > RFC series to enable raw capture on simple pipeline handler with
> >> > > > >> > SoftISP enabled. I had a few thoughts on design of the features
> >> > > > >> > when I was reviewing Milan's series [1]. Hence, I decided to crystallise
> >> > > > >> > my thoughts and posted as RFC to gather more feedback (and some real
> >> > > > >> > testing on supportedDevices[]).
> >> > > > >> >
> >> > > > >> > I have left few things to include from [1] for e.g. colorspace issues.
> >> > > > >> > My goal currently is to address the core issue of raw capture with this
> >> > > > >> > RFC.
> >> > > > >> >
> >> > > > >> > Testing branch [2]
> >> > > > >> > Setup: RPi 3b + IMX219
> >> > > > >> > Cases tested with cam:
> >> > > > >> >    - Single raw output
> >> > > > >> >    - Single processed output
> >> > > > >> >    - 1 Raw + 1 processed output
> >> > > > >> >
> >> > > > >> > One can propose more use-cases as part of their testing. If they turn
> >> > > > >> > out to be complex, we can probably document it as \todo, to get
> >> > > > >> > addressed at a later stage.
> >> > > > >> >
> >> > > > >> > cam Output:
> >> > > > >> > ===========
> >> > > > >> >
> >> > > > >> > uajain1@uajain:~/libcamera$ ./build-soft/src/apps/cam/cam -c1 -srole=raw,width=1920,height=1080 -C5
> >> > > > >> > [27:26:59.248233501] [2384]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/uajain1/libcamera/build-soft/src/ipa' to the IPA search path
> >> > > > >> > [27:26:59.262004021] [2384]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+63-c33be195
> >> > > > >> > [27:26:59.593703813] [2390]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
> >> > > > >> > [27:26:59.594050532] [2390]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'imx219.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
> >> > > > >> > [27:26:59.596620063] [2390]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
> >> > > > >> > Using camera /base/soc/i2c0mux/i2c@1/imx219@10 as cam0
> >> > > > >> > [27:26:59.625957459] [2384]  INFO Camera camera.cpp:1205 configuring streams: (0) 1920x1080-SRGGB10/RAW
> >> > > > >> > [27:26:59.627666834] [2390]  INFO SimplePipeline simple.cpp:1471 data->useConversion_ : 0
> >> > > > >> > cam0: Capture 5 frames
> >> > > > >> > 98819.750177 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 4147200
> >> > > > >> > 98819.783504 (30.01 fps) cam0-stream0 seq: 000001 bytesused: 4147200
> >> > > > >> > 98819.816829 (30.01 fps) cam0-stream0 seq: 000002 bytesused: 4147200
> >> > > > >> > 98819.850156 (30.01 fps) cam0-stream0 seq: 000003 bytesused: 4147200
> >> > > > >> > 98819.883484 (30.00 fps) cam0-stream0 seq: 000004 bytesused: 4147200
> >> > > > >> >
> >> > > > >> >
> >> > > > >> > uajain1@uajain:~/libcamera$ ./build-soft/src/apps/cam/cam -c1 -srole=viewfinder,width=1920,height=1080 -C5
> >> > > > >> > [27:27:09.663699747] [2391]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/uajain1/libcamera/build-soft/src/ipa' to the IPA search path
> >> > > > >> > [27:27:09.677708965] [2391]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+63-c33be195
> >> > > > >> > [27:27:09.933555736] [2392]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
> >> > > > >> > [27:27:09.933821778] [2392]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'imx219.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
> >> > > > >> > [27:27:09.936308080] [2392]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
> >> > > > >> > Camera configuration adjusted
> >> > > > >> > Using camera /base/soc/i2c0mux/i2c@1/imx219@10 as cam0
> >> > > > >> > [27:27:09.965424746] [2391]  INFO Camera camera.cpp:1205 configuring streams: (0) 1916x1080-XRGB8888/Unset
> >> > > > >> > [27:27:09.967118705] [2392]  INFO SimplePipeline simple.cpp:1471 data->useConversion_ : 1
> >> > > > >> > [27:27:09.970895528] [2392]  INFO IPASoft soft_simple.cpp:264 IPASoft: Exposure 4-1759, gain 1-10.6667 (0.0966667)
> >> > > > >> > cam0: Capture 5 frames
> >> > > > >> > 98830.200120 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 8277120
> >> > > > >> > 98830.233445 (30.01 fps) cam0-stream0 seq: 000001 bytesused: 8277120
> >> > > > >> > 98830.266770 (30.01 fps) cam0-stream0 seq: 000002 bytesused: 8277120
> >> > > > >> > 98831.033291 (1.30 fps) cam0-stream0 seq: 000025 bytesused: 8277120
> >> > > > >> > 98831.799810 (1.30 fps) cam0-stream0 seq: 000048 bytesused: 8277120
> >> > > > >> >
> >> > > > >> >
> >> > > > >> > uajain1@uajain:~/libcamera$ ./build-soft/src/apps/cam/cam -c1 -srole=raw,width=1920,height=1080 -srole=viewfinder,width=1920,height=1080 -C10
> >> > > > >> > [27:27:26.239476459] [2395]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/uajain1/libcamera/build-soft/src/ipa' to the IPA search path
> >> > > > >> > [27:27:26.253694063] [2395]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+63-c33be195
> >> > > > >> > [27:27:26.572276146] [2396]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
> >> > > > >> > [27:27:26.572539792] [2396]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'imx219.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
> >> > > > >> > [27:27:26.574856667] [2396]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
> >> > > > >> > Camera configuration adjusted
> >> > > > >> > Using camera /base/soc/i2c0mux/i2c@1/imx219@10 as cam0
> >> > > > >> > [27:27:26.603776146] [2395]  INFO Camera camera.cpp:1205 configuring streams: (0) 1920x1080-SRGGB10/RAW (1) 1916x1080-XRGB8888/Unset
> >> > > > >> > [27:27:26.605656303] [2396]  INFO SimplePipeline simple.cpp:1471 data->useConversion_ : 1
> >> > > > >> > [27:27:26.609448803] [2396]  INFO IPASoft soft_simple.cpp:264 IPASoft: Exposure 4-1759, gain 1-10.6667 (0.0966667)
> >> > > > >> > cam0: Capture 10 frames
> >> > > > >> > 98846.739621 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 4147200 cam0-stream1 seq: 000000 bytesused: 8277120
> >> > > > >> > 98846.772946 (30.01 fps) cam0-stream0 seq: 000001 bytesused: 4147200 cam0-stream1 seq: 000001 bytesused: 8277120
> >> > > > >> > 98846.806274 (30.00 fps) cam0-stream0 seq: 000002 bytesused: 4147200 cam0-stream1 seq: 000002 bytesused: 8277120
> >> > > > >> > 98846.839601 (30.01 fps) cam0-stream0 seq: 000003 bytesused: 4147200 cam0-stream1 seq: 000003 bytesused: 8277120
> >> > > > >> > 98847.539465 (1.43 fps) cam0-stream0 seq: 000024 bytesused: 4147200 cam0-stream1 seq: 000024 bytesused: 8277120
> >> > > > >> > 98848.206006 (1.50 fps) cam0-stream0 seq: 000044 bytesused: 4147200 cam0-stream1 seq: 000044 bytesused: 8277120
> >> > > > >> > 98848.872546 (1.50 fps) cam0-stream0 seq: 000064 bytesused: 4147200 cam0-stream1 seq: 000064 bytesused: 8277120
> >> > > > >> > 98849.539084 (1.50 fps) cam0-stream0 seq: 000084 bytesused: 4147200 cam0-stream1 seq: 000084 bytesused: 8277120
> >> > > > >> > 98850.205624 (1.50 fps) cam0-stream0 seq: 000104 bytesused: 4147200 cam0-stream1 seq: 000104 bytesused: 8277120
> >> > > > >> > 98850.872164 (1.50 fps) cam0-stream0 seq: 000124 bytesused: 4147200 cam0-stream1 seq: 000124 bytesused: 8277120
> >> > > > >> >
> >> > > > >> > [1]: https://patchwork.libcamera.org/project/libcamera/list/?series=5281
> >> > > > >> > [2]: https://gitlab.freedesktop.org/uajain/libcamera/-/commits/uajain/simple/raw
> >> > > > >> >
> >> > > > >> > Milan Zamazal (3):
> >> > > > >> >   libcamera: simple: Set the number of software ISP streams to 2
> >> > > > >> >   libcamera: simple: Exclude raw configurations from output conversions
> >> > > > >> >   libcamera: simple: Require metadata only when software ISP is used
> >> > > > >> >
> >> > > > >> > Umang Jain (3):
> >> > > > >> >   libcamera: simple: Support StreamRole::Raw in generateConfiguration()
> >> > > > >> >   libcamera: simple: Validate raw streams if requested
> >> > > > >> >   libcamera: simple: Enable capturing of raw streams
> >> > > > >> >
> >> > > > >> >  src/libcamera/pipeline/simple/simple.cpp | 166 ++++++++++++++++++-----
> >> > > > >> >  1 file changed, 131 insertions(+), 35 deletions(-)
> >> > > > >> 
> >> > > >
>
Milan Zamazal July 23, 2025, 6:18 p.m. UTC | #10
Hi,

I posted v11.  It's a conservative update, the most important change
being that the raw stream size is adjusted now if needed.  v11 seems to
work for me better than both my v10 and Umang's RFC but independent
testing is needed.

I picked some ideas from Umang's patches while leaving others out, most
notably identifying the raw streams, as discussed elsewhere.

Regards,
Milan
Umang Jain July 24, 2025, 2:50 a.m. UTC | #11
On 2025-07-23 23:48, Milan Zamazal wrote:
> Hi,
> 
> I posted v11.  It's a conservative update, the most important change
> being that the raw stream size is adjusted now if needed.  v11 seems to
> work for me better than both my v10 and Umang's RFC but independent
> testing is needed.

Perhaps you can briefly state / give testing commands to understand the
diff between the two implementations? 
I assume you will be using one of the supportedDevices[] platform,
right?

> 
> I picked some ideas from Umang's patches while leaving others out, most
> notably identifying the raw streams, as discussed elsewhere.
> 
> Regards,
> Milan
Milan Zamazal July 24, 2025, 7:12 a.m. UTC | #12
Hi Umang,

uajain <uajain@igalia.com> writes:

> On 2025-07-23 23:48, Milan Zamazal wrote:
>> Hi,
>> 
>
>> I posted v11.  It's a conservative update, the most important change
>> being that the raw stream size is adjusted now if needed.  v11 seems to
>> work for me better than both my v10 and Umang's RFC but independent
>> testing is needed.
>
> Perhaps you can briefly state / give testing commands to understand the
> diff between the two implementations? 

Examples of the commands I tried:

- cam -c1 -C -D
  Basic sanity check with a display output at the given display device
  resolution.

- cam -c1 -C8 -s role=raw -Ffoo#
  Simple raw output.

- ./build-swisp/src/apps/cam/cam -c1 -C8 -s role=raw,width=1920,height=1080,pixelformat=SRGGB10 -Ffoo#
  Raw output with exact and correct parameters.

- ./build-swisp/src/apps/cam/cam -c1 -C8 -s role=raw,width=2000,height=1000,pixelformat=RGB888 -Ffoo#
  Raw output with parameters that must be adjusted.
  (Example of a command that didn't work with v10.)

- cam -c1 -C8 -s role=viewfinder,width=3276,height=2464,pixelformat=BGR888 -s role=raw,width=3280,height=2464,pixelformat=SRGGB10 -Ffoo#
  Both processed and raw outputs, with correct parameters.
  (Example of a command that didn't work with your RFC.)

- cam -c1 -C8 -s role=viewfinder,width=3276,height=2464,pixelformat=BGR888 -s role=raw -Ffoo#
  The same with implicit raw parameters.

- cam -c1 -C8 -s role=viewfinder,width=3276,height=2464,pixelformat=BGR888 -s role=raw,width=640,height=480,pixelformat=SRGGB10 -Ffoo#
  The same with raw output parameters that must be adjusted.

> I assume you will be using one of the supportedDevices[] platform,
> right?

Right.

>> I picked some ideas from Umang's patches while leaving others out, most
>> notably identifying the raw streams, as discussed elsewhere.
>> 
>> Regards,
>> Milan
Umang Jain July 24, 2025, 2:41 p.m. UTC | #13
On Thu, Jul 24, 2025 at 09:12:22AM +0200, Milan Zamazal wrote:
> Hi Umang,
> 
> uajain <uajain@igalia.com> writes:
> 
> > On 2025-07-23 23:48, Milan Zamazal wrote:
> >> Hi,
> >> 
> >
> >> I posted v11.  It's a conservative update, the most important change
> >> being that the raw stream size is adjusted now if needed.  v11 seems to
> >> work for me better than both my v10 and Umang's RFC but independent
> >> testing is needed.
> >
> > Perhaps you can briefly state / give testing commands to understand the
> > diff between the two implementations? 
> 
> Examples of the commands I tried:
> 
> - cam -c1 -C -D
>   Basic sanity check with a display output at the given display device
>   resolution.
> 
> - cam -c1 -C8 -s role=raw -Ffoo#
>   Simple raw output.
> 
> - ./build-swisp/src/apps/cam/cam -c1 -C8 -s role=raw,width=1920,height=1080,pixelformat=SRGGB10 -Ffoo#
>   Raw output with exact and correct parameters.
> 
> - ./build-swisp/src/apps/cam/cam -c1 -C8 -s role=raw,width=2000,height=1000,pixelformat=RGB888 -Ffoo#
>   Raw output with parameters that must be adjusted.
>   (Example of a command that didn't work with v10.)
> 
> - cam -c1 -C8 -s role=viewfinder,width=3276,height=2464,pixelformat=BGR888 -s role=raw,width=3280,height=2464,pixelformat=SRGGB10 -Ffoo#
>   Both processed and raw outputs, with correct parameters.
>   (Example of a command that didn't work with your RFC.)
> 
> - cam -c1 -C8 -s role=viewfinder,width=3276,height=2464,pixelformat=BGR888 -s role=raw -Ffoo#
>   The same with implicit raw parameters.
> 
> - cam -c1 -C8 -s role=viewfinder,width=3276,height=2464,pixelformat=BGR888 -s role=raw,width=640,height=480,pixelformat=SRGGB10 -Ffoo#
>   The same with raw output parameters that must be adjusted.
> 

The last 3 you mentioned which doesn't seem to work, I happen to take a
high level look. If you request the raw stream first on commandline, it
seems works then - so it seems something silly.. or might be the order
of generateConfiguration for role is called with `cam` utility,
not sure.

uajain1@uajain:~/libcamera$ ./build-soft/src/apps/cam/cam -c1 -C10 -srole=raw,width=3280,height=2464,pixelformat=SBGGR10 -srole=viewfinder,width=3276,height=2464,pixelformat=BGR888 
[1:29:41.500713519] [678]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/uajain1/libcamera/build-soft/src/ipa' to the IPA search path
[1:29:41.524538571] [678]  INFO Camera camera_manager.cpp:326 libcamera v0.5.1+63-00e1f068
[1:29:41.778350967] [679]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
[1:29:41.782933936] [679]  WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'imx219.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
[1:29:41.785440915] [679]  INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/uajain1/libcamera/src/ipa/simple/data'
Camera configuration adjusted
Using camera /base/soc/i2c0mux/i2c@1/imx219@10 as cam0
[1:29:41.816448779] [678]  INFO Camera camera.cpp:1205 configuring streams: (0) 3280x2464-SRGGB10/RAW (1) 3276x2464-BGR888/Unset
[1:29:41.818171436] [679]  INFO SimplePipeline simple.cpp:1479 data->useConversion_ : 1
[1:29:41.821951019] [679]  INFO IPASoft soft_simple.cpp:264 IPASoft: Exposure 4-3522, gain 1-10.6667 (0.0966667)
cam0: Capture 10 frames
5382.047170 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 16163840 cam0-stream1 seq: 000000 bytesused: 24226048
5382.113823 (15.00 fps) cam0-stream0 seq: 000001 bytesused: 16163840 cam0-stream1 seq: 000001 bytesused: 24226048
5382.180477 (15.00 fps) cam0-stream0 seq: 000002 bytesused: 16163840 cam0-stream1 seq: 000002 bytesused: 24226048
5382.247130 (15.00 fps) cam0-stream0 seq: 000003 bytesused: 16163840 cam0-stream1 seq: 000003 bytesused: 24226048
5385.113249 (0.35 fps) cam0-stream0 seq: 000046 bytesused: 16163840 cam0-stream1 seq: 000046 bytesused: 24226048
...

Unfortunately, I don't have much time to test and fix all these
papercuts (which I expected for this quick implementation)
in the short term.

> > I assume you will be using one of the supportedDevices[] platform,
> > right?
> 
> Right.
> 
> >> I picked some ideas from Umang's patches while leaving others out, most
> >> notably identifying the raw streams, as discussed elsewhere.
> >> 
> >> Regards,
> >> Milan
>