| Message ID | 20260826082328.24176-1-robertbozik@gmail.com |
|---|---|
| Headers | show |
| Series |
|
| Related | show |
Hi Robert, On Wed, Aug 26, 2026 at 10:23:21AM +0200, Robert Bozik wrote: > Hi, > > this series comes out of bringing up the OmniVision OV32C4 - the > under-display camera of the Lenovo Yoga Slim 9 14ILL10, behind an Intel > IPU7 - on libcamera's software ISP. The kernel driver is under review on > linux-media [1]. The first two patches are specific to that sensor; the > other five are software ISP and simple pipeline changes the sensor needed > to be usable outside of good light, and they are meant to stand on their > own. > > Patches 1-2 add the sensor to camera_sensor_properties and libipa's > CameraSensorHelper. Both numbers in the helper were measured on the > sensor rather than copied: the analogue gain is code/256 (ov13b10 and > ov08x40 use the same register with /128, which does not hold for this > part), and the black level is 4096, i.e. 64 at 10 bits, stable to one > LSB across the whole gain range. The unit cell size comes from the > vendor's product brief. > > Patch 3 implements FrameDurationLimits for the software ISP through the > sensor's vertical blanking. The AGC lets the exposure grow up to the > longest allowed frame and moves V4L2_CID_VBLANK along with it, so the > frame rate only drops when the exposure actually needs it; VBLANK is > written through DelayedControls ahead of the exposure so the driver can > extend the exposure range first. FrameDuration is reported in the > metadata. This relies on the driver calling __v4l2_ctrl_modify_range() > on the exposure control when VBLANK changes, which is what the in-tree > sensor drivers do. > > Patch 4 passes the controls given to Camera::start() through the simple > pipeline handler to the software ISP IPA (they were dropped before), so > that libcamerasrc framerate caps and PipeWire can set > FrameDurationLimits at all. > > Patch 5 makes the AGC converge faster when far from its target (a ratio > step of at most 2x per iteration, the existing proportional step near > the target) and adds digital gain, applied in the ISP colour gains after > black level subtraction once exposure and analogue gain are exhausted. > It is bounded by a new maxDigitalGain tuning key, off by default, and > reported as DigitalGain in the metadata. > > Patch 6 adds a maxFrameDuration tuning key that bounds the default frame > duration for applications that never set FrameDurationLimits (PipeWire > 1.6.2 does not), so a sensor with a long VBLANK range does not fall to > sub-1 fps in the dark by default. An explicit FrameDurationLimits still > allows the full range. > > Patch 7 adds temporal noise reduction to the GPU debayer: a pass before > demosaicing blends the raw frame with the filtered history, with motion > detected per 4x4 block against a threshold derived from a sensor noise > model (variance = noiseSlope * signal + noiseFloor), configured through a > new Denoise algorithm in the tuning file. The CPU debayer ignores the > parameters for now. > > Everything was tested on the machine above against master 35c137c2 with > cam, gstreamer and PipeWire: exposure and frame length were verified > linear on the raw data up to 306 ms per frame, a 30 -> 15 fps request > moves VBLANK from 774 to 3389 lines, and the AGC settles in about a > second where it previously did not settle at all. > > Tooling disclosure: this series was written with the help of an AI > coding assistant (Claude, Anthropic; claude-opus-5 and claude-fable-5) > in an extended interactive session. The assistant drafted the code and > this cover letter from my descriptions and measurements; I ran every > test, reviewed all of the code and take responsibility for it. I have not > added Assisted-by trailers to the commits because utils/checkstyle.py > rejects that key; I am happy to add them if you would rather have them. I'm afraid we will have to refuse this contribution. I started drafting an AI policy for the libcamera project which I need to finish (sorry for not doing so before you spent time on this), and it will be along the line of forbidding generate AI usage to create any code, documentation or any material meant to be read by developers. > The sensor's tuning file is deliberately not part of this series: its > colour matrices are derived from the vendor's calibration data and I > have not established whether they can be redistributed. > > [1] https://lore.kernel.org/linux-media/20260826072002.14357-1-robertbozik@gmail.com/ > > Thanks, > Robert > > Robert Bozik (7): > libcamera: camera_sensor_properties: Add OV32C4 > ipa: libipa: camera_sensor_helper: Add OV32C4 > ipa: softisp: Support FrameDurationLimits through vertical blanking > pipeline: simple: Pass the startup controls to the software ISP IPA > ipa: softisp: Converge faster and add digital gain to the AGC > ipa: softisp: Let the tuning file bound the default frame duration > libcamera: software_isp: Add temporal noise reduction to the GPU > debayer > > .../internal/software_isp/debayer_params.h | 14 + > .../internal/software_isp/software_isp.h | 2 +- > include/libcamera/ipa/softisp.mojom | 2 +- > src/ipa/libipa/camera_sensor_helper.cpp | 17 + > src/ipa/softisp/algorithms/agc.cpp | 295 +++++++++++++++++- > src/ipa/softisp/algorithms/agc.h | 18 ++ > src/ipa/softisp/algorithms/denoise.cpp | 89 ++++++ > src/ipa/softisp/algorithms/denoise.h | 36 +++ > src/ipa/softisp/algorithms/meson.build | 1 + > src/ipa/softisp/ipa_context.h | 25 ++ > src/ipa/softisp/softisp.cpp | 60 +++- > src/libcamera/pipeline/simple/simple.cpp | 24 +- > .../sensor/camera_sensor_properties.cpp | 5 + > src/libcamera/shaders/meson.build | 1 + > src/libcamera/shaders/temporal.frag | 131 ++++++++ > src/libcamera/software_isp/debayer.cpp | 13 + > src/libcamera/software_isp/debayer_egl.cpp | 172 ++++++++++ > src/libcamera/software_isp/debayer_egl.h | 23 ++ > src/libcamera/software_isp/software_isp.cpp | 5 +- > 19 files changed, 907 insertions(+), 26 deletions(-) > create mode 100644 src/ipa/softisp/algorithms/denoise.cpp > create mode 100644 src/ipa/softisp/algorithms/denoise.h > create mode 100644 src/libcamera/shaders/temporal.frag > > > base-commit: 35c137c2f3e7104b96702c649344fafc91d8e233
Hi Laurent, understood, and thank you for saying so clearly and quickly. I will keep the series out of tree; it stays available in my repository for anyone with the same hardware. One question for when the policy is written, so that I do not waste your time again: patches 1 and 2 are only measured constants (unit cell size, analogue gain code/256, black level 4096 for the OV32C4). If a future sensor entry like that were written by hand, without any assistant, would it fall under the policy as well, or only work where an assistant was involved? No need to answer now if the policy will cover it. Thanks, Robert st 26. 8. 2026 o 11:44 Laurent Pinchart <laurent.pinchart@ideasonboard.com> napísal(a): > Hi Robert, > > On Wed, Aug 26, 2026 at 10:23:21AM +0200, Robert Bozik wrote: > > Hi, > > > > this series comes out of bringing up the OmniVision OV32C4 - the > > under-display camera of the Lenovo Yoga Slim 9 14ILL10, behind an Intel > > IPU7 - on libcamera's software ISP. The kernel driver is under review on > > linux-media [1]. The first two patches are specific to that sensor; the > > other five are software ISP and simple pipeline changes the sensor needed > > to be usable outside of good light, and they are meant to stand on their > > own. > > > > Patches 1-2 add the sensor to camera_sensor_properties and libipa's > > CameraSensorHelper. Both numbers in the helper were measured on the > > sensor rather than copied: the analogue gain is code/256 (ov13b10 and > > ov08x40 use the same register with /128, which does not hold for this > > part), and the black level is 4096, i.e. 64 at 10 bits, stable to one > > LSB across the whole gain range. The unit cell size comes from the > > vendor's product brief. > > > > Patch 3 implements FrameDurationLimits for the software ISP through the > > sensor's vertical blanking. The AGC lets the exposure grow up to the > > longest allowed frame and moves V4L2_CID_VBLANK along with it, so the > > frame rate only drops when the exposure actually needs it; VBLANK is > > written through DelayedControls ahead of the exposure so the driver can > > extend the exposure range first. FrameDuration is reported in the > > metadata. This relies on the driver calling __v4l2_ctrl_modify_range() > > on the exposure control when VBLANK changes, which is what the in-tree > > sensor drivers do. > > > > Patch 4 passes the controls given to Camera::start() through the simple > > pipeline handler to the software ISP IPA (they were dropped before), so > > that libcamerasrc framerate caps and PipeWire can set > > FrameDurationLimits at all. > > > > Patch 5 makes the AGC converge faster when far from its target (a ratio > > step of at most 2x per iteration, the existing proportional step near > > the target) and adds digital gain, applied in the ISP colour gains after > > black level subtraction once exposure and analogue gain are exhausted. > > It is bounded by a new maxDigitalGain tuning key, off by default, and > > reported as DigitalGain in the metadata. > > > > Patch 6 adds a maxFrameDuration tuning key that bounds the default frame > > duration for applications that never set FrameDurationLimits (PipeWire > > 1.6.2 does not), so a sensor with a long VBLANK range does not fall to > > sub-1 fps in the dark by default. An explicit FrameDurationLimits still > > allows the full range. > > > > Patch 7 adds temporal noise reduction to the GPU debayer: a pass before > > demosaicing blends the raw frame with the filtered history, with motion > > detected per 4x4 block against a threshold derived from a sensor noise > > model (variance = noiseSlope * signal + noiseFloor), configured through a > > new Denoise algorithm in the tuning file. The CPU debayer ignores the > > parameters for now. > > > > Everything was tested on the machine above against master 35c137c2 with > > cam, gstreamer and PipeWire: exposure and frame length were verified > > linear on the raw data up to 306 ms per frame, a 30 -> 15 fps request > > moves VBLANK from 774 to 3389 lines, and the AGC settles in about a > > second where it previously did not settle at all. > > > > Tooling disclosure: this series was written with the help of an AI > > coding assistant (Claude, Anthropic; claude-opus-5 and claude-fable-5) > > in an extended interactive session. The assistant drafted the code and > > this cover letter from my descriptions and measurements; I ran every > > test, reviewed all of the code and take responsibility for it. I have not > > added Assisted-by trailers to the commits because utils/checkstyle.py > > rejects that key; I am happy to add them if you would rather have them. > > I'm afraid we will have to refuse this contribution. I started drafting > an AI policy for the libcamera project which I need to finish (sorry for > not doing so before you spent time on this), and it will be along the > line of forbidding generate AI usage to create any code, documentation > or any material meant to be read by developers. > > > The sensor's tuning file is deliberately not part of this series: its > > colour matrices are derived from the vendor's calibration data and I > > have not established whether they can be redistributed. > > > > [1] > https://lore.kernel.org/linux-media/20260826072002.14357-1-robertbozik@gmail.com/ > > > > Thanks, > > Robert > > > > Robert Bozik (7): > > libcamera: camera_sensor_properties: Add OV32C4 > > ipa: libipa: camera_sensor_helper: Add OV32C4 > > ipa: softisp: Support FrameDurationLimits through vertical blanking > > pipeline: simple: Pass the startup controls to the software ISP IPA > > ipa: softisp: Converge faster and add digital gain to the AGC > > ipa: softisp: Let the tuning file bound the default frame duration > > libcamera: software_isp: Add temporal noise reduction to the GPU > > debayer > > > > .../internal/software_isp/debayer_params.h | 14 + > > .../internal/software_isp/software_isp.h | 2 +- > > include/libcamera/ipa/softisp.mojom | 2 +- > > src/ipa/libipa/camera_sensor_helper.cpp | 17 + > > src/ipa/softisp/algorithms/agc.cpp | 295 +++++++++++++++++- > > src/ipa/softisp/algorithms/agc.h | 18 ++ > > src/ipa/softisp/algorithms/denoise.cpp | 89 ++++++ > > src/ipa/softisp/algorithms/denoise.h | 36 +++ > > src/ipa/softisp/algorithms/meson.build | 1 + > > src/ipa/softisp/ipa_context.h | 25 ++ > > src/ipa/softisp/softisp.cpp | 60 +++- > > src/libcamera/pipeline/simple/simple.cpp | 24 +- > > .../sensor/camera_sensor_properties.cpp | 5 + > > src/libcamera/shaders/meson.build | 1 + > > src/libcamera/shaders/temporal.frag | 131 ++++++++ > > src/libcamera/software_isp/debayer.cpp | 13 + > > src/libcamera/software_isp/debayer_egl.cpp | 172 ++++++++++ > > src/libcamera/software_isp/debayer_egl.h | 23 ++ > > src/libcamera/software_isp/software_isp.cpp | 5 +- > > 19 files changed, 907 insertions(+), 26 deletions(-) > > create mode 100644 src/ipa/softisp/algorithms/denoise.cpp > > create mode 100644 src/ipa/softisp/algorithms/denoise.h > > create mode 100644 src/libcamera/shaders/temporal.frag > > > > > > base-commit: 35c137c2f3e7104b96702c649344fafc91d8e233 > > -- > Regards, > > Laurent Pinchart >
On Wed, Aug 26, 2026 at 12:04:43PM +0200, Róbert Božik wrote: > Hi Laurent, > > understood, and thank you for saying so clearly and quickly. I will keep > the series out of tree; it stays available in my repository for anyone > with the same hardware. > > One question for when the policy is written, so that I do not waste your > time again: patches 1 and 2 are only measured constants (unit cell size, > analogue gain code/256, black level 4096 for the OV32C4). If a future > sensor entry like that were written by hand, without any assistant, would > it fall under the policy as well, or only work where an assistant was > involved? No need to answer now if the policy will cover it. I'm not sure to understand the question. The policy will be about usage of generative AI. Writing patches by hand is obviously fine :-) If the question is about adding camera helper entries containing values that were measured instead of retrieved from datasheets, that's fine. As long, of course, as the values can be reasonably proven to be correct. > st 26. 8. 2026 o 11:44 Laurent Pinchart napísal(a): > > > Hi Robert, > > > > On Wed, Aug 26, 2026 at 10:23:21AM +0200, Robert Bozik wrote: > > > Hi, > > > > > > this series comes out of bringing up the OmniVision OV32C4 - the > > > under-display camera of the Lenovo Yoga Slim 9 14ILL10, behind an Intel > > > IPU7 - on libcamera's software ISP. The kernel driver is under review on > > > linux-media [1]. The first two patches are specific to that sensor; the > > > other five are software ISP and simple pipeline changes the sensor needed > > > to be usable outside of good light, and they are meant to stand on their > > > own. > > > > > > Patches 1-2 add the sensor to camera_sensor_properties and libipa's > > > CameraSensorHelper. Both numbers in the helper were measured on the > > > sensor rather than copied: the analogue gain is code/256 (ov13b10 and > > > ov08x40 use the same register with /128, which does not hold for this > > > part), and the black level is 4096, i.e. 64 at 10 bits, stable to one > > > LSB across the whole gain range. The unit cell size comes from the > > > vendor's product brief. > > > > > > Patch 3 implements FrameDurationLimits for the software ISP through the > > > sensor's vertical blanking. The AGC lets the exposure grow up to the > > > longest allowed frame and moves V4L2_CID_VBLANK along with it, so the > > > frame rate only drops when the exposure actually needs it; VBLANK is > > > written through DelayedControls ahead of the exposure so the driver can > > > extend the exposure range first. FrameDuration is reported in the > > > metadata. This relies on the driver calling __v4l2_ctrl_modify_range() > > > on the exposure control when VBLANK changes, which is what the in-tree > > > sensor drivers do. > > > > > > Patch 4 passes the controls given to Camera::start() through the simple > > > pipeline handler to the software ISP IPA (they were dropped before), so > > > that libcamerasrc framerate caps and PipeWire can set > > > FrameDurationLimits at all. > > > > > > Patch 5 makes the AGC converge faster when far from its target (a ratio > > > step of at most 2x per iteration, the existing proportional step near > > > the target) and adds digital gain, applied in the ISP colour gains after > > > black level subtraction once exposure and analogue gain are exhausted. > > > It is bounded by a new maxDigitalGain tuning key, off by default, and > > > reported as DigitalGain in the metadata. > > > > > > Patch 6 adds a maxFrameDuration tuning key that bounds the default frame > > > duration for applications that never set FrameDurationLimits (PipeWire > > > 1.6.2 does not), so a sensor with a long VBLANK range does not fall to > > > sub-1 fps in the dark by default. An explicit FrameDurationLimits still > > > allows the full range. > > > > > > Patch 7 adds temporal noise reduction to the GPU debayer: a pass before > > > demosaicing blends the raw frame with the filtered history, with motion > > > detected per 4x4 block against a threshold derived from a sensor noise > > > model (variance = noiseSlope * signal + noiseFloor), configured through a > > > new Denoise algorithm in the tuning file. The CPU debayer ignores the > > > parameters for now. > > > > > > Everything was tested on the machine above against master 35c137c2 with > > > cam, gstreamer and PipeWire: exposure and frame length were verified > > > linear on the raw data up to 306 ms per frame, a 30 -> 15 fps request > > > moves VBLANK from 774 to 3389 lines, and the AGC settles in about a > > > second where it previously did not settle at all. > > > > > > Tooling disclosure: this series was written with the help of an AI > > > coding assistant (Claude, Anthropic; claude-opus-5 and claude-fable-5) > > > in an extended interactive session. The assistant drafted the code and > > > this cover letter from my descriptions and measurements; I ran every > > > test, reviewed all of the code and take responsibility for it. I have not > > > added Assisted-by trailers to the commits because utils/checkstyle.py > > > rejects that key; I am happy to add them if you would rather have them. > > > > I'm afraid we will have to refuse this contribution. I started drafting > > an AI policy for the libcamera project which I need to finish (sorry for > > not doing so before you spent time on this), and it will be along the > > line of forbidding generate AI usage to create any code, documentation > > or any material meant to be read by developers. > > > > > The sensor's tuning file is deliberately not part of this series: its > > > colour matrices are derived from the vendor's calibration data and I > > > have not established whether they can be redistributed. > > > > > > [1] > > https://lore.kernel.org/linux-media/20260826072002.14357-1-robertbozik@gmail.com/ > > > > > > Thanks, > > > Robert > > > > > > Robert Bozik (7): > > > libcamera: camera_sensor_properties: Add OV32C4 > > > ipa: libipa: camera_sensor_helper: Add OV32C4 > > > ipa: softisp: Support FrameDurationLimits through vertical blanking > > > pipeline: simple: Pass the startup controls to the software ISP IPA > > > ipa: softisp: Converge faster and add digital gain to the AGC > > > ipa: softisp: Let the tuning file bound the default frame duration > > > libcamera: software_isp: Add temporal noise reduction to the GPU > > > debayer > > > > > > .../internal/software_isp/debayer_params.h | 14 + > > > .../internal/software_isp/software_isp.h | 2 +- > > > include/libcamera/ipa/softisp.mojom | 2 +- > > > src/ipa/libipa/camera_sensor_helper.cpp | 17 + > > > src/ipa/softisp/algorithms/agc.cpp | 295 +++++++++++++++++- > > > src/ipa/softisp/algorithms/agc.h | 18 ++ > > > src/ipa/softisp/algorithms/denoise.cpp | 89 ++++++ > > > src/ipa/softisp/algorithms/denoise.h | 36 +++ > > > src/ipa/softisp/algorithms/meson.build | 1 + > > > src/ipa/softisp/ipa_context.h | 25 ++ > > > src/ipa/softisp/softisp.cpp | 60 +++- > > > src/libcamera/pipeline/simple/simple.cpp | 24 +- > > > .../sensor/camera_sensor_properties.cpp | 5 + > > > src/libcamera/shaders/meson.build | 1 + > > > src/libcamera/shaders/temporal.frag | 131 ++++++++ > > > src/libcamera/software_isp/debayer.cpp | 13 + > > > src/libcamera/software_isp/debayer_egl.cpp | 172 ++++++++++ > > > src/libcamera/software_isp/debayer_egl.h | 23 ++ > > > src/libcamera/software_isp/software_isp.cpp | 5 +- > > > 19 files changed, 907 insertions(+), 26 deletions(-) > > > create mode 100644 src/ipa/softisp/algorithms/denoise.cpp > > > create mode 100644 src/ipa/softisp/algorithms/denoise.h > > > create mode 100644 src/libcamera/shaders/temporal.frag > > > > > > > > > base-commit: 35c137c2f3e7104b96702c649344fafc91d8e233
Hi Laurent, to be completely clear about how this work came about: I used an AI assistant to analyse and reverse-engineer the Windows camera driver, and together with it, through a very large number of measurements on the sensor, we worked out how the hardware behaves. The assistant then wrote most of the driver and the libcamera code, and I adapted it, changed values and tested it until it worked on my machine. That is the honest description :-) - it was not written by hand, and I do not want to argue about who typed which line. :-) I understand this means it cannot go into libcamera under the policy, and I will keep it out of tree. I will put it on GitHub with an install.sh so anyone with this laptop can install it. Thank you for taking the time. Have a nice day, Robert st 26. 8. 2026 o 14:54 Laurent Pinchart <laurent.pinchart@ideasonboard.com> napísal(a): > On Wed, Aug 26, 2026 at 12:04:43PM +0200, Róbert Božik wrote: > > Hi Laurent, > > > > understood, and thank you for saying so clearly and quickly. I will keep > > the series out of tree; it stays available in my repository for anyone > > with the same hardware. > > > > One question for when the policy is written, so that I do not waste your > > time again: patches 1 and 2 are only measured constants (unit cell size, > > analogue gain code/256, black level 4096 for the OV32C4). If a future > > sensor entry like that were written by hand, without any assistant, would > > it fall under the policy as well, or only work where an assistant was > > involved? No need to answer now if the policy will cover it. > > I'm not sure to understand the question. The policy will be about usage > of generative AI. Writing patches by hand is obviously fine :-) > > If the question is about adding camera helper entries containing values > that were measured instead of retrieved from datasheets, that's fine. As > long, of course, as the values can be reasonably proven to be correct. > > > st 26. 8. 2026 o 11:44 Laurent Pinchart napísal(a): > > > > > Hi Robert, > > > > > > On Wed, Aug 26, 2026 at 10:23:21AM +0200, Robert Bozik wrote: > > > > Hi, > > > > > > > > this series comes out of bringing up the OmniVision OV32C4 - the > > > > under-display camera of the Lenovo Yoga Slim 9 14ILL10, behind an > Intel > > > > IPU7 - on libcamera's software ISP. The kernel driver is under > review on > > > > linux-media [1]. The first two patches are specific to that sensor; > the > > > > other five are software ISP and simple pipeline changes the sensor > needed > > > > to be usable outside of good light, and they are meant to stand on > their > > > > own. > > > > > > > > Patches 1-2 add the sensor to camera_sensor_properties and libipa's > > > > CameraSensorHelper. Both numbers in the helper were measured on the > > > > sensor rather than copied: the analogue gain is code/256 (ov13b10 and > > > > ov08x40 use the same register with /128, which does not hold for this > > > > part), and the black level is 4096, i.e. 64 at 10 bits, stable to one > > > > LSB across the whole gain range. The unit cell size comes from the > > > > vendor's product brief. > > > > > > > > Patch 3 implements FrameDurationLimits for the software ISP through > the > > > > sensor's vertical blanking. The AGC lets the exposure grow up to the > > > > longest allowed frame and moves V4L2_CID_VBLANK along with it, so the > > > > frame rate only drops when the exposure actually needs it; VBLANK is > > > > written through DelayedControls ahead of the exposure so the driver > can > > > > extend the exposure range first. FrameDuration is reported in the > > > > metadata. This relies on the driver calling > __v4l2_ctrl_modify_range() > > > > on the exposure control when VBLANK changes, which is what the > in-tree > > > > sensor drivers do. > > > > > > > > Patch 4 passes the controls given to Camera::start() through the > simple > > > > pipeline handler to the software ISP IPA (they were dropped before), > so > > > > that libcamerasrc framerate caps and PipeWire can set > > > > FrameDurationLimits at all. > > > > > > > > Patch 5 makes the AGC converge faster when far from its target (a > ratio > > > > step of at most 2x per iteration, the existing proportional step near > > > > the target) and adds digital gain, applied in the ISP colour gains > after > > > > black level subtraction once exposure and analogue gain are > exhausted. > > > > It is bounded by a new maxDigitalGain tuning key, off by default, and > > > > reported as DigitalGain in the metadata. > > > > > > > > Patch 6 adds a maxFrameDuration tuning key that bounds the default > frame > > > > duration for applications that never set FrameDurationLimits > (PipeWire > > > > 1.6.2 does not), so a sensor with a long VBLANK range does not fall > to > > > > sub-1 fps in the dark by default. An explicit FrameDurationLimits > still > > > > allows the full range. > > > > > > > > Patch 7 adds temporal noise reduction to the GPU debayer: a pass > before > > > > demosaicing blends the raw frame with the filtered history, with > motion > > > > detected per 4x4 block against a threshold derived from a sensor > noise > > > > model (variance = noiseSlope * signal + noiseFloor), configured > through a > > > > new Denoise algorithm in the tuning file. The CPU debayer ignores the > > > > parameters for now. > > > > > > > > Everything was tested on the machine above against master 35c137c2 > with > > > > cam, gstreamer and PipeWire: exposure and frame length were verified > > > > linear on the raw data up to 306 ms per frame, a 30 -> 15 fps request > > > > moves VBLANK from 774 to 3389 lines, and the AGC settles in about a > > > > second where it previously did not settle at all. > > > > > > > > Tooling disclosure: this series was written with the help of an AI > > > > coding assistant (Claude, Anthropic; claude-opus-5 and > claude-fable-5) > > > > in an extended interactive session. The assistant drafted the code > and > > > > this cover letter from my descriptions and measurements; I ran every > > > > test, reviewed all of the code and take responsibility for it. I > have not > > > > added Assisted-by trailers to the commits because utils/checkstyle.py > > > > rejects that key; I am happy to add them if you would rather have > them. > > > > > > I'm afraid we will have to refuse this contribution. I started drafting > > > an AI policy for the libcamera project which I need to finish (sorry > for > > > not doing so before you spent time on this), and it will be along the > > > line of forbidding generate AI usage to create any code, documentation > > > or any material meant to be read by developers. > > > > > > > The sensor's tuning file is deliberately not part of this series: its > > > > colour matrices are derived from the vendor's calibration data and I > > > > have not established whether they can be redistributed. > > > > > > > > [1] > > > > https://lore.kernel.org/linux-media/20260826072002.14357-1-robertbozik@gmail.com/ > > > > > > > > Thanks, > > > > Robert > > > > > > > > Robert Bozik (7): > > > > libcamera: camera_sensor_properties: Add OV32C4 > > > > ipa: libipa: camera_sensor_helper: Add OV32C4 > > > > ipa: softisp: Support FrameDurationLimits through vertical blanking > > > > pipeline: simple: Pass the startup controls to the software ISP IPA > > > > ipa: softisp: Converge faster and add digital gain to the AGC > > > > ipa: softisp: Let the tuning file bound the default frame duration > > > > libcamera: software_isp: Add temporal noise reduction to the GPU > > > > debayer > > > > > > > > .../internal/software_isp/debayer_params.h | 14 + > > > > .../internal/software_isp/software_isp.h | 2 +- > > > > include/libcamera/ipa/softisp.mojom | 2 +- > > > > src/ipa/libipa/camera_sensor_helper.cpp | 17 + > > > > src/ipa/softisp/algorithms/agc.cpp | 295 > +++++++++++++++++- > > > > src/ipa/softisp/algorithms/agc.h | 18 ++ > > > > src/ipa/softisp/algorithms/denoise.cpp | 89 ++++++ > > > > src/ipa/softisp/algorithms/denoise.h | 36 +++ > > > > src/ipa/softisp/algorithms/meson.build | 1 + > > > > src/ipa/softisp/ipa_context.h | 25 ++ > > > > src/ipa/softisp/softisp.cpp | 60 +++- > > > > src/libcamera/pipeline/simple/simple.cpp | 24 +- > > > > .../sensor/camera_sensor_properties.cpp | 5 + > > > > src/libcamera/shaders/meson.build | 1 + > > > > src/libcamera/shaders/temporal.frag | 131 ++++++++ > > > > src/libcamera/software_isp/debayer.cpp | 13 + > > > > src/libcamera/software_isp/debayer_egl.cpp | 172 ++++++++++ > > > > src/libcamera/software_isp/debayer_egl.h | 23 ++ > > > > src/libcamera/software_isp/software_isp.cpp | 5 +- > > > > 19 files changed, 907 insertions(+), 26 deletions(-) > > > > create mode 100644 src/ipa/softisp/algorithms/denoise.cpp > > > > create mode 100644 src/ipa/softisp/algorithms/denoise.h > > > > create mode 100644 src/libcamera/shaders/temporal.frag > > > > > > > > > > > > base-commit: 35c137c2f3e7104b96702c649344fafc91d8e233 > > -- > Regards, > > Laurent Pinchart >
Hi, this series comes out of bringing up the OmniVision OV32C4 - the under-display camera of the Lenovo Yoga Slim 9 14ILL10, behind an Intel IPU7 - on libcamera's software ISP. The kernel driver is under review on linux-media [1]. The first two patches are specific to that sensor; the other five are software ISP and simple pipeline changes the sensor needed to be usable outside of good light, and they are meant to stand on their own. Patches 1-2 add the sensor to camera_sensor_properties and libipa's CameraSensorHelper. Both numbers in the helper were measured on the sensor rather than copied: the analogue gain is code/256 (ov13b10 and ov08x40 use the same register with /128, which does not hold for this part), and the black level is 4096, i.e. 64 at 10 bits, stable to one LSB across the whole gain range. The unit cell size comes from the vendor's product brief. Patch 3 implements FrameDurationLimits for the software ISP through the sensor's vertical blanking. The AGC lets the exposure grow up to the longest allowed frame and moves V4L2_CID_VBLANK along with it, so the frame rate only drops when the exposure actually needs it; VBLANK is written through DelayedControls ahead of the exposure so the driver can extend the exposure range first. FrameDuration is reported in the metadata. This relies on the driver calling __v4l2_ctrl_modify_range() on the exposure control when VBLANK changes, which is what the in-tree sensor drivers do. Patch 4 passes the controls given to Camera::start() through the simple pipeline handler to the software ISP IPA (they were dropped before), so that libcamerasrc framerate caps and PipeWire can set FrameDurationLimits at all. Patch 5 makes the AGC converge faster when far from its target (a ratio step of at most 2x per iteration, the existing proportional step near the target) and adds digital gain, applied in the ISP colour gains after black level subtraction once exposure and analogue gain are exhausted. It is bounded by a new maxDigitalGain tuning key, off by default, and reported as DigitalGain in the metadata. Patch 6 adds a maxFrameDuration tuning key that bounds the default frame duration for applications that never set FrameDurationLimits (PipeWire 1.6.2 does not), so a sensor with a long VBLANK range does not fall to sub-1 fps in the dark by default. An explicit FrameDurationLimits still allows the full range. Patch 7 adds temporal noise reduction to the GPU debayer: a pass before demosaicing blends the raw frame with the filtered history, with motion detected per 4x4 block against a threshold derived from a sensor noise model (variance = noiseSlope * signal + noiseFloor), configured through a new Denoise algorithm in the tuning file. The CPU debayer ignores the parameters for now. Everything was tested on the machine above against master 35c137c2 with cam, gstreamer and PipeWire: exposure and frame length were verified linear on the raw data up to 306 ms per frame, a 30 -> 15 fps request moves VBLANK from 774 to 3389 lines, and the AGC settles in about a second where it previously did not settle at all. Tooling disclosure: this series was written with the help of an AI coding assistant (Claude, Anthropic; claude-opus-5 and claude-fable-5) in an extended interactive session. The assistant drafted the code and this cover letter from my descriptions and measurements; I ran every test, reviewed all of the code and take responsibility for it. I have not added Assisted-by trailers to the commits because utils/checkstyle.py rejects that key; I am happy to add them if you would rather have them. The sensor's tuning file is deliberately not part of this series: its colour matrices are derived from the vendor's calibration data and I have not established whether they can be redistributed. [1] https://lore.kernel.org/linux-media/20260826072002.14357-1-robertbozik@gmail.com/ Thanks, Robert Robert Bozik (7): libcamera: camera_sensor_properties: Add OV32C4 ipa: libipa: camera_sensor_helper: Add OV32C4 ipa: softisp: Support FrameDurationLimits through vertical blanking pipeline: simple: Pass the startup controls to the software ISP IPA ipa: softisp: Converge faster and add digital gain to the AGC ipa: softisp: Let the tuning file bound the default frame duration libcamera: software_isp: Add temporal noise reduction to the GPU debayer .../internal/software_isp/debayer_params.h | 14 + .../internal/software_isp/software_isp.h | 2 +- include/libcamera/ipa/softisp.mojom | 2 +- src/ipa/libipa/camera_sensor_helper.cpp | 17 + src/ipa/softisp/algorithms/agc.cpp | 295 +++++++++++++++++- src/ipa/softisp/algorithms/agc.h | 18 ++ src/ipa/softisp/algorithms/denoise.cpp | 89 ++++++ src/ipa/softisp/algorithms/denoise.h | 36 +++ src/ipa/softisp/algorithms/meson.build | 1 + src/ipa/softisp/ipa_context.h | 25 ++ src/ipa/softisp/softisp.cpp | 60 +++- src/libcamera/pipeline/simple/simple.cpp | 24 +- .../sensor/camera_sensor_properties.cpp | 5 + src/libcamera/shaders/meson.build | 1 + src/libcamera/shaders/temporal.frag | 131 ++++++++ src/libcamera/software_isp/debayer.cpp | 13 + src/libcamera/software_isp/debayer_egl.cpp | 172 ++++++++++ src/libcamera/software_isp/debayer_egl.h | 23 ++ src/libcamera/software_isp/software_isp.cpp | 5 +- 19 files changed, 907 insertions(+), 26 deletions(-) create mode 100644 src/ipa/softisp/algorithms/denoise.cpp create mode 100644 src/ipa/softisp/algorithms/denoise.h create mode 100644 src/libcamera/shaders/temporal.frag base-commit: 35c137c2f3e7104b96702c649344fafc91d8e233