[RFC,00/12] Bayer Re-Processing
mbox series

Message ID 20250827090739.86955-1-david.plowman@raspberrypi.com
Headers show
Series
  • Bayer Re-Processing
Related show

Message

David Plowman Aug. 27, 2025, 9:07 a.m. UTC 
Hi everyone

Well, I'm pleased to let folks know that the hoary subject of Bayer
Re-Processing is finally making some progress. How much, I'll let you
judge for yourselves, but nonetheless I'm hoping that some progress is
better than none at all!

What do these patches do?

There are two main things going on here.

Firstly, there is the ability to make a "memory camera". A "memory
camera" is the gateway to Bayer re-processing, because it gives you
access to your hardware ISP as a memory-to-memory device. You can
queue requests to it much as you do with regular cameras, only in this
case, the raw stream is an input to the system and not an output
(there's a new "raw input" stream role).

Secondly, the Raspberry Pi pipeline handler is updated to cope with
memory cameras and these slightly different kinds of requests.

The first 5 patches or so are mostly introducing memory cameras, how
we create them, and adding some general helper classes (such as the
CameraSensorMemory to use in place of the CameraSensorRaw) which make
the rest of the code work with fewer interventions. They are not, or
are not particularly, concerned with the Raspberry Pi (though I do
take some Pi code from somewhere and move it to the BayerFormat
class).

The remaining patches are then pretty much all for the Pi PH and
IPA. Having thought initially that I might re-factor stuff more
generally, I've backed away from that and am using the same PH for
regular and memory cameras. There are some places where I've gathered
up the Front End / Back End stuff and moved them into separate
functions, which does limit the collateral damage somewhat. There is
support only for the PiSP platform (Pi 5) not the older VC4 platform
(all other devices).  I can't say it's all lovely, but you know, nor
is it the biggest skeleton ever to inhabit my closet!

As I said, this code runs and works. If you want to try it:

You can find a libcamera branch with these patches here:
https://github.com/davidplowman/libcamera/tree/reprocessing

You can find a program that uses a memory camera (to convert DNG files
to JPEGs) here:
https://github.com/davidplowman/libcamera-apps/blob/reprocessing/apps/rpicam_convert.cpp
(check out and build that code branch).

Obviously I'm always wanting feedback, particularly on the not-so-Pi
changes - namely how people would like to present memory cameras
within libcamera, and have users interact with them.

Thanks!

David

David Plowman (12):
  libcamera: Infrastructure to ask for "memory" cameras
  libcamera: Add BayerFormat::toMbusCode
  pipeline: rpi: pisp: Use the new BayerFormat::toMbusCode() instead
  libcamera: Add a direction (input or output) to the stream
    configuration
  libcamera: sensor: Add CameraSensorMemory class
  ipa: rpi: cam_helper: Add CamHelperDefault class
  libcamera: pipeline: rpi: Allow creation of the first "memory" camera
  pipeline: rpi: Allow generation of raw input configurations
  libcamera: Don't wait for input streams to complete
  ipa: rpi: Support memory cameras
  pipeline: rpi: Support memory cameras
  pipline: rpi: Support memory cameras processing requests

 include/libcamera/camera_manager.h            |   4 +
 include/libcamera/internal/bayer_format.h     |   1 +
 include/libcamera/internal/camera_manager.h   |   5 +
 include/libcamera/internal/camera_sensor.h    |   2 +
 .../libcamera/internal/camera_sensor_memory.h | 109 +++++
 include/libcamera/internal/meson.build        |   1 +
 include/libcamera/internal/pipeline_handler.h |   8 +
 include/libcamera/stream.h                    |  10 +
 src/ipa/rpi/cam_helper/cam_helper_default.cpp |  45 ++
 src/ipa/rpi/cam_helper/meson.build            |   1 +
 src/ipa/rpi/common/ipa_base.cpp               | 135 ++++--
 src/ipa/rpi/common/ipa_base.h                 |   5 +
 src/libcamera/bayer_format.cpp                | 124 +++++
 src/libcamera/camera_manager.cpp              |  40 ++
 .../pipeline/rpi/common/pipeline_base.cpp     |  99 +++-
 .../pipeline/rpi/common/pipeline_base.h       |   5 +-
 src/libcamera/pipeline/rpi/pisp/pisp.cpp      | 459 ++++++++++++------
 src/libcamera/request.cpp                     |   4 +-
 src/libcamera/sensor/camera_sensor_memory.cpp | 237 +++++++++
 src/libcamera/sensor/meson.build              |   1 +
 src/libcamera/stream.cpp                      |   7 +-
 21 files changed, 1101 insertions(+), 201 deletions(-)
 create mode 100644 include/libcamera/internal/camera_sensor_memory.h
 create mode 100644 src/ipa/rpi/cam_helper/cam_helper_default.cpp
 create mode 100644 src/libcamera/sensor/camera_sensor_memory.cpp