| Message ID | 20200806163639.12971-1-david.plowman@raspberrypi.com |
|---|---|
| Headers | show |
| Series |
|
| Related | show |
Hi everyone I was wondering if I could give this a gentle nudge? I also thought it might help if I explain a bit more what we're up to - it's a bit less random than it might seem! Some of you will know our existing "raspicam" apps, raspistill, raspivid etc. The plan is to introduce libcamera equivalents hoping to encourage more of our users to move to libcamera. (PiCamera - Python bindings for our existing stack - is also popular of course, but we're tackling the "raspicam" apps first.) The aim is therefore to plug the most obvious functionality gaps, which we've identified as: 1. Raw stream alongside video. This is addressed by some of Naush's recent patches on zero-copy raw buffers. 2. Ability to set camera configuration (esp. exposure/gain) before the camera starts. This is very important to many users. 3. User-level transforms of the image (e.g. 180 degree rotation). 4. Lack of digital zoom. Having addressed those we plan to start opening the apps to users to try, and being libcamera-based they could be helpful to non-Pi users too. I hope that extra information is useful! Thanks and best regards David On Thu, 6 Aug 2020 at 17:36, David Plowman <david.plowman@raspberrypi.com> wrote: > > Hi everyone > > Here's a second version of the 2d transform implementation (still for > discussion only). It follows the same principles as the previous > version, but the ALSC issues that I ran into are no longer part of > this set (indeed they have already been merged). > > There are now 5 patches. I'll add the relevant documentation once we > can feel confident there won't be too much more churn. That leaves: > > 1. The first merely adds the Transform enum. I to-ed and fro-ed a bit > here, and decided, after doing it both ways, that wrapping a class > round the enum didn't add a whole lot, and just left me with more > questions and more boilerplate. Anyway, see what you think... > > 2. The second patch adds the Transform enum to the > CameraConfiguration. I've also amended all the pipeline handlers > (including the Raspberry Pi one) simply to coerce it to the Identity > if it wasn't, marking the configuration as "adjusted". > > 3. This patch updates the Raspberry Pi pipeline handler to allow all > non-transposing transforms, and to set the flip bits in the sensor. > > 4. Here I just plumb the Transform through to the Raspberry Pi IPA > where it gets put into the CameraMode structure. This bothered me > first time round, but on reflection I think it's pragmatic to treat it > like that from the point of view of the IPAs. > > 5. This updates the ALSC algorithm to transform the calibrated tables > correctly. Note that we regenerate the tables from scratch if the > transform changes. > > Best regards > David > > David Plowman (5): > libcamera: Add Transform enum to represet 2d plane transforms. > libcamera: Add user Transform to CameraConfiguration > libcamera: raspberrypi: Set camera flips correctly from user transform > libcamera: raspberrypi: Plumb user transform through to IPA > libcamera: ipa: raspberrypi: ALSC: Handle user transform > > include/libcamera/camera.h | 3 + > include/libcamera/meson.build | 1 + > include/libcamera/transform.h | 58 +++++++++++++ > src/ipa/raspberrypi/controller/camera_mode.h | 4 + > src/ipa/raspberrypi/controller/rpi/alsc.cpp | 13 ++- > src/ipa/raspberrypi/raspberrypi.cpp | 48 ++++++----- > src/libcamera/camera.cpp | 2 +- > src/libcamera/meson.build | 1 + > src/libcamera/pipeline/ipu3/ipu3.cpp | 5 ++ > .../pipeline/raspberrypi/raspberrypi.cpp | 26 +++++- > src/libcamera/pipeline/rkisp1/rkisp1.cpp | 5 ++ > src/libcamera/pipeline/simple/simple.cpp | 5 ++ > src/libcamera/pipeline/uvcvideo/uvcvideo.cpp | 5 ++ > src/libcamera/pipeline/vimc/vimc.cpp | 5 ++ > src/libcamera/transform.cpp | 83 +++++++++++++++++++ > 15 files changed, 238 insertions(+), 26 deletions(-) > create mode 100644 include/libcamera/transform.h > create mode 100644 src/libcamera/transform.cpp > > -- > 2.20.1 >
Hi David, On Mon, Aug 17, 2020 at 06:02:52PM +0100, David Plowman wrote: > Hi everyone > > I was wondering if I could give this a gentle nudge? I also thought it > might help if I explain a bit more what we're up to - it's a bit less > random than it might seem! I've now reviewed the patches. Sorry again about the delay. > Some of you will know our existing "raspicam" apps, raspistill, > raspivid etc. The plan is to introduce libcamera equivalents hoping to > encourage more of our users to move to libcamera. (PiCamera - Python > bindings for our existing stack - is also popular of course, but we're > tackling the "raspicam" apps first.) > > The aim is therefore to plug the most obvious functionality gaps, > which we've identified as: > > 1. Raw stream alongside video. This is addressed by some of Naush's > recent patches on zero-copy raw buffers. > > 2. Ability to set camera configuration (esp. exposure/gain) before the > camera starts. This is very important to many users. > > 3. User-level transforms of the image (e.g. 180 degree rotation). > > 4. Lack of digital zoom. > > Having addressed those we plan to start opening the apps to users to try, > and being libcamera-based they could be helpful to non-Pi users too. I'll continue with the review of the other pending series :-) > I hope that extra information is useful! > > On Thu, 6 Aug 2020 at 17:36, David Plowman wrote: > > > > Hi everyone > > > > Here's a second version of the 2d transform implementation (still for > > discussion only). It follows the same principles as the previous > > version, but the ALSC issues that I ran into are no longer part of > > this set (indeed they have already been merged). > > > > There are now 5 patches. I'll add the relevant documentation once we > > can feel confident there won't be too much more churn. That leaves: > > > > 1. The first merely adds the Transform enum. I to-ed and fro-ed a bit > > here, and decided, after doing it both ways, that wrapping a class > > round the enum didn't add a whole lot, and just left me with more > > questions and more boilerplate. Anyway, see what you think... > > > > 2. The second patch adds the Transform enum to the > > CameraConfiguration. I've also amended all the pipeline handlers > > (including the Raspberry Pi one) simply to coerce it to the Identity > > if it wasn't, marking the configuration as "adjusted". > > > > 3. This patch updates the Raspberry Pi pipeline handler to allow all > > non-transposing transforms, and to set the flip bits in the sensor. > > > > 4. Here I just plumb the Transform through to the Raspberry Pi IPA > > where it gets put into the CameraMode structure. This bothered me > > first time round, but on reflection I think it's pragmatic to treat it > > like that from the point of view of the IPAs. > > > > 5. This updates the ALSC algorithm to transform the calibrated tables > > correctly. Note that we regenerate the tables from scratch if the > > transform changes. > > > > David Plowman (5): > > libcamera: Add Transform enum to represet 2d plane transforms. > > libcamera: Add user Transform to CameraConfiguration > > libcamera: raspberrypi: Set camera flips correctly from user transform > > libcamera: raspberrypi: Plumb user transform through to IPA > > libcamera: ipa: raspberrypi: ALSC: Handle user transform > > > > include/libcamera/camera.h | 3 + > > include/libcamera/meson.build | 1 + > > include/libcamera/transform.h | 58 +++++++++++++ > > src/ipa/raspberrypi/controller/camera_mode.h | 4 + > > src/ipa/raspberrypi/controller/rpi/alsc.cpp | 13 ++- > > src/ipa/raspberrypi/raspberrypi.cpp | 48 ++++++----- > > src/libcamera/camera.cpp | 2 +- > > src/libcamera/meson.build | 1 + > > src/libcamera/pipeline/ipu3/ipu3.cpp | 5 ++ > > .../pipeline/raspberrypi/raspberrypi.cpp | 26 +++++- > > src/libcamera/pipeline/rkisp1/rkisp1.cpp | 5 ++ > > src/libcamera/pipeline/simple/simple.cpp | 5 ++ > > src/libcamera/pipeline/uvcvideo/uvcvideo.cpp | 5 ++ > > src/libcamera/pipeline/vimc/vimc.cpp | 5 ++ > > src/libcamera/transform.cpp | 83 +++++++++++++++++++ > > 15 files changed, 238 insertions(+), 26 deletions(-) > > create mode 100644 include/libcamera/transform.h > > create mode 100644 src/libcamera/transform.cpp