| Message ID | 20230915-libyuv-convert-v1-0-1e5bcf68adac@baylibre.com |
|---|---|
| Headers | show |
| Series |
|
| Related | show |
Hi Mattijs, On Fri, Sep 15, 2023 at 09:57:24AM +0200, Mattijs Korpershoek via libcamera-devel wrote: > On some platforms, it's possible that the gralloc implementation > and the CSI receiver cannot agree on a pixel format. > When that happens, there is usually a m2m converter in the pipeline > which handles pixel format conversion. > > On platforms without pixel format converters, such as the AM62x, > we need to do software conversion. > > The AM62x platform: > * uses a CSI receiver (j721e-csi2rx), that only supports > packed YUV422 formats such as YUYV, YVYU, UYVY and VYUY. > * Has a gralloc implementation that only supports of semi-planar > YUV420 formats such as NV12. :-( Is this because the display hardware doesn't support YUYV, or because it's not implement in TI's gralloc ? A quick look at the TRM shows that the DSS supports packed YUV inputs. > This series add support for software conversion using libyuv. > Right now, only YUYV->NV12 is supported. > > This has been send as an RFC because: > > 1. I don't have much media knowledge, nor C++ knowledge so this is > probably not good enough for libcamera's high quality codebase. I'll take that as a compliment, but there's no need to denigrate your skills like that :-) > 2. I'm not sure that transforming the "main Type::Direct" stream into > an Internal stream has no side effects. > > 3. I'm unsure what to do with the gralloc size in case of conversion. > Right now, we always allocate HAL_PIXEL_FORMAT_YCBCR_420_888 in > CameraStream::getBuffer(). This should be adapted both on: > - the gralloc capabilities of the platform > - the conversion (if any) > > This has been tested on a AM62x SK EVM board with an Alinx > AN5641 (ov5640) sensor. That sensor should have disappeared a looooong time ago... > In Android 13. > It might work on BeaglePlay with the same sensor, but I have not > tested that. > > Both preview and still capture are functional with the > default com.android.camera2 app. > Note that video recording is not supported, because that would require > a similar conversion from YUYV->MJPEG which I did not investigate yet. > > I would appreciate some feedback. Thanks a lot in advance if you > have a look! > > Depends on: > - https://patchwork.libcamera.org/cover/18995/ > > Signed-off-by: Mattijs Korpershoek <mkorpershoek@baylibre.com> > --- > Mattijs Korpershoek (7): > android: yuv: separate source destination in length check > android: yuv: loop over each plane for size check > android: yuv: prepare support for other pixel formats > android: camera_device: support when no Direct stream is found > android: camera_stream: add add explicit input configuration > android: yuv: add YUYV -> NV12 conversion > WIP: android: add YUYV->NV12 format conversion via libyuv > > src/android/camera_capabilities.cpp | 90 ++++++++++++++++++- > src/android/camera_capabilities.h | 4 + > src/android/camera_device.cpp | 9 +- > src/android/camera_stream.cpp | 57 +++++++++++- > src/android/camera_stream.h | 5 ++ > src/android/yuv/post_processor_yuv.cpp | 156 ++++++++++++++++++++++----------- > src/android/yuv/post_processor_yuv.h | 12 ++- > 7 files changed, 270 insertions(+), 63 deletions(-) > --- > base-commit: 58e501c71c47e57f02afde1bd296a037038cd6d5 > change-id: 20230914-libyuv-convert-fec7082343e4
On some platforms, it's possible that the gralloc implementation and the CSI receiver cannot agree on a pixel format. When that happens, there is usually a m2m converter in the pipeline which handles pixel format conversion. On platforms without pixel format converters, such as the AM62x, we need to do software conversion. The AM62x platform: * uses a CSI receiver (j721e-csi2rx), that only supports packed YUV422 formats such as YUYV, YVYU, UYVY and VYUY. * Has a gralloc implementation that only supports of semi-planar YUV420 formats such as NV12. This series add support for software conversion using libyuv. Right now, only YUYV->NV12 is supported. This has been send as an RFC because: 1. I don't have much media knowledge, nor C++ knowledge so this is probably not good enough for libcamera's high quality codebase. 2. I'm not sure that transforming the "main Type::Direct" stream into an Internal stream has no side effects. 3. I'm unsure what to do with the gralloc size in case of conversion. Right now, we always allocate HAL_PIXEL_FORMAT_YCBCR_420_888 in CameraStream::getBuffer(). This should be adapted both on: - the gralloc capabilities of the platform - the conversion (if any) This has been tested on a AM62x SK EVM board with an Alinx AN5641 (ov5640) sensor. In Android 13. It might work on BeaglePlay with the same sensor, but I have not tested that. Both preview and still capture are functional with the default com.android.camera2 app. Note that video recording is not supported, because that would require a similar conversion from YUYV->MJPEG which I did not investigate yet. I would appreciate some feedback. Thanks a lot in advance if you have a look! Depends on: - https://patchwork.libcamera.org/cover/18995/ Signed-off-by: Mattijs Korpershoek <mkorpershoek@baylibre.com> --- Mattijs Korpershoek (7): android: yuv: separate source destination in length check android: yuv: loop over each plane for size check android: yuv: prepare support for other pixel formats android: camera_device: support when no Direct stream is found android: camera_stream: add add explicit input configuration android: yuv: add YUYV -> NV12 conversion WIP: android: add YUYV->NV12 format conversion via libyuv src/android/camera_capabilities.cpp | 90 ++++++++++++++++++- src/android/camera_capabilities.h | 4 + src/android/camera_device.cpp | 9 +- src/android/camera_stream.cpp | 57 +++++++++++- src/android/camera_stream.h | 5 ++ src/android/yuv/post_processor_yuv.cpp | 156 ++++++++++++++++++++++----------- src/android/yuv/post_processor_yuv.h | 12 ++- 7 files changed, 270 insertions(+), 63 deletions(-) --- base-commit: 58e501c71c47e57f02afde1bd296a037038cd6d5 change-id: 20230914-libyuv-convert-fec7082343e4 Cheers,